I'm looking for a solution to play raw h264 stream coming from a native server through WebSocket live on a browser. I tried many third party h264 decoders in JavaScript and each one has its own issues. Decoders based on broadway cannot decode main and high profile h264. Other decoders are too slow to decode 1080p frames. I tried converting raw h264 into fragmented mp4 in JavaScript but the playback is very ugly when decoding bidirectional frames. I also tried webrtc but it seems impossible to implement peer-connection between browser and a native server. Any suggestions?
-
1You will need to mux your h.264 stream into an MP4/ISO BMFF container, and then use MediaSource Extensions to play it back. You mentioned you tried this... can you show the code you tried?– BradCommented Jan 2, 2019 at 17:25
-
Yes i tried it. I used this project github.com/xevokk/h264-converter to do the muxing. It does not play well in chrome due to the fact that chrome decoder refers decoding timestamp instead of presentation timestamp for decoding b-frames.– Kiran RajCommented Jan 3, 2019 at 13:46
-
MSE in Chrome can handle b-frames just fine (unlike WebRTC), you just need to provide composition times in your fragmented mp4. So whatever library you are using for muxing, make sure to set composition times which are the difference between presentation and decoder timestamps.– user1390208Commented Jan 3, 2019 at 16:38
-
Thanks for the idea @user1390208. Can u suggest a standard muxing library that handles cts properly– Kiran RajCommented Jan 7, 2019 at 6:32
-
@user1390208 by "unlike WebRTC", do you mean that WebRTC doesn't handle b frames? Does WebRTC support decoding raw h264 stream?– ZipCommented Apr 26, 2019 at 5:34
|
Show 4 more comments
1 Answer
The best I have seen used (not had hands-on experience using it my self) is https://github.com/samirkumardas/jmuxer
There is an example of how to handle streaming data via WebSockets at https://github.com/samirkumardas/jmuxer/blob/master/example/index-h264.html
var socketURL = 'ws://localhost:8080';
var jmuxer = new JMuxer({
node: 'player',
mode: 'video',
flushingTime: 1000,
fps: 30,
debug: true
});
var ws = new WebSocket(socketURL);
ws.binaryType = 'arraybuffer';
ws.addEventListener('message',function(event) {
jmuxer.feed({
video: new Uint8Array(event.data)
});
});
ws.addEventListener('error', function(e) {
console.log('Socket Error');
});
