8

I want to achieve the following, but it remains unclear if this is possible.

The current scenario:

Someone is streaming a video with audio through OBS to a media server, clients connect through a website.

[OBS Stream/Video Stream] -> [AWS/External Streaming Service] -> Clients

The wanted scenario:

capture this stream through a custom media server and manipulate it by injecting certain metadata at certain moments during the livestream. Note the importance of live.

[OBS Stream/Video Stream] -> [My Custom Node.js Server to insert metadata] -> [AWS/External Streaming Service] -> Clients

The idea:

The idea is that I want to synchronize the stream to some popup for example. The default protocol stream seems to be RTMP from OBS, but maybe this can be changed. At a given time during the livestream, an html5 videoplayer on the website can read these tags from the livestream (through some additional library such as video.js) and tell the JS application to show some text. In the end, it boils down to synchronizing the video stream to a text stream (eg from a websocket connection)

Potential solutions:

  • ID3 tags. I read about ID3 tags in MP3 files, but this does not seem to be what i'm looking as it needs a complete .mp3 file upfront and is not used for streams (Dynamically Inject ID3 in FFMPEG Live Stream). What I want is to dynamically inject metadata into this stream. For example, inject an id at any time (dynamically chosen) which references to a database for example should suffice.

  • LTC/Linear Time Code/SMPTE is this possible to embed that in a video stream somehow with node.js? that would enable me to match timings with an id on the client.

Is this possible to do given an incoming video stream with audio? and if so, what is the format of the stream and how do I inject metadata?

EDIT: it seems RTMP is not supported without flash in the browser. This is a no-go so I will need to use another stream format such as HLS/FLV?

Improve this question
14
  • what is the format of the stream What is the format the player expects? I assume you want to read the metadata back out?
    – szatmary
    Commented Apr 17, 2020 at 22:11
  • @szatmary I don't really care I guess, any decent supported format that a HTML5 video tag can play.
    – Captain Obvious
    Commented Apr 18, 2020 at 9:11
  • There is no supported inband metadata format for MSE. You will need to research players that support such things. I’m not sure if hls.js supports in band ID3, but it may.
    – szatmary
    Commented Apr 19, 2020 at 1:56
  • MSE? Also, reading ID3 is not a problem (if this is the solution). However the problem is inserting these tags dynamically into a video livestream as from what I gathered this is done upfront (and not timed?, eg i want 5 ID3 tags every 10 seconds from a certain point, it seems ID3 tags is just metadata for one file).
    – Captain Obvious
    Commented Apr 19, 2020 at 13:02
  • @CaptainObvious According to the comment dated the 15th March 2020 on this site , this feature you want is not yet available with obs, but is in progress. You can add metadata when creating your clip, but at present, this can't be edited once the video is playing. These features are only available for outgoing videos in any case, i,e, where you are the publisher, not incoming streams
    – Rachel Gallen
    Commented Apr 21, 2020 at 14:32

1 Answer 1

Reset to default
5
+200

Sounds like using something like Liquidsoap as your streaming server would do the trick for inserting the metadata into the stream. Plenty of options for manipulating metadata for you to explore.

As for client side decoding you could perhaps use a javascript readable stream within a service worker to split the server output into metadata/video and process as you see fit.

I did a similar thing for processing inband metadata on an infinite mp3 stream which might give you some ideas on where to start. You can find the code for that here

Improve this answer
3
  • 1
    Can you provide more details? What should the format of the incoming videostream be? What kind of meta data can I insert? In what form, ID3 tags? It seems your proposed a solution for mp3 files, but not for videostreams?
    – Captain Obvious
    Commented Apr 25, 2020 at 9:03
  • Liquidsoap can cope with many formats. I can't give you an example specific to your use case as I haven't had to implement one but the project is extensively documented. My service worker which parses in band metadata was written for use with an mp3 stream but all its doing is consuming a stream of data from a server and splitting it into file and metadata based upon the metadata being spliced in to the original file at set byte intervals. Whether the original file is mp3 or something else isn't really relevant as to how it works.
    – miknik
    Commented Apr 27, 2020 at 17:07
  • I've rewarded you with the bounty since this is the only answer. However, I'll dive a bit deeper in the implementation later since i'm still not convinced this is answer to my question.
    – Captain Obvious
    Commented Apr 28, 2020 at 19:28

Not the answer you're looking for? Browse other questions tagged or ask your own question.