I've been trying to setup a baby monitor with a Raspberry Pi Zero and a Logitech C920 webcam. It does work with VLC (cvlc) but it lags too much and gets worse over time.
So I am playing around with ffmpeg and I am getting some better results. This is what I've done so far.
First I set the webcam to output h264 1080p natively (the Pi Zero W can't afford to do any transcoding).
v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=1
Now, if I stream audio only with
ffmpeg \
-f alsa \
-i hw:1,0 \
-vn \
-flags +global_header \
-acodec aac \
-ac 1 \
-ar 16000 \
-ab 16k \
-f rtp rtp://192.168.0.10:5002 > audio.sdp
It works great and the lag is about 1 second (definitely acceptable).
If I stream video only with
ffmpeg \
-f v4l2 \
-vcodec h264 \
-i /dev/video0 \
-an \
-vcodec copy \
-pix_fmt yuv420p \
-r 30 \
-b:v 512k \
-flags +global_header \
-f rtp rtp://192.168.0.10:5000 > video.sdp
Same result, very little lag (for some reason the first -vcodec is necessary to force the webcam to output h264).
However, when I stream both with
ffmpeg \
-f v4l2 \
-vcodec h264 \
-i /dev/video0 \
-f alsa \
-i hw:1,0 \
-an \
-preset ultrafast \
-tune zerolatency \
-vcodec copy \
-pix_fmt yuv420p \
-r 30 \
-b:v 512k \
-flags +global_header \
-f rtp rtp://192.168.0.10:5000 \
-vn \
-flags +global_header \
-acodec aac \
-ac 1 \
-ar 16000 \
-ab 16k \
-f rtp rtp://192.168.0.10:5002 > both.sdp
The lag ramps up to ~10 seconds and audio and video are out of sync. Does anybody know why?
I've tried UDP and TCP instead of RTP but then the lag is always high, even with audio/video only.
Any suggestion is much appreciated.
P.S. On the client side (MacOS) I'm receiving with
ffplay -protocol_whitelist file,rtp,udp -i file.sdp
