ffmpeg RTSP error while decoding MB

Question

I\'m using ffmpeg to read an h264 RTSP stream from a Cisco 3050 IP camera and reencode it to disk as h264 (there are reasons why I\'m not just using -codec:copy).

Answer 1

It does sound like packet loss is an issue. Higher video resolution and greater motion both increase the bitrate of the encoded video stream which will increase your packet loss. Depending on which packet is lost, you will see varying errors in the decoding process as you indicated in your post.

The higher system load running ffmpeg also indicates that your network card might be dropping packets, when e.g. ffmpeg takes too long to read them while it is busy transcoding the video.

First question is what is your network topology? Streaming over the public Internet is a lot harder than streaming over your LAN. What kind of switches/routers are in the network?

Next question, what bitrate is your camera streaming at? Try reducing this and check the results. Be systematic in your approach i.e.

• don't transcode at first.
• just receive the video.
• write it to file.
• Check for packet loss/video artifacts.
• start at lower bitrates e.g. 100kbps and increase this if no loss is evident

The next thing I would try to do is to increase the size of the receiver buffers. While I am not that familiar with ffmpeg, it looks like you can set it via recv_buffer_size as indicated here. You then need to work out a reasonably big enough size based on your camera configuration to store e.g. a couple (5?) of seconds of video data. Check if there are less artifacts as you increase the receiver buffer size or longer periods without artifacts.

Of course if your processor is too slow to transcode the video in real-time, you will run out of space sooner or later, in which case, you might have to transcode to a lower resolution/bitrate or use less intensive encoder settings, etc or run the transcoding on a faster machine.

Also, note that adjusting receiver buffer size will not compensate for packet loss occurring on the public Internet so the above will help assuming you're streaming on a local network that supports the bitrate of the camera. If you exceed the bandwidth of the network you can expect packet loss. In that case streaming over TCP could help somewhat (at least until the receiver buffer overruns eventually).

More things you can try if the above does not help or solve the problem completely:

• Sniff the incoming traffic with wireshark or tcpdump. Have a look at the traces. Filter the trace using "RTSP". You should be able to see the RTP traffic where consecutive RTP packets have increasing sequence numbers e.g. 20, 21, 22, 23, etc. If you see missing sequence numbers, then you've got packet loss and try streaming over TCP. Repeat the trace while streaming over TCP. Also, remember to increase the receiver buffer size also when streaming over TCP.

In summary you have a pipeline architecture and you need to determine where in the pipeline the loss is occurring:

camera -> network -> receiver buffer (OS) -> application (ffmpeg)

Answer 2

Looking at the initial error message:

[rtsp @ 0x7f268c5e9220] max delay reached. need to consume packet
[rtsp @ 0x7f268c5e9220] RTP: missed 40 packets

I guess that you are loosing UDP packets. The rest of the H.264 error messages are caused by receiving an incomplete bitstream. Now key is to isolate the issue. Is your network dropping packets? Or is your sever too slow or overloaded receiving the UDP (RTP).

First I'd check the UDP buffer size of your OS. https://access.redhat.com/documentation/en-US/JBoss_Enterprise_Web_Platform/5/html/Administration_And_Configuration_Guide/jgroups-perf-udpbuffer.html

If increasing the UDP buffer size doesn't help - use ffmpeg with -codec:copy to lower the CPU load. Do you still get errors? Since you want to reencode consider using Intel Quicksync -vcodec h264_qsv or some other hardware encoder lowering your CPU load.

The question is not so much about if the PC has enough CPU. But more about identifying the bottle neck in the processing pipeline. Your H.264 encoder (x264) may over subscribe your CPU so that you get momentary peak loads that result in packet drops. Try limiting the number of threads for x264 and/or lower the quality to 'fast' or 'faster'.