FFmpeg can't decode H264 stream/frame data

Question

Recently I had chance to work with two devices that are streaming the H264 through RTSP. And I\'ve ran into some problem trying to decompress this stream using FFmpeg librar

Answer 1

Ok, managed to make things working.

• I needed to include the sequence (SPS) and picture parameter sets (PPS) for my frame data before sending frame to the FFmpeg.
• I needed to add 4 extra bytes "00 00 00 01" after SPS and PPS data.

Here is a little picture showing what I mean:

Bytes "65 88..." is where my original frame data begins.

This SPS and PPS information was not included in RTP packet. I'm using Live555 library for RTSP streaming, so I've used subsessions "fmtp_spropparametersets" function to get what I need. This information was Base64 encoded. (Sample: Something like this "Z0KAKNoC0EkQ,aM48gA==") Note that there are two "parameters" SPS and PPS seperated by "," and those parameters doesn't have a "00 00 00 01" included, so you need to add them.

Some code sample (I'm using Qt library here):

QByteArray        ba          = pSubSession->fmtp_spropparametersets();
QList<QByteArray> recordsList = ba.split(',');

for (int i = 0; i < recordsList.size(); i++)
{
   mExtraData.append(char(0x00));
   mExtraData.append(char(0x00));
   mExtraData.append(char(0x00));
   mExtraData.append(char(0x01));

   mExtraData += QByteArray::fromBase64(recordsList.at(i));
}

Now for every frame I do something like this:

QByteArray ba = QByteArray(4, 0); // Prepare the "00 00 00 01"
           ba[3] = 0x01;

mpTrackVideo->buffer.insert(0, mExtraData);
mpTrackVideo->buffer.insert(mExtraData.size(), ba);

Year ago I thought I had H264 stream support integrated in my project till I've had chance to test it with some other devices... So you need to keep in mind that some devices might send SPS and PPS data for every I frame... and some might not!