Decode audio and video and process both streams — ffmpeg, sdl, opencv

Question

My goal is to proceed on audio and video of mpeg-2 file independently, and to keep synchronicity on both flows. Duration of video is about 1 or 2 minutes maximum.

Answer 1

See here for syncing:

http://dranger.com/ffmpeg/tutorial06.html

and here:

http://dranger.com/ffmpeg/tutorial07.html

Answer 2

I solved the "Unsupported codec" error. Remplace AVMEDIA_TYPE_VIDEO with AVMEDIA_TYPE_AUDIO in the following line :

if (pFormatCtx->streams[i]->codec->codec_type == AVMEDIA_TYPE_VIDEO && audioStream < 0){
          audioStream = i;
}

Grayscale conversion is not true due to uchar opencv pixel format. I still have the synchro not working ... any help ?

Thanks