Key Frame Extraction From Video

Question

I need Need To extract Key Frames from Video/Stream.So is there any standard implementation. I am using open CV. (Currently i am extracting frames each seco

Answer 1

I will assume that a keyframe is a frame presenting content a lot different from the previous ones (it's not a formal definition, but it fits). Take frames i and i+1. Use cv2.absDiff to compute the difference between the frames and cv2.sumElems to get the sum of all pixels differences. Do this for all frames i. This will reduce your video stream to an one dimensional signal. Look for peaks in this signal and pick keyframes corresponding to these peaks. To find peaks choose a threshold on this signal either manually by finding a frame you deem to be key, and letting its error be the error threshold or automatically using statistics (e.g. any frame i+1 where the error is greater than 1 stdev from the mean error).

Answer 2

Using ffmpeg you can extract all key frames using the following code:

ffmpeg -vf select="eq(pict_type\,PICT_TYPE_I)" -i yourvideo.mp4 -vsync 2 -s 160x90 -f image2 thumbnails-%02d.jpeg

What follows -vf in a ffmpeg command line is a Filtergraph description. The select filter selects frames to pass in output. The constant of the filter is “pict_type” and the value “ PICT_TYPE_I”. So ffmpeg is only passing key frames to the output.

-vsync 2 prevents ffmpeg to generate more than one copy for each key frame.

-f image2 writes video frames to image files. The output filenames are specified by a pattern, which can be used to produce sequentially numbered series of files. The pattern may contain the string "%d" or "%0Nd".

Reference: http://www.videoproductionslondon.com/blog/scene-change-detection-during-encoding-key-frame-extraction-code

Answer 3

You can use ffprobe to extract key frames. It is a tool in ffmpeg.

Use the command:

ffprobe in.mp4 -select_streams v -show_entries frame=key_frame,pkt_pts_time -of csv=nk=1:p=0 | findstr "1,"

Answer 4

The ffmpeg solution should work well.

For someone facing problems with the select filter 'eq(pict_type\,PICT_TYPE_I)', you might want to try the filter as 'eq(pict_type\,I)'. This was broken for a while so some versions of ffmpeg might not recognize the constant. The fix can be seen here.

The final command which worked for me finally was:

ffmpeg -vf select='eq(pict_type\,I)' -i yourVideo.mp4 -vsync 2 -s 160x90 -f image2 thumbnails-%02d.jpeg

Answer 5

If something wrong with the above code, try this argument order instead.

ffmpeg -i yourVideo.mp4 -vf select='eq(pict_type\,I)' -vsync 2 -s 160x90 -f image2 thumbnails-%02d.jpeg