How to determine a video file's framerate with MediaCodec, MediaExtractor or MediaMetadataRetriever?

Question

How to I extract the frame rate of a recorded video file? I know that there is MediaFormat.KEY_FRAME_RATE and that I can access MediaFormat objects through MediaExtractor. H

Answer 1

When using MediaExtractor, if your video track has a stable FPS, then you know for sure your MediaExtractor is advancing by the frame duration you are searching for.

Before doing anything, just after having set up your MediaExtractor, you can do the following:

mediaExtractor.Advance();
var fps = 1000000f / (float) mediaExtractor.SampleTime;
mediaExtractor.SeekTo(0, MediaExtractorSeekTo.None);

Like I said, you can't take for grant that all your frames have the same duration. Frame presentation time are totally arbitrary, but I feel it's not common to not have a stable FPS.

Answer 2

It looks like there is no way to get framerate through any of official API functions. It might require some extra logic - count times between timestamps or to parse headers info. in general h.264 standard allows variable framerate, so frame times could differ from one to one. for example if can show some static picture for several seconds or so

Answer 3

You can use ffmpeg to get the frame rate of a video (https://www.ffmpeg.org) - subject to the comments that fadden and Marlon make on variable frames rates in the comments to the accepted answer.

There are a number of ways to include ffmpeg in Android - the one I found the best compromise between complexity and functionality was to wrap the command line ffmpeg program in a Java JNI wrapper.

There are a couple of issues to look out for, including an issue with invoking ffmpeg twice via the wrapper, so it is worth either using or studying existing Android wrapper solutions - e.g.:

• http://writingminds.github.io/ffmpeg-android-java/

From the documentation there:

FFmpeg ffmpeg = FFmpeg.getInstance(context);
try {
  // to execute "ffmpeg -version" command you just need to pass "-version"
  ffmpeg.execute(cmd, new ExecuteBinaryResponseHandler() {

    @Override
    public void onStart() {}

    @Override
    public void onProgress(String message) {}

    @Override
    public void onFailure(String message) {}

    @Override
    public void onSuccess(String message) {}

    @Override
    public void onFinish() {}
  });
} catch (FFmpegCommandAlreadyRunningException e) {
  // Handle if FFmpeg is already running
}

An example of ffmpeg cmd to get the frame rate, replacing filename with the name of the file you want to check:

cmd = "-i filename";

Your code can then parse the output for the fps - see example output below:

ffmpeg version 2.6.1 Copyright (c) 2000-2015 the FFmpeg developers
  built with llvm-gcc 4.2.1 (LLVM build 2336.11.00)
  configuration: --prefix=/Volumes/Ramdisk/sw --enable-gpl --enable-pthreads --enable-version3 --enable-libspeex --enable-libvpx --disable-decoder=libvpx --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libx264 --enable-avfilter --enable-libopencore_amrwb --enable-libopencore_amrnb --enable-filters --enable-libgsm --enable-libvidstab --enable-libx265 --disable-doc --arch=x86_64 --enable-runtime-cpudetect
  libavutil      54. 20.100 / 54. 20.100
  libavcodec     56. 26.100 / 56. 26.100
  libavformat    56. 25.101 / 56. 25.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 11.102 /  5. 11.102
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  3.100 / 53.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/Users/BBB/Desktop/vid_bigbuckbunny.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 1
    compatible_brands: mp42avc1
    creation_time   : 2010-02-09 01:55:39
  Duration: 00:01:00.10, start: 0.000000, bitrate: 733 kb/s
    Stream #0:0(eng): Audio: aac (LC) (mp4a / 0x6134706D), 22050 Hz, stereo, fltp, 65 kb/s (default)
    Metadata:
      creation_time   : 2010-02-09 01:55:39
      handler_name    : Apple Sound Media Handler
    Stream #0:1(eng): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p(tv, smpte170m/smpte170m/bt709), 640x360, 612 kb/s, 23.96 fps, 24 tbr, 600 tbn, 1200 tbc (default)
    Metadata:
      creation_time   : 2010-02-09 01:55:39
      handler_name    : Apple Video Media Handler
    Stream #0:2(eng): Data: none (rtp  / 0x20707472), 45 kb/s
    Metadata:
      creation_time   : 2010-02-09 01:55:39
      handler_name    : hint media handler
    Stream #0:3(eng): Data: none (rtp  / 0x20707472), 5 kb/s
    Metadata:
      creation_time   : 2010-02-09 01:55:39
      handler_name    : hint media handler

Bear in mind that ffmpeg syntax and output format can change over time, so for the parsing in particular you need to allow for this, although looking for 'fps' in the output will probably generally work anyway.

Also be aware that many video 'containers' (e.g. mp4, often referred to as the video file) may contain multiple video streams and the different streams may have different frame rates. This will be included in the output from ffmpeg so something else to check for when parsing the output.

And finally, just a reminder again that the frame rate may not be constant for a video so use with this in mind.