Encode a series of Images into a Video

Question

How can I create a video file from a series of images/photos on Android? Can this be done with the current SDK? or do I need a codec library?

Answer 1

You can use a free open source library called JCodec ( http://jcodec.org ) that contains pure java implementations of some popular video formats and codecs, including: H.264 ( AVC ), MPEG 1/2, Apple ProRes, JPEG, MP4 ( ISO BMF ), MPEG PS, MPEG TS, Matroska.
You can use the CORRECTED class below that utilizes JCodec low-level API:

public class SequenceEncoder {
    private SeekableByteChannel ch;
    private Picture toEncode;
    private RgbToYuv420 transform;
    private H264Encoder encoder;
    private ArrayList<ByteBuffer> spsList;
    private ArrayList<ByteBuffer> ppsList;
    private CompressedTrack outTrack;
    private ByteBuffer _out;
    private int frameNo;
    private MP4Muxer muxer;

    public SequenceEncoder(File out) throws IOException {
        this.ch = NIOUtils.writableFileChannel(out);

        // Transform to convert between RGB and YUV
        transform = new RgbToYuv420(0, 0);

        // Muxer that will store the encoded frames
        muxer = new MP4Muxer(ch, Brand.MP4);

        // Add video track to muxer
        outTrack = muxer.addTrackForCompressed(TrackType.VIDEO, 25);

        // Allocate a buffer big enough to hold output frames
        _out = ByteBuffer.allocate(1920 * 1080 * 6);

        // Create an instance of encoder
        encoder = new H264Encoder();

        // Encoder extra data ( SPS, PPS ) to be stored in a special place of
        // MP4
        spsList = new ArrayList<ByteBuffer>();
        ppsList = new ArrayList<ByteBuffer>();

    }

    public void encodeImage(BufferedImage bi) throws IOException {
        if (toEncode == null) {
            toEncode = Picture.create(bi.getWidth(), bi.getHeight(), ColorSpace.YUV420);
        }

        // Perform conversion
        for (int i = 0; i < 3; i++)
            Arrays.fill(toEncode.getData()[i], 0);
        transform.transform(AWTUtil.fromBufferedImage(bi), toEncode);

        // Encode image into H.264 frame, the result is stored in '_out' buffer
        _out.clear();
        ByteBuffer result = encoder.encodeFrame(_out, toEncode);

        // Based on the frame above form correct MP4 packet
        spsList.clear();
        ppsList.clear();
        H264Utils.encodeMOVPacket(result, spsList, ppsList);

        // Add packet to video track
        outTrack.addFrame(new MP4Packet(result, frameNo, 25, 1, frameNo, true, null, frameNo, 0));

        frameNo++;
    }

    public void finish() throws IOException {
        // Push saved SPS/PPS to a special storage in MP4
        outTrack.addSampleEntry(H264Utils.createMOVSampleEntry(spsList, ppsList));

        // Write MP4 header and finalize recording
        muxer.writeHeader();
        NIOUtils.closeQuietly(ch);
    }

    public static void main(String[] args) throws IOException {
        SequenceEncoder encoder = new SequenceEncoder(new File("video.mp4"));
        for (int i = 1; i < 100; i++) {
            BufferedImage bi = ImageIO.read(new File(String.format("folder/img%08d.png", i)));
            encoder.encodeImage(bi);
        }
        encoder.finish();
    }
}

Answer 2

There is no built-in support for this. You would need to either find some Java source code that does what you need, or some C/C++ code you can turn into a library and use via the NDK. Or, if the end goal is for the video to be on a server, upload the images/photos and have the server create the video.

Answer 3

ffmpeg is your rescue ranger. You can download the FFMPEG port for android. Refer to the question FFmpeg on Android for more details.

The port supports almost everything that you'll ever need, including the formats - input image formats being GIF, JPG, PNG, BMP etc and output video formats being AVI, MP4 (containers) with a lot of codecs.

Answer 4

I agree with Mark. Among the C/C++ libraries are ffmpeg here or X264 here. I have to say I havn't found them or the Android NDK easy to use but that's probably because I don't really know much about C/C++ and JNI. If you're interested in this route, then the RockPlayer free app for Android has the pre-built shared libraries from ffmpeg ready for use on Android. They claim these libraries only use LGPL components but you'd have to satisfy yourself on this, I guess. With regards to Java, there is a port (of sorts) of ffmpeg called, unsurprisingly, jffmpeg which you can access here but it still calls out to much of the existing ffmpeg framework, so you're back in NDK land. It is possible to convert a series of images to video with the Java Media Framework (JMF) but it has the following drawbacks:

1. Limited number of video formats.
2. Doesn't produce videos which can be played back on most (if not all) Android phones e.g. H264 or MPEG4.
3. The JMF code is now quite old, hasn't progressed in a long time, is bloated and not well structured so there may be additional Java dependencies which are not packaged in Android.

Another option I've seen used is Adobe Air, but it's a 17Mb payload which some users complain about.

There are lots of other questions here on Stack Overflow regarding ffmpeg and Android NDK.

Good luck with the project.