Regarding Android's Mediacodec speed concerns and bottlenecks

问题

Currently, my encoding a 30 FPS video consisting of 90 frames using Mediacodec takes 2100-2400 ms. I'm using the code found here, except with the generateSurfaceFrame(i) part being replaced with:

private void generateFrame(Bitmap bitmap, Rect source)
    {
        long drawFrameStartTime = System.currentTimeMillis();
        Canvas canvas = mInputSurface.lockCanvas(null);
        canvas.drawRect(0, 0, mSquareDim, mSquareDim, clearPaint);
        //Process the canvas below
        try
        {
            canvas.drawBitmap(bitmap, source, new Rect(0, 0, mSquareDim, mSquareDim), antiAliasPaint);
        }
        //Process the canvas above
        catch(Exception e) {Log.e("renderExc", e.toString());}
        finally {mInputSurface.unlockCanvasAndPost(canvas);}
        long drawFrameEndTime = System.currentTimeMillis();
        Log.i("frame_draw_time", (drawFrameEndTime - drawFrameStartTime)+"");
    }

And the putting the frames into the MediaMuxer part with the code found and adapted from here - the one using the CircularBuffer class from Grafika. The muxer had to be released independently from the rest using that code.

I'm still concerned about Mediacodec's other bottlenecks when it comes to speed, though, and I'm targeting API 18 (minimum) at the moment. My questions are:

Should I start using Asynchronous mode, and how much faster can it be than Synchronous mode?
Is drawing the frames with OpenGL faster than the Surface-Canvas method described above?
Are there other bottlenecks in Mediacodec I should be concerned about?

Full source code will be provided when asked.

回答1:

Asynchronous mode in itself isn't faster than the synchronous mode (used correctly), but the asynchronous interface makes it clearer how it is supposed to be used. In particular, whichever interface you're using, do not wait for an output buffer directly after passing in one buffer/surface to the encoder - you can (and should) check for output, but don't block waiting for one - instead proceed to provide the next input buffer/surface instead (as long as the input doesn't block).
Rendering with OpenGL instead of Canvas should most probably be faster. See e.g. Android Graphics architecture, which says:

Note in particular that while the Canvas provided to a View's onDraw() method may be hardware-accelerated, the Canvas obtained when an app locks a Surface directly with lockCanvas() never is.

回答2:

@mstorsjo hit the high points. You can find an example of GLES-based video generation in Grafika, e.g. MovieEightRects uses the GeneratedMovie helper class.

If you measure the end-to-end video encoding time you will be measuring both throughput and latency. MediaCodec talks to a separate process (mediaserver) through IPC, which has to allocate hardware resources through an OMX driver. It takes a little time for this to warm up, and there's some amount of latency shoving frames through the codec.

Generating frames faster won't affect the overall encoding speed so long as you're generating as fast as the encoder can encode. The occasional stall when sending data to MediaMuxer will plug up the pipeline, hence the Horizon Camera blog post, so it's reasonable to worry about that (especially if your source drops frames when the encoding pipeline stalls).

来源：https://stackoverflow.com/questions/35769857/regarding-androids-mediacodec-speed-concerns-and-bottlenecks

标签

android

mediacodec