Crop video before encoding with MediaCodec for Grafika's “Continuous Capture” Activity

Question

I am learning about Grafika's "Continuous Capture" Activity, It is about recording a video with MediaCodec.

The activity source code is at https://github.com/google/grafika/blob/master/src/com/android/grafika/ContinuousCaptureActivity.java

The program uses a SurfaceTexture obj to receive data from camera and creates 2 EGLSurface obj with this SurfaceTexture obj, one EGLSurface obj feeds the data to MediaCodec and the other feeds data to SurfaceView for camera preview. MediaCodec encodes the data to h264 data and the MediaMuxer obj writes h264 data to a mp4 file.

But there is a problem, the preview size supported by the camera is landspace (width > height) such as 1920*1080, 1440*1080,720*480 and so on. Usually, we take the phone in portrait orientation when we record a video, so we should use API: Camera.setDisplayOrientation(90) to rotate the picture to a portrait one, then a portrait video will be recorded.

But I want to record a landscape video with the phone portrait in my hand, I have to crop every frame from the camera. My method is that cutting off the bottom and top of every frame picture and retain the middle of the picture, then the left picture will be a landscapce one.

But I am not familiar with opengl, I do not know how to crop the SurfaceTexture data. Could anyone who is good at opengl give me some help?

Answer 1

Take a look at the "texture from camera" activity. Note it allows you to manipulate the image in various ways, notably "zoom". The "zoom" is done by modifying the texture coordinates.

The ScaledDrawable2D class does this; the setScale() call changes the "zoom", rather than scaling the rect itself. Texture coordinates range from 0.0 to 1.0 inclusive, and the getTexCoordArray() method modifies them to span a subset of the texture.

To clip the frames, you'd need to modify the texture coordinates proportionally. For example, if the input video is portrait 720x1280, and you want 720x720, you would change the coordinates from this:

[0.0, 0.0]  [1.0, 0.0]
[0.0, 1.0]  [1.0, 1.0]

to this:

[0.0, 280/1280.0]  [1.0, 280/1280.0]
[0.0, 1000/1280.0] [1.0, 1000/1280.0]

and then render that on a square rather than a rectangle.