Android Camera原理之setRepeatingRequest与capture模块

Camera操作过程中最重要的四个步骤：

• CameraManager-->openCamera ---> 打开相机
• CameraDeviceImpl-->createCaptureSession ---> 创建捕获会话
• CameraCaptureSession-->setRepeatingRequest ---> 设置预览界面
• CameraDeviceImpl-->capture ---> 开始捕获图片

之前我们介绍了openCamera流程和createCaptureSession流程，如下：
《Android Camera原理之openCamera模块(一)》
《Android Camera原理之openCamera模块(二)》
《Android Camera原理之createCaptureSession模块》
至此，Camera 会话已经创建成功，接下来我们可以开始预览了，预览回调onCaptureCompleted之后就可以拍照(回调到onCaptureCompleted，说明capture 完整frame数据已经返回了，可以捕捉其中的数据了。)，由于预览和拍照的很多流程很相似，拍照只是预览过程中的一个节点，所以我们把预览和拍照放在一文里讲解。

1.预览

预览发起的函数就是CameraCaptureSession-->setRepeatingRequest，本文我们就谈一下Camera 是如何发起预览操作的。
CameraCaptureSession-->setRepeatingRequest是createCaptureSession(List<Surface> outputs, CameraCaptureSession.StateCallback callback, Handler handler)中输出流配置成功之后执行CameraCaptureSession.StateCallback.onConfigured(@NonNull CameraCaptureSession session)函数中执行的。

            mCameraDevice.createCaptureSession(Arrays.asList(surface, mImageReader.getSurface()),
                    new CameraCaptureSession.StateCallback() {

                        @Override
                        public void onConfigured(@NonNull CameraCaptureSession cameraCaptureSession) {
                            // The camera is already closed
                            if (null == mCameraDevice) {
                                return;
                            }

                            // When the session is ready, we start displaying the preview.
                            mCaptureSession = cameraCaptureSession;
                            try {
                                // Auto focus should be continuous for camera preview.
                                mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AF_MODE,
                                        CaptureRequest.CONTROL_AF_MODE_CONTINUOUS_PICTURE);
                                // Flash is automatically enabled when necessary.
                                setAutoFlash(mPreviewRequestBuilder);

                                // Finally, we start displaying the camera preview.
                                mPreviewRequest = mPreviewRequestBuilder.build();
                                mCaptureSession.setRepeatingRequest(mPreviewRequest,
                                        mCaptureCallback, mBackgroundHandler);
                            } catch (CameraAccessException e) {
                                e.printStackTrace();
                            }
                        }

                        @Override
                        public void onConfigureFailed(
                                @NonNull CameraCaptureSession cameraCaptureSession) {
                            showToast("Failed");
                        }
                    }, null
            );

最终执行了
mCaptureSession.setRepeatingRequest(mPreviewRequest, mCaptureCallback, mBackgroundHandler);
来执行camera preview操作。像对焦等操作就可以在这个onConfigured回调中完成。

• onConfigured回调表示当前的配置流已经完成，相机已经显示出来了，可以预览了。
• onConfigureFailed配置流失败，相机黑屏。

    public int setRepeatingRequest(CaptureRequest request, CaptureCallback callback,
            Handler handler) throws CameraAccessException {
        checkRepeatingRequest(request);

        synchronized (mDeviceImpl.mInterfaceLock) {
            checkNotClosed();

            handler = checkHandler(handler, callback);

            return addPendingSequence(mDeviceImpl.setRepeatingRequest(request,
                    createCaptureCallbackProxy(handler, callback), mDeviceExecutor));
        }
    }

• 第一个参数CaptureRequest 标识当前capture 请求的属性，是请求一个camera还是多个camera，是否复用之前的请求等等。
• 第二个参数CaptureCallback 是捕捉回调，这是开发者直接接触的回调。

    public interface CaptureCallback {
        public static final int NO_FRAMES_CAPTURED = -1;
        public void onCaptureStarted(CameraDevice camera,
                CaptureRequest request, long timestamp, long frameNumber);
        public void onCapturePartial(CameraDevice camera,
                CaptureRequest request, CaptureResult result);
        public void onCaptureProgressed(CameraDevice camera,
                CaptureRequest request, CaptureResult partialResult);
        public void onCaptureCompleted(CameraDevice camera,
                CaptureRequest request, TotalCaptureResult result);
        public void onCaptureFailed(CameraDevice camera,
                CaptureRequest request, CaptureFailure failure);
        public void onCaptureSequenceCompleted(CameraDevice camera,
                int sequenceId, long frameNumber);
        public void onCaptureSequenceAborted(CameraDevice camera,
                int sequenceId);
        public void onCaptureBufferLost(CameraDevice camera,
                CaptureRequest request, Surface target, long frameNumber);
    }

这需要开发者自己实现，这些回调是如何调用到上层的，请看《Android Camera原理之CameraDeviceCallbacks回调模块》，都是通过CameraDeviceCallbacks回调调上来的。
下面我们从camera 调用原理的角度分析一下
mCaptureSession.setRepeatingRequest
--->CameraDeviceImpl.setRepeatingRequest
--->CameraDeviceImpl.submitCaptureRequest
其中CameraDeviceImpl.setRepeatingRequest中第3个参数传入的是true。之所以这个强调一点，因为接下来执行CameraDeviceImpl.capture的时候也会执行setRepeatingRequest，这里第3个参数传入的就是false。第3个参数boolean repeating如果为true，表示当前捕获的是一个过程，camera frame不断在填充；如果为false，表示当前捕获的是一个瞬间，就是拍照。

    public int setRepeatingRequest(CaptureRequest request, CaptureCallback callback,
            Executor executor) throws CameraAccessException {
        List<CaptureRequest> requestList = new ArrayList<CaptureRequest>();
        requestList.add(request);
        return submitCaptureRequest(requestList, callback, executor, /*streaming*/true);
    }
    private int submitCaptureRequest(List<CaptureRequest> requestList, CaptureCallback callback,
            Executor executor, boolean repeating)  {
//......
    }

CameraDeviceImpl.submitCaptureRequest核心工作就是3步：

• 1.验证当前CaptureRequest列表中的request是否合理：核心就是验证与request绑定的Surface是否存在。
• 2.向底层发送请求信息。
• 3.将底层返回的请求信息和传入的CaptureCallback 绑定，以便后续正确回调。

而这三步中，第二步却是核心工作。

1.1 向底层发送captureRequest请求

            SubmitInfo requestInfo;

            CaptureRequest[] requestArray = requestList.toArray(new CaptureRequest[requestList.size()]);
            // Convert Surface to streamIdx and surfaceIdx
            for (CaptureRequest request : requestArray) {
                request.convertSurfaceToStreamId(mConfiguredOutputs);
            }

            requestInfo = mRemoteDevice.submitRequestList(requestArray, repeating);
            if (DEBUG) {
                Log.v(TAG, "last frame number " + requestInfo.getLastFrameNumber());
            }

            for (CaptureRequest request : requestArray) {
                request.recoverStreamIdToSurface();
            }

• 执行request.convertSurfaceToStreamId(mConfiguredOutputs);将本地已经缓存的surface和stream记录在内存中，并binder传输到camera service层中，防止camera service端重复请求。
• requestInfo = mRemoteDevice.submitRequestList(requestArray, repeating);这儿直接调用到camera service端。这儿需要重点讲解一下的。
• request.recoverStreamIdToSurface();回调成功，清除之前在内存中的数据。

CameraDeviceClient::submitRequest
--->CameraDeviceClient::submitRequestList
这个函数代码很多，前面很多执行都是在复用检索之前的缓存是否可用，我们关注一下核心的执行：预览的情况下传入的streaming是true，执行上面；如果是拍照的话，那就执行下面的else。
err = mDevice->setStreamingRequestList(metadataRequestList, surfaceMapList, &(submitInfo->mLastFrameNumber));
传入的submitInfo就是要返回上层的回调参数，如果是预览状态，需要不断更新当前的的frame数据，所以每次更新最新的frame number。

    if (streaming) {
        err = mDevice->setStreamingRequestList(metadataRequestList, surfaceMapList,
                &(submitInfo->mLastFrameNumber));
        if (err != OK) {
            String8 msg = String8::format(
                "Camera %s:  Got error %s (%d) after trying to set streaming request",
                mCameraIdStr.string(), strerror(-err), err);
            ALOGE("%s: %s", __FUNCTION__, msg.string());
            res = STATUS_ERROR(CameraService::ERROR_INVALID_OPERATION,
                    msg.string());
        } else {
            Mutex::Autolock idLock(mStreamingRequestIdLock);
            mStreamingRequestId = submitInfo->mRequestId;
        }
    } else {
        err = mDevice->captureList(metadataRequestList, surfaceMapList,
                &(submitInfo->mLastFrameNumber));
        if (err != OK) {
            String8 msg = String8::format(
                "Camera %s: Got error %s (%d) after trying to submit capture request",
                mCameraIdStr.string(), strerror(-err), err);
            ALOGE("%s: %s", __FUNCTION__, msg.string());
            res = STATUS_ERROR(CameraService::ERROR_INVALID_OPERATION,
                    msg.string());
        }
        ALOGV("%s: requestId = %d ", __FUNCTION__, submitInfo->mRequestId);
    }

Camera3Device::setStreamingRequestList
--->Camera3Device::submitRequestsHelper

status_t Camera3Device::submitRequestsHelper(
        const List<const PhysicalCameraSettingsList> &requests,
        const std::list<const SurfaceMap> &surfaceMaps,
        bool repeating,
        /*out*/
        int64_t *lastFrameNumber) {
    ATRACE_CALL();
    Mutex::Autolock il(mInterfaceLock);
    Mutex::Autolock l(mLock);

    status_t res = checkStatusOkToCaptureLocked();
    if (res != OK) {
        // error logged by previous call
        return res;
    }

    RequestList requestList;

    res = convertMetadataListToRequestListLocked(requests, surfaceMaps,
            repeating, /*out*/&requestList);
    if (res != OK) {
        // error logged by previous call
        return res;
    }

    if (repeating) {
        res = mRequestThread->setRepeatingRequests(requestList, lastFrameNumber);
    } else {
        res = mRequestThread->queueRequestList(requestList, lastFrameNumber);
    }
//......
    return res;
}

预览的时候会执行mRequestThread->setRepeatingRequests(requestList, lastFrameNumber);
拍照的时候执行mRequestThread->queueRequestList(requestList, lastFrameNumber);

mRequestThread->setRepeatingRequests

status_t Camera3Device::RequestThread::setRepeatingRequests(
        const RequestList &requests,
        /*out*/
        int64_t *lastFrameNumber) {
    ATRACE_CALL();
    Mutex::Autolock l(mRequestLock);
    if (lastFrameNumber != NULL) {
        *lastFrameNumber = mRepeatingLastFrameNumber;
    }
    mRepeatingRequests.clear();
    mRepeatingRequests.insert(mRepeatingRequests.begin(),
            requests.begin(), requests.end());

    unpauseForNewRequests();

    mRepeatingLastFrameNumber = hardware::camera2::ICameraDeviceUser::NO_IN_FLIGHT_REPEATING_FRAMES;
    return OK;
}

将当前提交的CaptureRequest请求放入之前的预览请求队列中，告知HAL层有新的request请求，HAL层连接请求开始工作，源源不断地输出信息到上层。这儿是跑在Camera3Device中定义的RequestThread线程中，可以保证在预览的时候不断地捕获信息流，camera就不断处于预览的状态了。

1.2 将返回请求信息和 CaptureCallback 绑定

            if (callback != null) {
                mCaptureCallbackMap.put(requestInfo.getRequestId(),
                        new CaptureCallbackHolder(
                            callback, requestList, executor, repeating, mNextSessionId - 1));
            } else {
                if (DEBUG) {
                    Log.d(TAG, "Listen for request " + requestInfo.getRequestId() + " is null");
                }
            }

    /** map request IDs to callback/request data */
    private final SparseArray<CaptureCallbackHolder> mCaptureCallbackMap =
            new SparseArray<CaptureCallbackHolder>();

1.向底层发送captureRequest请求--->回调的requestIinfo表示当前capture request的结果，将requestInfo.getRequestId()与CaptureCallbackHolder绑定，因为Camera 2架构支持发送多次CaptureRequest请求，如果不使用这种绑定机制，后续的回调会造成严重的错乱，甚至回调不上来，那么开发者无法继续使用了。
我们看看使用这些回调的地方的代码：
《Android Camera原理之CameraDeviceCallbacks回调模块》已经说明了CameraDeviceCallbacks.aidl才是camera service进程与用户进程通信的回调，到这个回调里面，再取出CaptureRequest绑定的CaptureCallback回调，调用到CaptureCallback回调函数，这样开发者可以直接使用。
下面看看CameraDeviceCallbacks的onCaptureStarted回调---->

        public void onCaptureStarted(final CaptureResultExtras resultExtras, final long timestamp) {
            int requestId = resultExtras.getRequestId();
            final long frameNumber = resultExtras.getFrameNumber();

            if (DEBUG) {
                Log.d(TAG, "Capture started for id " + requestId + " frame number " + frameNumber);
            }
            final CaptureCallbackHolder holder;

            synchronized(mInterfaceLock) {
                if (mRemoteDevice == null) return; // Camera already closed

                // Get the callback for this frame ID, if there is one
                holder = CameraDeviceImpl.this.mCaptureCallbackMap.get(requestId);

                if (holder == null) {
                    return;
                }

                if (isClosed()) return;

                // Dispatch capture start notice
                final long ident = Binder.clearCallingIdentity();
                try {
                    holder.getExecutor().execute(
                        new Runnable() {
                            @Override
                            public void run() {
                                if (!CameraDeviceImpl.this.isClosed()) {
                                    final int subsequenceId = resultExtras.getSubsequenceId();
                                    final CaptureRequest request = holder.getRequest(subsequenceId);

                                    if (holder.hasBatchedOutputs()) {
                                        // Send derived onCaptureStarted for requests within the
                                        // batch
                                        final Range<Integer> fpsRange =
                                            request.get(CaptureRequest.CONTROL_AE_TARGET_FPS_RANGE);
                                        for (int i = 0; i < holder.getRequestCount(); i++) {
                                            holder.getCallback().onCaptureStarted(
                                                CameraDeviceImpl.this,
                                                holder.getRequest(i),
                                                timestamp - (subsequenceId - i) *
                                                NANO_PER_SECOND/fpsRange.getUpper(),
                                                frameNumber - (subsequenceId - i));
                                        }
                                    } else {
                                        holder.getCallback().onCaptureStarted(
                                            CameraDeviceImpl.this,
                                            holder.getRequest(resultExtras.getSubsequenceId()),
                                            timestamp, frameNumber);
                                    }
                                }
                            }
                        });
                } finally {
                    Binder.restoreCallingIdentity(ident);
                }
            }
        }

holder = CameraDeviceImpl.this.mCaptureCallbackMap.get(requestId);然后直接调用

holder.getCallback().onCaptureStarted(
                                                CameraDeviceImpl.this,
                                                holder.getRequest(i),
                                                timestamp - (subsequenceId - i) *
                                                NANO_PER_SECOND/fpsRange.getUpper(),
                                                frameNumber - (subsequenceId - i));

简单明了，脉络清楚。

2.拍照

开发者如果想要拍照的话，直接调用
mCaptureSession.capture(mPreviewRequestBuilder.build(), mCaptureCallback, mBackgroundHandler);
拍照的调用流程和预览很相似，只是在调用函数中个传入的参数不同。

    public int capture(CaptureRequest request, CaptureCallback callback, Executor executor)
            throws CameraAccessException {
        if (DEBUG) {
            Log.d(TAG, "calling capture");
        }
        List<CaptureRequest> requestList = new ArrayList<CaptureRequest>();
        requestList.add(request);
        return submitCaptureRequest(requestList, callback, executor, /*streaming*/false);
    }

拍照的时候也是调用submitCaptureRequest，只不过第3个参数传入的是false，表示不用循环获取HAL调用上来的帧数据，只获取瞬间的帧数据就可以。
拍照和预览调用的区分在：CameraDeviceClient::submitRequestList

    if (streaming) {
//......
    } else {
        err = mDevice->captureList(metadataRequestList, surfaceMapList,
                &(submitInfo->mLastFrameNumber));
        if (err != OK) {
            String8 msg = String8::format(
                "Camera %s: Got error %s (%d) after trying to submit capture request",
                mCameraIdStr.string(), strerror(-err), err);
            ALOGE("%s: %s", __FUNCTION__, msg.string());
            res = STATUS_ERROR(CameraService::ERROR_INVALID_OPERATION,
                    msg.string());
        }
        ALOGV("%s: requestId = %d ", __FUNCTION__, submitInfo->mRequestId);
    }

接下里调用到
mDevice->captureList
--->Camera3Device::submitRequestsHelper

status_t Camera3Device::submitRequestsHelper(
        const List<const PhysicalCameraSettingsList> &requests,
        const std::list<const SurfaceMap> &surfaceMaps,
        bool repeating,
        /*out*/
        int64_t *lastFrameNumber) {
//......
    RequestList requestList;
//......
    if (repeating) {
        res = mRequestThread->setRepeatingRequests(requestList, lastFrameNumber);
    } else {
        res = mRequestThread->queueRequestList(requestList, lastFrameNumber);
    }
//......
    return res;
}

执行Camera3Device::RequestThread线程中的queueRequestList。

status_t Camera3Device::RequestThread::queueRequestList(
        List<sp<CaptureRequest> > &requests,
        /*out*/
        int64_t *lastFrameNumber) {
    ATRACE_CALL();
    Mutex::Autolock l(mRequestLock);
    for (List<sp<CaptureRequest> >::iterator it = requests.begin(); it != requests.end();
            ++it) {
        mRequestQueue.push_back(*it);
    }

    if (lastFrameNumber != NULL) {
        *lastFrameNumber = mFrameNumber + mRequestQueue.size() - 1;
        ALOGV("%s: requestId %d, mFrameNumber %" PRId32 ", lastFrameNumber %" PRId64 ".",
              __FUNCTION__, (*(requests.begin()))->mResultExtras.requestId, mFrameNumber,
              *lastFrameNumber);
    }

    unpauseForNewRequests();

    return OK;
}

*lastFrameNumber = mFrameNumber + mRequestQueue.size() - 1;
这里有关键的执行代码，表示当前取最新的capture frame数据。

拍照的时候在什么地方捕捉image？

camera1的时候提供了PictureCallback回调方式来提供实时预览回调，可以在这里获取image数据回调。
camera2没有这个接口，但是提供了ImageReader.OnImageAvailableListener来实现回调。

    public interface OnImageAvailableListener {
        /**
         * Callback that is called when a new image is available from ImageReader.
         *
         * @param reader the ImageReader the callback is associated with.
         * @see ImageReader
         * @see Image
         */
        void onImageAvailable(ImageReader reader);
    }

还记得《Android Camera模块解析之拍照》中提到openCamera之前要设置

                mImageReader = ImageReader.newInstance(largest.getWidth(), largest.getHeight(),
                        ImageFormat.JPEG, /*maxImages*/2);
                mImageReader.setOnImageAvailableListener(
                        mOnImageAvailableListener, mBackgroundHandler);

ImageReader中有一个getSurface()函数，这是ImageReader的拍照输出流，我们拍照的时候一般有两个输出流(outputSurface对象)，一个是预览流，还有一个是拍照流。不记得可以参考《Android Camera原理之createCaptureSession模块》，ImageReader设置的拍照流会设置到camera service端。

    public Surface getSurface() {
        return mSurface;
    }

我们看看在什么时候回调这个接口。

ImageReader回调接口.jpg

看上面的调用流程，调用到ImageReader.OnImageAvailableListener->onImageAvailable中，我们获取ImageReader->acquireNextImage可以获取采集的image图片。其实ImageReader中也可以获取预览的流式数据。SurfacePlane 封装了返回的ByteBuffer数据，可供开发者实时获取。

 

private class SurfacePlane extends android.media.Image.Plane {
            private SurfacePlane(int rowStride, int pixelStride, ByteBuffer buffer) {
                mRowStride = rowStride;
                mPixelStride = pixelStride;
                mBuffer = buffer;
                /**
                 * Set the byteBuffer order according to host endianness (native
                 * order), otherwise, the byteBuffer order defaults to
                 * ByteOrder.BIG_ENDIAN.
                 */
                mBuffer.order(ByteOrder.nativeOrder());
            }

            @Override
            public ByteBuffer getBuffer() {
                throwISEIfImageIsInvalid();
                return mBuffer;
            }
            final private int mPixelStride;
            final private int mRowStride;

            private ByteBuffer mBuffer;
}

Note：很多开发者在camera1使用Camera.PreviewCallback的
void onPreviewFrame(byte[] data, Camera camera)
可以获取实时数据，但是在camera2中没有这个接口了，虽然camera1的接口方法也能用，camera2替代的接口就是ImageReader.OnImageAvailableListener->onImageAvailable

来源：oschina

链接：https://my.oschina.net/u/920274/blog/3078089