Setting larger GOP size in MediaFoundation hardware MFT

问题

I'm trying to live stream the desktop that's captured through Desktop duplication API. H264 encoding works fine, except the fact that the Desktop duplication API delivers frames only when there is a screen change, but video encoders expect the frames to be delivered at a constant frame rate. So, I'm forced to save the previous sample to feed the encoder at a constant rate when there is no screen change triggered. This works, I could see live output at the other end.

One problem though, the encoder produces a large sample equal to the size of a fresh full-screen sample (which is probably a key-frame) at a constant rate. I have also noticed that an I frame (That large sample) is produced exactly once every 1 second (I guess, it could possibly the default GOP size) even when there is no screen change and I'm providing only the sample that I previously created and literally no diff between them except the sample time I'm setting. This is costly for a live stream, I'm not expecting the decoder to be able to seek or join the stream at the midst of the stream (At least, I have control over it), is there a way to get around this by setting a larger GOP?

I tried all the below settings, but nothing seems to be changing.

FPS: 30

CHECK_HR(pMFTOutputMediaType->SetUINT32(CODECAPI_AVEncMPVGOPSize, 1024), "Failed to set GOP size");
CHECK_HR(pMFTOutputMediaType->SetUINT32(CODECAPI_AVEncMPVGOPSInSeq, 1024), "Failed to set GOPInSeq");
CHECK_HR(pMFTOutputMediaType->SetUINT32(MF_MT_MAX_KEYFRAME_SPACING, 1024), "Failed to set keyframe spacing");

I have tried setting CODECAPI_AVEncCommonRealTime property too, are these settings incompatible with each other?

I have also tried the below code (copied from chromium https://github.com/chromium/chromium/blob/master/media/gpu/windows/media_foundation_video_encode_accelerator_win.cc). There is still no change in the keyframes count, it still generates an I frame every one second once. I think I'm missing something.

Here is the code review link for the commit in chromium that contains their discussion regarding this particular configuration. Reading this discussion gave me some hope, but no luck yet.

void SetEncoderModes() {

    VARIANT var = { 0 };

    if (!mpCodecAPI) {
        CHECK_HR(_pTransform->QueryInterface(IID_PPV_ARGS(&mpCodecAPI)), "Failed to get codec api");
    }

    var.vt = VT_UI4;
    var.lVal = 1024;

    CHECK_HR(mpCodecAPI->SetValue(&CODECAPI_AVEncMPVGOPSize, &var), "Failed to set GOP size");
}

Any help would be appreciated.

回答1:

The code snippet extracted from Chromium is the way to do it: you have to use ICodecAPI interface.

As documented on MSDN:

Certified Hardware Encoder

[...]

The following is the set of required and optional ICodecAPI properties for encoders to pass the HCK encoder certification.

The following Windows 8 and Windows 8.1 ICodecAPI properties are required:

[...]

CODECAPI_AVEncMPVGOPSize

So you will have the property present in most cases.

Note that you might need to set the property before you start actual streaming.

来源：https://stackoverflow.com/questions/59006207/setting-larger-gop-size-in-mediafoundation-hardware-mft