Set rate at which AVSampleBufferDisplayLayer renders sample buffers

Question

I am using an AVSampleBufferDisplayLayer to display CMSampleBuffers which are coming over a network connection in the h.264 format. Video playback is smooth and working corr

Answer 1

Hit with the same issue, managed to play several streams one after another without lags with following timing in CMSampleBufferCreate creation

CMSampleTimingInfo timingdata ={
 .presentationTimeStamp = CMTimeMakeWithSeconds(self.frame0time+(1/self.frameFPS)*self.frameActive, 1000),
 .duration =  CMTimeMakeWithSeconds(1/self.frameFPS, 1000),
 .decodeTimeStamp = kCMTimeInvalid
};

No need to use kCMSampleAttachmentKey_DisplayImmediately with this approach, you just have to self.frameActive++ on every Iframe and BFrame and make self.frame0time = CACurrentMediaTime(); on first frame

Answer 2

The Timebase needs to be set to the presentation time stamp (pts) of the first frame you intend to decode. I was indexing the pts of the first frame to 0 by subtracting the initial pts from all subsequent pts and setting the Timebase to 0. For whatever reason, that didn't work.

You want something like this (called before a call to decode):

CMTimebaseRef controlTimebase;
CMTimebaseCreateWithMasterClock( CFAllocatorGetDefault(), CMClockGetHostTimeClock(), &controlTimebase );

displayLayer.controlTimebase = controlTimebase;

// Set the timebase to the initial pts here
CMTimebaseSetTime(displayLayer.controlTimebase, CMTimeMake(ptsInitial, 1));
CMTimebaseSetRate(displayLayer.controlTimebase, 1.0);

Set the PTS for the CMSampleBuffer...

CMSampleBufferSetOutputPresentationTimeStamp(sampleBuffer, presentationTimeStamp);

And maybe make sure display immediately isn't set....

CFDictionarySetValue(dict, kCMSampleAttachmentKey_DisplayImmediately, kCFBooleanFalse);

This is covered very briefly in WWDC 2014 Session 513.

Answer 3

Set output presentation timestamp on sample buffer before enqueuing it to AVSampleBufferDisplayLayer. For FPS = 30:

CMSampleBufferSetOutputPresentationTimeStamp(sampleBuffer,    CMTimeAdd(kCMTimeZero, CMTimeMake(1 * _frameCount, 30)));

Set the timestamp on first buffer to 0 (kCMTimeZero) and increment the timestamp on subsequent buffers by 1/30th of a second.

Also, need to set timebase on AVSampleBufferDisplayLayer instance so that buffer with presentation timestamp set to 0 is displayed first.

CMTimebaseRef controlTimebase;
CMTimebaseCreateWithMasterClock(CFAllocatorGetDefault(),    CMClockGetHostTimeClock(), &controlTimebase);

_videoLayer.controlTimebase = controlTimebase;
CMTimebaseSetTime(_videoLayer.controlTimebase, kCMTimeZero);
CMTimebaseSetRate(_videoLayer.controlTimebase, 1.0);