How to synchronize OpenGL drawing with UIKit updates

Question

In our app we have UIScrollView above CAEAGLLayer. UIScrollView contains some UIViews (red rectangles). In CAEAGLLayer we draw white rectangles. Centers of white rectangles

Answer 1

After digging around a little I'd like to extend my previous answer:

Your OpenGL rendering is immediately done from within the scrollViewDidScroll: method while the UIKit drawing is performed later, during normal CATransaction updates from the runloop.

To synchronize UIKit updates with the OpenGL rendering just enclose both in one explicit transaction and flush it to force UIKit to commit the changes to backbaordd immediately:

- (void)scrollViewDidScroll:(UIScrollView *)aScrollView {
    [CATransaction begin];
    [overlayView updateVisibleRect:CGRectMake(...)];
    [openGlView updateVisibleRect:CGRectMake(...)];
    [CATransaction flush]; // trigger a UIKit and Core Animation graphics update
    [CATransaction commit];
}

Answer 2

In order to synchronize UIKit and OpenGL drawing, you should try to render where Core Animation expects you to draw, i.e. something like this:

- (void)displayLayer:(CALayer *)layer
{
    [self drawFrame];
}

- (void)updateVisibleRect:(CGRect)rect {
    _visibleRect = rect;
    [self.layer setNeedsDisplay];
}

Answer 3

In fact, you can't synchronize them using current APIs. MobileMaps.app and Apple Map Kit use private property on CAEAGLLayer asynchronous to workaround this issue.

Here is related radar: http://openradar.appspot.com/radar?id=3118401

Answer 4

In iOS 9 CAEAGLLayer has presentsWithTransaction property that synchronizes the two.

Answer 5

In lack of a proper answer I'd like to share my thoughts:

There are two ways of drawing involved: Core Animation (UIKit) and OpenGL. In the end, all drawing is done by OpenGL but the Core Animation part is rendered in backboardd (or Springboard.app, before iOS 6) which serves as a kind of window server.

To make this work your app's process serializes the layer hierarchy and changes to its properties and passes the data over to backboardd which in turn renders and composes the layers and makes the result visible.

When mixing OpenGL with UIKit, the CAEAGLLayer's rendering (which is done in your app) has to be composited with the rest of the Core Animation layers. I'm guessing that the render buffer used by the CAEAGLLayer is somehow shared with backboardd to have a fast way of transferring your application's rendering. This mechanism does not necessarily has to be synchronized with the updates from Core Animation.

To solve your issue it would be necessary to find a way of synchronizing the OpenGL rendering with the Core Animation layer serialization and transfer to backboardd. Sorry for not being able to present a solution but I hope these thoughts and guesses help you to understand the reason for the problem.

Answer 6

I am trying to solve exactly same issue. I tried all method described above but nothing was acceptable.

Ideally, this can be solved by accessing Core Animation's final compositing GL context. But we can't access the private API.

One another way is transferring GL result to CA. I tried this by reading frame-buffer pixels to dispatch to CALayer which was too slow. About over 200MB/sec data should be transferred. Now I am trying to optimize this. Because this is the last way that I can try.

Update

Still heavy, but reading frame-buffer and setting it to a CALayer.content is working well and shows acceptable performance on my iPhone 4S. Anyway over 70% of CPU load is used only for this reading and setting operation. Especially glReadPixels I this most of them are just waiting time for reading data from GPU memory.

Update2

I gave up last method. It's cost is almost purely pixel transferring and still too slow. I decided to draw every dynamic objects in GL. Only some static popup views will be drawn with CA.