How to properly use setZOrderMediaOverlay on Android?

Question

Like many others, I am trying to draw 3D objects (using GLSurfaceView) on camera preview (using SurfaceView), along with some buttons placed on top. I actually got a prototype working, but I could not get the onResume working correctly. After a resume, GLSurfaceView stays behind and is not visible anymore. I know that it is working, because I can see the drawing fine if I remove the SurfaceView from the layout.

The target is Nexus One, running stock 2.3.3 ROM.

Here's the layout xml:

<com.example.cameratest.GLPreview 
        android:id="@+id/cubes" 
        android:layout_width="fill_parent" 
        android:layout_height="fill_parent"/> 

<com.example.cameratest.Preview 
        android:id="@+id/camera" 
        android:layout_width="fill_parent" 
        android:layout_height="fill_parent"/> 

<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:id="@+id/buttongroup" 
    android:orientation="vertical"
    android:layout_width="wrap_content"
    android:layout_height="fill_parent"
    android:layout_gravity="right" android:gravity="center">

    <Button android:layout_width="fill_parent" android:layout_height="wrap_content" android:layout_gravity="center" android:id="@+id/buttonClose"></Button>
    <Button android:layout_width="fill_parent" android:layout_height="wrap_content" android:layout_gravity="center" android:id="@+id/buttonMode"></Button>    
    <Button android:layout_width="fill_parent" android:layout_height="wrap_content" android:layout_gravity="center" android:id="@+id/buttonCameraInfo"></Button>
</LinearLayout>

All of this is wrapped in a <merge> block, but for some reason, I could not include it in the above <code> listing. Preview is the class that extends SurfaceView and implements the camera preview logic.

This works fine when the activity launches. After I launch another activity (for example, by pressing one of the buttons) and finish that activity, the main activity resumes but the 3D objects are not visible anymore.

I found this discussion where it is mentioned that setZOrderMediaOverlay() will help fix this z-ordering issue. Also, the Android document for setZOrderMediaOverlay() says this function should be called "before the surface view's containing window is attached to the window manager". I do not know how I should interpret that statement, so I put debug statements in the code to see the sequence of events. Here's how the GLSurfaceView code looks like:

public class GLPreview extends GLSurfaceView
{
    private static final String TAG = "GLPreview";

    public GLPreview(Context context, AttributeSet attrs)
    {
        super(context, attrs);
        // this.setZOrderMediaOverlay(true);        
    }

    public void onPause()
    {
        super.onPause();
        Log.d(TAG, "GLPreview: OnPause");
    }

    public void onResume()
    {
        // this.setZOrderMediaOverlay(true);        
        super.onResume();
        Log.d(TAG, "GLPreview: OnResume");
    }

    public void surfaceCreated(SurfaceHolder holder)
    {
        Log.d(TAG, "GLPreview: surfaceCreated");
        super.surfaceCreated(holder);
    }

    public void surfaceChanged(SurfaceHolder holder, int format, int w, int h)
    {
        Log.d(TAG, String.format("GLPreview: surfaceChanged, format=%d, w=%d, h=%d", format, w, h));
        super.surfaceChanged(holder, format, w, h);
    }

    public void surfaceDestroyed(SurfaceHolder holder)
    {
        Log.d(TAG, "GLPreview: surfaceDestroyed");
        super.surfaceDestroyed(holder);
    }

    protected void onAttachedToWindow()
    {
        Log.d(TAG, "GLPreview: onAttachedToWindow");
        super.onAttachedToWindow();
    }

    protected void onDetachedFromWindow()
    {
        Log.d(TAG, "GLPreview: onDetachedFromWindow");
        super.onDetachedFromWindow();
    }

    protected void onWindowVisibilityChanged (int vis)
    {
        String newVisibility;
        switch (vis)
        {
            case View.GONE:
                newVisibility = "GONE";
                break;
            case View.INVISIBLE:
                newVisibility = "INVISIBLE";
                break;
            case View.VISIBLE:
                newVisibility = "VISIBLE";
                break;
            default:
                newVisibility = String.format("Unknown constant %d", vis);
        }

        Log.d(TAG, String.format("GLPreview: onWindowVisibilityChanged -> %s", newVisibility));
        super.onWindowVisibilityChanged(vis);
    }
}

Here's the sequence of events when I start the application:

GLPreview: OnResume
GLPreview: onAttachedToWindow
GLPreview: onWindowVisibilityChanged -> VISIBLE
GLPreview: surfaceCreated
GLPreview: surfaceChanged, format=1, w=800, h=480

So I assumed that setZOrderMediaOverlay() needs to be called in either onResume() or in the constructor. However, I tried various combinations of setZOrderMediaOverlay() with true or false, in the GLSurfaceView or the SurfaceView classes, but none of it helped with the problem.

I am relatively new when it comes to Android programming. I have come so far but at this point I need some help. If someone who used setZOrderMediaOverlay() successfully in this scenario and got it working with onResume() let me know how this needs to be done.

Thanks a lot in advance!

Answer 1

I have struggled with the same issue for some time, but believe the code below now works. (It continues to work after pausing/ locking regardless of whether the camera is on or off).

Firstly, I haven't got any views etc defined in the layout file - they're created in code. The following is called from the main onCreate() method - Note the setZOrderMediaOverlay() on the augScreen.

I have done nothing special within onPause() or onResume() apart from passing onPause() and onResume() through to the augScreen.

        // 1 - Camera Preview
        camScreen = new CameraPreview(this);
        setContentView(camScreen, new LayoutParams(
                LayoutParams.FILL_PARENT, LayoutParams.FILL_PARENT));

    // 2 - 3D object display
        augScreen = new AR_SurfaceView(this, getViewRange());
        addContentView(augScreen, new LayoutParams(
                LayoutParams.WRAP_CONTENT, LayoutParams.WRAP_CONTENT));
        augScreen.setZOrderMediaOverlay(true);

    // 3 - Permanent overlay
    RelativeLayout overlayScreen = new OverlayView(this);
    addContentView(overlayScreen, new LayoutParams(
            LayoutParams.FILL_PARENT, LayoutParams.FILL_PARENT));
    overlayScreen.setVisibility(View.VISIBLE);

    // 4 - UI buttons (toggleable)
        uiScreen = new UserInterfaceView(this);
        addContentView(uiScreen, new LayoutParams(
                LayoutParams.FILL_PARENT, LayoutParams.FILL_PARENT));

Answer 2

Awila_Tech's answer helped me as well. I used a GLSurfaceView and neatly called its onPause and onResume in the Activities's onPause and onResume, but 1 out of 5 times my screen stayed black while the actual onDrawFrame was being called.

To summarize:

public class OpenGLApp extends Activity implements GLSurfaceView.Renderer {
    private GLSurfaceView mGLSurfaceView;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);

        mGLSurfaceView = new GLSurfaceView(this);
        mGLSurfaceView.setEGLConfigChooser(8, 8, 8, 8, 16, 8);
        mGLSurfaceView.setRenderer(this);
        mGLSurfaceView.getHolder().setFormat(PixelFormat.TRANSLUCENT);
        mGLSurfaceView.setKeepScreenOn(true);
        mGLSurfaceView.setDrawingCacheEnabled(true);
        mGLSurfaceView.setZOrderOnTop(true);

        setContentView(mGLSurfaceView);
        //...
    }

    @Override
    protected void onPause() {
        //...
        mGLSurfaceView.onPause();
        super.onPause();
    }

    @Override
    protected void onResume() {
        mGLSurfaceView.onResume();
        //...
        super.onResume();
    }
}