Using OpenGL in Matlab to get depth buffer

Question

Ive asked a similar question before and didnt manage to find a direct answer.

Could someone provide sample code for extracting the depth buffer of the rendering of an object into a figure in Matlab?

So lets say I load an obj file or even just a simple surf call, render it and now want to get to its depth buffer then what code will do that for me using both Matlab and OpenGL. I.e. how do I set this up and then access the actual data?

I essentially want to be able to use Matlabs powerful plotting functions and then be able to access the underlying graphics context for getting the depth buffer out.

NOTE: The bounty specifies JOGL but that is not a must. Any code which acts as above and can provide me with the depth buffer after running it in Matlab is sufficient)

Answer 1

Today, I went drinking with my colleagues, and after five beers and some tequillas I found this question and thought, "have at ya!" So I was struggling for a while but then I found a simple solution using MEX. I theorized that the OpenGL context, created by the last window, could be left active and therefore could be accessible from "C", if the script ran in the same thread.

I created a simple "C" program which calls one matlab function, called "testofmyfilter" which plots frequency response of a filter (that was the only script I had at hand). This is rendered using OpenGL. Then the program uses glGetViewport() and glReadPixels() to get to the OpenGL buffers. Then it creates a matrix, fills it with the depth values, and passes it to the second function, called "trytodisplaydepthmap". It just displays the depthmap using the imshow function. Note that the MEX function is allowed to return values as well, so maybe the postprocessing would not have to be another function, but I'm in no state to be able to understand how it's done. Should be trivial, though. I'm working with MEX for the first time today.

Without further delay, there are source codes I used:

testofmyfilter.m

imp = zeros(10000,1);
imp(5000) = 1;
% impulse

[bwb,bwa] = butter(3, 0.1, 'high');
b = filter(bwb, bwa, imp);
% filter impulse by the filter

fs = 44100; % sampling frequency (all frequencies are relative to fs)
frequency_response=fft(b); % calculate response (complex numbers)
amplitude_response=20*log10(abs(frequency_response)); % calculate module of the response, convert to dB
frequency_axis=(0:length(b)-1)*fs/length(b); % generate frequency values for each response value
min_f=2;
max_f=fix(length(b)/2)+1; % min, max frequency

figure(1);
lighting gouraud
set(gcf,'Renderer','OpenGL')

semilogx(frequency_axis(min_f:max_f),amplitude_response(min_f:max_f),'r-') % plot with logarithmic axis using red line
axis([frequency_axis(min_f) frequency_axis(max_f) -90 10])  % set axis limits

xlabel('frequency [Hz]');
ylabel('amplitude [dB]'); % legend

grid on % draw grid

test.c

//You can include any C libraries that you normally use
#include "windows.h"
#include "stdio.h"
#include "math.h"
#include "mex.h"   //--This one is required

extern WINAPI void glGetIntegerv(int n_enum, int *p_value);

extern WINAPI void glReadPixels(int     x, 
    int     y, 
    int     width, 
    int     height, 
    int     format, 
    int     type, 
    void *      data);

#define GL_VIEWPORT                       0x0BA2
#define GL_DEPTH_COMPONENT                0x1902
#define GL_FLOAT                          0x1406

void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
    int viewport[4], i, x, y;
    int colLen;
    float *data;
    double *matrix;
    mxArray *arg[1];

    mexCallMATLAB(0, NULL, 0, NULL, "testofmyfilter");
    // call an .m file which creates OpenGL window and draws a plot inside

    glGetIntegerv(GL_VIEWPORT, viewport);
    printf("GL_VIEWPORT = [%d, %d, %d, %d]\n", viewport[0], viewport[1], viewport[2], viewport[3]);
    // print viewport dimensions, should be [0, 0, m, n]
    // where m and n are size of the GL window

    data = (float*)malloc(viewport[2] * viewport[3] * sizeof(float));
    glReadPixels(0, 0, viewport[2], viewport[3], GL_DEPTH_COMPONENT, GL_FLOAT, data);
    // alloc data and read the depth buffer

    /*for(i = 0; i < 10; ++ i)
        printf("%f\n", data[i]);*/
    // debug

    arg[0] = mxCreateNumericMatrix(viewport[3], viewport[2], mxDOUBLE_CLASS, mxREAL);
    matrix = mxGetPr(arg[0]);
    colLen = mxGetM(arg[0]);
    printf("0x%08x 0x%08x 0x%08x %d\n", data, arg[0], matrix, colLen); // debug
    for(x = 0; x < viewport[2]; ++ x) {
        for(y = 0; y < viewport[3]; ++ y)
            matrix[x * colLen + y] = data[x + (viewport[3] - 1 - y) * viewport[2]];
    }
    // create matrix, copy data (this is stupid, but matlab switches
    // rows/cols, also convert float to double - but OpenGL could have done that)

    free(data);
    // don't need this anymore

    mexCallMATLAB(0, NULL, 1, arg, "trytodisplaydepthmap");
    // pass the array to a function (returnig something from here
    // is beyond my understanding of mex, but should be doable)

    mxDestroyArray(arg[0]);
    // cleanup

    return;
}

trytodisplaydepthmap.m:

function [] = trytodisplaydepthmap(depthMap)

figure(2);
imshow(depthMap, []);
% see what's inside

Save all of these to the same directory, compile test.c with (type that to Matlab console):

mex test.c Q:\MATLAB\R2008a\sys\lcc\lib\opengl32.lib

Where "Q:\MATLAB\R2008a\sys\lcc\lib\opengl32.lib" is path to "opengl32.lib" file.

And finally execute it all by merely typing "test" in matlab console. It should bring up a window with filter frequency response, and another window with the depth buffer. Note the front and back buffers are swapped at the moment "C" code reads the depth buffer, so it might be required to run the script twice to get any results (so the front buffer which now contains the results swaps with back buffer again, and the depth can be read out). This could be done automatically by "C", or you can try including getframe(gcf); at the end of your script (that reads back from OpenGL as well so it swaps the buffers for you, or something).

This works for me in Matlab 7.6.0.324 (R2008a). The script runs and spits out the following:

>>test
GL_VIEWPORT = [0, 0, 560, 419]
0x11150020 0x0bd39620 0x12b20030 419

And of course it displays the images. Note the depth buffer range depends on Matlab, and can be quite high, so making any sense of the generated images may not be straightforward.

Answer 2

the swine's answer is the correct one. Here is a slightly formatted and simpler version that is cross-platform.

Create a file called mexGetDepth.c

#include "mex.h"   

#define GL_VIEWPORT                       0x0BA2
#define GL_DEPTH_COMPONENT                0x1902
#define GL_FLOAT                          0x1406

void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
    int viewport[4], i, x, y;
    int colLen;
    float *data;
    double *matrix;

    glGetIntegerv(GL_VIEWPORT, viewport);
    data = (float*)malloc(viewport[2] * viewport[3] * sizeof(float));
    glReadPixels(0, 0, viewport[2], viewport[3], GL_DEPTH_COMPONENT, GL_FLOAT, data);

    plhs[0] = mxCreateNumericMatrix(viewport[3], viewport[2], mxDOUBLE_CLASS, mxREAL);
    matrix = mxGetPr(plhs[0]);
    colLen = mxGetM(plhs[0]);

    for(x = 0; x < viewport[2]; ++ x) {
        for(y = 0; y < viewport[3]; ++ y)
            matrix[x * colLen + y] = data[x + (viewport[3] - 1 - y) * viewport[2]];
    }

    free(data);
    return;
}

Then if youre on windows compile using

mex mexGetDepth.c "path to OpenGL32.lib"

or if youre on a nix system

mex mexGetDepth.c "path to opengl32.a"

Then run the following small script to test out the new function

peaks;
figure(1);
depthData=mexGetDepth;
figure
imshow(depthData);