How does the gluProject function work? I can't understand it

Question

I need to show a square polygon with the 100% of the width of the screen, then, i supose that i must zoom it (with Z axis) until the polygon borders are tounching the screen

Answer 1

gluProject does exactly what the fixed function transformation pipeline would do, too:

1. The 3D vertex is expanded to homogeneous coordinates by appending a 1 as fourth coordinate: v[3]=1.

2. Then this homogenous vertex is multiplied by the modelview matrix and the projection matrix: v'=P*M*v.

3. Then comes the persepctive division. By dividing by the fourth coordinate we account for perspective distortion (if you have an orthographic projection e.g. using glOrtho, then v'[3]==1 and there is no perspective distortion): v"=v'/v'[3].

4. Now everything in your viewing volume (the visible area of your scene) has been transformed into normalized device coordinates, the [-1,1]-cube. So what needs to be done is transform this into screen coordinates [0,w] x [0,h]: x=w * (v"[0]+1) / 2 and y = h * (v"[1]+1) / 2. And finally, the z-coordinate is transformed from [-1,1] to [0,1] to give the normalized depth value that is written into the depth buffer: z = (v"[2]+1) / 2.

So the key to understand what happens to the z value is to realize, that the distance to the camera (the z value in view space) is first transformed into the [-1,1] range by the projection matrix, depending on the near-far range (the near and far values you put into glOrtho, glFrustum or gluPerspective). Then this normalized value is transformed into the [0,1] range to result in the final depth value that gets written into the depth buffer and that gluProject computes as z-value of the window coordinates.

So what you actually got out (0, 0, 0.5) is the lower left corner of your screen and with a depth of 0.5. With an orthographic matrix (without any perspective distortion) and an identity modelview matrix this would be equal to a coordinate of (left, bottom, (far-near)/2), where bottom, left, near and far are the corresponding arguments you put into the glOrtho function call (or something with similar functionality). So the vertex is in the middle of the near-far-range and in the lower left corner of the viewing volume (as seen from the camera). But this won't hold for a perspective projection, as in this case the transformation from the view-space z-coordinate to the depth value is not linear (though still monotonic, of course).

Since you put in the vertex (-1, -1, 0), this could mean your modelview matrix is identity and your projection matrix corresponds to a matrix created with glOrtho(-1, 1, -1, 1, -1, 1), which is also nearly the identity matrix (though with a mirrored z value, but because the input z is 0, you might not notice it). So if these are not the values you would have awaited (after understanding the workings of gluProject, of course), it may also just be that your matrices haven't been retrieved correctly and you just got identity matrices instead of your actual modelview and projection matrices.

So I think there is nothing wrong with your gluProject function. You might also look at the answers to this question to gain some more insight into OpenGL's default transformation pipeline. Although with the advent of vertex shaders some of the stages can be computed differently, you normally still follow the idiomatic model -> view -> projection approach.