问题

If linear interpolation happens during the rasterization stage in the OpenGL pipeline, and the vertices have already been transformed to screen-space, where does the depth information used for perspectively correct interpolation come from?

Can anybody give a detailed description of how OpenGL goes from screen-space primitives to fragments with correctly interpolated values?

回答1:

The output of a vertex shader is a four component vector, vec4 gl_Position. From Section 13.6 Coordinate Transformations of core GL 4.4 spec:

Clip coordinates for a vertex result from shader execution, which yields a vertex coordinate gl_Position.

Perspective division on clip coordinates yields normalized device coordinates, followed by a viewport transformation (see section 13.6.1) to convert these coordinates into window coordinates.

OpenGL does the perspective divide as

device.xyz = gl_Position.xyz / gl_Position.w

But then keeps the 1 / gl_Position.w as the last component of gl_FragCoord:

gl_FragCoord.xyz = device.xyz scaled to viewport
gl_FragCoord.w = 1 / gl_Position.w

This transform is bijective, so no depth information is lost. In fact as we see below, the 1 / gl_Position.w is crucial for perspective correct interpolation.

Short introduction to barycentric coordinates

Given a triangle (P0, P1, P2) one way to parametrize the points inside the triangle is by choosing one vertex (here P0) and expressing each other point as:

P(u,v) = P0 + (P1 - P0)u + (P2 - P0)v

where u ≥ 0, v ≥ 0 and u + v ≤ 1. Given an attribute (f0, f1, f2) on the vertices of the triangle, we can use u, v to interpolate it over the interior:

f(u,v) = f0 + (f1 - f0)u + (f2 - f0)v

All math can be done using the above parametrization, and in fact is sometimes preferable due to faster calculations. However it is less convenient and has numerical issues (e.g. P(1,0) might not equal P1).

Instead barycentric coordinates are usually used. Every point inside the triangle is a weighted sum of the vertices:

P(b0,b1,b2) = P0*b0 + P1*b1 + P2*b2
f(b0,b1,b2) = f0*b0 + f1*b1 + f2*b2

where b0 + b1 + b2 = 1, b0 ≥ 0, b1 ≥ 0, b2 ≥ 0 are the barycentric coordinates of the point in the triangle. Each bi can be thought of as 'how much of Pi has to be mixed in'. So b = (1,0,0), (0,1,0) and (0,0,1) are the vertices of the triangle, (1/3, 1/3, 1/3) is the barycenter, and so on.

Perspective correct interpolation

So let's say we fill a projected 2D triangle on the screen. For every fragment we have its window coordinates. First we calculate its barycentric coordinates by inverting the P(b0,b1,b2) function, which is a linear function in window coordinates. This gives us the barycentric coordinates of the fragment on the 2D triangle projection.

Perspective correct interpolation of an attribute would vary linearly in the clip coordinates (and by extension, world coordinates). For that we need to get the barycentric coordinates of the fragment in clip space.

As it happens (see [1] and [2]), the depth of the fragment is not linear in window coordinates, but the depth inverse (1/gl_Position.w) is. Accordingly the attributes and the clip-space barycentric coordinates, when weighted by the depth inverse, vary linearly in window coordinates.

Therefore, we compute the perspective corrected barycentric by:

     ( b0 / gl_Position[0].w, b1 / gl_Position[1].w, b2 / gl_Position[2].w )
B = -------------------------------------------------------------------------
      b0 / gl_Position[0].w + b1 / gl_Position[1].w + b2 / gl_Position[2].w

and then use it to interpolate the attributes from the vertices.

Note: GL_NV_fragment_shader_barycentric exposes the device-linear barycentric coordinates through gl_BaryCoordNoPerspNV and the perspective corrected through gl_BaryCoordNV.

Implementation

Here is a C++ code that rasterizes and shades a triangle on the CPU, in a manner similar to OpenGL. I encourage you to compare it with the shaders listed below:

struct Renderbuffer {
    int w, h, ys;
    void *data;
};

struct Vert {
    vec4f position;
    vec4f texcoord;
    vec4f color;
};

struct Varying {
    vec4f texcoord;
    vec4f color;
};

void vertex_shader(const Vert &in, vec4f &gl_Position, Varying &out)
{
    out.texcoord = in.texcoord;
    out.color = in.color;
    gl_Position = { in.position[0], in.position[1], -2*in.position[2] - 2*in.position[3], -in.position[2] };
}

void fragment_shader(vec4f &gl_FragCoord, const Varying &in, vec4f &out)
{
    out = in.color;
    vec2f wrapped = vec2f(in.texcoord - floor(in.texcoord));
    bool brighter = (wrapped[0] < 0.5) != (wrapped[1] < 0.5);
    if(!brighter)
        (vec3f&)out = 0.5f*(vec3f&)out;
}

void store_color(Renderbuffer &buf, int x, int y, const vec4f &c)
{
    // can do alpha composition here
    uint8_t *p = (uint8_t*)buf.data + buf.ys*(buf.h - y - 1) + 4*x;
    p[0] = linear_to_srgb8(c[0]);
    p[1] = linear_to_srgb8(c[1]);
    p[2] = linear_to_srgb8(c[2]);
    p[3] = lrint(c[3]*255);
}

void draw_triangle(Renderbuffer &color_attachment, const box2f &viewport, const Vert *verts)
{
    Varying perVertex[3];
    vec4f gl_Position[3];

    box2f aabbf = { viewport.hi, viewport.lo };
    for(int i = 0; i < 3; ++i)
    {
        // invoke the vertex shader
        vertex_shader(verts[i], gl_Position[i], perVertex[i]);

        // convert to device coordinates by perspective division
        gl_Position[i][3] = 1/gl_Position[i][3];
        gl_Position[i][0] *= gl_Position[i][3];
        gl_Position[i][1] *= gl_Position[i][3];
        gl_Position[i][2] *= gl_Position[i][3];

        // convert to window coordinates
        auto &pos2 = (vec2f&)gl_Position[i];
        pos2 = mix(viewport.lo, viewport.hi, 0.5f*(pos2 + vec2f(1)));
        aabbf = join(aabbf, (const vec2f&)gl_Position[i]);
    }

    // precompute the affine transform from fragment coordinates to barycentric coordinates
    const float denom = 1/((gl_Position[0][0] - gl_Position[2][0])*(gl_Position[1][1] - gl_Position[0][1]) - (gl_Position[0][0] - gl_Position[1][0])*(gl_Position[2][1] - gl_Position[0][1]));
    const vec3f barycentric_d0 = denom*vec3f( gl_Position[1][1] - gl_Position[2][1], gl_Position[2][1] - gl_Position[0][1], gl_Position[0][1] - gl_Position[1][1] );
    const vec3f barycentric_d1 = denom*vec3f( gl_Position[2][0] - gl_Position[1][0], gl_Position[0][0] - gl_Position[2][0], gl_Position[1][0] - gl_Position[0][0] );
    const vec3f barycentric_0 = denom*vec3f(
        gl_Position[1][0]*gl_Position[2][1] - gl_Position[2][0]*gl_Position[1][1],
        gl_Position[2][0]*gl_Position[0][1] - gl_Position[0][0]*gl_Position[2][1],
        gl_Position[0][0]*gl_Position[1][1] - gl_Position[1][0]*gl_Position[0][1]
    );

    // loop over all pixels in the rectangle bounding the triangle
    const box2i aabb = lrint(aabbf);
    for(int y = aabb.lo[1]; y < aabb.hi[1]; ++y)
    for(int x = aabb.lo[0]; x < aabb.hi[0]; ++x)
    {
        vec4f gl_FragCoord;
        gl_FragCoord[0] = x + 0.5;
        gl_FragCoord[1] = y + 0.5;

        // fragment barycentric coordinates in window coordinates
        const vec3f barycentric = gl_FragCoord[0]*barycentric_d0 + gl_FragCoord[1]*barycentric_d1 + barycentric_0;

        // discard fragment outside the triangle. this doesn't handle edges correctly.
        if(barycentric[0] < 0 || barycentric[1] < 0 || barycentric[2] < 0)
            continue;

        // interpolate inverse depth linearly
        gl_FragCoord[2] = dot(barycentric, vec3f(gl_Position[0][2], gl_Position[1][2], gl_Position[2][2]));
        gl_FragCoord[3] = dot(barycentric, vec3f(gl_Position[0][3], gl_Position[1][3], gl_Position[2][3]));

        // clip fragments to the near/far planes (as if by GL_ZERO_TO_ONE)
        if(gl_FragCoord[2] < 0 || gl_FragCoord[2] > 1)
            continue;

        // convert to perspective correct (clip-space) barycentric
        const vec3f perspective = 1/gl_FragCoord[3]*barycentric*vec3f(gl_Position[0][3], gl_Position[1][3], gl_Position[2][3]);

        // interpolate the attributes using the perspective correct barycentric
        Varying varying;
        for(int i = 0; i < sizeof(Varying)/sizeof(float); ++i)
            ((float*)&varying)[i] = dot(perspective, vec3f(
                ((const float*)&perVertex[0])[i],
                ((const float*)&perVertex[1])[i],
                ((const float*)&perVertex[2])[i] 
            ));

        // invoke the fragment shader and store the result
        vec4f color;
        fragment_shader(gl_FragCoord, varying, color);
        store_color(color_attachment, x, y, color);
    }
}



int main()
{
    Renderbuffer buffer = { 512, 512, 512*4 };
    buffer.data = malloc(buffer.ys * buffer.h);
    memset(buffer.data, 0, buffer.ys * buffer.h);

    // interleaved attributes buffer
    Vert verts[] = {
        { { -1, -1, -2, 1 }, { 0, 0, 0, 1 }, { 0, 0, 1, 1 } },
        { { 1, -1, -1, 1 }, { 10, 0, 0, 1 }, { 1, 0, 0, 1 } },
        { { 0, 1, -1, 1 }, { 0, 10, 0, 1 }, { 0, 1, 0, 1 } },
    };

    box2f viewport = { 0, 0, buffer.w, buffer.h };
    draw_triangle(buffer, viewport, verts);

    lodepng_encode32_file("out.png", (unsigned char*)buffer.data, buffer.w, buffer.h);
}

OpenGL shaders

Here are the OpenGL shaders used to generate the reference image.

Vertex shader:

#version 450 core
layout(location = 0) in vec4 position;
layout(location = 1) in vec4 texcoord;
layout(location = 2) in vec4 color;

out gl_PerVertex {
    vec4 gl_Position;
};

layout(location = 0) out PerVertex {
    vec4 texcoord;
    vec4 color;
} OUT;

void main() {
    OUT.texcoord = texcoord;
    OUT.color = color;
    gl_Position = vec4(position[0], position[1], -2*position[2] - 2*position[3], -position[2]);
}

Fragment shader:

#version 450 core
layout(location = 0) in PerVertex {
    vec4 texcoord;
    vec4 color;
} IN;
layout(location = 0) out vec4 OUT;

void main() {
    OUT = IN.color;
    vec2 wrapped = fract(IN.texcoord.xy);
    bool brighter = (wrapped[0] < 0.5) != (wrapped[1] < 0.5);
    if(!brighter)
        OUT.rgb *= 0.5;
}

Results

Here are the almost identical images generated by the C++ (left) and OpenGL (right) code:

The differences are caused by different precision and rounding modes.

For comparison, here is one that is not perspective correct (uses barycentric instead of perspective for the interpolation in the code above):

回答2:

The formula that you will find in the GL specification (look on page 427; the link is the current 4.4 spec, but it has always been that way) for perspective-corrected interpolation of the attribute value in a triangle is:

   a * f_a / w_a   +   b * f_b / w_b   +  c * f_c / w_c
f=-----------------------------------------------------
      a / w_a      +      b / w_b      +     c / w_c

where a,b,c denote the barycentric coordinates of the point in the triangle we are interpolating for (a,b,c >=0, a+b+c = 1), f_i the attribute value at vertex i, and w_i the clip space w coordinate of vertex i. Note that the barycentric coordinates are calculated only for the 2D projection of the window space coords of the triangle (so z is ignored).

This is what the formulas that ybungalowbill gave in his fine answer boils down to, in the general case, with an arbitrary projection axis. Actually, the last row of the projection matrix defines just the projection axis the image plane will be orthogonal to, and the clip space w component is just the dot product between the vertex coords and that axis.

In the typical case, the projection matrix has (0,0,-1,0) as the last row, so it transfroms so that w_clip = -z_eye, and this is what ybungalowbill used. However, since w is what we actually will do the division by (that is the only nonlinear step in the whole transformation chain), this will work for any projection axis. It will also work in the trivial case of orthogonal projections where w is always 1 (or at least constant).

Note a few things for an efficient implementation of this. The inversion 1/w_i can be pre-calculated per vertex (let's call them q_i in the following), it does not have to be re-evaluated per fragment. And it is totally free since we divide by w anyway, when going into NDC space, so we can save that value. The GL spec does never describe how a certain feature is to be implemented internally, but the fact that the screen space coordinates will be accessible in glFragCoord.xyz, and gl_FragCoord.w is guaranteed to give the (lineariliy interpolated) 1/w clip space coordinate is quite revealing here. That per-fragment 1_w value is actually the denominator of the formula given above.
The factors a/w_a, b/w_b and c/w_c are each used two times in the formula. And these are also constant for any attribute value, now matter how many attributes there are to be interpolated. So, per fragment, you can calculate a'=q_a * a, b'=q_b * b and c'=q_c and get
```
  a' * f_a + b' * f_b + c' * f_c
f=------------------------------
           a' + b' + c'
```