OpenGL Stencil - Exclude transparent pixels

问题

I have this texture that I want to use as as mask using the Stencil buffer :

Then I want to draw an image IMG on the screen which should appear only where the above image is visible (where there's color), thus excluding transparent pixels above the top edge and below the gradient.

The problem is whenever I draw my image IMG on the screen it appears everywhere where the image has been drawn, whether the pixels are transparent or not.

So I thought about using the ALPHA_TEST but it's gone in the latest OpenGL versions, so I tried to discard fragment with v_color.a == 0 or v_color.r == 0 in my fragment shader but it does nothing...

Fragment shader :

#ifdef GL_ES
    #define LOWP lowp
    precision mediump float;
#else
    #define LOWP
#endif

uniform sampler2D u_texture;

varying LOWP vec4 v_color;
varying vec2 v_texCoords;

void main() {
    if (v_color.a == 0) {
        discard;
    }

    gl_FragColor = v_color * texture2D(u_texture, v_texCoords);
}

Vertex shader :

attribute vec4 a_color;
attribute vec4 a_position;
attribute vec2 a_texCoord0;

uniform mat4 u_projTrans;

varying vec4 v_color;
varying vec2 v_texCoords;

void main() {
    v_color = a_color;
    v_texCoords = a_texCoord0;
    gl_Position = u_projTrans * a_position;
}

Code :

@Override
public void draw(final DrawConfig drawConfig) {
    this.applyTransform(drawConfig.getBatch(), this.computeTransform());

    // Draw mask to stencil buffer

    drawConfig.end();
    final GL20 gl = Gdx.gl;

    gl.glEnable(GL20.GL_STENCIL_TEST);
    gl.glStencilFunc(GL20.GL_ALWAYS, 1, 0xFF);
    gl.glStencilOp(GL20.GL_REPLACE, GL20.GL_REPLACE, GL20.GL_REPLACE);
    gl.glStencilMask(0xFF);
    gl.glColorMask(false, false, false, false);
    gl.glClearStencil(0);
    gl.glClear(GL20.GL_STENCIL_BUFFER_BIT);

    drawConfig.begin(Mode.BATCH);
    this.mBackground.setClippingEnabled(true); // The clipping area is a rectangle centered in the screen
    this.mBackground.draw(drawConfig);
    this.mBottomBorder.draw(drawConfig); // Border below the rectangle area
    this.mTopBorder.draw(drawConfig); // Border above the rectangle area
    drawConfig.end();

    gl.glColorMask(true, true, true, true);
    gl.glStencilFunc(GL20.GL_EQUAL, 1, 0xFF);
    gl.glStencilMask(0x00);

    // Draw elements

    this.mBackground.setClippingEnabled(false); // No clipping area, use stencil test
    this.mBackground.draw(drawConfig);
    this.mBottomBorder.draw(drawConfig);
    this.mTopBorder.draw(drawConfig);

    drawConfig.end();
    Gdx.gl.glDisable(GL20.GL_STENCIL_TEST);

    this.resetTransform(drawConfig.getBatch());
}

Result :

The red arrows show the parts that go outside of the borders and that I want to make disappear. The yellow square is the clipping area, for your convenience.

回答1:

The fixed-function alpha test (GL_ALPHA_TEST) you are trying to duplicate was a per-fragment test. It used to occur after the fragment shader finished, and you can implement it yourself using discard.

However, your current solution is using the interpolated per-vertex alpha value, and that has nothing to do with the texture you are trying to use as a mask. You need to test the alpha value assigned to gl_FragColor to do this properly:

void main() {
    gl_FragColor = v_color * texture2D(u_texture, v_texCoords);

    if (gl_FragColor.a == 0.0) {
        discard;
    }
}

That is the modern shader-based equivalent of:

glAlphaFunc (GL_NOTEQUAL, 0.0f); // Reject fragments with alpha == 0.0
glEnable    (GL_ALPHA_TEST);

来源：https://stackoverflow.com/questions/29304423/opengl-stencil-exclude-transparent-pixels

标签

opengl

shader

mask

stencil-buffer