Overlay texture using GLES20 and apply rotation

问题

I am working on a project in which I am using GLES20 to render a texture. I used GLUtils.texImage2D() to draw 2d texture image from the bitmap.

What do I need to implement to make these changes:

Place the texture to the right corner of the screen instead of stretching it to the full screen.
Provide incremental rotation frame by frame to the texture.

Shaders

private final static String FRAGMENT_SHADER =
            "precision mediump float;\n" +
                    "varying vec2 vTextureCoord;\n" +
                    "uniform lowp sampler2D sTexture;\n" +
                    "uniform lowp sampler2D oTexture;\n" +
                    "void main() {\n" +
                    "   lowp vec4 textureColor = texture2D(sTexture, vTextureCoord);\n" +
                    "   lowp vec4 textureColor2 = texture2D(oTexture, vTextureCoord);\n" +
                    "   \n" +
                    "   gl_FragColor = mix(textureColor, textureColor2, textureColor2.a);\n" +
                    "}\n";

 protected static final String DEFAULT_VERTEX_SHADER =
            "attribute highp vec4 aPosition;\n" +
                    "attribute highp vec4 aTextureCoord;\n" +
                    "varying highp vec2 vTextureCoord;\n" +
                    "void main() {\n" +
                    "gl_Position = aPosition;\n" +
                    "vTextureCoord = aTextureCoord.xy;\n" +
                    "}\n";

Texture Generation

int[] textures = new int[1];

@Override
public void setup() {
    super.setup();// 1
    GLES20.glGenTextures(1, textures, 0);
    GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, textures[0]);
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);
}

@Override
public void onDraw() {
    if (bitmap == null) {
        return;
    }
    int offsetDepthMapTextureUniform = getHandle("oTexture");
    GLES20.glActiveTexture(GLES20.GL_TEXTURE3);
    GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, textures[0]);
    if (bitmap != null && !bitmap.isRecycled()) {
        GLUtils.texImage2D(GLES20.GL_TEXTURE_2D, 0, GLES20.GL_RGBA, bitmap, 0);
    }
    GLES20.glUniform1i(offsetDepthMapTextureUniform, 3);
    // Recycle the bitmap, since its data has been loaded into OpenGL.
    releaseBitmap();
}

UPDATED

I added the mMVPMatrix for the texture transformation as suggested by alexrnov. Both, camera feed and overlay texture are rotating now.

protected static final String VERTEX_SHADER ="attribute highp vec4 aPosition;\n" +
            "uniform mat4 uMVPMatrix;\n" +
            "attribute highp vec4 aTextureCoord;\n" +
            "varying highp vec2 vTextureCoord;\n" +
            "void main() {\n" +
            "gl_Position = uMVPMatrix * aPosition;\n" +
            "vTextureCoord = aTextureCoord.xy;\n" +
            "}\n";

@Override
public void onDraw() {
    ....
    Matrix.setIdentityM(mvpMatrix, 0);
    Matrix.rotateM(mvpMatrix, 0, rotation, 0.0f, 0.0f, 1.0f);
    rotation++;
    GLES20.glUniformMatrix4fv(mMVPMatrixHandle, 1, false, getMVPMatrixAsFloatBuffer(mvpMatrix));
    ...
}

The project structure is quite complicated. I can't write all the code here. Please refer the Github Project

回答1:

Maybe there is another way, but you can overlay the texture on the rectangle, and apply transformations (movement/rotate) to this rectangle:

Vertex shader:

#version 100
uniform mat4 u_mvpMatrix; // MVP-matrix for moving and rotating texture

attribute vec4 a_position; // data of vertices rectangle 
attribute vec2 a_textureCoordinates; 

varying vec2 v_textureCoordinates; 

void main() {
    v_textureCoordinates = a_textureCoordinates;
    gl_Position = u_mvpMatrix * a_position;
}

Fragment shader:

#version 100
varying vec2 v_textureCoordinates;
uniform sampler2D s_texture;

void main() {
    gl_FragColor = texture2D(s_texture, v_textureCoordinates);
}

Approximate Java-code:

private final int textureID;
...
textureID = loadTexture(R.raw.texture);
...
@Override
public void draw() {
    GLES20.glUseProgram(programObject);
    GLES20.glEnableVertexAttribArray(positionLink); 
    GLES20.glBindBuffer(GLES20.GL_ARRAY_BUFFER, VBO[0]);
    GLES20.glVertexAttribPointer(positionLink, VERTEX_COMPONENT, 
        GLES20.GL_FLOAT, false, VERTEX_STRIDE, 0);
    GLES20.glBindBuffer(GLES20.GL_ARRAY_BUFFER, 0);

    GLES20.glEnableVertexAttribArray(textureCoordinatesLink);
    GLES20.glBindBuffer(GLES20.GL_ARRAY_BUFFER, VBO[1]);
    GLES20.glVertexAttribPointer(textureCoordinatesLink, TEXTURE_COMPONENT, 
        GLES20.GL_FLOAT, false, TEXTURE_STRIDE, 0);
    GLES20.glBindBuffer(GLES20.GL_ARRAY_BUFFER, 0);

    GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
    GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, textureID);
    GLES20.glUniform1i(samplerLink, 0);

    GLES20.glUniformMatrix4fv(mvpMatrixLink, 1, false,
        RectangleObject3D.getMVPMatrixAsFloatBuffer());

    GLES20.glBindBuffer(GLES20.GL_ELEMENT_ARRAY_BUFFER, VBO[2]);
    GLES20.glDrawElements(GLES20.GL_TRIANGLES, NUMBER_INDICES, 
        GLES20.GL_UNSIGNED_INT, 0);
    GLES20.glBindBuffer(GLES20.GL_ELEMENT_ARRAY_BUFFER, 0);

    GLES20.glDisableVertexAttribArray(positionLink); 
    GLES20.glDisableVertexAttribArray(textureCoordinatesLink); 
}

MVP-matrix:

import android.opengl.Matrix

protected val viewMatrix = FloatArray(16)
protected val projectionMatrix = FloatArray(16)
protected val modelMatrix = FloatArray(16)
protected val modelViewMatrix = FloatArray(16)
protected val mvpMatrix = FloatArray(16)
...
Matrix.setLookAtM(viewMatrix, 0, 0f, 0f, 0f,
        0f, 0f, -4f, 0f, 1.0f, 0.0f) // camera
...
// parameters different for portrait and landscape orientation screen android
Matrix.frustumM(projectionMatrix, 0, left, right, bottom, top, near, far)
...
fun spotPosition() { // invoke on every frame
    Matrix.setIdentityM(modelMatrix, 0)
    Matrix.translateM(modelMatrix, 0, x, y, z) // move object
    Matrix.rotateM(modelMatrix, 0, angleX, 0.0f, 1.0f, 0.0f) // rotate object
    Matrix.scaleM(modelMatrix, 0, 4f, 4f, 4f) // scale object 
    Matrix.multiplyMM(modelViewMatrix, 0, viewMatrix, 0, modelMatrix, 0)
    Matrix.multiplyMM(mvpMatrix, 0, projectionMatrix, 0, modelViewMatrix, 0)
}
fun getMVPMatrixAsFloatBuffer(): FloatBuffer = floatBuffer(mvpMatrix)

get FloatBuffer (essential in Java SDK):

public static FloatBuffer floatBuffer(float[] data) {
    ByteBuffer byteBuffer = ByteBuffer.allocateDirect(data.length * 4)
                        .order(ByteOrder.nativeOrder());
    FloatBuffer returnBuffer = byteBuffer.asFloatBuffer();
    returnBuffer.put(data).position(0);
    return returnBuffer;
}

来源：https://stackoverflow.com/questions/61118884/overlay-texture-using-gles20-and-apply-rotation

标签

android

opengl-es