Getting Tango's camera stream data

问题

I'm trying to get the Tango's camera stream in order to combine an homemade AR Kit to Tango.

I'm stuck at a point where everything works as intended in Tango's editor emulation, but not in the app pushed to the tablet.

The code I'm using is the following:

YUVTexture yuvTexture = m_tangoApplication.GetVideoOverlayTextureYUV();
Texture2D yTexture = yuvTexture.m_videoOverlayTextureY;
// m_videoOverlayTextureCr is not used by Tango yet for some reason
Texture2D uvTexture = yuvTexture.m_videoOverlayTextureCb;

// convert from YV12 to RGB
for (int i = 0; i < yTexture.height; ++i)
{
    for (int j = 0; j < yTexture.width; ++j)
    {
        Color yPixel = yTexture.GetPixel(j, i);
        Color uvPixel = uvTexture.GetPixel(j, i);

        m_texture.SetPixel(4 * j + 0, yTexture.height - i - 1, YUV2Color(yPixel.r, uvPixel.r, uvPixel.g));
        m_texture.SetPixel(4 * j + 1, yTexture.height - i - 1, YUV2Color(yPixel.g, uvPixel.r, uvPixel.g));
        m_texture.SetPixel(4 * j + 2, yTexture.height - i - 1, YUV2Color(yPixel.b, uvPixel.b, uvPixel.a));
        m_texture.SetPixel(4 * j + 3, yTexture.height - i - 1, YUV2Color(yPixel.a, uvPixel.b, uvPixel.a));
    }
}

YUV2Color (extracted from Tango's YUV2RGB Shader):

public static Color YUV2Color(float y_value, float u_value, float v_value)
{
    float r = y_value + 1.370705f * (v_value - 0.5f);
    float g = y_value - 0.698001f * (v_value - 0.5f) - (0.337633f * (u_value - 0.5f));
    float b = y_value + 1.732446f * (u_value - 0.5f);

    return new Color(r, g, b, 1f);
}

Did someone already solved this problem? I've seen a lot of post related to it when the ITangoVideoOverlay was mostly used, but nothing with the current IExperimentalTangoVideoOverlay

I've experimented a lot of things, so far it has been the closest I got to what I expected ... Any help would be highly appreciated.

回答1:

You are using the Texture ID method to get the YUV texture color, this is not very common to do. A easier path would be using the Raw Byte buffer method to get color camera image, to do that:

On TangoManager prefab, enable video overlay, and select Raw Byte method from the drop down box.
Register to ITangoVideoOverlay interface.
Convert the image buffer data from YUV to RGB, this part is exactly like the YUV2Color function, but use data from TangoImageData.data

来源：https://stackoverflow.com/questions/38189628/getting-tangos-camera-stream-data

标签

unity3d

google-project-tango