Android SensorManager strange how to remapCoordinateSystem

Question

API Demos -> Graphics -> Compass

It works properly only, until you don't change the device natural orientation. In most phones is the portrait and most 10 inch tablets are the landscape. If you change than need to rotate this with 90 degree. I would like to see a 3D fix for that system.

100% sure need to use remapCoordinateSystem() method.

I would like to see how ( code ) if I could see an explanation with how is calculated those axes mapping ( theoretical math ) it would be nice.

I have tryed to understand, but I forgot all linear algebra.

Here it says why must we use, but doesn't telling how!

float R[] = new float[9];
// X  (product of Y and Z) and roughly points East
// Y: points to Magnetic NORTH and tangential to ground
// Z: points to SKY and perpendicular to ground
float I[] = new float[9];
boolean success = SensorManager.getRotationMatrix(R, I, mGravity, mGeomagnetic);

It seems those coordinates are to this position: - the device say in a table (x, and y axes are on table)

Only and only if

getWindowManager().getDefaultDisplay().getRotation() == Surface.ROTATION_0

The question is how to complete this code: - those case branches

switch (mScreenRotation) {
    case Surface.ROTATION_0:
    Log.v("SurfaceRemap", "0 degree");
    axisX = SensorManager.AXIS_X;// is this valid?
    axisY = SensorManager.AXIS_Y;// is this valid?
    break;

    case Surface.ROTATION_90:
    Log.v("SurfaceRemap", "90 degree");
    // examples says remapCoordinateSystem(inR, AXIS_Y, AXIS_MINUS_X, outR);
    axisX = SensorManager.AXIS_Y;
    axisY = SensorManager.AXIS_MINUS_X;
    break;

    case Surface.ROTATION_180:
    Log.v("SurfaceRemap", "180 degree");
    break;

    case Surface.ROTATION_270:
    Log.v("SurfaceRemap", "270 degree");
    break;

    default:
    Log.v("SurfaceRemap", "don't know the mScreenRotation value: "+mScreenRotation+" you should never seen this message!");
    break;
}


boolean remapped = SensorManager.remapCoordinateSystem(R, axisX, axisY, R);

float orientation[] = new float[3];

SensorManager.getOrientation(R, orientation);// All three angles above are in radians and positive in the counter-clockwise direction.
inclination = SensorManager.getInclination(I);

Edit: I wrote a little test application, where on the screen it display the screen rotation: 0, 90, 270 degrees ( can't make 180 now)

It seems, if Rotation 0 is unchanged (axisX = SensorManager.AXIS_X;axisY = SensorManager.AXIS_Y;) than the 90 degree should be:

axisX = SensorManager.AXIS_MINUS_Y;
axisY = SensorManager.AXIS_X;

than the Google documentation says somewhere wrong values! Question is where?!

getRotationMatrix return this:

X is defined as the vector product Y.Z (It is tangential to the ground at the device's current location and roughly points East).

Y is tangential to the ground at the device's current location and points towards the magnetic North Pole.

Z points towards the sky and is perpendicular to the ground.

See the phone above! I want to to to right from left, with back camera to ground.

getOrientation return this:

X is defined as the vector product Y.Z (It is tangential to the ground at the device's current location and roughly points West).

Y is tangential to the ground at the device's current location and points towards the magnetic North Pole.

Z points towards the center of the Earth and is perpendicular to the ground.

values[0]: azimuth, rotation around the Z axis.

values[1]: pitch, rotation around the X axis.

values[2]: roll, rotation around the Y axis.

How should be the phone?

Finally I would like to have values of angles like aircrafts. My phone (me) heading to North: (yaw is azimuth)

              if ScreenRotation = 0 degree
Pitch axis = -orientationAxisX  =  rotationAxisX
Roll axis  =  orientationAxisY  =  rotationAxisY 
Yaw axis   =  orientationAxisZ  = -rotationAxisZ

Answer 1

To complete the switch branches I just try to think following the remapCoordinateSystem method javadoc:

X defines on which world axis and direction the X axis of the device is mapped.
Y defines on which world axis and direction the Y axis of the device is mapped.

So take your device rotate it from its natural orientation (90, 180 or 270 degrees) and ask yourself: The X positive axis in the original device orientation to which axis corresponds in the current device orientation?. And same for the Y axis.

So in case your device is rotated 90 degrees you will see that the original X positive axis corresponds to the current positive Y axis and the original positive Y axis corresponds to the current orientation negative X axis.

So It should be:

switch (mScreenRotation) {
    case Surface.ROTATION_0:
        axisX = SensorManager.AXIS_X;
    axisY = SensorManager.AXIS_Y;
        break;

    case Surface.ROTATION_90:
        axisX = SensorManager.AXIS_Y;
    axisY = SensorManager.AXIS_MINUS_X;
        break;

    case Surface.ROTATION_180:
        axisX = SensorManager.AXIS_MINUS_X;
    axisY = SensorManager.AXIS_MINUS_Y;
        break;

    case Surface.ROTATION_270:
        axisX = SensorManager.AXIS_MINUS_Y;
    axisY = SensorManager.AXIS_X;
        break;

    default:
        break;
}

That worked for me, hope that helps.

Answer 2

Thanks keianhzo, your answer works great with phones flat on the ground. For AR-applications where you look "through" the display, I found this to work: use the proper axis:

int screenRotation = mActivity.getWindowManager().getDefaultDisplay().getRotation();
//use the correct axis
int axisX = SensorManager.AXIS_X;
int axisY = SensorManager.AXIS_Y;
switch (mMode) {
    case LOOK_THROUGH: {
        // look through always uses x and z
        axisX = SensorManager.AXIS_X;
        axisY = SensorManager.AXIS_Z;
        break;
    }
    case FLAT: {
        // flat changes the x axis depending on rotation state
        switch (screenRotation) {
            case Surface.ROTATION_0:
                axisX = SensorManager.AXIS_X;
                axisY = SensorManager.AXIS_Y;
                break;
            case Surface.ROTATION_90:
                axisX = SensorManager.AXIS_Y;
                axisY = SensorManager.AXIS_MINUS_X;
                break;
            case Surface.ROTATION_180:
                axisX = SensorManager.AXIS_MINUS_X;
                axisY = SensorManager.AXIS_MINUS_Y;
                break;
            case Surface.ROTATION_270:
                axisX = SensorManager.AXIS_MINUS_Y;
                axisY = SensorManager.AXIS_X;
                break;
            default:
                break;
        }
        break;
    }
    default:
        break;
}

Get the orientation degrees:

boolean success = SensorManager.remapCoordinateSystem(getQuaternion().getMatrix4x4().getMatrix(), axisX, axisY, mRotationMatrixTransformed);
if (success) {
    SensorManager.getOrientation(mRotationMatrixTransformed, mOrientationValues);

    for (int i = 0; i < 3; i++) {
        mOrientationDegrees[i] = (float) Math.toDegrees(mOrientationValues[i]);
    }
//And for look through, add the rotation state
    if (mMode == MODE.LOOK_THROUGH) {
    // look through has different angles depending on rotation state
    switch (screenRotation) {
        case Surface.ROTATION_90: {
            mOrientationDegrees[2] += 90;
            break;
        }
        case Surface.ROTATION_180: {
            mOrientationDegrees[2] += 180;
            break;
        }
        case Surface.ROTATION_270: {
            mOrientationDegrees[2] += 270;
            break;
        }
    }
}

Answer 3

This is how I do the magic in my application:

        float[] rotationMatrixOrig = new float[9];
        SensorManager.getRotationMatrix(rotationMatrixOrig, null, lastAccelerometerValue, lastMagnetometerValue);

        int screenRotation = app.getCurrentActivity().getWindowManager().getDefaultDisplay().getRotation();
        int axisX, axisY;
        boolean isUpSideDown = lastAccelerometerValue[2] < 0;

        switch (screenRotation) {
            case Surface.ROTATION_0:
                axisX = (isUpSideDown ? SensorManager.AXIS_MINUS_X : SensorManager.AXIS_X);
                axisY = (Math.abs(lastAccelerometerValue[1]) > 6.0f ? 
                        (isUpSideDown ? SensorManager.AXIS_MINUS_Z : SensorManager.AXIS_Z) :
                        (isUpSideDown ? SensorManager.AXIS_MINUS_Y : SensorManager.AXIS_Y));
                break;
            case Surface.ROTATION_90:
                axisX = (isUpSideDown ? SensorManager.AXIS_MINUS_Y : SensorManager.AXIS_Y);
                axisY = (Math.abs(lastAccelerometerValue[0]) > 6.0f ? 
                        (isUpSideDown ? SensorManager.AXIS_Z : SensorManager.AXIS_MINUS_Z) :
                        (isUpSideDown ? SensorManager.AXIS_X : SensorManager.AXIS_MINUS_X));
                break;
            case  Surface.ROTATION_180:
                axisX = (isUpSideDown ? SensorManager.AXIS_X : SensorManager.AXIS_MINUS_X);
                axisY = (Math.abs(lastAccelerometerValue[1]) > 6.0f ? 
                        (isUpSideDown ? SensorManager.AXIS_Z : SensorManager.AXIS_MINUS_Z) :
                        (isUpSideDown ? SensorManager.AXIS_Y : SensorManager.AXIS_MINUS_Y));
                break;
            case Surface.ROTATION_270:
                axisX = (isUpSideDown ? SensorManager.AXIS_Y : SensorManager.AXIS_MINUS_Y);
                axisY = (Math.abs(lastAccelerometerValue[0]) > 6.0f ? 
                        (isUpSideDown ? SensorManager.AXIS_MINUS_Z : SensorManager.AXIS_Z) :
                        (isUpSideDown ? SensorManager.AXIS_MINUS_X : SensorManager.AXIS_X));
                break;
            default:
                axisX = (isUpSideDown ? SensorManager.AXIS_MINUS_X : SensorManager.AXIS_X);
                axisY = (isUpSideDown ? SensorManager.AXIS_MINUS_Y : SensorManager.AXIS_Y);
        }

        float[] rotationMatrix = new float[9];
        SensorManager.remapCoordinateSystem(rotationMatrixOrig, axisX, axisY, rotationMatrix);

Answer 4

If the phone UI locked to the rotation 0, I am getting the following values without remapCoordinateSystem()

Pitch (phone) = -Pitch   (API)
Roll  (phone) =  Roll     (API)
Yaw   (phone) =  Azimuth  (API)

• at least near 0,0,0 values.

If the phone UI forced to rotation 90:

Yaw value has -90 degree ( - PI/2 ) at old orientation!!! => I will go to East in reality instead of North.

If I take phone to 0,0,0 position:

Pitch (phone) = -Roll    (API)
Roll  (phone) = -Pitch   (API)
Yaw   (phone) =  Azimuth (API)

If the phone UI forced to rotation 180:

Yaw value has +/-180 degree ( +/- PI ) at old orientation!!! => I will go to South in reality instead of North.

If I take phone to 0,0,0 position:

Pitch (phone) =  Pitch   (API)
Roll  (phone) = -Roll    (API)
Yaw   (phone) =  Azimuth (API)

If the phone UI forced to rotation 270:

Yaw value has +90 degree ( + PI/2 ) at old orientation!!! => I will go to West in reality instead of North.

If I take phone to 0,0,0 position:

Pitch (phone) =  Roll    (API)
Roll  (phone) =  Pitch   (API)
Yaw   (phone) =  Azimuth (API)

I wrote a little fix, and tested with: android:screenOrientation="fullSensor"

public static final void fixRotation0(float[] orientation) { //azimuth, pitch, roll
    orientation[1] = -orientation[1]; // pitch = -pitch
}

public static final void fixRotation90(float[] orientation) { //azimuth, pitch, roll
    orientation[0] += Math.PI / 2f; // offset
    float tmpOldPitch = orientation[1];
    orientation[1] = -orientation[2]; //pitch = -roll
    orientation[2] = -tmpOldPitch; // roll  = -pitch    
}

public static final void fixRotation180(float[] orientation) { //azimuth, pitch, roll
    orientation[0] = (float)(orientation[0] > 0f ? (orientation[0] - Math.PI) : (orientation[0] + Math.PI)); // offset
    orientation[2] = -orientation[2]; // roll = -roll
}

public static final void fixRotation270(float[] orientation) { //azimuth, pitch, roll
    orientation[0] -= Math.PI / 2; // offset
    float tmpOldPitch = orientation[1];
    orientation[1] = orientation[2]; //pitch = roll
    orientation[2] = tmpOldPitch; // roll  = pitch  
}

In most cases is working. When you rotate quickly 180 degree around 1 axis, than the system will be screwed!

The full code available at Github

Answer 5

What I do is

1. remapping coordinate system like keianhzo suggested in his answer, to remap coordinates according to screen-rotation
2. then I remap resulting coordinate system again with

   SensorManager.remapCoordinateSystem(rotationMatrixScreenRemapped, SensorManager.AXIS_X, SensorManager.AXIS_Z, rotationMatrixCameraRemapped);

   to remap coordinates according to camera (AR-like), as suggested in documentation

So far I hope it works well!