How do I get touch coordinates with respect to canvas after scaling and translating?

Question

I need to get the touch x and y with respect to the canvas to check for collisions and things like that after I have moved and scaled the canvas.

I already managed t

Answer 1

This should help:

float px = e.getX() / mScaleFactorX;
float py = e.getY() / mScaleFactorY;
int ipy = (int) py;
int ipx = (int) px;
Rect r = new Rect(ipx, ipy, ipx+2, ipy+2);

And where the canvas is:

    final float scaleFactorX = getWidth()/(WIDTH*1.f);
    final float scaleFactorY = getHeight()/(HEIGHT*1.f);

    if(mScaleFactorX == INVALID){
        mScaleFactorX = scaleFactorX;
        mScaleFactorY = scaleFactorY;
    }

This is a really simple way, and it works because it scales down the onTouch coordinates to be the same min/max as the canvas, causing them to be scaled. Do NOT use getRawX and getRawY because that will return wrong coordinates if you are using a custom layout with the view added and other elements around it. getX and getY returns the accurate coordinates scaled to your view.

This is really simple and does not take up a lot of code. scaleFactor can be used elsewhere to handle zoom(you take care of that code) but what this code does is to handle the issue of getting the pointer coordinates to match the scaled canvas

Answer 2

 private float convertToCanvasXCoordinate(float touchx,float offsetx,float viewportVisibleWidth){
        float newx=(touchx*viewportVisibleWidth)/getWidth()+offsetx;
        return newx;
    }

    private float convertToCanvasYCoordinate(float touchy,float offsety,float viewportVisibleHeight){
        float newy=(touchy*viewportVisibleHeight)/getHeight()+offsety;
        return newy;
    }

i just found out there is a function canvas.getClipBound() which is a rectangle representing the visible viewport that includes the offsetx offset y (the left and top of the rectangle respectively) and the viewport width and height

simply call these functions and it will get you touchx and touchy with respect to canvas

Answer 3

Updated, super simple example:

The most basic way to properly do a scene in android is to use a matrix to modify the view and the inverse of that matrix to modify your touches. Here's a simplified answer. Kept very short.

public class SceneView extends View {
    Matrix viewMatrix = new Matrix(), invertMatrix = new Matrix();
    Paint paint = new Paint();
    ArrayList<RectF> rectangles = new ArrayList<>();
    RectF moving = null;

    public SceneView(Context context) { super(context); }
    public SceneView(Context context, AttributeSet attrs) { super(context, attrs); }
    public SceneView(Context context, AttributeSet attrs, int defStyleAttr) { super(context, attrs, defStyleAttr); }

    @Override
    public boolean onTouchEvent(MotionEvent event) {
        event.transform(invertMatrix);
        switch (event.getAction()) {
            case MotionEvent.ACTION_DOWN:
                moving = null;
                for (RectF f : rectangles) {
                    if (f.contains(event.getX(), event.getY())) {
                        moving = f;
                        return true;
                    }
                }
                viewMatrix.postTranslate(50,50);
                viewMatrix.postScale(.99f,.99f);
                viewMatrix.postRotate(5);
                invertMatrix = new Matrix(viewMatrix);
                invertMatrix.invert(invertMatrix);
                break;
            case MotionEvent.ACTION_MOVE:
                if (moving != null) {
                    moving.set(event.getX() - 50, event.getY() - 50, event.getX() + 50, event.getY() + 50);
                }
                break;
            case MotionEvent.ACTION_UP:
                if (moving == null) {
                    rectangles.add(new RectF(event.getX() - 50, event.getY() - 50, event.getX() + 50, event.getY() + 50));
                }
                break;
        }
        invalidate();
        return true;
    }

    @Override
    protected void onDraw(Canvas canvas) {
        canvas.concat(viewMatrix);
        for (RectF f : rectangles) {
            canvas.drawRect(f,paint);
        }
    }

This is rather minimalist, but it shows all the relevant aspects. Moving the view, touch modification, collision detection. Each time you touch the screen it will move diagonally, zoomout, and rotate (basically moves in a spiral), and create a black rectangle. If you touch the rectangles you can move them around to your heart's content. When you click the background, more spiraling the view, dropping black rectangles.

See: https://youtu.be/-XSjanaAdWA

---

The line in the other answer here is " Given that we're scaling relative to the origin." Which is to say that our scale is already relative to the origin. Scale is relative to the origin because the matrix code simply multiplies the x and y coords.

When things are scaled relative to anything else, they are really translated, scaled, translated back. That's just how the math has to be. If we apply a scale to the canvas, that scale is already relative to the origin. Sometimes you can scale relative to a point, but that's just matrix math.

In most implementations of views as such, generally perform a zoom on a point by zooming in. Then panning the given viewport. This is because the viewports are similar scaled rectangles.

So what we do is. figure out how much we need to pan to keep the point in the same place, both the prezoomed view and postzoomed relative to the viewport. That code is:

scalechange = newscale - oldscale;
offsetX = -(zoomPointX * scalechange);
offsetY = -(zoomPointY * scalechange);

Then we do canvas.translate(offsetX,OffsetY);

---

The question here though, is how to do translate that back for the given touch events for Android. And for that the answer we apply all the same operations we applied to the view to the touch positions in reverse order.

Basically the way matrix math works you have to apply reverse operations in reverse order to get the inversion. Though this is why we tend to get inverted matrices for our Matrix transformation. And in Android, we have a lot of stuff done for us. And if you get your head around what's going on, we can solve all of these problems and, really don't have to worry about any of this.

---

You can check a well done implementation of this at this project (MIT license, I coded the relevant part): https://github.com/Embroidermodder/MobileViewer

The MotionEvent class can be very importantly modified by the Matrix. And the matrices can be inverted. If we understand this, we understand that all the work is done for us. We simply take whatever matrix we made, and apply that to the View. We get the inverse of that matrix, and apply that inverted Matrix to the touch events, as they happen. -- Now our touch events happen in scene space.

We can also, if we want the position of something call matrix.mapPoints() will let us simply convert these back and forth, as needed.

The other way of doing this would be to take the scene we want and convert that via the View class rather than in the canvas. This would make the touch events occur in the same space as the screen. But Android will void out touch events that occur outside of the view, So MotionEvents that begin outside of the original clipped part of the view will be discarded. So this is a non-starter. You want to translate the canvas. And counter translate the MotionEvent.

We'll need a couple classes. We can define a view port, and use that to build our matrices:

private RectF viewPort;

Matrix viewMatrix;
Matrix invertMatrix;

The viewPort certainly isn't needed, but conceptually it can help a lot.

Here we build the matrix from the viewPort. Which is to say, whatever rectangle we set that to, it will be the the part of the scene we can view.

public void calculateViewMatrixFromPort() {
    float scale = Math.min(_height / viewPort.height(), _width / viewPort.width());
    viewMatrix = new Matrix();
    if (scale != 0) {
        viewMatrix.postTranslate(-viewPort.left, -viewPort.top);
        viewMatrix.postScale(scale, scale);

    }
    calculateInvertMatrix();
}

If we modify the viewMatrix, we can use that to derive the port, by simply setting the original screen then using the Matrix to put that Rectangle the size of the screen in the terms of the screen.

public void calculateViewPortFromMatrix() {
    float[] positions = new float[] {
            0,0,
            _width,_height
    };
    calculateInvertMatrix();
    invertMatrix.mapPoints(positions);
    viewPort.set(positions[0],positions[1],positions[2],positions[3]);
}

This assumes we have the _width and _height of the view we're working with, we can simply pan and scale the viewbox. If you wanted something fancier like to apply a rotation to the screen, you would need to use 4 points, 1 for each of the corner, and then apply the matrix to the points. But, you can basically easily add such things, because we don't deal with the heavy lifting directly but rely heavily on the matrix.

We also need to be able to calculate the inverted matrix, so that we can reverse the MotionEvents:

public void calculateInvertMatrix() {
    invertMatrix = new Matrix(viewMatrix);
    invertMatrix.invert(invertMatrix);
}

And then we apply these matrices to the canvas and the invert matrix to the MotionEvent

@Override
public boolean onTouchEvent(MotionEvent event) {
    //anything happening with event here is the X Y of the raw screen event.

    event.offsetLocation(event.getRawX()-event.getX(),event.getRawY()-event.getY()); //converts the event.getX() to event.getRaw() so the title bar doesn't fubar.

    //anything happening with event here is the X Y of the raw screen event, relative to the view.
    if (rawTouch(this,event)) return true;

    if (invertMatrix != null) event.transform(invertMatrix);
    //anything happening with event now deals with the scene space.

    return touch(this,event);
}

One of the notable deficits in the MotionEvent class is that getRawX() and getRawY() (which are the actual raw touches on the screen rather than the touches in the view, only allow you to do a single finger location. Really that's pretty crippling, but we can simply put in an offset to the MotionEvent so that the getX(3) and various points properly overlap where getRawX(3) would be. This will properly let us deal with title bars etc, as the MotionEvents are technically in relative to the view, and we need them relative to the screen (sometimes these are the same, such as with full screen mode).

Now, we're done.

So we can apply these Matrices and remove them and switch our contexts very easily, without needing to knowing what they are, or what our current view is looking at and get all the different touch events and various pointerCounts of Touch Events properly.

We also can draw our stuff at different translations. Such as if we want an overlay of tools that do not move with the scene, but rather relative to the screen.

@Override
public void onDraw(Canvas canvas) {
        //Draw all of our non-translated stuff. (under matrix bit).
        canvas.save();
        if (viewMatrix != null) canvas.setMatrix(viewMatrix);
        //Draw all of our translated stuff.
        canvas.restore();
        //Draw all of our non-translated stuff. (over matrix bit).
}

It's best to save and restore the canvas so that the matrix we apply gets removed. Especially if things are going to get complicated by passing the draw event around to different classes. Sometimes these classes might add in a matrix to the canvas which is the reason the View classes source code itself looks a bit like:

        int level = canvas.getSaveCount();
        canvas.save();
        //does the drawing in here, delegates to other draw routines.
        canvas.restoreToCount(level);

It saves count for how many states are stacked up in the canvas. Then after delegating to who knows what, it restores back to that level in case some class called .save() but didn't call restore(). You might want to do the same.

If we want the full pan and zoom code, we can do that too. There's some tricks with regard to setting the zoom point as the midpoint between the various touches, etc.

---

float dx1;
float dy1;
float dx2;
float dy2;
float dcx;
float dcy;

@Override
public boolean rawTouch(View drawView, MotionEvent event) {
//I want to implement the touch events in the screen space rather than scene space.
//This does pinch to zoom and pan.
    float cx1 = event.getX();
    float cy1 = event.getY();
    float cx2 = Float.NaN, cy2 = Float.NaN;
    float px = cx1;
    float py = cy1;
    if (event.getPointerCount() >= 2) {
        cx2 = event.getX(1);
        cy2 = event.getY(1);
        px = (cx1 + cx2) / 2;
        py = (cy1 + cy2) / 2;
    }


    switch (event.getActionMasked()) {
        case MotionEvent.ACTION_MOVE:
            float deltascale = (float) (distance(cx1,cy1,cx2,cy2) / distance(dx1,dy1,dx2,dy2));
            float dpx = px-dcx;
            float dpy = py-dcy;
            if (!Float.isNaN(dpx)) pan(dpx, dpy);
            if (!Float.isNaN(deltascale)) scale(deltascale, px, py);
            view.invalidate();
            break;
        default:
            cx1 = Float.NaN;
            cy1 = Float.NaN;
            cx2 = Float.NaN;
            cy2 = Float.NaN;
            px = Float.NaN;
            py = Float.NaN;
            break;
    }
    dx1 = cx1;
    dy1 = cy1;
    dx2 = cx2;
    dy2 = cy2;
    dcx = px;
    dcy = py;
    return true;
}

@Override
public boolean touch(View drawView, MotionEvent event) {
    //if I wanted to deal with the touch event in scene space.
    return false;
}

public static double distance(float x0, float y0, float x1, float y1) {
    return Math.sqrt(distanceSq(x0, y0, x1, y1));
}
public static float distanceSq(float x0, float y0, float x1, float y1) {
    float dx = x1 - x0;
    float dy = y1 - y0;
    dx *= dx;
    dy *= dy;
    return dx + dy;
}



public void scale(double deltascale, float x, float y) {
    viewMatrix.postScale((float)deltascale,(float)deltascale,x,y);
    calculateViewPortFromMatrix();
}

public void pan(float dx, float dy) {
    viewMatrix.postTranslate(dx,dy);
    calculateViewPortFromMatrix();
}