Transform screen coordinates to model coordinates

Question

I've got some sort of newbie question.

In my application (processingjs) i use scale() and translate() to allow the user to zoom and scroll through the scene. As long as i keep the scale set to 1.0 i've got no issues. BUT whenever i use the scale (i.e. scale(0.5)) i'm lost...

I need the mouseX and mouseY translated to the scene coordinates, which i use to determine the mouseOver state of the object I draw on the scene.

Can anybody help me how to translate these coordinates? Thanks in advance! /Richard

Answer 1

Unfortunately for me this required a code modification. I'll look at submitting this to the Processing.JS code repository at some point, but here's what I did.

First, you'll want to use modelX() and modelY() to get the coordinates of the mouse in world view. That will look like this:

float model_x = modelX(mouseX, mouseY);
float model_y = modelY(mouseX, mouseY);

Unfortunately Processing.JS doesn't seem to calculate the modelX() and modelY() values correctly in a 2D environment. To correct that I changed the functions to be as follows. Note the test for mv.length == 16 and the section at the end for 2D:

p.modelX = function(x, y, z) {
  var mv = modelView.array();
  if (mv.length == 16) {
    var ci = cameraInv.array();
    var ax = mv[0] * x + mv[1] * y + mv[2] * z + mv[3];
    var ay = mv[4] * x + mv[5] * y + mv[6] * z + mv[7];
    var az = mv[8] * x + mv[9] * y + mv[10] * z + mv[11];
    var aw = mv[12] * x + mv[13] * y + mv[14] * z + mv[15];
    var ox = 0, ow = 0;
    var ox = ci[0] * ax + ci[1] * ay + ci[2] * az + ci[3] * aw;
    var ow = ci[12] * ax + ci[13] * ay + ci[14] * az + ci[15] * aw;
    return ow !== 0 ? ox / ow : ox
  }
  // We assume that we're in 2D
  var mvi = modelView.get();
  // NOTE that the modelViewInv doesn't seem to be correct in this case, so
  // having to re-derive the inverse
  mvi.invert();
  return mvi.multX(x, y);
};
p.modelY = function(x, y, z) {
  var mv = modelView.array();
  if (mv.length == 16) {
    var ci = cameraInv.array();
    var ax = mv[0] * x + mv[1] * y + mv[2] * z + mv[3];
    var ay = mv[4] * x + mv[5] * y + mv[6] * z + mv[7];
    var az = mv[8] * x + mv[9] * y + mv[10] * z + mv[11];
    var aw = mv[12] * x + mv[13] * y + mv[14] * z + mv[15];
    var oy = ci[4] * ax + ci[5] * ay + ci[6] * az + ci[7] * aw;
    var ow = ci[12] * ax + ci[13] * ay + ci[14] * az + ci[15] * aw;
    return ow !== 0 ? oy / ow : oy
  }
  // We assume that we're in 2D
  var mvi = modelView.get();
  // NOTE that the modelViewInv doesn't seem to be correct in this case, so
  // having to re-derive the inverse
  mvi.invert();
  return mvi.multY(x, y);
};

I hope that helps someone else who is having this problem.

Answer 2

Have you tried another method?

For example, assume that you are in a 2D environment, you can "map" all the frame in a sort of matrix. Something like this:

int fWidth = 30;
int fHeight = 20;
float objWidth = 10;
float objHeight = 10;

void setup(){
  fWidth = 30;
  fHeight = 20;
  objWidth = 10;
  objHeight = 10;
  size(fWidth * objWidth, fHeight * objHeight);
}

In this case you will have a 300*200 frame, but divided in 30*20 sections. This allows you to move in somewhat ordered way your objects.

When you draw an object you have to give his sizes, so you can use objWidth and objHeight.

Here's the deal: you can make a "zoom-method" that edit the value of the object sizes. In this way you drew a smaller/bigger object without editing any frame property.

This is a simple example because of your inaccurate question. You can do it [in more complex ways], in a 3D environment too.