Struggeling translating the position of the mouse to the location of the tiles in my grid. When it\'s all flat, the math looks like this:
this.position.x = Math.
Here is the grid input I would define for the sake of this discussion. The output should be some tile (coordinate_1, coordinate_2) based on visibility on the users screen of the mouse:
I can offer two solutions from different perspectives, but you will need to convert this back into your problem domain. The first methodology is based on coloring tiles and can be more useful if the map is changing dynamically. The second solution is based on drawing coordinate bounding boxes based on the fact that tiles closer to the viewer like (0, 0) can never be occluded by tiles behind it (1,1).
Approach 1: Transparently Colored Tiles
The first approach is based on drawing and elaborated on here. I must give the credit to @haldagan for a particularly beautiful solution. In summary it relies on drawing a perfectly opaque layer on top of the original canvas and coloring every tile with a different color. This top layer should be subject to the same height transformations as the underlying layer. When the mouse hovers over a particular layer you can detect the color through canvas and thus the tile itself. This is the solution I would probably go with and this seems to be a not so rare issue in computer visualization and graphics (finding positions in a 3d isometric world).
Approach 2: Finding the Bounding Tile
This is based on the conjecture that the "front" row can never be occluded by "back" rows behind it. Furthermore, "closer to the screen" tiles cannot be occluded by tiles "farther from the screen". To make precise the meaning of "front", "back", "closer to the screen" and "farther from the screen", take a look at the following:
.
Based on this principle the approach is to build a set of polygons for each tile. So firstly we determine the coordinates on the canvas of just box (0, 0) after height scaling. Note that the height scale operation is simply a trapezoid stretched vertically based on height.
Then we determine the coordinates on the canvas of boxes (1, 0), (0, 1), (1, 1) after height scaling (we would need to subtract anything from those polygons which overlap with the polygon (0, 0)).
Proceed to build each boxes bounding coordinates by subtracting any occlusions from polygons closer to the screen, to eventually get coordinates of polygons for all boxes.
With these coordinates and some care you can ultimately determine which tile is pointed to by a binary search style through overlapping polygons by searching through bottom rows up.