How to find the 3D coordinates of a surface from the click location of the mouse on the ILNumerics surface plots?

问题

Currently our system uses the ILNumerics 3D plot cube class with an ILNumerics surface component to display a 3D meshed surface. An aim for our system is to be able to interrogate individual points on the surface from a mouse click on the plot. We have the MouseClick event set up on our plot the problem is I am unsure on how to get the values for the particular point on the surface that has been clicked, could anyone help with this issue?

回答1:

The conversion from 2D mouse coordinates to 3D 'model' coordinates is possible - under some limitations:

The conversion is not unambiguous. The mouse event only provides 2 dimensions: X and Y screen coordinates. In the 3D model there might be more than one point 'behind' this 2D screen point. Therefore, the best you can get is to compute a line in 3D, starting at the camera and ending in infinite depth.
While in theory it would be possible at least to try to find the crossing of the line with the 3D objects, ILNumerics currently does not. Even in the simple case of a surface it is easy to construct a 3D model which crosses the line at more than one point.

For a simplified situation a solution exists: If the Z coordinate in 3D does not matter, one can use common matrix conversions in order to acquire the X and Y coordinates in 3D and use these only. Let's say, your plot is a 2D line plot or a surface plot - but only watched from 'above' (i.e. The unrotated X-Y plane). The Z coordinate of the point clicked may not be of interest. Let's further assume, you have setup an ILScene scene in a common windows application with ILPanel:

private void ilPanel1_Load(object sender, EventArgs e) {

var scene  = new ILScene() {
    new ILPlotCube(twoDMode: true) {
        new ILSurface(ILSpecialData.sincf(20,30)) 
    }
}; 
scene.First<ILSurface>().MouseClick += (s,arg) => {
    // we start at the mouse event target -> this will be the 
    // surface group node (the parent of "Fill" and "Wireframe")
    var group = arg.Target.Parent;
    if (group != null) {
        // walk up to the next camera node 
        Matrix4 trans = group.Transform; 
        while (!(group is ILCamera) && group != null) {
            group = group.Parent;
            // collect all nodes on the path up
            trans = group.Transform * trans; 
        }
        if (group != null && (group is ILCamera)) {
            // convert args.LocationF to world coords
            // The Z coord is not provided by the mouse! -> choose arbitrary value
            var pos = new Vector3(arg.LocationF.X * 2 - 1, arg.LocationF.Y * -2 + 1, 0); 
            // invert the matrix.
            trans = Matrix4.Invert(trans); 
            // trans now converts from the world coord system (at the camera) to 
            // the local coord system in the 'target' group node (surface). 
            // In order to transform the mouse (viewport) position, we 
            // left multiply the transformation matrix.
            pos = trans * pos; 
            // view result in the window title
            Text = "Model Position: " + pos.ToString(); 
        }
    }
};
ilPanel1.Scene = scene;

}

What it does: it registers a MouseClick event handler on the surface group node. In the handler it accumulates the transformation matrices on the path from the clicked target (the surface group node) up to the next camera node the surface is a child of. While rendering, the (model) coordinates of the vertices are transformed by the local coordinate transformation matrix, hosted in every group node. All transformations are accumulated and so the vertex coordinates end up in the 'world coordinate' system, established by every camera. So rendering finds the 2D screen position from the 3D model vertex positions.

In order to find the 3D position from the 2D screen coordinates - one must go the other way around. In the example, we acquire the transformation matrices for every group node, multiply them all up and invert the resulting transformation matrix. This is needed, because such transforms naturally describe the conversion from the child node to the parent. Here, we need the other way around - hence the inversion is necessary.

This method gives the correct 3D coordinates at the mouse position. However, keep the limitations in mind! Here, we do not take into account any rotation of the plot cube (the plot cube must be left unrotated) and no projection transforms (plot cubes do use orthographic transform by default, which basically is a noop). In order to recognize those variables as well, you may extend the example accordingly.

来源：https://stackoverflow.com/questions/17676981/how-to-find-the-3d-coordinates-of-a-surface-from-the-click-location-of-the-mouse

标签

ilnumerics