Moving a vertex through the cursor

Question

There is a way to move a selected point in a mesh with a cursor(assuming a camera that doesn;t change between frames).

The way I remember the algorithm (but seems to be wrong) is:

• Unproject the cursor position onto the world position
• Use this and the camera position to create a ray $R$
• for every vertex in the mesh, see if $R$ intersects a small sphere centered at the vertex.
• If you find a valid vertex, mark it as selected
• project the selected vertex onto the screen using the view_projection transformation of the camera
• store the resulting depth (z component) into a variable.
• Now when you move the mouse, you create a fabricated position consisting of the cursor position and the stored projected depth, then you unproject that.

The resulting point should be constrained on a plane in 3D that is parallel to the screen. I have coded the procedure but it seems I am mathematically wrong. So I am not entirely sure if I forgot a step.

To be clear, the objective is to drag the mouse around and have the selected point move in 3D such that it is rendered at the same position as the cursor is on the screen.

Answer 1

Figured it out, I did skip a step, you are supposed to multiply by the homogeneous coordinate.

Basically the actually projected point is (courtesy of GLM):

        vec<4, T, Q> tmp = vec<4, T, Q>(obj, static_cast<T>(1));
        tmp = model * tmp;
        tmp = proj * tmp;

        tmp /= tmp.w;

Kinda obvious in hindsight.