I need to create a ray to pick objects in 3d environment using cursor.
I'm not going to go into detailed math for this question. There's tons of information on that. In short terms, for orthogonal projection we accomplish this by taking the cursor position, and we unproject it to the near and far plane of the view frustum. The direction of the ray is then the normalized difference of the unprojected points $p_{far}-p_{near}$. The origin of the ray should be $p_{near}$.
However I'm not completely sure about how we should approach this problem for perspective projection. Should we consider the camera position as the ray origin or should it be $p_{near}$? And should the direction then be the normalized difference `$p_{far}-p_{near}$ or $p_{far}-p_{camera}$?