How to apply rotation to extrinsic and intrinsic camera parameters post-calibration?

Question

I currently have a set of intrinsic / extrinsic camera parameters, and am using them to project 3D world coordinates to the 2D image, using projection matrix $P = K[R | t]$.

My problem is that this projects them onto the horizontal image, however I want the image to be in portrait. I can easily just rotate the image after projection, but I want to manually rotate all the images to portrait beforehand and alter the projection matrix so that it can project directly to the portrait image.

I've tried the technique from this post, swapping the focal lengths and adjusting the principal point, however that did not seem to give the desired affect. Do I also need to apply a transformation to the extrinsic matrix?

Answer 1

I managed to figure it out. In addition to altering the intrinsic matrix, I also had to apply a 3D rotation around the z axis, in camera space. (since z is the forward pointing axis)

So I just altered the projection matrix to be:

$$ P = K' \cdot rot_z \cdot [R|t] $$

where,

$$ rot_z = \begin{bmatrix} cos(90°) & -sin(90°) & 0 \\ sin(90°) & cos(90°) & 0 \\ 0 & 0 & 1 \end{bmatrix} $$

and $K'$ is the 'portrait' version of the original 'horizontal' intrinsic matrix

$$ K' = \begin{bmatrix} f_y & 0 & portrait\_width - c_y \\ 0 & f_x & c_x \\ 0 & 0 & 1 \end{bmatrix} $$