I am working with the standford2d3d dataset http://buildingparser.stanford.edu/dataset.html which contains panoramas of various scenes and a .mat file of the point cloud information of all the scenes. I have been able to write a script that will extract the point cloud information from each scene according to the position of the camera. So for example there might be an RGB panorama of a specific office (let's say office_26 for example). I can extract the office_26 point cloud information so that now I have the panorama and the point cloud information of the scene.
Ultimately what I want to do is get each face of the panorama using a CMP script I wrote but also get the point cloud information of each face. Something to note is that my CMP script allows me to have different width and height dimensions (the theta shift is just to shift the panorama x degrees to the right):
def spherical_to_cube(face_width, face_height, img, theta_shift):
"""
Converts an equirectangular (spherical) panorama image into a cube map representation.
"""
The route I have tried to go down is to create a view frustum and extract the points inside the frustum but I haven't figured out a proper way of doing it and keep getting weird behaviour. There is also the fact that I have no way of aligning the panorama with the point cloud data properly.
Some useful information: the point cloud data includes the camera position and rotation.
