Photogrammetry: Structure from Motion Project

In the last article, we saw different methods to create 3D point clouds from pictures. In this one, we will talk about a specific method used to do that: Structure from Motion. We computed this method by hand on Python. Here, I will detail the different part of my Python's code.

---

Images Preprocessing

The first step of this project is to take all the images, to categorize all the camera intrinsic parameters, and to sort the pictures. These data will be able to compute the essential matrix for example. Here is the associated code:

---

SIFT and RANSAC

Now that we have all the pictures and the information, we have to take two consecutives images (this algorithm works iteratively). We can reduce the size of the images to accelerate the calculations. We must detect the 2D locations of the features points in the two images using the SIFT technic. Thanks to that, we can compute the Essential Matrix by corresponding the important points (we use a RANSAC algorithm here). I did this code for this part:

For the SIFT and the image reduction, I created theses functions:

---

Camera Pose and Triangulation

Thanks to the Essential Matrix that we computed in the last part and to the camera parameters, we can now estimate the camera pose, and the different translation and rotation matrices between the two points. After that, in order to avoid ambiguity we practice triangulation, knowing the two camera poses. It will help us knowing the exact position of the 3D points thanks to the two points of view.

Here is the code to do that:

For the triangulation part, I created this function:

---

Reprojection Error and Perspective-n-Points

After that, we can compute the reprojection error, that we can try to minimize by using Perspective-n-Points. It is useful to obtain the final camera poses, and to obtain the 3D points. I used this code here:

---

Same for all the points

Now, we want to do the same, but not only for two points. We have an incremental part were we add the images one by one, to have a more detailed model at the end. Here is the code:

Finally, we convert the point cloud into a .ply file in order to be able to read it.

---

Results

I tried this code with different sets of images. The first one is a set of photographies, representing a statue.

With the code, we obtain this 3D model:

This is quite convincing. Let's try with a building:

We can obtain this reconstruction:

Finally, I tried with a water tower, but the pictures were taken only from one face:

Here, the result from this side was pretty good:

However, if we turn around, the result is really bad (because we didn't turn around the structure during the photo acquisition):

To conclude, using this Structure from Motion method, we were able to create 3D point clouds from photographies. The results are great, but in fact if we want a better result, we must have more pictures, in better quality.