How it works
DirtScan is at a high level, a customised Photogrammetry pipeline. For those interested, it is based off of the COLMAP software.
Photogrammetry, is the process of reconstructing 3D models from lots of individual images.
I won’t go into detail, but at a high level these are the main steps to be aware of. Understanding this will allow you to understand the why behind lots of the decisions made.
Main Steps
1 - Photo taking
Photos are taken of the intended object, here, a mountain bike trail. Lots of photos are taken, from as many angles as possible.
The rule is, if the camera can’t see it, it doesn’t end up in the 3D model - the camera has no concept of space like a human does.
2 - Feature Extraction
Features here are simply parts of the photo, that are stable under different lighting conditions. Think: leaves, rocks, sticks on the ground; the corner of a log; the edge of a tree.
A algorithm runs over all the photos and finds what could be useful reference points in each photo.
3 - Matching
After features are extracted, another algorithm tries to pair images up by comparing their features. Basically, it tries to work out what images saw the same objects.
4 - Mapping
After matching, an algorithm then runs and tries to place these photos in 3D space, such that their matching features line up. It tries to build a “map” of where you walked with the camera. This is not a perfect process - often features are ever so slightly misaligned due to external factors such as lens distortion, bluriness, lighting, etc. The mapping algorithm tries to minimise the error, but it can never fully remove it.
5 - Dense
Now that each photo is placed in a 3D world, another algorithm then runs and projects every (or nearly every) pixel into the 3D world. It compares the output of all the cameras that see that pixel, and tries to find the optimal location.
This produces a dense point cloud - a large (sometimes 10+ million) cloud of 3D points, that form the “surface” of the model.
6 - Meshing
This step takes the dense point cloud and runs an algorithm over it to get a proper 3D mesh.
7 - Texturing
This step takes the mesh, and the actual photos from the cameras, and projects the actual images to form actual textures on the surface of the model.
Additional optional steps
There are other tangential/optional steps that can be run, such as:
- Cleaning the point cloud to remove tree foliage (this often appears as blobs)
- Removing sun-cast shadows from textures
- Selectively improving detail in some parts of the model, either via more detailed dense triangulation, or by supplying additional photos.
- Automatic detection of objects in the images/model (coming soon)