In this episode of the basic turntable photogrammetry series, we will analyze the fundamental features of a Meshroom project for reconstructing geometry from photographs.
Video Transcript
Hello everyone! In this episode of the basic turntable photogrammetry series, we will analyze the fundamental features of a Meshroom project for reconstructing geometry from photographs.
I will first give an overview of Meshroom’s interface and workflow, then move on to the practical steps of loading images.
After downloading and installing Meshroom, let's launch it. Two windows will open. The console window displays data and information about the program’s execution, while the main window is where we work on our projects. Closing either of these windows will cause the program to shut down, so both must remain open.
Meshroom actually allows for different types of projects, as you can see by clicking New Pipeline in the File menu. Of course, we are interested in a Photogrammetry project.
The first thing to do is save the project. I recommend creating a dedicated folder where you can save the project file, as running the photogrammetry process will generate an entire folder structure, which we will examine shortly.
The project file will have the MG extension, which stands for Meshroom Graphs. The terms Pipeline and Graphs emphasize that processes in Meshroom follow a sequential, step by step order. This can be visually understood in the Graph Editor panel at the bottom, where nodes and their connections are displayed.
Before examining this pipeline, let’s take a quick look at the program’s interface.
The Image Gallery panel will contain the images we want to use for reconstruction. Initially, it is empty and indicates that we can also drag and drop files from a folder into this space. If I remember correctly, this drag and drop feature does not work in Windows if you run Meshroom in admin mode. In any case, images can also be imported via Import Images in the File menu.
In the Image Viewer panel, we can view the loaded images one by one, allowing us to discard poor quality ones or extract relevant data. This window will also display some data generated by Meshroom, such as the Depth Map, as we will see later.
The 3D Viewer panel will display the point cloud, which represents the reconstructed scene. Here, we can also redefine the volume to be reconstructed, known as the bounding box, allowing us to focus only on specific elements.
The Graph Editor panel shows the sequence of nodes to be processed and their connections. Each node represents a specific task that is executed step by step, using incoming data from the input ports and producing one or more outputs from the output ports.
A new Photogrammetry project will have a basic pipeline with default parameters, which will work well for many projects. When we start the project, Meshroom will create a subfolder called Cache, inside which it will generate additional subfolders, one for each node present in the pipeline.
The pipeline does not have to remain as the default one; it can be modified by adding new nodes or changing the connections between them, as we will see later in this episode.
In any case, by clicking on a node, we can view its parameters and settings in the Node panel on the right. Each node provides access to multiple types of information, displayed in the Attributes, Log, and other tabs at the bottom.
The last panel in the default interface is the Task Manager, which is initially empty. Here, we will later see execution details for the various nodes.
In the View menu, you may notice that the Live Reconstruction panel is missing. However, I won’t be using it in this introductory series, so we can leave it disabled.
The Fullscreen option, as expected, allows the main program window to be displayed in full screen mode.
Before moving on to the practical steps, let me briefly introduce the pipeline, which represents the sequence of execution for the nodes.
The pipeline can be divided into two parts. In the first part, which runs up to the Structure From Motion node, Meshroom loads the images and extracts specific information, called Features, which allow it to identify reference points across multiple photographs.
These Features will then be used to reconstruct the scene's geometry, but this process occurs after the Structure From Motion node. Specifically, in the second phase, Meshroom calculates depth data for the detected Features, generating a 3D point cloud. It then reconstructs the geometry, which is called a Mesh because it consists of vertices connected by edges, which in turn define triangular faces.
Finally, the original photographs will be combined into an image called a Texture, which represents the colors of the various parts of the original 3D object, but arranged in a 2D image.
From the Features detected in the photographs, we start with points in space that are initially unconnected. These points are then linked together to form geometry, made up of surfaces, onto which a Texture image is applied to give the reconstructed geometry the original object's colors.
There is much more to discuss about the pipeline, the nodes, and the overall execution process, but I will explain these details as we progress with the project.
Now, let’s move on to the practical steps. First, I drag the original photographs into the Image Gallery panel.
In the Image Gallery panel, we can see the total number of photos added and how many cameras were recognized by Meshroom, based on the intrinsic data extracted from the images. This number is displayed next to the shutter icon at the bottom of the panel.
For each photograph, we can examine this metadata, if available, by hovering the mouse over the shutter icon in the image preview. In this case, the metadata is present, so the shutter icon appears green. If the metadata is missing, the icon will be red, and Meshroom will proceed with the reconstruction as best as it can without that information.
In a turntable photogrammetry project, it is recommended to select the Feature Matching node and set the Minimal 2D Motion parameter to 2. Once this adjustment is made, I save the project.
In a project where a green screen background was used in the photos, as in this case, it is possible to try to have Meshroom exclude the green background before extracting the Features. This can be done by adding an Image Masking node in the Graph Editor, by right clicking on an empty area of the panel and inserting the node.
In the Structure From Motion Data input of the new node, we need to connect the Structure From Motion Data output from the Camera Init node, by clicking and dragging between the two ports.
The output of the new node should then be connected to the Mask Folder inputs of both the Feature Extraction and Prepare Dense Scene nodes.
Now, by selecting the Image Masking node, we can define the color to exclude and adjust the tolerance values for hue, saturation, and intensity in the Node panel. However, this method for excluding the green background is not very practical, as it does not provide good feedback on the result and does not allow for customized parameters for each photo. This last point could be useful in our case since the lighting on the background cloth is not uniform.
For these reasons, I am removing the Image Masking node by right clicking and selecting Remove Node. For reference, a connection between nodes can be removed by right clicking on the link and selecting Remove.
I am also removing all the photographs, because in the next episode, I will show you a small setup in Blender that will allow us not only to remove the green background more easily, but also to eliminate the green spill, which is the green tint that affects objects photographed against a green screen.
The only custom setting I am keeping is the value 2 for the Minimal 2D Motion parameter in the Feature Matching node.
I am now saving the project with these settings, closing Meshroom, and I’ll see you in the next episode!