In this episode of the basic turntable photogrammetry series, we will see how to remove the green background and Color Spill from photographs in Blender.
Video Transcript
Hello everyone! In this episode of the basic turntable photogrammetry series, we will see how to remove the green background and Color Spill from photographs in Blender.
The images obtained at the end of this process will be PNG files with transparency, which Meshroom will be able to use while focusing, so to speak, only on the object of interest during the feature extraction phase.
This tutorial was recorded using Blender 4.3, but the Compositing nodes used have been available in several previous versions of the software and will likely continue to be available in future releases as well.
All the work takes place in a Compositor Editor, so a virtual camera inside the scene is not even required. In fact, I can also delete the Render Layers node, which is present by default in new Compositing setups.
First, I need to set the output format of the project with the resolution values of the photographs, which in my case are 4000 and 3000 for the X and Y resolutions.
In the Output tab of the Properties Editor, I also need to set the output format to PNG with RGBA, meaning with transparency, and with a low compression value.
Within this tab, I also need to set the disk path where the processed files will be saved.
As the last preliminary step, before loading the images into the project, I need to set an End Frame value equal to the number of images to be processed, which in my case is 146.
Regarding the images, Blender will recognize the image sequence in the folder as long as the images are numbered sequentially.
In a Windows environment, there is a very simple way to achieve this numbering. Select all the files in the folder containing the original photographs. Right click on the first file and choose the Rename option. Then, type a name and press Enter. As you can see, Windows will automatically add a progressive numerical suffix to all the file names in the folder.
Now, I can drag the first file into the Compositor Editor in Blender and change the node type to Image Sequence.
Inside the node, I enter the number of files to be processed in the Frames field. This value should be the same as the one previously entered in the End field of the Timeline.
I connect the Image output of the image node to the Viewer node so that I can see the image in the Backdrop of the Compositor Editor.
When I took the photographs, I sometimes rotated my smartphone, which means some images appear upside down. This issue can be easily fixed with a Rotate node, where the Degrees field can be animated so that the node is used only when needed. However, these rotations won’t be a problem for Meshroom, so it’s not strictly necessary to perform the rotations at this stage.
I insert a Color Key node and a Color Spill node into the setup.
The Backdrop preview is useful for selecting a shade of green from the image for the Color Key node. I click on the default color in the Key Color field of the Color Key node, then click on the Eyedropper tool and select a green point in the photograph.
Now, I connect the Image output of the image node to the Image input of the Color Key node.
The Matte output of the Color Key node should be connected to the Alpha input of both the Viewer and Composite nodes, as it represents the transparency mask of the images.
The Image output of the Color Key node, on the other hand, should be connected to a Color Spill node, which allows for the removal of green reflections on objects that are photographed against a green background. The output of the Color Spill node should then be connected to the Image inputs of both the Composite and Viewer nodes.
In our case, the Despill Channel of the node will obviously be green, while the algorithm should be chosen between Simple and Average, depending on the quality of the result, which can be previewed in the Backdrop.
At this point, I can start adjusting the tolerance values for the Hue, Saturation, and Value channels in the Color Key node. These tolerances define how much the pixel color can deviate, in those three information channels, from the color specified in the Key Color field of the node.
The interesting thing about performing this operation in Blender is that the node values can be animated, meaning I can set values that work well for the first frame, keep them as long as they are effective, and then modify them if they no longer work for a specific image.
However, be careful: if you realize that the values set do not work well for another image, you need to go back to the previous frame and insert keyframes there to lock in the values that worked up to that point. Once this is done, you can move to the next frame, adjust the values to meet the new requirements, and record the new values with new keyframes. This step of inserting a keyframe in the previous frame before making changes is necessary because otherwise, Blender will create interpolations between frames, which might have negative effects on their masks.
In general, it is best to use low values for the Hue parameter, as there is a risk of excluding other colors that should actually be kept. It’s better to maintain a base hue with low tolerance and adjust the other two parameters. Most of the time, the uneven lighting on the green screen will cause variations in brightness but will not significantly change the hue.
When adjusting values, you should always keep an eye on the object of interest, because while we want to remove all unwanted areas, sometimes this is not entirely possible, and you might accidentally remove parts of the object you need!
To eliminate more complex areas, you can use one or more Box Mask nodes, connecting them in sequence to the Matte output of the Color Key node. There are also other Masking tools in the Compositor, but a couple of Box Mask nodes will be more than sufficient for our purposes.
These nodes create rectangular masks that can be positioned and rotated on the input mask using parameters that allow you to adjust their position, size, and rotation. The masks have different application modes. For our purposes, I will use the Subtract mode, but keep in mind that this will make the affected areas black rather than transparent.
To solve this issue, simply insert a Set Alpha node before the Image inputs of both the Composite and Viewer nodes. First, I disconnect the Alpha connections from Composite and Viewer. Then, I connect the Image output of the Color Spill node to the Image input of the new Set Alpha node. Finally, I connect the Alpha input of the Set Alpha node to the output of the last Box Mask node in the setup.
Now, I am skipping ahead to the final result of the Compositing setup just to show you that it is essentially a combination of keyframes, both for the tolerance values in the Color Key node and for the position and size values of any masks used.
The position and size parameters of the masks can also be animated, which allows us to bring them into the workspace only when they are actually needed, avoiding the risk of covering the object of interest in other frames.
The same warning given for keyframing the Color Key node values applies here: it is crucial to insert keyframes before modifying a frame to prevent incorrect interpolations between mask positions throughout the animation.
This process of analyzing and adjusting the Color Key values and masks may take some time, but it will save a lot more time later when Meshroom processes these cleaned-up images instead of the original ones.
However, it is not a big issue if you cannot remove every single unwanted element, as Meshroom will still interpret some small artifacts as noise and automatically ignore them. The more we can refine at this stage, the better, but there’s no need to overcomplicate things.
After verifying the quality of the Color Keying and masks across multiple frames, I can finally start the rendering of the sequence, producing the images that will be imported into Meshroom. This rendering process will not take long, as it is simply processing external images rather than rendering a full 3D scene. The images produced are the same ones included in the free package, so you can replicate the steps we will cover in the next tutorials. That’s all for this episode! See you next time!