In this tutorial, I’ll show you how to replace a planar portion of real-world footage in Blender using Motion Tracking, without having to perform the Solving step.
Video Transcript
Hello everyone! In this tutorial, I’ll show you how to replace a planar portion of real-world footage in Blender using Motion Tracking, without having to perform the Solving step. I’ll use only 4 Track Points and the Plane Track tool, which works specifically on planar surfaces, so it’s great for replacing or modifying the appearance of a wall, a painting, or other flat surfaces in general, possibly using semi-transparent images.
Before we begin, I need to make two preliminary remarks that will help you understand whether the tool shown in this tutorial is suitable for your needs. The Plane Track is not a 3D object, so you cannot apply UV Mapping to this Plane for the images or video clips you want to overlay onto the original footage. This means that the images or footage must have the same aspect ratio as the Plane Track. Otherwise, distortions will occur. Secondly, since the Plane Track is not a 3D object, it doesn’t have a Material, so the replacement won’t show any specular reflections or other shading effects. However, you can still add Color Grading, Motion Blur, and other effects in Compositing. If these limitations are an issue, it’s better to perform a full Track with Solving and use an actual Plane object with its own Material. If not, then let’s move on with the tutorial!
First, I open a new Blender project, in this case version 4.3, and switch the Workspace to Motion Tracking. You’ll find this option in the VFX section of the Workspace list.
In the Movie Clip Editor, I load the footage I’ve saved on disk. I want to replace the image shown in the painting, and the four corners of that image are perfect for placing Track Points because they’re high-contrast and easy for the tracking algorithm to detect throughout the clip.
I then place 4 Track Points by clicking Add Marker on the left and clicking on the image to define the initial position of each Track Point. I fine-tune their placement using the zoom window in the Track tab on the right-hand panel. Once I’m done, I select all the Track Points by pressing the A key while the mouse is over the image.
I click on Set Scene Frames in the Track Clip tab to set the length of the Blender clip so that it matches the duration of the video clip. Then I click the Track Markers button so Blender will try to track the 4 Track Points throughout the entire clip.
I now open the Solve tab on the left and the Plane Track section within it, at the top. With the four Track Points selected, I add a Plane Track object to the scene by clicking the Create Plane Track button. This object has four vertices that get associated with the four original Track Points. The match isn’t always perfect, but you can manually fix it by selecting and dragging each vertex of the Plane using left-click and mouse drag. In this case, the tracking was so accurate that there are no major mismatches between the Plane’s vertices and the Control Points, but I’ll explain how to fix it anyway in case it happens.
In the Plane Track section of the Track tab on the right, there’s a button called Auto Keyframe. When this option is enabled, whenever we move a Plane vertex at a frame in the timeline, the new position of that vertex is automatically saved. This is very useful when you need to apply manual corrections.
We can now load the image to be overlaid by clicking Open in the Plane Track section. The image can also be semi-transparent, for example if you want to add a logo or text over the original content instead of replacing it completely.
The Opacity section in the Plane Track tab lets you reduce the opacity of the new image. This parameter is not animatable. In any case, what we’re seeing here is just a preview of the effect, because to actually perform the replacement and render the final video, we need to move over to the Compositor.
In a Compositing editor, I replace the default Render Layers node with a Movie Clip node, into which I load the original footage. I then connect the Movie Clip node to a Reroute node. The Reroute node is then connected to both a Composite node and a Viewer node. The Viewer node will allow us to preview the result in the background of the editor, using the Backdrop option enabled at the top right.
In the editor, I also add an Image input node, where I obviously load the image I want to apply over the original footage. This Image node is then connected to a Plane Track Deform node. Connecting this node to the Viewer will give us a preview of the new image, but without any deformations yet, because we haven’t configured the node settings.
First, I load the original footage into the Plane Track Deform node. Then, I set the other two fields in the node: the camera used in the footage and the Plane Track. You can include multiple such Planes in a single project. Note that both the camera and the Plane are objects from the Movie Clip Editor, not 3D objects in the Blender scene.
To overlay the image onto the original footage, I use an Alpha Over node, specifically by connecting the image to the second Image input of that node. Now the preview is clearly visible in the editor’s Backdrop. I then set the output resolution to match the original footage, along with the other output file settings, before starting the animation rendering from the Render menu.
Here is the resulting output video.
If you want to animate the opacity of the image you’re applying, for example to create a fade between the original and the modified clip, you can animate the Factor value of the Alpha Over node. Set this value to 0 at the first frame of the animation and press the I key while the mouse is hovering over the parameter, or right-click and choose Insert Keyframes. Then go to the last frame, change the Factor value to 1, and insert a new keyframe for the parameter.
Here is the final video with the fade effect applied.
Before moving on to another example, here’s a small extra tip. If you click the drop-down menu in the Plane Track tab and select New Image From Plane Marker, Blender will generate a new temporary image based on the area enclosed by the Plane Track, and name it after the Plane Track. You can then edit this image directly in Blender or save it to disk and modify it with an external program. This way, you can edit the image visible in the original clip, load the edited version, and use it to replace the original.
You can also load video clips into the Plane Track and into the Compositor using another Movie node; however, these clips must already have dimensions proportional to those of the Plane Track, so keep that in mind.
Here’s the clip with the animated texture. That’s it for this tutorial! I hope it was helpful! See you next time!