Does it drive you crazy to see somebody use something the wrong way, then declare it doesn’t work? That’s how I feel about the Roto Brush tool introduced in After Effects CS5. This semi-automated tool helps you separate the foreground (i.e. an actor) from a complex background (i.e. not greenscreen) – “all” you have to do is make a couple quick brush strokes defining where those areas are. Well, not exactly. But when you follow the correct process, it can work rather well, and save you a lot of time in the process.
To use Roto Brush effectively, you need to follow a several step process, including:
- choosing a good “base frame” as a starting point
- defining the foreground and background on this base frame with as much precision as possible
- adjusting the propagation settings that help that semi-automated part
- making corrective strokes over subsequent frames away from the base frame
- refining the matte using a variety of additional parameters
Even then, not every shot is a good candidate for the Roto Brush – for example, you need some contrast in value or color for it to be able to distinguish the foreground from the background. But when you do follow these steps on an appropriate shot, Roto Brush often requires a lot less work to create a good matte than the traditional paint or mask methods.
The movie below details the middle step in this process, which many miss: refining the Propagation Settings so Roto Brush knows how to take the information you defined on the base frame and best use it on subsequent frames:
This video is from our updated video course After Effects Apprentice 14: Paint, Roto, and Puppet. In it (as well as the companion book After Effects Apprentice 3rd Edition) we walk you through two Roto Brush exercises – one easy, one challenging.
Fellow PVC author and After Effects guru Mark Christiansen also has a pair of videos about Roto Brush which are available on lynda.com for free, covering steps 2 and 5 of the above recommended workflow.