In the first installment, we looked at the basic set up for shooting Greenscreen. Most of what I covered is exactly what we see people skip everyday. What I’m going to talk about next is what everyone else ignores at their peril. We’re talking about capture systems and codecs.
As with color correction, capturing greenscreen plates is more than just resolution. Color depth and, more importantly, color scaling make a big difference. How you capture your precious cargo can make the difference between getting by and getting ahead.
Most people capture to their camera’s internal storage – Generally – P2, SxS, HDV Tape, or SD tape. Getting video onto these tiny cards (or tapes0 requires some compromise. Here’s why…
When a camera’s HD chip captures the world in front of it, the data pipe can be roughly 180 megabytes to over 400 megabytes per second. At this rate, most memory cards, if they could write that fast, would fill up in less than a minute. To get more data onto the card, and to allow it to keep up at all, color scaling and compression is applied. For many video projects, these manipulations are not apparent. Most of what we focus on in an image is the luma detail (black and white information) and the primary subject of the video. Much of this is left intact. BUT as compositors, we need more. When a camera throws away color data, it is throwing away exactly what we need to do our work.
Here’s the filesize differences between popular capture formats. While size isn’t everything… it’s something to consider.
Color Scaling
We’ve all heard the terms… “4:1:1”, “4:2:2”, “4:4:4”, “4:2:0”. For many, this is some course measure of quality. For greenscreen compositors, this is the terrain: From the smooth slopes of 4:4:4 to the rocky coasts of 4:1:1. Color Scaling will set the tone of your composite and ultimately define how far you will go, and is the first “handle” that you will have to get ahold of when determining the quality and file size of what you capture.
In a 4:1:1 frame, you have luma information for every pixel across the image, but color information for only one quarter of the pixels (above). This may work okay for normal footage when it is not as obvious (our eyes are more sensitve to luma than to chroma), but produces a terrible color key (below).
Color Scaling is simple enough. We start with YUV, Y is the Luma, U and V represent the color information. The magic numbers represent the ratio of YUV. So, if every 4 pixels of luma are matched with 2 in the UV… we have 4:2:2.
We generally keep all of the luma information (because it will look soft if we don’t); we then throw away color information. If you’re lucky, the process only throws away half the information but it’s usually more. HDV is 4:2:0, as is XDCAM. This is an overview. Adam Wilt has covered this in great detail in the past in his various camera reviews; you should read his articles on the subject if you want to dig deeper.
Half the color information in 4:2:2 means you are keying an image half the size of the full frame and then mapping that back to your selection. When you see the stair steps down the side of your next key, this is why.
There are 4:4:4 options. Sony F-950s, F-23s, and F-35s along with a few others all shoot 4:4:4. But you pay a dear price this color scale. Most of these camera bodies start at $100k or more. The Red is one exception but it reaches 4:4:4 at the cost of definition… we’ll talk about that later.
Color Depth
But it’s worse than all of this… 4:2:2 is the least of your problems. In fact, a large amount of our keying is done on 4:2:2 footage. The difference in our case is that the footage is uncompressed. Uncompressed?? You thought 4:2:2 was compression… no, that’s just scaling the color channels. You actually have four handles that control the quality and file size. These handles are Color Scaling, Color Depth, Temporal Compression, and Spatial Compression. We just covered Color Scaling, where the camera or system actually scales the color information (throwing away up to 75% of it) to save space. This is generally not something you can choose to do after you choose what type of camera or capture system that you are using.
Color Depth is the number of bits used to describe each pixel. Typically this is 8 bit or 10 bit in capture formats. 8 bits provide 256 levels in each channel. 10 bits provides 1024 levels in each channel. Why does this matter? If the footage is well shot (see Part 1 of this series), it doesn’t really matter. BUT, if the footage has a large variance of lighting across the screen, more bit depth gives you more data to work with.
Compression
The last two handles are temporal and spatial compression. Temporal compression looks at successive frames and tries to intelligently throws away data used in all the frames. This can create relatively small file sizes but it also can create files that are difficult to edit because of the lack of keyframes in important areas. Spatial compression looks at the colors within a single frame and tries to clump similar colors together as discretely as possibly. Unfortunately, this can often leads to square edges on subjects as the macro blocks (the clumped areas) are designed not to noticeable in uncorrected images. When we change the color levels, these perceptual cheats show up. We change the levels a lot in keying.
Whew… why do we need to know all of this? Because it factors into all of our shooting calculations. As we all know… life, and production, is imperfect. We really get to operate in a pristine environment. We need to make compromises. Understanding all of the factors allows us to compromise effectively. For instance…
next page: our setups
What We Use Where
In our studio, we have a Sony F-950 and a capture RAID. When we are able, we will shoot 4:4:4 Uncompressed, 10-bit video… from which we will quickly and easily pull keys that include every hair of our subject. But it’s rarely this simple…
1) Our big camera goes into the shop. We now need to shoot on our little EX-1s. We can now do no better than 4:2:2. BUT we’ll use its Uncompressed HD-SDI output to capture uncompressed 10-bit video. We’ll need to soften the alpha channel a little to make it work but if we’re delivering 960×540 (web)… no one will notice. Solved!
2) We’re shooting on the road. We can’t tote our capture system to every location. We’ll use an AJA IO to capture Apple ProRes HQ to an external drive via a laptop. We’re throwing away 75% of the data but it’s better than XDCAM (the EX-1’s native format).
3) Don’t have an AJA IO or a laptop available… just the EX-1. We’ll be very careful about our lighting and make sure the subject isn’t wearing black (because the compression’s sharpening processes will cause ringing). I won’t have the subject move alot because the temporal compression will cause the edges to break up. It’s won’t be perfect but it will still be better than 90% of the greenscreen keys on VH1 (if someone from VH1 is reading this… call me… really. I will work with you for free… not doing for you, I’m doing it for me. Soft keys hurt my eyes and make me sad).
4) We’re in Africa with only an HV20. It’s native HDV format is, but… we can get a Blackmagic Intensity Card and a local MacPro (sounds crazy but it’s happened). I know the HDMI port is uncompressed 4:2:2. We’ll capture this stream to the hard drive and get a broadcast-quality key from a $500 camera.
The HDMI output: 4:2:2 compressed (this image has been scaled down 70%). Not 4:4:4 perfect, but good enough for an acceptable key.
By understanding all the variables, we can minimize the damage that each compromise presents. This allows us to get the maximum quality in as many situations as possible.
If we have to rely on HV20’s internal HDV format rather than external HDMI, the result is considerably worse.
next page: more detail on our capture systems
I’ve alluded to a few of our capture systems. Here’s the breakdown…
Studio
Camera – F-950, Red, or EX-1 – For our internal shows, we generally use the F-950 or, if we know we will really not being going above 960×540, the EX-1 (we’re 4:4:4 snobs). We have some clients that prefer the Red. It’s also good when we may be shooting a lot of footage. The one big no-no with the Red is shooting Bluescreen: Its compression is very savage to the blue record. In reality, we really could use the EX-1 for almost everything (using the HD-SDI out) but… we have a 950 so… why not use it?
Capture Tool – Final Cut Pro (Red is direct to drive). Final Cut isn’t perfect but it’s serviceable and it’s generally stable (apart from an odd frame dropping issue when the cursor floats over the capture window during 4:4:4 capture – easy enough to avoid). We haven’t really found that we trust any of the other options. We do use Scopebox or Conduit to check the scopes and Conduit to do live key tests (this saves a great deal of upset later).
Capture Interface – Kona 3 Card via either Dual HD-SDI (F-950) or Single HD-SDI (EX-1). Red is direct to Red Drive. We have used both Blackmagic and AJA Kona cards over the last 5 years. We love the Blackmagic products in many areas… but not direct capture. This is for 2 reasons… 1) Blackmagic doesn’t provide an equivalent to the AJA Control Panel. When something isn’t showing up, we can’t see a live update of inputs. 2) Blackmagic, at least when we were testing it, didn’t down convert a live stream. We use this often.
Capture Medium – Hard Drive (uncompressed) (Red Drive or MacPro Internal 3-disk RAID which yields 300 MB/sec with off-the-shelf hard drives). Our standard studio MacPro includes a 3-drive RAID 0. If yours doesn’t, you should consider it. It’s like having a Honda Accord with a Ferrari engine. We call these “Scary RAIDs” for good reason… you could lose everything if they go bad. This doesn’t usually affect us because we have 2-3 copies of everything important. We’re crazy that way.
Situations are often “imperfect” which is exactly when you want to have uncompressed footage. Adding compression the poor lighting already present is a recipe for disaster (actually… more accurately… it was a recipe for disaster).
Remote
Camera – Red or EX-1 (Red is same as Studio) – The Red is a great remote camera for high-quality 4:4:4 capture with small data footprints. The one cost is its weight. Throwing the Red, and often teleprompter gear, on a small tripod (if your head cost less than $5000 – it’s “small”) will often cause unexpected tilts and will, over time, strip the brakes on the head. Hence the use of EX-1s in some cases. And no, we don’t carry expensive O’connor 2060 heads on a plane…that’s what rental companies are for.
Shooting with the RED. You’re probably wondering how we keyed the little car area. Just remember 80% of all composited plates include some Roto…
Capture Tool – Final Cut Pro – See Above.
Capture Interface – AJA IO HD (We’re looking forward to testing the IO Express). The AJA IO HD box is a Swiss army knife of video throughput. We use it to conform signals, capture to MacPros, Capture to Laptops, and even tie together teleprompters. It’s awesome.
Capture Medium – External Hard drive (Apple ProRes). One the road, capturing to Apple ProRes is not as good as full Uncompressed Video but it’s pretty close to even with 4:2:2 Uncompressed and close enough for most uses.
Remote with HV20 – We use Blackmagic Intensity Card with local MacPro via Final Cut Pro. The Blackmagic Intensity Card is the best deal around for capturing high quality footage from consumer cameras (many of which do not compress the HDMI signal). You really can get a broadcast quality signal from an HV20 and an Intensity Card.
So there’s an overview of the codecs and systems we use to capture greenscreen. In Part 3, we’re talk about the actual techniques of capture. If you have comments or questions… fire away!