This is the fourth installment of six-part HDR “survival guide.” Over the course of this series, I hope to impart enough wisdom to help you navigate your first HDR project successfully. Each day I’ll talk about a different aspect of HDR, leaving the highly technical stuff for the end. You can find part 3 here.
Thanks much to Canon USA, who responded to my questions about shooting HDR by sponsoring this series.
SERIES TABLE OF CONTENTS
1. What is HDR?
2. On Set with HDR
3. Monitor Considerations
4. Artistic Considerations < You are here
5. The Technical Side of HDR
6. How the Audience Will See Your Work
HIGH CONTRAST IS HDR’S KEY STRENGTH
Specific code values in HDR data are mapped to specific nit/brightness values on the monitor. Choose a 10-bit code value at random, and that code value will cause an HDR monitor to emit a specific, repeatable amount of light. (This is true of both Dolby Vision and HDR10, but not technically true of HLG, which is an HDR format meant primarily for over-the-air broadcasting. It’s extremely unlikely that you’ll ever monitor in HLG on set. These formats are covered in Part 6.)
Without grading, an object exposed at five stops (reflected) above middle gray will generate a brightness level that corresponds to five stops above middle gray on a consumer HDR TV (if the consumer TV is capable of emitting 800+ nits). The dynamic range captured during photography will map very closely to the dynamic range displayed on the end-viewers TV: if two objects differ in brightness by one stop on set, they will also differ in brightness by one stop on a consumer HDR television.
Also, whereas we’re used to seeing reduced color saturation in SDR as exposure increases, this is less likely to happen in HDR.
For this reason it’s important to understand your camera’s dynamic range and color space, from brightest highlight to darkest black and most saturated hue to least saturated hue. There is no hiding at the extremes of exposure. We’re no longer exposing for middle gray and letting the rest “roll off.” Every stop of dynamic range counts. Highlights should always be protected, and shadows lit to the point that some detail is visible.
SHOOT FOR DELIVERY, OR SHOOT FOR GRADING?
There are two theories about shooting for HDR:
- Don’t shoot for delivery; capture the most data possible for post manipulation.
This method dictates capturing imagery with less dynamic range than might be optimal for delivery, so the colorist has more room to push bits around in post. Rather than play highlights or shadows on the edge, you’d light and expose a little flatter so there’s plenty to manipulate later. The overall “shape” of the look still has to be created on set, so “layers” within the image (foreground, mid-ground, background) should be separated in color, contrast or brightness such that the colorist can enhance their separation, rather than try to create separation from scratch.
- Shoot for delivery; make the image appear exactly the way you want.
This is a little trickier. It helps to know what peak brightness level to shoot for (4,000 nits, 1,000 nits, 400 nits, etc.) in order to properly monitor the image. This can be difficult, as there are a limited number of monitors on the market that are set ready, affordable and can display brightness at higher nit levels.
The answer seems to be to shoot with a “fat digital negative.” (In film terms, this refers to overexposing film slightly, robbing from the highlights in order to make the shadow density deeper and darker, which results in reduced graininess.) In general, it is a good idea to rate the camera slower in order to crush noise, while recording to an RGB log or raw codec at the highest bit depth possible. (Recording codecs will be covered in Part 5, “The Technical Side of Shooting HDR.”)
Regardless of the technique chosen, one has to be aware that the colorist has more control than ever before while also being severely constrained. There is very little shoulder or toe to the exposure curve that can hide clipped highlights or conceal noise in shadows. HDR’s constant contrast means that there’s little roll-off, or compression, at either end of the exposure curve from which more information can be pulled, or uncompressed. A 14-stop image on a six stop SDR monitor allows for a lot of leeway, but a 14-stop image on a 14-stop monitor offers considerably less.
At the same time, HDR isn’t HDR without taking full advantage of its capabilities. Director of photography Jimmy Matlosz, who has photographed several HDR test projects for Dolby® Laboratories, told me he prefers to use the entire dynamic range of the camera when possible. “I try to make sure that every f/stop of dynamic range is represented.”
This won’t work for all HDR material—I doubt showrunners for the typical sitcom would appreciate this approach—but, in general, this technique should produce the most stunning images.
I anticipate the greatest challenge will be convincing our bosses that darkness is our friend. More often than not my clients speak of dark areas as if they are holes in the image, as to them black symbolizes a lack of information. “If it’s dark, something is missing. I want to see it!”
Rather than talking about black as a gap in the image, I try to talk about it as a color. Dark areas aren’t missing detail, they are accents—just like red, or yellow, or green. “I’m going to jazz this up a bit by adding some black.” It’s exactly like painting, where black is just another artistic choice on my palette.
HIGHLIGHTS AND SHADOWS TAKE ON NEW IMPORTANCE
Large highlight areas may have an adverse effect on viewers. At the very least they may cause the rest of the shot to appear considerably darker than a light meter might indicate due to the principle of simultaneous contrast (where bright objects make adjacent dark objects appear darker, and vice versa). An on-set monitor helps in this situation, as it concentrates our attention on the image in a manner similar to how the audience will see it. Simultaneous contrast works differently between looking at a set and viewing a small, bright image surrounded by black in a dark room.
It’s important to think critically about what objects or surfaces in the frame will be brighter than two stops above middle gray (or brighter than diffuse white) as that’s the realm of highlights in HDR. They should almost never be clipped as their slightest detail—or lack thereof—will be evident to the consumer. (Tiny areas of clipping may be okay.)
There is so much range for correction in HDR that underexposing overall may be preferable to clipping highlights as long as shadow detail isn’t completely lost.
Solid HDR blacks can be disconcerting and, in many cases, are less pleasing than seeing some detail in the darkest shadows. Back in the days of film it wasn’t unusual to “light for black”: rather than let shadows fall off into darkness, a DP might set a small light to impart just a hint of illumination. This gave film emulsion “something to do” and resulted in richer and more interesting shadows. HDR is similar: often it is more pleasing to see some small amount of detail in the shadows than to see nothing at all.
Dolby’s Dolby Vision Source Format document (a portion of which is reproduced in the appendix) notes that an ASC cinematographer found that plus or minus four stops from middle gray is the “sweet spot.” A subject can walk from sunlight into shadow and no stop change is necessary if the dynamic range of the scene stays within that eight stop range. At plus or minus six stops one can see easily into the toe or shoulder but highlight and shadow detail will start to roll off, depending on the final grade.
I spoke with Bill Villarreal, Dolby’s Senior Director of Content Development, who specializes in remastering feature films for HDR. He tells me that most interior scenes don’t exhibit so much contrast that they automatically fall into HDR territory. Often the brightest thing in the frame is a wall sconce, practical lamp or window. His team spends a lot of time reducing the intensity of interior highlights, as they tend to be more distracting in HDR than when projected from film.
Daytime work is more dramatic: glints on chrome bumpers and tree leaves tend to “pop” and make the image feel sharper, as small points of bright light stand out well against dark backgrounds without overwhelming them. Light streaming through windows and backlit dust in air look amazing.
His biggest pet peeve is that they spend a lot of time removing movie lights and other equipment from windows that appear overexposed in SDR but are perfectly exposed in HDR.
He echoed a sentiment communicated by several other colorists: HDR shadows can be very disturbing if they are completely without detail. His feeling is that we don’t see deep, inky blacks in daily life, and they can feel wrong in a medium that reproduces (and mimics) the dynamic range of human vision.
I view this as an artistic choice: yes, most scenes work better if there’s some detail in the darkest shadows, as HDR reproduces those with amazing depth and clarity… but this is not a hard and fast rule, and can be broken for dramatic effect. An on-set monitor can be helpful in placing detail just at the very edge of blackness, or judging the psychological effect of featureless black.
Mr. Villarreal mentioned a grading session with a famous director who looked at an actor’s closeup and commented that he’d prefer to see softer lighting in HDR, as tonal transitions appear sharper due to higher contrast. In particular, he felt lighting on faces should be much softer. He commented that the specular highlight on an actress’s forehead looked fine in SDR but was distractingly bright and contrasty in HDR, and would have looked better had it been more diffuse and a bit dimmer.
At the same time, it’s not unusual for HDR colorists to “stretch,” or add contrast, to flesh tones—making highlights a little brighter and shadows a little darker—as this makes faces appear richer.
TAKE ADVANTAGE OF TEMPORAL STORYTELLING TECHNIQUES
HDR is not simply about monitors that are brighter and darker than ever before. New technologies naturally create new storytelling opportunities for creative minds. Dolby® colorist Shane Mario Ruggieri shared his thoughts on what he calls “temporal HDR,” which is the use of HDR to create complex emotional responses in an audience over time through changes in contrast and brightness.
He posits four types of temporal HDR:
INTRA-FRAME. This is what we think of as “traditional” HDR, where a camera’s full dynamic range is accurately reproduced on a consumer television. Highlights are brighter than ever before, and shadows are deeper. Imagery takes on an almost three-dimensional feel.
INTER-FRAME. This type of HDR takes advantage of the human visual system’s ability to adapt to large changes in brightness over time. Imagine a scene in a western where the camera follows a character from a bright street, lit by noon-day sun, into a dark saloon illuminated only by indirect light filtering through dusty windows. Normally we’d hide a change in f/stop during that transition, as SDR televisions don’t have enough contrast to produce a usable image otherwise. HDR, however, does allow for this, and it’s possible to expose the image such that no stop pull is necessary and the audience adapts naturally to the brightness change. (Use of an on-set monitor might be wise to ensure that critical action takes place after the visual adaptation is complete.)
INTRA-SCENE. Dramatically different brightness levels in adjacent shots communicate emotion over time. The transitions can take place slowly, where some element of the scene (overall exposure, highlights alone, shadows alone, or overall contrast) changes incrementally across cuts, for a subtle effect, or quickly, through jarring smash cuts. Once again, the audience adapts naturally to the changes in brightness, and it’s up to the creative team to determine the forcefulness of the transition.
INTER-SCENE. Brightness levels change across scenes. Often color is used to communicate location cues to the audience, but now it’s possible to use extreme changes in brightness as well. For example, scenes set in a major metropolis might be underexposed to emotionally communicate the shady environment that exists between tall buildings, whereas a scene set in a scorching hot desert might be consistently overexposed by one, two or three stops—which is possible in HDR without losing highlight detail that might otherwise be crushed in Rec 709.
VARIABLES: Through all of these techniques, it’s important to recognize that our old friend and exposure aid, middle (18%) gray, will be of limited usefulness, as middle gray will shift with the audience’s adaptation to brightness changes. It may be possible for a cinematographer to “chase” middle gray with a light meter, but they’ll likely need to find their new middle gray value by sitting in front of an HDR monitor long enough that their vision adapts to the brighter or darker image in the same manner as the intended audience. The cinematographer can then visually identify a new middle gray value in the scene and adjust their light meter accordingly.
While HDR content can be exposed solely by light meter, an on-set monitor is the best way to evaluate whether such temporal changes produce the desired emotional effect, and to help ensure that critical action occurs only after the audience has time to adapt to large overall changes in brightness.
THINGS TO REMEMBER
- HDR’s strength is high dynamic range and extended color saturation, but this strength reveals weaknesses in optics, filtration, and how those interact with a camera’s sensor design. Test in advance to check that your particular combination of these will work to your satisfaction.
- Shooting for the grade—by capturing lower contrast images that don’t push the extremes of exposure—gives a colorist more control in post. Shooting for delivery may limit the colorist’s choices at the extremes of dynamic range (mostly in the deepest darkest shadows just above the noise floor). Middle gray plus/minus four stops is the “sweet spot” of exposure, although HDR can be pushed much further.
- Clipped highlights should be avoided. Underexposure may be preferable to clipped highlights.
- Completely black shadows can be disturbing as we never see them in nature. Shadows might need additional illumination to bring out textures near black.
- Don’t hide mistakes behind overexposure or underexposure. This works for SDR and film, but does not work in HDR.
- HDR’s higher contrast makes highlight/shadow transitions look harder than they do in SDR. Softer light may be preferable in HDR, particularly on faces.
- HDR can be exposed solely by meter, but it helps to have a monitor nearby to see the image the way consumers will in order to better judge physiological and emotional responses to the picture.
- HDR within the frame is the most basic implementation, but HDR can also be used to impart emotion through changes in contrast, brightness, darkness and saturation across frames, shots and scenes. It helps to have a monitor on set to assess how these transitions play against each other.
SERIES TABLE OF CONTENTS
1. What is HDR?
2. On Set with HDR
3. Monitor Considerations
4. Artistic Considerations < You are here
5. The Technical Side of HDR < Next in series
6. How the Audience Will See Your Work
The author wishes to thank the following for their assistance in the creation of this article.
Canon USA
David Hoon Doko
Larry Thorpe
Dolby Laboratories
Bill Villarreal
Shane Mario Ruggieri
Cinematographers
Jimmy Matlosz
Bill Bennett, ASC
Disclosure: I was paid by Canon USA to research and write this article.