(Page 3 of 3 pages for this article  <  1 2 3)

Filed under: Audio •Editing •Motion Graphics •

Timing Video to Audio

Cut Time

Whenever possible, we like to start our visual work with the music, and look at this to decide where edit and action points in our video and animations will be. At the most simplistic, we will look at the downbeats of the music as our first cuts at where edits should go. If we are building 2D or 3D animations, we look at the finer resolution of the beats inside the measures of music as first choices to place keyframes for times when flying logos hit their mark, text pops on, elements fall into place, etc.

Beats should not be thought of as just points in time: You can think of the length of a beat as a good duration for events that happen over time. For example, the length of one beat is a good time to stretch a transition over. One or two beats, or even a whole measure, are good lengths to fade or zoom up or down elements such as text, and also to stretch special effects such as particle system outbursts or lighting sweeps. Typically, these transitions or effects will start or end on a downbeat; occasionally, straddling the downbeat also works. The main thing is to think of downbeats as your emotional or dynamic pivot points.

Whether your events start or end with the downbeat depends on the psychological interaction you are trying to evoke between the music and the visuals: Is the music motivating the visuals, or are the visuals motivating the music? For example, if there is a build-up in the energy of the resolving in a big cymbal crash, you might start a transition or effect on that crash, and make it last the duration of the crash. This will make it seem that the music caused the visuals to change. On the other hand, if you are approaching a change in the music - such as the start of the guitar solo - you might have the transition, move, or effect end when this new musical section starts. This will make it seem the visuals built up the tension, and caused the music to change in reaction. Either way, the interaction between the music and video will increase their overall impact.

Going back to the micro level, tension games can also be played with exactly when you “hit” the individual beats. As mentioned earlier, sound events happen more quickly and with finer resolution than video or film frames. You might find an audio peak occurs somewhere between two frames. Which side should you fall on? If you hit before the peak, you will increase the tension - it will seem the video is impatiently rushing the audio. If you hit after the peak, it will relax the tension, because it will seem the video is reacting to the audio. In general, don’t be afraid to run lots of test previews, trying out tweaks in your timing to see how the end result feels.

A Case Study

Sometimes, a blank slate can make it hard to get started on a job. Referring to the audio to lay down grid marks on that blank slate can help move things along - particularly when on deadline.

Several years ago, we worked on an animation-intense video opener for the Radius (RIP) Creative Expo tour. The original plans to create the animation fell through, and we were handed the job with not much time until the curtain rose in the first city. The first thing we did was get approval on the music to use. Next, we took it into Adobe Premiere (our main editing tool back then) and spotted out all of the downbeats in the music, as well as the start and end to major sections and any special events such as sound effects or interesting words.

It so happens the tour was going to visit eight cities; we thought it would be good to have a section for each city. Therefore, we looked at our chart of downbeats and musical sections, and tried to break the music down into eight sections plus an intro and ending. To get a feeling whether or not it would work, we created a solid color matte for each section in Premiere, and trimmed them to fit these proposed sectional breaks. We used another color to mark transitions between these sections. An example of this sort of “timing storyboard” is in shown above. In some cases, we used still images representative of the section (such as photo of the Golden Gate Bridge for San Francisco).

Next, we previewed this simple “storyboard” in Premiere, seeing if the sections and the music seemed to work together, or if we should adjust where some of the breaks fell. If it works at this stage, it should be easy to make it work with real animations. If it doesn’t work emotionally at this stage, then you have to work harder to make your animations cover for the shortcomings.

A sample “timing storyboard” in an ancient copy of Premiere where a solid color matte marks the different sections of music for a different project (gray marks the transitions between segments - areas where we will need extra handle in the source modules). We preview this to see if the general feel is going to work.

Once happy with this timing storyboard, we then looked at the duration of each of the sections and transitions, and now knew exactly how long each section should be. We could then divide up the sections between us, each taking a few cities, knowing their duration plus the handles we needed for transitions. We already had our timing charts of beats and events inside the sections, and spotted additional beats as needed inside After Effects where we did our work. Once done, it was a relatively simple matter of pasting each rendered section over its corresponding solid matte in Premiere. 

After we’re done working on the individual modules, we replaced the placeholders in our timing storyboard with the finished sections.

Working this way, our final edit was actually finished first; all we had to do was render a composite movie in Premiere when our sections were in place. The time spent up front was more than saved on the back end, since we didn’t have to try to edit “finished” animations to fit the music, or go back and have to re-animate any of the sections to make them work.

Again, remember that all of these techniques are just suggestions for starting points - don’t let a formula dictate the way you work. If the piece demands something different, break out of the form. However, when audio and video work together, the result is indeed greater than the sum of the parts. It might also save you some time while you work.

The content contained in our books, videos, blogs, and articles for other sites are all copyright Crish Design, except where otherwise attributed.

(Page 3 of 3 pages for this article  <  1 2 3)

Name:

Email:

Location:

URL:

Smileys

Remember my personal information

Notify me of follow-up comments?

Submit the word you see below: