As I was making my way around the show floor of NAB 2009 my friend Shane Ross said that I had to check out this tiny little booth over at the plug-in pavilion and demo this product called PluralEyes from a little company called Singular Software. My first thought was … “PluralEyes, that’s an odd name.” Seconds later I headed straight to the booth. Automatic synchronization without timecode. Is this for real? It is.
NAB’s plug-in pavilion is a large booth sub-divided into smaller spaces where some of the smaller creators of software-based plug-ins can exhibit without the high cost of renting larger NAB floor space all by themselves. The plug-in pavilion is where vendors like Red Giant Software, Automatic Duck and CoreMelt (to name a few that I visited) all setup shop. There was also Bruce Sharpe and his company Singular Software who had a tiny space facing the convention center wall, a small vinyl banner, a couple of computers and a killer product.
As I watched the demo of PluralEyes in action I immediately had flashbacks to multicam concert edits of the past and how this little application would have made life much easier. Syncing multiple cameras isn’t really that difficult after you get the hang of it and the occasional 4, 6 or 8 camera event can be ready to edit without too much difficulty in a minimal amount of time by an experienced “syncer.” But then I thought about a 26 camera behemoth that once took us about as long to sync as it did to edit. Oh how PluralEyes would have been helpful there. That show was cut on Avid but my mind immediately started devising ways of how using PluralEyes to get timecode numbers could have at least got us in the ballpark and saved a lot of time. That is until the day when PluralEyes might be available for Avid Media Composer. After that I thought of an editor named Neal Usatin and how if he had had a copy of PluralEyes he could have quickly set up his massive edit and thought: Awesome: PluralEyes helped me f******g sync that.
But then the NEXT thing that popped into my mind was how this little application could help in the syncing of music videos:
It’s not uncommon in a music video edit to have a lot of takes to sync as in the above 38 layer timeline
Bruce sent me a copy of the software to try out and I thought syncing music videos would be a great test. A video is a different type of sync-beast than a multicamera concert. Instead of 2, 4, 6, 8, 22, 24, 26, or 28 different cameras to sync you often have 52, 54, 56, 58 + angles to sync. I say + as that number can move to the triple digits. A beast indeed. Unlike a concert where all of the cameras have essentially the same audio, often at just a different proximity to the camera, a music video can have audio anomalies (to say the least) as the camera is moved closer to the singer or the drummer for instance. They will usually be performing lip sync-style to the onset playback but that camera’s mic will often be recording the singing of the singer or the banging of the drums as well as the playback. That could conceivably cause some problems for anything that is dependent on the captured audio. More on that in a bit.
The idea behind PluralEyes is that it “listens” to the audio that has been captured with the video and then lines all of the video clips up, in sync, in a new Final Cut Pro timeline. Just like an assistant editor would do (if you have one)! If it can’t figure out where a clip should fall then it places those clips into an unsynced timeline:
With that unsynced timeline the editor has to resort to old old-school eyeball syncing. It does all of this in what is not at all an unreasonable amount of time. All the editor has to do is place all of the clips to by synced (both the audio and video) onto their own tracks in a single FCP timeline titled pluraleyes (as in the 38 layer timeline above), launch the PluralEyes application, choose your settings and then wait for the task to complete.
PluralEyes, an ideal companion for music video
As my use of PluralEyes has thus far been limited to music videos then that’s what I’ll talk about here. The first couple of attempts using version 1 of the software, to be quite honest, didn’t work very well. If you think about the audio that is happening on a music video shoot it can be quite a cacophony of sound. There’s the official playback of the band’s track that is being lip synced to by the band and then there’s the sound that the band is producing (singing, guitars and especially drums) as they lip sync along with the song. Then take into account the commands that might be shouted from the director as well as the volume of the sounds as a handheld or camera dolly moves closer to and further away from the band and you can see where it might confuse PluralEyes.
But one of the advantages of working with software from a small developer like Singular Software is they are often very responsive to questions, suggestions and complaints from the user. As I was trying to make PluralEyes work with my music video edits I was in contact with Bruce Sharpe, PluralEye’s creator. Describing what I was tying to sync, Bruce was confident that it should work. I sent him several examples of the material I was working with and in return PluralEyes received some minor updates and added a few new features. These features proved to be the missing link in working with these extreme cases.
One of the first updates was the Level audio command which is very important to music video work since the camera can often vary at the distance from which it is to the audio source. Use markers is an option that might at first seem to defeat the purpose of an auto-syncing program because if you are going to take time to place markers then why not go ahead and sync by those markers? You don’t have to be as exact when placing the PluralEyes markers as you do when manually syncing so they can be added quickly. Finally there is the nicely named “Try really hard.” With this option checked PluralEyes does some extra processing in an attempt to better sync problem clips. This “Try really hard” option was the magic fairy dust that made PluralEyes sync the more problematic music videos. They claim an extra 5-10 times as long when processing with “Try really hard.” If I have the time and there is any hint that the audio is problematic I will use this option. If there’s a ton of takes, as if often the case on a music video, I let it process over lunch or at night. It works well and is worth the extra time if I have any doubt. Clips are in chronological order doesn’t have as much use in music video editing as you don’t usually have camera starts and stops as you do with long concert recording.
PluralEyes will also make a multiclip of your takes as well. This saves yet another step in the music video edit process. If you don’t use multiclips then it’s worth a try as the old fashioned multilayer clip stack can get unwieldy very fast when you have 20 – 30 + takes.
Room for improvement?
How can PluralEyes be improved? It’s a little strange that you have to name the sequence to be synced “pluraleyes.” It would be nice if PluralEyes gave you the option to specify which sequence to sync when you hit the Sync button. But that’s just a minor thing. It would also be nice if you didn’t have to stack the clips on different video layers. While that’s not a difficult task it can take quite some time with 20 – 30 + takes of a music video. If you could select all the takes and drag them into a timeline at once, which would put them in the order from your bin on track V1, and have PluralEyes do the stacking of the clips as part of its operation that would be a nice timesaver as well.
The other downside of an automagic tool like PluralEyes is it might be an excuse to become lazy with shooting and not plan properly. It can almost eliminate the need for a smart-slate and the traditional audio master slating on a music video shoot. As long as you have quality playback and are sure you are recording scratch audio then you can sync with PluralEyes. That is provided you are using Final Cut Pro which is the only support app at this writing though Sony Vegas support has been announced. As I said before I hope Avid Media Composer isn’t far behind as there’s a lot of music videos in the world cut on Avid. If you could send a line out of the audio playback directly to the line in of your camera then you could eliminate the problem audio as discussed above and really make syncing easy. Shooting film? Just use that smart-slate to sync during telecine and you can use PluralEyes with film as well.
As for the future of PluralEyes … as I said, I hope that it does come to Avid Media Composer. Given Apple’s history of buying cool post-production tools it wouldn’t surprise me at all if it was purchased and folded into Final Cut Studio. I really hope this doesn’t happen as that would deprive editors using other apps from working with such a useful tool.
I’ll close with a couple of illustrations of how well this little app automagically works. I was asked to edit a music video but at the last moment I didn’t have time. I wanted to help the director meet his deadline so I ran the takes through PluralEyes as fast as I could. When it finished performing it’s operation I noticed it had created several sequences. I didn’t have time to examine the results so I passed the hard drive over to the director. I learned later there were two different versions of the song that was performed. PluralEyes recognized this and created different sequences for each version.
On another recent video the production had used a proper smart-slate for syncing but out of curiosity I wondered how close PluralEyes would get the sync to the smart-slate. I ran the clips through PluralEyes and it placed all of the clips within nearly a single frame of the smart-slate if not right on. Considering how it does its operation I thought that was pretty damn good. Automagic indeed.