The new Netflix series “Undone,” like “A Scanner Darkly” before it, creates a compelling visual style by rotoscoping live action actors, and painting animation frames over the top. But what if you could create the effect without all that pesky manual labor? After all, this is 2019–aren’t robots supposed to be doing the tedious stuff for us by now? Enter EbSynth.
EbSynth is a cool app developed by some very bright people at the Czech Technical University in Prague and the good folks at Adobe Research. And like so much cool software in the modern era, it’s a free download.
While there’s not much to the process of authoring animation using EbSynth, there also isn’t any documentation on the site. So I’ve compiled a short (5 minute) video taking you through the process of turning your boring video footage into compelling animated art.
Now if you’re a skilled ‘oil and canvas’ artist, you can paint your own keyframes and EbSynth will adapt your style to the rest of the video sequence. However, if stick figures are more your speed (as you can see from the sketch to the right, I fall into that ‘cat’egory), you can use a style transfer site like deepdreamgenerator.com to paint the keyframes for you.
The system works surprisingly well. If subjects move from forward facing to a profile during the shot you’ll need to create additional keyframes and blend between renders (you may want to use a morph tool to make this look seamless). You may also need to mask out actors and track in static backgrounds if there’s a lot of camera movement.
This really does open up a lot of creative possibilities. Since the quality of the shot texture and lighting is derived from the painted keyframes, you can spend a lot less time on set lighting for video you’re planning to convert. There are also some possible tricks that could be performed on 3D animated renders (like rendering out high quality keyframes, then applying those to a quicker, untextured render). I’ve yet to experiment there, so stay tuned.
Ready to dive into a new realm of hybrid animation? Watch the five minute Impossible Shots video below.
