Tracking gestures and screen replacement in After Effects
for iOS and beyond
By Rich Young | January 08, 2014
trackFinger AE is new iOS app (on the App Store) that allows you to record the movements of up to 10 fingers on an iPad, and then send tracking data as keyframes in e-mail text to paste into Adobe After Effects. But wait, there's more!
◊ A few weeks after this post, Chris Meyer published a whole course at Lynda.com that discusses mocha AE and the RotoBrush to solve some of the pieces of the puzzle. Screen Replacement in After Effects introduced the course.
The trackFinger AE app, noted by kita keiju, is designed by some Italian fellows (@trackFinger or Facebook) and should work on any iPad running iOS 6.0 or later. A free update coming in a few weeks should work on the iPhone 3GS or later. Here's the promo and one of the tutorials. They use a camera to record the scene, while hand movement for sprite animation is recorded on the device itself.
Later, another iOS app was available, VFX screens by Codega Studio. There's not much information beyond "Compatible AirPlay with AppleTV for TV screen replacement, Compatible AirPlay with AirServer for PC screen replacement."
Other apps record an iPhone, iPod or iPad screen, like Reflector, which works with Mac OS or Windows 8. The same company makes other similar useful apps as well.
There are similar apps, like Reflection, as shown in iPad Tip: Screen Capture by Ryan Boyle. He shows how to wirelessly mirror an iPad or iPhone screen on a Mac desktop through Reflection, which you can then screen capture (even with QuickTime Player).
Screen replacement and tracking tutorials for After Effects has been its own cottage industry since the early years, but manipulating screen content (the goal of trackFinger AE) has not been a focus of tutorials. In the trackFinger AE tutorial above though, it seems that a different finger could be tracked and offset to match the index finger.
For more on tracking and screen replacement in general, check out these resources (though none involve animating touch screen gestures):
- Tracking and stabilizing motion in After Effects Help covers the gamut well
- [later] Screen Replacement in After Effects, which introduces a course at Lynda.com by Chris and Trish Meyer, discusses mocha AE and Roto Brush to solve this common VFX challenge
- [later] Using Reversible Stabilization in Warp Stabilizer VFX for Screen Replacement was also posted by Chris and Trish Meyer
- After Effects Apprentice Free Video: Tracking with mochaAE by Chris and Trish Meyer introduces mochAE for screen replacement
- Motion Tracking and Green Screen with After Effects & Mocha AE (track, insert and matte - while retaining reflection integrity on shiny surfaces) by Jeff Foster
- mocha AE: iPhone screen replace tutorial by Martin Brenand shows how to track an iPhone and export tracking and mask XML data to finish. (see also Dealing with Foreground Obstructions)
- Learn mocha AE - Online Training Webinar by Mathias Mohl at Mamoworld
- 14. Compositing tricks for screen replacement is a short course from Digital Tutors.
- thevfxbro posted a quickie on replacing a painting, After Effects Tutorial - Screen Replacement (Motion Tracking w/Mocha)
- Adam Everett Miller posted some Free Tracking Clips for Training (Non-Commercial) though Imagineer Systems.
- Mary Poplin posted Tracking Difficult Screen Inserts with mocha AE CC (embedded below)
- Video Tutorial: Cell Phone & Tablet Screen Replacement in After Effects by Evan Abrams (embedded below)
Get articles like this in your inbox: Sign Up