Photogrammetry and Visual Effects

DISCLOSURE: I have no business affiliation with any of the companies mentioned nor am I being paid by any of them to write this article. I chose to write about this subject matter on my own due to a passion I have for the subject of Photogrammetry. I have done much research over the years on Photogrammetry and it’s great potential use in my Visual Effects work since 2012. The information here is provided as a general explanation of the process, and possible uses of Photogrammetry. I hope that you find it interesting and perhaps informative enough to try it for yourself, and encourage you to do your own research as well if pursuing this process. It is continually changing and becoming more accessible as technology progresses.

• What is photogrammetry?:
What is this photogrammetry I speak of? Well for a more scientific type of explanation of what photogrammetry is, take a peak at the ISPRS site.

Some more resources are here as well:
photogrammetry.com
Wikipedia/Photogrammetry

Usually when I explain what Photogrammetry is, I describe it as the process of converting 2D photographic images into a computer generated 3D model. 2D becomes 3D, basically magic. This magic happens via software, but you do need to feed the software the right kind of information for it to work properly. Like most processes, starting with a bad foundation will ultimately result in an unstable and potentially disastrous final product. While perhaps the roof won’t cave in on you from taking bad photos during the capture process, it can mean you’ve wasted a lot of time and money. Starting over from scratch being the only next solution.

There are also alternative ways to gather similar type of data using 3D scanners like the Artec, even using Microsoft Kinect, as well as a process using lasers called Lidar. I won’t be going into detail with those methods as I have little to no experience with them.

I first got hooked on the process when I initially saw Lee Perry-Smith’s Infinite Realities site around 2012. You might actually recognize Lee without even knowing it. He released a scan of his head that has been used all over the planet thousands of times. If you are a Cinema4D user you can even find the model in the Render Examples folder of the Content Browser.

I was immensely mesmerized by what he was creating. I had already been doing digital photography for over sixteen years at that point as a relaxing hobby and for texture capturing to use in my work. Now while Lee may not the inventor of Photogrammetry itself, he most certainly pioneered the human capturing process. He also continues to make new breakthroughs in that process. I personally find his work very inspiring, and it is what pushed me to find my own way in creating a capture setup for myself with the limited budget I had. Having camera equipment already from all those years of photography didn’t hurt either. I quickly found out the limitations of my equipment and the headaches resulting from that.

Photogrammetry most recently in the last six years or so has been getting more popular in consumer markets and with creatives, especially with the emergence of 3D printers. To most consumers though, the process may not have much appeal, but it’s resulting products usually do. The resulting data, once processed and cleaned up, can be used in 3D printing to create figurines of yourself or family members. It actually branched a whole new market for capturing people in a different way than just simple photos. That then lead to things like wedding cake toppers of the actual bride and groom. My wife and I actually considered this at one point for our wedding, but eventually chose a different route.

Photogrammetry has played a big hand in creating topographic mapping, preserving and studying archaeological finds, as well as architecture.

threedscans.com
photomodeler.com

• What uses are there for Visual Effects?:
How is photogrammetry helpful for VFX? Well there are a number of VFX uses both pre-production and post-production related. Capturing a potential filming location or built set can be very helpful in pre-visualizing (previz) for planning out action or layout of a scene. On the post side it most definitely can be used as an actual 3D background element in the compositing process, very helpful with matchmoving difficult scenes since it is can have relatively accurate scale data. I’ve used them as HDRI sources to light a 3D scene, as well as a way to get an generic 360 chrome ball lighting pass after the fact. Motion graphics can easily benefit from it by using the 3D result as an element in the animation, or a source to emit animation from. The same goes if you capture a human instead of scenery, you can use that data for a digital stunt double, character based emission or interaction for dynamics based effects. It could also be used for crowd replication, and morphing effects of a character into something else.

• How do I capture something?:
There are a number of ways to do photogrammetry captures, it all depends on your subject matter, production needs, and final use of the data. The basic premise is that you are gathering visual information in slices from numerous angles. Imagine a ball being placed on a copy machine.

You can only get one segment of visual information at a time. To get the whole ball, you would have to rotate it a lot of times to copy every side of it. With photogrammetry capturing, the subject is stationary and you move around it snapping a photo for every segment. The more area you cover, the better. You also want to make sure you enough overlap between images. Making sure features on one image are seen in at least two other images will help. Each subject will of course present it’s own issues and force you to change up your process on the fly. Below are some graphics showing some typical subjects and the general approach concepts.

You can see from these graphics that there can be a lot of photos to take. Ideally you would like to capture all the photos at once to make life easy, but that’s just not always possible.

Continued on page 2.