With all the recent interest in 360º VR technology, cameras like the Ricoh Theta S and the new GoPro Fusion, we’re only just beginning to scratch the surface on the latest craze. After Facebook introduced the ability to view 360 panorama images directly in the page feeds, they not only allow you to upload directly from your cameras or iPhones, but also to create spherical panos that you can capture professionally with your drone and some computer magic and bringing old print panos to new life in the process. So let’s take a look at what’s involved:
With all the recent interest in 360º VR technology, cameras like the Ricoh Theta S and the new GoPro Fusion, we’re only just beginning to scratch the surface on the latest craze. After Facebook introduced the ability to view 360 panorama images directly in the page feeds, they not only allow you to upload directly from your cameras or iPhones, but also to create spherical panos that you can capture professionally with your drone and some computer magic and bringing old print panos to new life in the process. So let’s take a look at what’s involved:
Shooting your aerial pano
There are several ways to capture imagery to create your panorama image with a camera drone. You can manually position and trigger the camera to take the shots you need, record 4K video while panning the craft and pull the desired frames out later or you can utilize an app such as DronePan if you’re flying a DJI Phantom Mavic Pro or Inspire.
If you’re shooting manually or selecting frames from a video clip, then make sure you have about 50-70% overlap in the images as you frame them up during your panning. This will ensure you’ll have enough data for the pano stitching software to do its job.
Manual overlap of aerial panorama frames
I will typically do a full rotate with the camera facing straight out at 90º, then do another full rotation down about 60º, then another about 30º and finally one straight down at 0º (nadir). If it’s a clear blue sky day above you, then you may not need to also capture an image straight up at 180º (zenith) but hard to do with a drone so at least capture one with your iPhone and you can edit it in later.
Depending on the complexity of the shot and how wide the FOV of your camera is (like a GoPro for instance), you can also just do three complete pans at 90/45/0.
The DronePan app allows you to set variables in automated pan capture and will automatically detect which craft you’re flying. If using the DJI Inspire, then you can also add a row of +30º “sky” shots that will help fill in the upper portion of your spherical pano.
It’s important to keep your elevation and position locked as close as possible, so your foreground images don’t shift too much from frame to frame. Having a good GPS lock helps and avoid really heavy winds.
As far as exposure goes, I typically will shoot with an ND8-PL filter from PolarPro on a DJI Phantom 3,4 Pro or Inspire 1, depending on the location and time of day, and let the camera auto-adjust the exposures as I pan around.
This allows me to capture crisp details, even if I’m facing the rising or setting sun in one direction. Fortunately, today’s stitching software will compensate the differences around the pano and still provide a smooth completed image without any over/underexposed regions in the finished product. Everything else can be tweaked further in Photoshop/Lightroom.
Stitching your aerial pano image
The term “stitching” goes back a ways, when photographic montages were combined to create one large image for the early days of VR imaging – primarily with Apple’s QTVR and scanned film was used as our source material. Many photographers have been using several tools for years to create panoramas for long prints, but there are more available to us now that are just brilliant for an easier workflow. There are so many free/low-cost options out there as well as more sophisticated professional options. Here’s a list with comparisons with links, pricing and more.
Some more popular are Panoweaver, PTGui and Kolor Autopano/APG (my personal favorite). You can also use Adobe Photoshop, Lightroom and ACR (Adobe Camera Raw) to stitch many styles of panos. My friend and colleague Colin Smith with PhotoshopCAFE has a great video tutorial showing you how to create a pano in Lightroom with the new Boundary Warp function that uses Content-aware Fill to fill in the missing data around the edges.
My friend and colleague Colin Smith with PhotoshopCAFE has a great video tutorial showing you how to create a pano in Lightroom with the new Boundary Warp function that uses Content-aware Fill to fill in the missing data around the edges. Link to it here…
My personal preference is to use Kolor APG to create my panoramas for both print and 360 spherical VRs. The UI takes a bit of a learning curve, but the tools are quite sophisticated and it provides really solid results. Kolor has some documentation and manuals on their website if you’re looking for specifics on using the software. Otherwise, whichever pano stitching software you use should provide basically the same results.
Once you’ve selected your pano frames in APG and made all the adjustments to get the best results for a spherical pano, just render it out as a PNG file to preserve the transparency around the processed image so it will be easier to fix in Photoshop.
Kolor APG stitched pano ready to export
At this point I typically render out my panos very large, because a great deal of my work ends up printed on large format printers or I print them myself on the Epson Stylus R2000 7 color inkjet printer on roll photo stock or canvas. I continue working in this hi-res format until I need to resize for the 360 VR step below.
Cleaning up/correcting your aerial pano image
The most important part of creating a professional panorama – whether for VR or print display, is to clean up any error points in the stitching process, filling in holes in the ground or sky and overall color correction. This is best done with an image editing software such as Adobe Photoshop or Lightroom. Some have had some success with GIMP as well, but I use Photoshop exclusively.
I open the PNG file in Photoshop and load the selection of the transparent space. I often expand the selection by just a few pixels to make sure there’s no ghosting along the edge when I’m done.
I then apply a Content-Aware Fill to the selection which will complete the sky area with a smooth sky line across the top – even if there are clouds present.
Often times, there may be a small stitching alignment errors in your pano that will need to be corrected in Photoshop. You really have to zoom in and examine your pano image to find them at times. Look along the horizon line first as this is typically where the errors may occur and anything in the foreground with strong horizontal or vertical lines/edges.
Sometimes it’s as simple as using the Healing Brush or Patch Tool to fix an errant area or you may have to resort to opening up the matching original frame from your shots and copy/paste a larger patch to manipulate and cover up the problem. It’s a purely subjective process and requires a bit of scaling/positioning and applying Warp Transform to the pasted layer to align perfectly, so take your time.
Next comes color correction and my go-to tool is the Adobe Camera Raw adjustment which allows you to knock down harsh highlights and bring up the shadows if the frames were shot dark. I typically like a warmer image than what the camera captures, so I adjust that and add some Clarity, Vibrance and Saturation, maintaining natural levels so it doesn’t look too processed.
At this point, I will save the hi-res file for printing and the proceed to scaling a copy of the image for 360 VR in the next steps.
Scaling and processing the image for proper 360 VR
The “sweet spot” for most online VR panorama players is 6000 x 3000 (2:1 aspect ratio) for the best results. 4000 x 2000 is acceptable but the image quality may not be as good. Sites like DJI’s SkyPixel will provide a much higher quality VR playback that allow the viewer to zoom in, while Facebook’s 360 VR compresses the snot out of your image (especially noticeable in gradient sky textures) but produce a quick way to distribute your panos to a much larger audience in the social media universe.
A great tool provided by Steven Van De Staak at stfoto.nl is this Photoshop template you can download for Facebook 360 VRs. It has guides and overlays to help you correctly scale, position and build your pano so the horizon and POI are in the correct placement upon viewing on Facebook. It even includes the correct EXIF data so when you save your final JPG image, it will be ready to post on Facebook directly.
Using this template, I am able to continue building my 360 VR pano. Even though it was stitched in Kolor APG with a Sphere model projection, it isn’t a complete 360º sphere yet.
As with some of my earliest tests that didn’t have a complete panel across the top (zenith) region or below (nadir), the pixels get stretched like a ballon to compensate and fill in those gaps, so everything looks like it’s getting sucked up through a perspective point straight up in the sky. Or my horizon was off and the initial view angle was distorted and off-centered.
To avoid this phenomenon, it’s important to fill in and stretch out to those areas in Photoshop and this template file helps us do just that.
First, I just copy/paste my scaled 6000 x 3000 pano I created previously, into this template file in Photoshop, and align the horizon line of my pano to the guide line in the file. Then I use the selection box and drag-select the bottom portion of the image layer to select it and use the Transform function to stretch it downward to close up the gap at the bottom. Take care not to leave ANY WHITE at the bottom or you’ll create a white spot at the bottom of your 360 VR! Same is true if you stretch it to far – you’ll see artifacts at the bottom and will lose image data.
Then, if I haven’t shot/stitched a complete spherical panorama with a top (zenith) image, I’ll need to fill in the top to avoid a big white hole or stretching. First, I drag-select an area about 1/2 of the height of the white space remaining at the top of the image layer with the box selection tool and then choose Content-Aware Fill to fill it in. If your image has a lot of detailed clouds in this sky selection, then you may need to do some cloning and healing to fix the repetition regions before proceeding. Once you’ve filled the region, you’ll want to use a very wide feathered brush to paint across the rest of the top of the image layer with a solid blue color that matches your sky. Take care to feather it and blend it well so there’s no abrupt transition from clouds or lighter areas of sky. It’s important that the very top is a solid color all across to eliminate any anomalies in the final pano.
To verify you have a good 360º pano at the “seams”, we’ll need to shift the pano from side to side to view/repaid any off-aligned pixels or hard edged color shifts – usually in the sky.
You’ll need to use the Offset plugin to do this. Select Filter > Other > Offset from the top pull-down menu.
You’ll be able to enter in values that will shift the entire image layer left or right in your image window. keep adjusting the amounts in either -/+ increments until you either see an edge that needs to be repaired or your POI fits exactly inside the box designated in the template guides.
If we zoom in to the sky in the left 1/3 we can see an anomaly that needs to be painted out in the image above. Simply use a big feathered brush and carefully paint out or clone/heal over areas that need repairs. If you’ve offset your image to repair the edge seams and int he process moved the POI out of the box, simply open up the Offset dialog box again after you’ve made the repairs and shift the image back into position. Sometimes just changing the – to a + (or vice-versa) will do the trick.
Finally, I’ve found that since Facebook does compress your images so much when you upload them, large soft gradients such as blue sky in this image will cause harsh pixelation and banding in the final published 360 VR, so we have to “trick” the compression algorithm a bit by adding some noise to those areas.
Start with selecting the sky area with the Quick Selection tool and then choose Filter > Noise > Add Noise to bring up the dialog box. For an image this size you’ll want to use something around 1-2 pixels and Gaussian, Monochromatic so you don’t introduce patterns or moire into your sky texture. You want just enough grain in there to barely see it but it will be enough to help eliminate over-compression in your final 360 VR on Facebook.
When you’re done editing your pano image, be sure to Save As to a JPG file on your hard drive. If you used Steve’s template file you’re all set and no need to continue on to the next steps – you’re ready to upload directly to Facebook!
Modifying the EXIF data in your VR image file
It’s important to modify the EXIF data in your finished image (or use the template image you can download from stfoto) so that Facebook and other sources like SkyPixel.com can recognize it as a 360º spherical panorama when you upload it. Otherwise it will just show up flat in your feed like this:
Where the intended result should be displayed more like this on Facebook:
My friend and colleague Eric Cheng is currently serving as the Head of Immersive Imaging at Facebook and has provided some great tips for modifying your images in his blog post on Facebook, which will help you even further if you want to explore more about this process to get the proper EXIF data into your final pano image.
Another great resource is from Steven Van De Staak’s site from the Netherlands, stfoto – where he has provided a great tutorial for getting the EXIF data into a pano image for use on Facebook 360 VR. In addition, he has provided a download link from this page for a PSD file that already has the EXIF data embedded into it, as I used in my examples above – plus several guides to help you format your pano for the best results – which is the one that I’ve used for this tutorial project.
If you have your pano already created in a 2:1 format and just wish to quickly and easily add the EXIF data to your file, just go to thexifer.net website and drag your image into the provided space, chose “eXif.me” and input “Ricoh” as the Make and for the Model use “Ricoh Theta S”. GEOtag your image on the map if you wish and then “Go.eXifing” and download the finished file as instructed. I’ve found this to work on all the panos I’ve attempted that aren’t too large in scale.
The “Tiny Planet” – a creative alternative
Once you have your spherical panorama stitched together, there are a couple different methods for creating this fun and creative look at the world below! I created this one by choosing the projection option in Kolor APG and then finishing in Photoshop, but there’s a fairly simple process to create these from a finished wide 360º pano image as well.
Colin Smith of PhotoshopCAFE.com created a quick tutorial on his site that will give you step-by-step instructions to create this effect directly in Photoshop, so check it out to get more info – and have fun! Be creative!
Coming soon in Part 2…
360 Spherical Video and updated advanced shooting/editing techniques – stay tuned!