Virtual Reality offers a unique way to experience narratives, but it’s also a tool that can help filmmakers create classic “flat screen” films. Paired with a game engine, VR offers a tool for indies and big studios.

From Plato’s Allegory of the Cave to Star Trek’s Holodeck or the film The Matrix, Virtual Reality has been around us for a while, even if the term itself only appears in 1987, coined by Jaron Lanier. A founder of the field of VR, Lanier was also a pioneer, and together with Thomas G. Zimmerman founded VPL Research, the first company to sell VR equipment for personal use.

We’ve not yet reached the stage that Lanier imagined for VR in the 1980s, with the DataGlove, EyePhone, and DataSuit – terms you can Google after, to know more – but these developments paved the way for early prototypes of systems like the original Oculus Rift and HTC Vive. Lanier said, in an interview to the New Scientist, in 2013, that “the Oculus Rift folks remind me of the kids we were 30 years ago, and I am rooting for them to succeed. It would be awfully nice to see this latest wave of VR make a lot of people happy.”

Well, Lanier’s wishes are turning reality, and the numbers show that VR audiences are growing, and the medium is, as he wished, being used for more than entertainment. During the interview mentioned above, Lanier said that “as a practical technology, VR is already widely successful and adopted. My favourite applications are in medicine, particularly in surgical training and in augmenting surgery itself. For the last couple of decades every new vehicle, whether it rolls, floats or flies, has been designed in part in VR. Scientific visualisation apps are also extensively used in chemistry, neuroscience and other fields.”

Walk in VR like you are on a real set

Furthermore, VR is being used in entertainment, but not just to create games or titles that audiences can explore, like some of the storytelling experiences ProVideo Coalition has mentioned in recent months. Films like The Lion King have set a milestone and revealed how VR has become a key part of Hollywood productions, while many filmmakers have adopted VR as a platform to share their narratives.

Rob Legato, who reteamed with director Jon Favreau to bring the characters of the Disney classic back to the screen, said this about the process: “the VR component we introduced after The Jungle Book allows you to walk in like you would on a real set, put the camera where you want, and the physicality of it helps you to psychologically root yourself. You start doing very natural camera work because you’re in an environment that you’re familiar with. You know where the light goes, you know where the camera goes, you know where the actors are – and all of a sudden on day one you’re off to the races.”

He then added that “when we’re in VR, which is different from what we did on The Jungle Book, the VR experience gives you the visceral feeling of being there, of being up high or at a faraway distance, all those various things, you feel it. So Jon, Caleb, James Chinlund, Andy Jones and I could all walk in together as a group of filmmakers, a group of collaborators, and see things for the first time, look at things from different angles.”

From The Lion King to The Mandalorian

“Caleb might have a better idea to shoot something – he continues – to take advantage of a sunset that’s happening, and again it’s like music, where you’re bouncing off of each other. And when you’re done it’s a very successful location scout that otherwise you could never have done together in a CG world. But in the real world, or in our case a virtual world, we all can share in it and we all can respond to what we’re looking at – at the same time.”

The knowledge acquired through the experiences of The Jungle Book and The Lion King allowed MPC to refine constantly the tools used for Virtual Production and now, elaborates Favreau, “MPC has a suite of tools that are available to any filmmaker based on the innovations that we made on The Lion King.” Favreau continued to a game engine for a better workflow in film production and The Mandalorian is another example.

While on The Lion King and The Jungle Book Jon Favreau used the Unity game engine, for this new Disney title the team went with another solution, Epic’s Unreal Engine. One of the reasons is the Virtual Camera Plugin, which enables a user to drive a Cine Camera in Unreal Engine 4 (UE4) using an iPad Pro in a virtual production environment.

Game engines offer filmmakers the ability to replace classic previsualization methods with an immersive experience that cuts production time, but they also give independent creators and small teams options that until not long ago were only available to big studios. That explains why VR is becoming popular in film festivals around the world. Together or used on their own, these technologies open new options for creators.

VR became a key part of film festivals

Virtual Reality, which at first was only associated with “experiences” and video games, appeared as a key component of film festivals, from Sundance  to Tribeca or SXSW, which showed everything from VR films, interactive or not, covering diverse areas, from documentary or animation to pure creative exercises. The second step of that experience was triggered by the Covid-19 pandemic that forced many events to move online. Although the change created some chaos, it has also revealed the potential to reach international audiences… using Virtual Reality.

In fact, throughout 2020, as many events moved online, organizers discovered a solution to bolster audiences and attract attention of filmgoers around the world: that explains why the already existing Virtual Reality events associated with festivals gained a new dimension, offering their content to anyone with a VR headset. The Cannes XR Virtual had its first edition this year, and events like the Venice VR Expanded or the LFF Expanded demonstrate the potential for this new way to share narratives from different filmmakers and artists.

No one knows if and when the world returns to the “normal” we knew before the pandemic, but whatever the future brings us, the couple of years, and in some cases the recent months, have moved technology forward more than many imagined. As Aaron Linne, Microsoft Teams Senior Product Manager, said recently, “we did two years or more of digital evolution in about two weeks or two months and that’s going to continue”, while talking about the growing use of communication tools.

Watching movies at home in VR

Film festivals will return in their previous form, but it’s highly probable that the virtual side of them will continue and expand. Industry events like SIGGRAPH 2021, for example, are planned to go back to Los Angeles, but the organizers already confirmed that the aim is to have a hybrid physical and virtual event. Filmmaking will also resume, while the growing use of tools based on game engines and Virtual Reality will contribute to make production faster and more accessible to many.

As new VR headsets enter the market, some, as the HP Reverb G2, offering resolutions – 2160 x 2160 pixels per eye – that suggest better ways to watch content, when connected to a PC capable of deliver the best visual experience, both consumers and professionals using the tools have access to better technology and, one hopes, more immersive experiences, suggesting we’re getting closer to Jaron Lanier’s dream.

As we move into the Winter months and more people stay inside, also due to the situation caused by the pandemic, it’s only natural that more people want to try the options offered by VR. As apps like BigScreen continue to offer an alternative to going to the movies in VR, we may see the interest in VR headsets to rise next Christmas. I must confess I am eager to receive my HP Reverb G2 headset and discover how watching films in VR has improved.