Lucy, who won an Emmy in 2019, was at the Sundance Film Festival to show her short film, “Dracula”. Lucy is an artificial character, so this was the first AI Virtual Being Premiere at Sundance 2021, a virtual event.

2020 was the year of virtual, as the pandemic moved many events online, and did the same to our work routines and meetings. Virtual became a keyword for many of us, and it drastically changed the way we work and connect. Even when we return to normal life as it was before, many of the things we learned to use the last 12 months – yes COVID-19 has been around so long – will stay around. In that sense, we’ve learned to use technology. Filmmakers even learned how to make movies keeping their crews safe, and solutions as Virtual Production helped to adhere to the strict rules of social distancing.

As ProVideo Coalition revealed recently, LG introduced its own virtual influencer at CES 2021. Reah Keem, says the company, impressed audiences with her distinctive personality and honest charm. Reah represents the next evolutionary step forward from purely voice recognition-based AI systems, as it is understood that virtual humans bring a more “personal” element to human-AI interactions. Made possible through the evolution of extended reality (XR), or the blending of the virtual and the real, these virtual humans are entering our daily digital lives.

A previous article published here at ProVideo Coalition – LG introduced at CES 2021 its first virtual influencer -, revealed a series of cases that show the number of artificial characters is growing, from Digital Deepak, based on the world-renowned writer and teacher Dr. Deepak Chopra, to Shudu Gram, a computer generated social media personality and model, or Miquela Sousa or Lil Miquela, who has over 2.9 million followers on Instagram and in 2020 signed an agreement with the Creative Artists Agency in areas “including TV, film, and brand strategy and commercial endorsements, raising the prospect of a movie or show featuring the character”.

Gollum should have an Oscar; Lucy won an Emmy

If you think virtual characters are a thing of modern times… think again. The first commercial film to attempt to make a computer-generated, three-dimensional, solid-looking model of a whole human body is Looker, from 1981, an American science-fiction thriller–horror film written and directed by Michael Crichton. Virtual humans employed as actors, usually known as synthespians, virtual actors, vactors, cyberstars, or “silicentric” actors are nothing new, but the first humans digitally duplicated appear in a 1987 film, Rendez-vous in Montreal: the characters rendered in three dimensions depict Marilyn Monroe and Humphrey Bogart!

While virtual actors have been used in everything from creating the crowds in the movie Titanic to populate the battles at Pelennor Fields in The Lord of the Rings, the best example of a virtual actor in a major supporting role continues to be Gollum, which saw its debut in the 2002 movie The Lord of The Rings: The Two Towers. Gollum’s performance was so impressive that there were online petitions asking the Academy Awards judges to honor the character with an Oscar (and with him the actor Andy Sarkis, who provided motion capture movements and voice – and emotions – for the CGI character).

It’s not an Oscar, but Lucy and her first series of narratives – Wolves in the Walls – won an Emmy in 2019, as ProVideo Coalition noted previously. Technology has evolved, and tools like Reallusion’s Character Creator, a digital human creation system, bring the technology to desktop-level, allowing more people to experiment with the creation of digital doubles or synthespians. With the pandemic reshaping the way productions are made, the need for avatars and virtual characters has grown, as some scenes will be created in computer, reducing the need for physical contact between actors. We’re, in fact, entering a new phase in movie production, one in which the tools developed and perfected in recent years will become invaluable.

A groundbreaking event in terms of technology

It’s as part of that process that Lucy’s appearance at the 2021 Sundance Film Festival represents a milestone. Last January Edward Saatchi, CEO of Fable, the company who created the virtual being Lucy, participated in a live VR event under the title  “The Metaverse — The Future of VR and Virtual Beings” , to talk about the issues related to virtual beings. Fable’s team was also at the virtual Sundance Film Festival, in a Zoom meeting, which was, in fact, a groundbreaking event in terms of technology: the first AI Virtual Being Premiere at Sundance 2021.

Lucy was at Sundance to present her film “Dracula.” Over the course of this year’s first virtual festival, Lucy talked to selected community members to get feedback and notes as she finalizes “Dracula” before her screening. Festival attendees were invited to be a part of Lucy’s first screening at Sundance, followed by a moderated conversation with Lucy herself. The Fable team also joined for Q&A and discussion around virtual beings and the future of AI-driven storytelling and authoring.

Lucy was first at Sundance to show “Wolves in the Walls” VR. In fact, Fable’s VR experience “Wolves in the Walls” (Chapter 1) – adapted from the book The Wolves in the Walls, written by Neil Gaiman and illustrated by Dave McKean – premiered at New Frontier at the Sundance Film Festival in 2018. Wolves in the Walls earned, in 2019, a Primetime Emmy for Outstanding Innovation in Interactive Media, an award that recognizes exceptional distinctiveness, inventiveness and relevance in expanding and redefining the conventions of interactive media experiences.

Charlie and Beck, two more Virtual Beings

As Fable continues to expand its Virtual Beings project, with characters as Charlie, – a French musician and poet – or Beck, a Canadian athlete who comes with an invitation to watch the Japan Olympics 2021 together, the studio is inviting people to interact with its Virtual Beings. The characters are powered by the Fable Wizard, an AI tool to bring characters to life through visuals, dialogue, voice, and animation, the same technology behind Lucy’s appearance at Sundance.

Virtual Beings open exciting options in terms of narrative building, as anyone who has tried Wolves in the Walls knows. There we’re just touching the surface, but more recent videos from Fable showing how Lucy interacts point to the direction the technology is moving. Another option also being explored is represented by Digital Humans. During a video presentation at the 2020’s VR Days conference Piotr Uzarowicz shows how digital humans are created, modified, and delivered via volumetric video and HoloSuite. ProVideo Coalition recently mentioned the HoloSuite in one article titled Arcturus brings volumetric video to 5G mobile devices.

The behaviour of virtual crowds

Arcturus is the company behind the technology used to show Madonna’s volumetric holograms at the 2019 Billboard Music Awards, the first ever live broadcast of a volumetric performance and the winner of the Advanced Imaging Society Lumiere award. The video presentation at VRDays shows how the technology allows digital humans to react and respond to their viewers in real time. Piotr Uzarowicz says that “we’re just beginning to discover all the use cases for digital humans but at the very least we have robust methods to build and deliver them” pointing to the technology from Arcturus recently announced.

The future of Virtual Beings and Digital Humans is inching closer and we may well, some believe, see a digital actor scoop an Oscar, in a not-so-distant future. In fact, back in 2004, Stephen Rymill from the University of Cambridge Computer Lab, suggested the possibility. Rymill has been involved with advanced research into virtual crowds and their behaviour, having published, with N. Dodgson, studies about the control of virtual crowds to avoid collision, and a system designed to improve the simulation of human crowd behaviour by considering the visual perception and attention of each individual in the crowd.

Virtual Beings want to be friends with you

The techniques used for the second study – A Psychologically-Based Simulation of Human Behaviour -, all based on psychology research, allow the system, which runs in real-time, to test different options. According to the authors, “the attention shifts for each actor, along with the associated gaze shifts, are controlled by a set of communicating agents. The agents can make requests for shifts of gaze and a gaze control agent sorts these according to priority. In our system each actor builds its own mental model of the world, based on its monitoring of other actors in the crowd. We argue that because we are using only the limited information from this mental model to decide the actor’s behaviour, our simulation produces more realistic behaviour than previous work.”

Stephen Rymill’s studies date back to 2005, and the technology has evolved at a fast pace in the last 15 years, with the introduction of Artificial Intelligence, a variety of tools for animation, motion capture and everything else used to make storytelling even more compelling, many of which has been covered by ProVideo Coalition.

While there are negative aspects that still must be considered, like the fact that stunt specialists and secondary actors may find less work, as digital actors are able to do the most dangerous stunts and integrate crowds of thousands – the truth is that digital doubles or synthespians or whatever you want to call them, are here to stay, and everything is being made to make them look more human. In some cases, like with Virtual Beings, they even want to be friends with you!