A performance of the Noh play Lady Aoi, directed by the Hoshokai, is the inaugural project for Canon and IBM’s partnership to promote visual and performing arts using volumetric video.

Canon and IBM Japan form joint business operation utilizing volumetric video technology to promote the visual and performing arts. The first project created through this corporate partnership is the release, this July, of a performance of the Noh theatre work “Lady Aoi,” produced by Hoshokai and utilizing volumetric video technology.

Directed by Kazufusa Hosho, Head of the Hosho Noh Style, the performance gracefully flows between scenes of the Noh stage and the spiritual world. For the creation of the video, Canon obtained 3DCG data of a real-life Noh stage, generated dynamic backgrounds and CG animations, and captured video of the Noh actors’ performance at the Volumetric Video Studio – Kawasaki. These resources were then smoothly integrated into the completed product, right there at the studio.

Capturing a traditional performing art with 600 years of history with modern technology opens new options in terms of narrative. Kazufusa Hosho, Head of the Hosho Noh Style says that “the emotional state of Lady Rokujo, the main character in Lady Aoi, is beautifully expressed through the integration of volumetric visuals and 3D graphics that depict traversing back and forth from the Noh stage and the spiritual world. During the production process, I placed strong importance on how to convey the powerful impact of the story when it is finally ready for audiences to enjoy. Thanks to this volumetric video technology, I believe we have created a way of viewing performances unlike anything on the traditional stage.”

Exploring new kinds of visual expression

Leveraging its know-how and experience in the field of art, IBM Japan provides consulting services that include business strategies and project execution support for companies involved in the arts in order to help the visual and performing arts achieve a digital transformation. In addition, IBM Japan provided high-speed infrastructure and technological support through its IBM Power Systems AC922 advanced dedicated parallel processing server and IBM Elastic Storage System broadband storage technology, enabling the use of essential responsiveness required for real-time video rendering.

Equipped with an IBM POWER9 processor and V100 GPU accelerator, the IBM Power System AC922 vastly improves system performance of deep learning, analytics and other applications by connecting the processor and GPU, or connecting two GPUs, through NVIDIA NVLink.  IBM Elastic Storage System (ESS) is a modern implementation of software-defined storage, making it easier for you to deploy fast, highly scalable storage for big data. This data management solution maximizes the performance of high-speed SSD/NVMe storage technology and has a proven record of success as a high-performance parallel solution that manages massive quantities of data using a global file system.

Canon’s Volumetric Studio Kawasaki is a visual recording studio, that began operations in July 2020, can capture performers’ movements from every possible angle. More than 100 4K cameras are utilized to capture video that is subsequently rendered by computers into 3D data inside a virtual space. Inside this virtual space, content can be viewed from any angle, making possible new kinds of visual expression. Recording and content generation can be performed separately, helping contribute to more efficient workflows.

Preserving Japanese performing arts

ProVideo Coalition announced, last May, that seven companies launched the Volumetric Format Association, with Canon as one of its founding members. At the time, Mr. Atsushi Date, senior general manager, Canon SV Business Development Center said “Canon is a strong believer in volumetric video technology as evidenced by the new kind of sporting experience we delivered at the 2019 Rugby World Cup in Japan. In addition, we opened a volumetric video studio in Kawasaki, Japan that is used by the most advanced creators. Canon is confident that volumetric video technology will enable the world to enjoy sports and entertainment more in many ways that have never been experienced before, and we have high expectations for the results of VFA standardization. This is a big first step of many to come in the rollout of the Canon Volumetric Business.”

Through this partnership, both Canon and IBM Japan will aim to create new value within the visual and performing arts world while expanding the potential use cases of volumetric video technology. To date, Canon’s Volumetric Studio Kawasaki has completed such projects as music-related content, variety TV programs, documentaries, TV commercials, promotional videos and collaborations with the traditional Japanese performing art of Kabuki and Noh theatre. In addition, Canon uses its capabilities to generate volumetric video in real time to create new forms of experiential visual entertainment including such events as live-streamed performances.

To view the movie, please visit the Volumetric Studio Kawasaki website.