During the recent Sony Technology Day the company showed, among other technologies, how Virtual Production is made possible with Sony’s own Crystal LED and cinema cameras like the VENICE 2.

During the Sony Technology Day event, held this December, Sony introduced eight technologies, under the theme “Technology that inspires emotion,” some of which are related to media and entertainment, from the EPTS and data visualization technology”, to a prototype of a VR headset and, most important to filmmakers, how Virtual Production is made possible with Crystal LED and modern digital cinema cameras.

The Electronic Performance Tracking Systems (EPTS) used in football, which captures the movements of players and the match ball from live video feeds taken by dedicated tracking cameras, collecting skeletal data in real-time and with millimeter accuracy, was one of the highlights of the show. The system, from Sony Group company Hawk-Eye Innovations, uses Hawk-Eye’s data visualization technology, Virtual Recreation, to manipulate the data obtained from EPTS and video review systems to create virtual outputs.

A LED display installed in the studio

The advanced image processing, AI recognition technologies perfected by Hawk-Eye, fused with Sony’s specialty broadcast output capabilities and image sensor technology, entire games, including details down to the posture of the players, are converted into data, making it possible to visualize performances that were previously impossible to capture. In the future, the Sony Group and Hawk-Eye aim to apply this technology and contribute to various entertainment businesses, not limited to sports.

Sony also showed a prototype of a VR headset – not connected with its PlayStation VR system – that achieves a high-resolution of 4K with one eye and 8K with both eyes, using OLED microdisplays, and which, if developed further, can have multiple applications, from industry and entertainment to… Virtual Production.

In fact, one of the key themes of the event was Virtual Production, a general term for new video production technology that synthesizes live-action video and computer graphics (CG) in real time. With “In-Camera VFX,” one form of Virtual Production, a 3DCG image linked with the movement of the camera is projected as a background on an LED display installed in the studio behind the performer. This frees creators from the hassle of CG composition required for conventional green background shooting, as well as restrictions such as weather, time and place.

Crystal LED technology

Sony showed at the event how its own Crystal LED technology can be used. Crystal LED is an LED display that combines Sony’s unique LED control technology and signal processing technology cultivated in BRAVIA, and projects high-resolution images with overwhelming reality with high brightness and a wide viewing angle. The recently announced digital cinema camera “VENICE 2,” equipped with the newly developed 8.6K image sensor, offers high-resolution and a wide color gamut for rich expression, and is capable of expressing rich gradation even under a wide range of brightness conditions.

Sony says that the combination of these in-house displays and cameras has a high affinity for color reproduction and tonal expression, and makes possible more realistic content production. Furthermore, the Sony Group will deepen collaboration with creators inside and outside the Sony Group, as well as with engineers involved in actual filming, to promote the development of Virtual Production solutions that can efficiently produce high-quality content.

Naoki Tokunaga, from Imaging Products & Solutions Business Group, and Hitoshi Teshirogi, from Home Entertainment & Sound Products Business Group at Sony Corporation, explain in the video how the VENICE 2 camera, which will be available February 2022, and the Crystal LED displays give filmmakers new tools to explore creativity. The video shows different examples of how the combination of displays and cameras allows for new, efficient workflows.