Topaz Labs have become one of the leading names in Upscaling in the past few years with their Video Enhance AI v2 standalone software – doing very well in my head to head tests.
It is a piece of software designed to bring back details when moving from HD to 4K say, by cleverly estimating what should have been there through machine learning – i.e. studying thousands and thousands of video pairs at HD and 4K and learning how to get from one to the other. This is also very popular for photos – Topaz have their three software programs – Gigapixel AI, DeNoise AI and Sharpen AI – that they’ve recently combined into one called Topaz Photo AI, which always seemed to be where it would go. I used it recently for a blurry photo of a face and with “Recover Faces” turned on, it was remarkable the difference it made.
What is Topaz Video AI v3
Topaz Labs have now released Topaz Video AI v3. According to Topaz, this is more than an upgrade from v2 to v3 and this is why presumably they’ve renamed it, from Video Enhance AI to Video AI – it’s a completely new version of the software.
As Topaz state on their FAQ:
“After reviewing the list of pain points our users experienced in VEAI, as well as the feature requests that were received, it became clear that VEAI had to change significantly to evolve into a product enjoyed by both beginners, and experts alike.
Most of the issues stemmed from the fact video was being treated as a series of constant-rate images. All the current models operate on this assumption and as a result, had issues processing videos with variable frame rates, repeated frames, scene changes, etc. This was hampering the development of new models and processes that require multiple passes, like stabilization, scene detection, etc. To address these concerns and allow the addition of new features, we decided to rewrite the code base.
This approach allows us to focus on our core expertise to create and improve our models to perform better on all user videos.
Topaz Video AI consists of two independent parts: the Graphical user interface (GUI) and the backend FFmpeg filters. The GUI will merely call the FFmpeg backend and spawn separate processes. Topaz Video AI v3 unveiled a Stabilization model that allows users to stabilize their videos and keep the original resolution.”
What’s it like?
Some months back I noticed Topaz had early beta access to this software so I’ve been running it through its paces. Certainly they had some bugs to iron out during that time, but overall I didn’t personally see that much difference over Video Enhance AI v2 aside from the differently designed GUI – at least not as much as I was expecting from the version and name change. I think whether this release is a big deal to you will depend a lot on whether you are one of those users who had those “pain points” they mentioned in their FAQ or whether you are someone who can take advantage of the increased access to the backend FFmpeg filters. That said there are a few new features which most will be able to take advantage of.
Stacked filters
Aside from stability and under the hood improvements, probably the biggest change is that instead of only choosing one model, whether it was an upscaler or timewarp, you can now turn on both at the same time – for example if you wanted to slow down and upscale some footage – bearing in mind that Topaz also slows down cleverly too, interpolating new frames based on what in theory should be there instead of just repeating them (like the new feature in Nvidia’s DLSS 3 for games).
Parallel Tasks
Another fairly big change is better support for working with multiple videos. You can now set up multiple previews to render out when you are at the testing stage trying to work out which model is best. Sadly I didn’t see any major speed improvements overall to the export speed, but that may come in time.
New stabilization mode
Topaz state:
“Traditional methods rely on finding and matching individual feature points between frames to determine the camera motion. Still, it may be unreliable if the frames contain challenging scenes such as low contrast regions, illumination changes, motion blur, and moving objects.
The new stabilization AI model considers the entire frame instead of an individual point. We trained the model with challenging video samples to perform better than the traditional feature-matching algorithm.”
This feels a bit like a feature no-one asked for as you would normally do your stabilization in your NLE, but I must admit it is excellent so I don’t think this is just marketing chat. I saw none of the artefacts you can get in Premiere’s Warp Stabilizer or even sometimes in Davinci Resolve’s Stabilization and it felt very smooth and invisible. And the fact that you can stack it with your upscaling model of choice means it’s a great addition. So if you don’t feel like cropping in 10-20% to cover your stabilization, you can upscale just that amount to keep your framing the same.
“Most stabilization methods need to crop the composition to remove the missing regions created after transforming the frames, which can affect the footage’s intended framing. Therefore, in addition to the Auto-Crop option, Topaz Video AI offers Full-Frame stabilization, which generates the missing region using neighboring frames and AI models.”
And they even claim the ability to solve the dreaded rolling shutter – nice!
“Topaz Video AI also offers options for correcting rolling shutter effect and reducing jittery motions. The “Reduce Jitter Motions” option uses another AI model to reduce the spatial jitters among the neighboring frames, minimizing 3D camera rotations and parallax effects.”
CLI is back!
Running Topaz Video AI from the command line interface will be one of those features that some will kill for and others would never use. This did appear at some stage in v2 and was taken away as it wasn’t working well enough. Well now it’s back, hopefully to stay.
How much does it cost?
The big news is this costs nothing if you currently own Topaz Video Enhance v2 and are within your active upgrade plan. Download away!
If you are a new user, it costs $299, though as of writing is on sale at $159 which stays til 4th November. Though not perfect in all scenarios, I think this software is well worth the money if you can get use out of it (and charge your clients for the added value you’re bringing them of course). But as always, try before you buy – there is a 30 day trial so it’s best to make use of that to see if it’s compatible with your hardware (it does require a decent Mac or PC to run).