Upscaling footage is important when it comes to refreshing older footage, or simply intercutting lower resolution footage with more modern shots. Here's our review of Topaz Video Enhance AI, a piece of software that seems to be performing Voodoo magic.
Most people probably don’t think too much about upscaling video because they are usually filming everything from the same camera. However upscaling can be incredibly important, particularly if you are utilising older, lower resolution footage within an edit that contains more recently shot images. It might also be that you want to refresh an old edit from analogue or older digital and make it available on more modern platforms and larger screens.
But not all scaling methods are the same, and there’s some software out there that can help improve things even if you don’t scale the footage at all. Why would you want to do this? I have some older productions that I no longer have the masters for, or they are on obsolete media and not easily accessible to me (XDCAM disc anyone?) Furthermore if I shoot and reframe even a modern 360 camera I am generally still limited to a 1080p output.
That’s more than adequate, I hear you say. Ordinarily maybe, if it was shot on a high quality camera to begin with. However most 360 cameras and action cameras do suffer from compression artefacts and noise, which makes things more difficult for online streaming systems like YouTube. Additionally if you upload a 1080p video to YouTube it doesn’t make your video available in the most advanced codec unless you happen to be one of the lucky ones with a few gajillion subscribers. For most mortals you need to make sure you upload at a resolution well above 1080p to force it to encode in the better codec.
Upscaling well has always traditionally been extremely pricey. The gold standard has usually always been the Teranex system. Now owned by Blackmagic Design the Teranex hardware still performs some fantastic upscaling and standards conversions. Although you'll pay for it, it does do everything in realtime.
The sledgehammer way to upscale is simply to put your footage into a higher resolution timeline and hope for the best. Needless to say that if your primary concern is quality this is not a very good way of doing things.
One of the most accessible ways to upscale footage well is if you have DaVinci Resolve Studio Edition. Right click on a clip, go to its attributes, and enable Super Scale. This uses some advanced processing to make a really good upscale of your footage to a dimension ratio of your setting. You can set the degree of additional sharpening it applies as well as noise reduction. It works pretty well, although there are limitations. It's a really good option if you are in a rush though because it processes pretty quickly.
If you have the Creative Cloud suite you could use the "upscale while preserving details" option within After Effects. This isn't necessarily an inexpensive method, but if you already own the software it is a method of doing it without having to spend more money.
Another, if very slow method, is if you own the latest version of Photoshop you can also use its upscaling feature called Preserve Details 2.0. This does an absolutely incredible job of increasing the resolution of images, although you’ll have to get a batch workflow going by exporting each frame and processing them with a Photoshop script. This is not ideal, but it is a method.
There is another way however, and it’s by using dedicated software that not only performs an upscale, but it uses AI to rid the footage of artefacts such as moire, macro blocking, aliasing, and other issues that can afflict various lower quality cameras and footage. The software also uses AI to introduce detail where there was none before, and as you’ll see it’s no gimmick. In many instances it really does appear to be performing Voodoo magic. Below is a video I shot a few years back on the original BMPCC camera in 1080p unconverted to 4K using Topaz Video Enhance AI.
Like any software of this sort the better the quality of the imagery you put in, the better you get out. I tried out a number of different videos to see what Video Enhance AI could do with everything from some of the lowest resolution and worst compression imaginable through to high quality images shot on a Sony FS7 II and the original BMPCC, a camera that produced great quality but that could be prone to moire and aliasing due to not having any form of OLPF in front of the sensor.
Topaz Video Enhance AI has gained some notoriety recently from some rather good upscales of old television series such as Star Trek Voyager and Rick Astley music videos of all things. Interestingly we had one comment from someone in our comments section when we published the latter saying that the Topaz software was ‘snake oil’. This did pique my interest to see how well the software worked on my own footage, despite having seen quite clearly what it was capable of on others.
It’s the AI aspect of Video Enhance AI that gets the most attention, and it is this aspect that sets it apart from competitors that simply perform a straightforward upscale. For example it knows what trees look like, and can intelligently enhance the texture of them in images that contain various foliage. You can use the software to restore old, low resolution videos, or you can use it to help match new footage. For example you might be filming a project mostly in 4K, but your camera loses resolution once it shoots in slow motion. Using Video Enhance AI can allow those slow motion shots to match your 4K footage much better, particularly if you are using a camera that produces a lot more aliasing in those modes, such as the Panasonic GH5S or BMPCC 4K.
Video Enhance AI uses a variety of different models to cope with different scenarios, from low quality, highly compressed MPEG style footage from ancient YouTube, medium quality footage with average compression artefacts and detail, and high quality footage with minimal compression issues. There are many more modes to cater for CGI generated footage and analogue interlaced video.
It pays to play around with the different modes and previews to see which one suits your footage best. For example when I upconverted some footage from the Insta360 GO 2 it was suggested to me to use the Low Quality AI model. This worked well, however I found in the end that the Medium Quality model did a better job at preserving some textures, particularly with subtle moving water.
I gave the software footage of varying quality to see what it could do, and here are some of the results. When looking at some of these I will emphasise that you should pay very close attention to the original footage it was working with and understand that in many instances it barely has anything to work with. So please bear that in mind when judging the results. It might use AI, but it's not a robot Rembrandt. There are limitations. Please also consider that these are 100% still crops, which is not how you'd be viewing a video in motion.
These shots were taken from very low resolution, very highly compressed video. These were set using the Low Quality model and sizing was kept at 100% with no resolution upconversion. Aside from getting rid of the roughness of the text on the side of the boat, the foliage in the background is enhanced and more defined as well. Note also how the haloing around the bottom of the hull has been subdued too.
Before:
After:
This next example is a test of the impossible. It was a video from YouTube from around 10 years ago and is both incredibly low resolution and compressed to within an inch of its life. Surely there would be no hope at any sort of rescue?
Well, yes and no. I don't think anyone could disagree that the end result is certainly an improvement. It was never going to be a miracle recovery, but the software has even made an attempt on the foliage in the background.
Before:
After:
This shot was taken from the Insta360 One X 2 after it had been reframed to a 1080p cinemascope style ratio.
1080p before:
1080p to 4K after:
This was 1080p output from the original BMPCC camera from footage taken in 2013. Consider that at 100% size the trees are pretty much just mush, but when put through Topaz Video Enhance AI detail is seemingly brought from nowhere!
Before:
After:
Before:
After:
The thing I found most amazing about Video Enhance AI is that it does the upscaling and artefact reduction in the most natural way possible. It doesn’t introduce objectionable halos around edges when enhancing the detail, in fact it does a great job of reducing them in some modes. The other great thing about the software is that while you can use some of the set it and forget it models, you can select others that are extremely fine tuneable, so you can experiment with the best balance of artefact reduction and detail enhancement. In order to help keep things looking more natural you can get the software to add some degree of grain, although I would recommend leaving this stage to the final grade of an edit.
The other thing I should stress is how good the software is at simply enhancing low quality footage while keeping the original resolution. As you can see above in examples of low resolution Facebook video Video Enhance AI manages to very effectively reduce the compression artefacts while sharpening up edges and getting rid of aliasing.
Does it work miracles? Actually I would say yes, it does. Sometimes if you look at the upscale results you could be tempted to say that it hasn’t done a lot. However if you look closely at the original footage and then the end result you can clearly see how detail has been introduced into an image that was, to put it mildly, complete mush. On extremely highly compressed footage it can’t get around some warping issues. We can’t really expect it to. But it does make such footage much more watchable and usable. Give Topaz Video Enhance AI high quality input footage though, and then it really flies.
Okay, I've sung the software's praises, but surely there has to be a gotcha? You've got me in a corner with that because, yes, there is a fairly large gotcha in all of this, and that's processing power. Video Enhance AI is slow. Very slow. If you're in a rush, even on a very powerful machine, you'll be waiting a fair while for footage to render out. But that's the price for the sheer amount of work that's going on under the hood. Processing power is increasing all the time, and it will be interesting to see how the software works when it is optimised for both the Apple ARM based chips and any new developments from PC chip manufacturers. I will say this, though. If you need a high quality upscale then you'll make time for the processing.
For many purposes the Super Scale function in Resolve Studio will suffice for many people's requirements. But the ability of Video Enhance AI to do a lot more corrective work than simple upscaling makes it stand out amongst competitors.
If you have some very high quality HD or even SD masters from a few years back and you want to bring them up to date with the newer compression on YouTube, or simply repurpose the footage, then Topaz Video Enhance AI could well be an incredibly useful and valuable tool for you to use. At US$299.99 it isn’t the cheapest option, although it it is much less expensive than a Teranex. It is extremely powerful and having experimented with it quite a bit I can honestly say that it does exactly what the makers claim it does. This is no snake oil, it’s the real deal.
Find out more on the Topaz website.