AI editing is already here. So, how do we cope with it?

In editing, as elsewhere, AI offers both intriguing possibilities and huge potential pitfalls. Perhaps though the biggest issue that we face is that AI editing is already here, and it is getting better and better all the time.

At a base level AI editing would seem to offer some fantastic possibilities. After all, which editor hasn’t ended up at one point or another, being faced with having to sift through hours of, possibly rather dull, footage? Wouldn’t it be great if your computer could sift through it in minutes or seconds and pick out the best, or most interesting footage, for you to choose from?

Recently, IBM did this very thing for the film trailer for ‘Morgan’. After being fed with 100 different horror movie trailers, IBM’s AI system, Watson, went through the trailers and rated it according to a statistical selection method. This drew upon a list of 24 emotions and 22,000 types of scene category. Watson also analysed the voices, music, and scene compositions and rated them on a scale of scariness.

All of this gave Watson the basis from which to choose footage from Morgan itself, finally choosing the top 10 scenes that could be used in a trailer. While the AI did not perform the actual editing, it did pick out what it thought were the best shots and scenes. It was up to a human to provide the final trailer.

In this capacity, weeding out the best footage, such a system could be very useful indeed. But AI is making advances in terms of editing in general. For example there is Magisto, an online system that works by users uploading their footage to it. The system then goes through the files and analyses their content in a way that the company claims can edit footage based upon emotion, content and timing.

To be fair the example videos on the site are pretty impressive. They look like any other low budget edits performed by humans. Of course the company would be expected to choose the best examples possible, so we can never be sure how it would deal with particularly problematic footage. But it is intriguing nonetheless.

It would be all too easy to dismiss AI editing for all sorts of reasons that I will delve into in just a moment. But technological history always teaches us that convenience usually always trumps outright quality.

AI editing: quantity over quality

The democratisation of video creation is ongoing, and video producers are being squeezed from all sides both from competition, and from the increasing number of companies that produce in-house. With artificial intelligence this squeeze will only become worse, because to a small company it means that they can shoot some semi-competent footage themselves, feed it into a computer, and let it do all the hard work. And no professional needs to be hired.

That the video may be generic and not as creative as a dedicated, but comparatively pricey, human editor, may be neither here nor there. Convenience may well win out. In other words it is that age old question of value. If a company pays, say £1000 for a short video to be edited, do they really get a video that is £1000 better than the one edited by the AI system?

For higher end producers the development of advanced artificial intelligence editing may mean an increased project work load. No bad thing if you can edit more in less time, much more easily. But the amount of work on offer is often finite. Could it simply mean even lower rates due to the speed on offer? Less income as well as less stress?

Questions multiply

Could the increase in AI capabilities mean that it is possible that there could be an increasing gap between the low-end and the high-end tiers of production? Would competitiveness be based upon how good your AI system is? As technology develops, many different systems will become available, and picking the wrong one could put you at a disadvantage. When AI becomes really good, even if it is just to help you with the initial shot selection, or even an assemble edit, it will still offer a drastic increase in speed to the process.

AI is most definitely already capable of editing footage to a script that is fed to it, further speeding up the process. But will it ever totally replace the human input? Currently there are obstacles for AI that are preventing a full take over right now, such as the ability to truly understand timing. Could comic or dramatic timing, for instance, ever be reduced to an algorithm?

Theoretically I suppose it could. After all, most things in nature can be reduced to mathematics. But sometimes actors’ reactions and facial expressions can be so subtle, that it could be a very long time before it is possible that a computer could pick out all those nuances.

Then we have to think about creativity. Could a computer ever be truly creative, thinking up a new editing style or method to approach an edit? Maybe it could, but ultimately it would be up to a human to determine whether it was any good or not.

And finally there is the question of trust. Would we ever want to fully trust a computer to select and edit our footage? How can we be absolutely sure that the best takes are being chosen? If we cannot trust the system it then means sifting through the footage again ourselves, which would defeat the whole object of the AI in the first place. Furthermore, there may always be a nagging feeling in the back of our minds that we could be making a better job of the edit ourselves, but are being prevented from doing so due to the increased intensity of the production schedule brought about by the speed increases enabled by AI?

Currently, there are limitations. Sunspring is a film scripted entirely by an AI system by the name of Jetson and it makes no sense whatsoever. If anything it emphasises how far AI has to go if it is to understand the complexities of human communication.

But, as I alluded to earlier in this article, AI could certainly make the more drudging aspects of editing more bearable, particularly where processing a lot of footage is concerned. In an ideal world the AI would be there to truly assist us, and compliment our skills. However I also think that the march of convenience may well advance things in ways that as video professionals, we are not fully comfortable with. The likelihood though is will have no choice in the matter, whichever direction it takes.