Adobe MAX starts today in Miami and one of the big announcements from the show is the addition of a new Generative Extend beta to Premiere Pro powered by the Firefly Video model.
The list of Adobe’s Firefly genAI models keeps growing and the latest to be added is the Firefly Video Model (beta), which enables users to generate video from text and image prompts, as well as extend video clips and smooth out transitions. More to the point, it is released with deep integration into Premiere Pro from the get go.
Generative Extend was one of Adobe’s stand out ‘sneaks’ from earlier this year (there will be more from Adobe MAX, we’ll keep an eye on them) and following an intensive period of iteration the company reckons it’s onto something rather special here.
“Your feedback has helped us create our most magical new editing tool ever,” it says. “Generative Extend is our first generative AI feature crafted by and for video professionals. And this is just the beginning.”
Generative Extend has been positioned right in the toolbar, so you can access it as easily as adjusting an edit point. The idea is that it will sort out all those myriad times when the camera starts too late or cuts too soon, leaving you with the perfect clip that’s just a little bit too short. Now, in a similar way to what you’ve been able to do for a while now with the audio Remix tool, you can just click and drag the beginning or end of a video or audio clip to add what Adobe says are lifelike, photorealistic video and audio extensions. These are created in the background so you can keep working.
There are some qualifications here. Generative Extend won’t create or extend spoken dialogue, so it’ll be muted. Music is also not supported due to potential copyright issues, meanwhile that you will have to default to automatically lengthening and shortening tracks with the existing Remix tool. It will also only extend clips by about two seconds.
Adobe acknowledges that Generative Extend has some limitations in beta. In beta, video and audio extensions are limited to:
But the company says it is rapidly innovating and expanding its capabilities for professional use cases with user feedback. As a result, on each generation you can right click right and rate the result and play a small part in nudging it in the direction you want to go.
Since it was announced in September, the Firefly Video model has continued to evolve at the usual breakneck pace we tend to associate with AI nowadays.
Some of the stuff it's able to do is impressive. Take a look at the following video generated from the prompt: Point of view shot inside a beautiful, gently lit cenote in Mexico. The water is clear and blue with a sparkle from the late afternoon sun. The color is warm and magic hour style. High quality, cinematic.
It's interesting to hear from Adobe how the usage of the tools is developing too. Visualisation seems to be a key aspect, generating b-roll from still images is popular, while even rough prompts seem to perform better than 'insert Oscar-winning vfx sequence here' placeholders.
"There is a lot of excitement from our community over the ability to add visual depth to existing content by using Firefly Video to generate atmospheric elements like fire, water, light leaks, and smoke," writes the company. "Generating these elements on a black or green background means you can then layer them over existing video using blend modes or keying inside Adobe’s tools like Adobe Premiere Pro or Adobe After Effects."
Here's a quick example. The following was generated from the simple prompt: Filmic light leaks on black background, organic texture, realistic.
This was then composited in Premier Pro using the screen blending mode.
More on these use cases, and how to generate them, no doubt as MAX rumbles on in Miami for the next few days.