Adobe previews genAI video tools and third party options for Premiere Pro

Adobe is not only developing a video model for its Firefly genAI, but is also investigating ways of bringing third-party genAI models from OpenAI, Pika Labs and Runway directly into Premiere Pro.

Once OpenAI’s Sora was out in the open, so to speak, it was only really a matter of time before other major players in the market joined in, and Adobe promises a whole range of new generative AI tools coming to Premiere Pro this year. And beyond that it is laying out an eminently sensible vision for a genA tool ecosystem rather than the current silos the different developers have erected around themselves.

So, here’s what’s coming to Premiere Pro later this year powered by a new video model for Firefly that will join the family of existing models including Image, Vector, Design and Text Effects.

• Generative Extend: Seamlessly add frames to make clips longer, so it’s easier to perfectly time edits and add smooth transitions
• Object Addition & Removal: Simply select and track objects, then replace them. Remove unwanted items, change an actor’s wardrobe, or quickly add set dressings

• Text to Video: Generate entirely new footage directly within Premiere Pro. Simply type text into a prompt or upload reference images. These clips can be used to ideate and create storyboards, or to create B-roll for augmenting live action footage

All of which is rather neat. Here's it all laid out in a video.

As to opening up Adobe tools to third party genAI models, the company reasons that Creative Cloud has always had a rich partner and plugin ecosystem, and this is just an extension of that concept. Users want the best tools for the job, and whether that means Adobe Firefly or other specialized AI models, the company is working to make the integration process as seamless as possible from within Adobe applications themselves.

Early explorations show how professional video editors could, in the future, leverage video generation models from OpenAI and Runway, integrated in Premiere Pro, to generate B-roll to edit into their project. It also shows how Pika Labs could be used with the Generative Extend tool to add a few seconds to the end of a shot.

Of course, genAI tools are slightly different to a denoiser or transitions plugin, and to its credit Adobe highlights this aspect of the development work too. 

“Adobe has developed its own AI models with a commitment to responsible innovation and plans to apply what its learned to ensure that the integration of third-party models within its applications is consistent with the company’s safety standards,” it says. “As one of the founders of the Content Authenticity Initiative, Adobe pledges to attach Content Credentials – free, open-source technology that serves as a nutrition label for online content – to assets produced within its applications so users can see how how content was made and what AI models were used to generate the content created on Adobe platforms.”

We’re filming on the Adobe booth at NAB soon. Watch the site for video on all this, the newly rewritten frame.io, and more.