Artificial intelligence has met its new frontier: video. From generating custom footage, to trimming long-form video into shareable clips, to creating entire videos from simple text prompts… the technologies are undergoing rapid evolution and daily improvement. As this transformation occurs, Signal is working to master and incorporate video AI into our current systems along with anticipating how it will change our future workflows.
Generating Footage with Text-to-Video Models
The days of stock footage libraries and b-roll shoots may soon be behind us. Technologies like Sora, Runway and Lumiere give users the flexibility to produce unique, tailored content, providing a personal touch for branding and marketing efforts. Right now, these models generate footage without sound, so keep concepts with voiceovers and background music at the forefront of the mind when brainstorming use-cases.
Sora
Made using visual “patches” (like the text “tokens” used to develop LLMs), OpenAI’s highly anticipated contribution to the video AI landscape, Sora, is to be released to the public in the second half of 2024. Sora can currently develop videos up to a minute long of high-fidelity (720p) video. Among its competitors, Sora stands out as most photorealistic and accessible model, and may become the industry go-to for generating high-quality stock-like footage for b-roll.
Runway
Runway Research has been at the forefront of image and video neural networks since its founding in 2018 – Runway-Gen-1 and Gen-2 were both released at the start of 2023. Like Sora, Runway-Gen-2 allows users to generate text-to-video prompts. However, Runway-Gen-2 offers more flexibility and an entire editing suite for designers and artists to work with. Additionally, the platform has a variety of different methods for video generation, allowing image inputs and making storyboards come to life. With flexibility comes variability in quality in output, as some results aren’t photo-realistic. Additionally, Gen-2 is presently capable of generating videos no more than 16 seconds long.
Lumiere
Google’s addition to the scene, Lumiere, seemingly aims to offer the artistic flexibility of Runway-Gen-2 combined with the photo-realistic nature of Sora. The research Google published in early 2024 indicated that they’re still fine-tuning the product – current videos generated by the model are 480p and only up to 5 seconds. As of this writing, Lumiere has no announced release date.
Converting long-form video to short-form
Have you recorded a webinar, conference or podcast chock-full of great insights? No need to waste time or budget on sifting for the shareable bits for your LinkedIn or Instagram. There are several platforms, like Munch or Vidyo, that take the legwork out of developing clips. Of course, a human is needed to referee and catch any mistakes, but having a sidekick to jumpstart the process and speed up workflow is always welcome.
Creating videos with no existing assets
All-in-one AI video generation plaforms – like Invideo AI, Synthensia and Kapwing to name a few – allows users to create script, voiceover and footage with a single prompt. Of course, output quality is unpredictable and varies, but with intuitive web-based WYSIWYG editors at the user’s disposal, these tools may be a good option for jobs when creators have flexibility and need something quick and simple.
Enhancing audio
Adobe offers audio editors tools to quickly elevate audio to create professional-sounding voiceovers and podcasts with Adobe Podcast. Read how Signal has been incorporating Adobe Podcast features into video editing.
Conclusion
Video content development has historically been a high-budget item for marketers. With the expense of scripts, shoots, talent, animation and editing, making even short clips for social can be costly and laborious. New AI tools seek to make video more affordable with shortcuts that optimize workflows and save time.
Signal will continue to closely follow the developments in video AI and carefully and cautiously incorporate emerging technologies into our routines to remain faithful to our core principle of Creative Intelligence.
