Discover the First Music Video Created Using OpenAI's Unreleased Sora Model

OpenAI has captivated the tech community, along with professionals in media and the arts, with its new AI model, Sora. This innovative technology produces realistic, high-resolution videos up to 60 seconds long, although it remains unreleased to the public. As of February 2024, OpenAI announced that Sora was available for testing by selected "red teamers" and a small group of visual artists, designers, and filmmakers. Nevertheless, early adopters have begun creating and sharing projects using Sora.

Among them is writer/director Paul Trillo, who was one of the first to showcase third-party videos generated by Sora. He has created what is being hailed as the “first official music video made with OpenAI’s Sora” for indie chillwave artist Washed Out (Ernest Weatherly Greene Jr.) and his single “The Hardest Part.” The 4-minute video features a series of quick, interconnected zoom shots across various scenes, seamlessly stitched together to create the illusion of a continuous zoom.

On the social media platform X, Trillo revealed that he initially conceived the video idea a decade ago but set it aside. He shared that the final video was crafted from 55 clips generated by Sora out of a total of 700, which were then edited together in Adobe Premiere.

In related news, Adobe has announced plans to integrate Sora and other AI video generation models into its Premiere Pro software. However, no timeline has been confirmed. For those eager to replicate Trillo’s workflow in the meantime, they would need to create AI video clips using other tools like Runway or Pika, as Sora is not publicly accessible yet.

In an interview with the Los Angeles Times, Washed Out expressed enthusiasm for incorporating new technologies into his work, indicating a desire to explore pioneering techniques. The interview detailed the specific prompts used to generate footage: Greene emphasized the importance of including precise details about the images, shot angles, and character movements. For instance, Trillo described one scene with, “We zoom through the bubble, it pops, and we zoom through the bubblegum into an open football field,” capturing the dynamic and immersive nature of the video.

Trillo also noted that he exclusively utilized Sora's text-to-video capabilities, opting not to incorporate still images generated elsewhere, a common approach among artists in the evolving AI video landscape.

The emergence of Sora illustrates the potential of AI in media creation, countering reports that previous demo videos—like “Air Head” by Canadian studio Shy Kids—heavily relied on VFX and traditional editing tools such as rotoscoping in Adobe After Effects. Furthermore, it highlights the ongoing interest among creatives to leverage AI tools for storytelling, despite criticism from others who view such technologies, particularly OpenAI's, as exploitative. Concerns arise over the use of artists' prior works for training without consent or compensation.