OpenAI’s video generation tool, Sora, made waves in February with its remarkably fluid and realistic video capabilities, outpacing its competitors. However, the tool’s highly curated launch left many details unexplored—details which a Toronto-based production team, Shy Kids, has since unveiled through their creative project, “Air Head.” Selected by OpenAI to produce promotional short films, Shy Kids had significant creative freedom while utilizing Sora during their project. In an insightful interview with visual effects outlet fxguide, post-production artist Patrick Cederberg shared his experiences of working with Sora.
One key takeaway is this: Although OpenAI's promotional posts suggest that the shorts emerged fully formed from Sora, the reality is that these were professional productions involving extensive pre-production processes like storyboarding, meticulous editing, color correction, and refined visual effects work such as rotoscoping. Similar to how Apple’s “shot on iPhone” campaign doesn’t showcase the studio environments and post-production efforts behind the scenes, OpenAI’s communications focus primarily on what Sora enables rather than the intricate processes that shaped the final products.
Cederberg's interview, engaging yet accessible, offers fascinating insights for those curious about Sora's practical applications. Here are some key highlights that reveal the model's capabilities—while also suggesting it may not be as revolutionary as initially anticipated.
Firstly, achieving consistent control over elements remains a major challenge. Cederberg noted, “The closest we could get was just being hyper-descriptive in our prompts.” This involved meticulously detailing wardrobe choices and props to maintain consistency, as Sora generates each shot independently, lacking full control over continuity.
Additionally, Sora's outputs required scrutiny to eliminate unwanted elements. Cederberg described how the model frequently created faces on balloons or awkward strings hanging down—issues that then had to be corrected in post-production, adding to the workload.
When it comes to timing and character movements, precision is lacking. Cederberg explained, “There’s a little bit of temporal control, but it’s not precise … it’s kind of a shot in the dark.” For instance, timing an action like a wave gesture is more of a suggestion than a manual adjustment. Consequently, the team found themselves cropping and adjusting shots after generating them, especially since many clips inexplicably rendered in slow motion.
Using common filmmaking terminology, such as "panning right" or "tracking shot," was often inconsistent, which surprised the team. “The researchers hadn’t really been thinking like filmmakers,” Cederberg remarked, highlighting a disconnect in the tool's development.
Ultimately, the team generated hundreds of clips—each lasting 10 to 20 seconds—yet used only a few, with a ratio estimated at 300:1. This example underscores the extensive effort required, reminiscent of the challenges faced during traditional shoots.
Curious audiences may find Shy Kids’ behind-the-scenes video enlightening. However, like much AI-related content, the project has faced critical scrutiny, albeit without the harshness observed in some other recent AI-assisted controversies.
A noteworthy aspect involves copyright restrictions: when users request specific content—such as a “Star Wars” clip—Sora refuses. Attempts to circumvent this by describing elements in familiar franchises also trigger denials, indicating the model's ability to recognize potentially infringing requests. This raises important questions about whether Sora was trained using such content, allowing it to identify infringement. OpenAI's reticence in revealing its training data raises further intrigue.
In conclusion, while Sora demonstrates considerable potential as a filmmaking tool, it is not yet capable of generating films from scratch. As the dialogue around generative AI continues to evolve, the filmmakers' journey with Sora suggests that significant advancements lie ahead. As a famous character once proclaimed, “that comes later.”
Ranking
