Photo credit: venturebeat.com
A few years back, the concept of a “generative AI video model” was virtually nonexistent, but that has dramatically changed. Nowadays, dozens of these models exist, with capabilities to generate ultra-high-definition, high-quality video within moments based on text inputs, user-submitted images, or existing video snippets. Recent articles, especially in platforms like VentureBeat, have highlighted various companies and their contributions to this burgeoning field, such as Runway’s Gen-3, Google’s Veo 2, OpenAI’s Sora, and newer entities like Luma AI and Pika, alongside Chinese competitors Kling and Hailuo. Notably, even major firms like Alibaba and the startup Genmo have begun releasing open-source video models.
These generative models have been utilized in significant film productions, including Everything, Everywhere All At Once and HBO’s True Detective: Night Country, as well as in various music videos and advertisements for Cokes and Toys R’ Us. Despite this swift adoption of AI technology within Hollywood, a considerable challenge looms in the form of copyright issues.
Most generative AI models obtain their training data from a vast array of videos sourced from the internet and archived collections. This raises concerns regarding copyrights, especially as owners of this content may not have consented to its use in training AI systems. Notably, Runway is currently embroiled in a class-action lawsuit focused on this issue, while reports indicate that Nvidia also utilized an extensive number of YouTube videos to train its models. Legal debates regarding whether this data “scraping” constitutes fair or transformative use are still ongoing.
In response to concerns over copyright and the ethics of model training, a startup named Moonvalley has introduced a solution: Marey, a generative AI video model tailored specifically for filmmakers, studios, and enterprise brands. This model claims to prioritize ethical practices by being trained exclusively on owned and licensed materials.
“Critics argued that it wasn’t technically possible to create a cutting-edge AI video model without using scraped content, and we demonstrated otherwise,” stated Moonvalley’s CEO and cofounder Naeem Talukdar during a recent discussion with VentureBeat.
Marey, currently available on an invitation-only basis, is positioned alongside Adobe’s Firefly Video model, which also emphasizes licensed data usage. Adobe has asserted that it provides indemnification for enterprises utilizing its model, a promise that Moonvalley echoes, pledging to defend its clients against legal repercussions regarding their content usage.
Advancing Ethical AI Video Production
Marey is a product of Moonvalley’s partnership with Asteria, a studio specializing in artist-led AI filmmaking and animation. The model is intended to enhance rather than replace the creative workforce, equipping filmmakers with advanced tools that align with traditional industry values.
“Our belief is that mainstream adoption in the industry can only be achieved by collaborating with industry professionals. They’ve explicitly communicated that a clean model is essential for them to consider using our technology,” Talukdar noted.
Instead of relying on automated content scraping, Moonvalley has invested significant time in forming direct relationships with content creators to license their work, ensuring all data used in training is legally obtained.
Furthermore, the firm is committed to supporting creators by ensuring they receive compensation for their footage contributions. Talukdar mentioned, “Many creators have reached out to us once they’ve learned of our approach. For local creators, it offers a chance to monetize footage that would otherwise be idle. We aim to create mutually beneficial partnerships.”
The company’s compensation model primarily pays creators based on how long their footage is used, with agreements lasting specific periods that allow for ongoing payments if the content remains in use.
Improving Cinematic Control
In a departure from the consumer-driven focus of many existing generative video companies, Moonvalley’s Marey has been developed with an eye toward professional production needs. Talukdar emphasized the necessity to create technology tailored for major studios and advertisers rather than merely offering simplified consumer applications.
Marey boasts several innovative features:
Native HD Generation — It produces high-definition video without requiring post-processing upscaling, which minimizes visual distortions.
Extended Video Length — Marey can generate 30-second segments in a singular process, unlike many models that can only produce short clips.
Layer-Based Editing — Users can edit the video’s foreground, midground, and background as distinct layers for enhanced compositional precision.
Storyboard and Sketch Inputs — Marey allows the use of storyboards, sketches, and real-world references, in addition to traditional text prompts, making it easier for professionals to utilize.
Responsive Conditioning Inputs — The model is designed for better interpretation of visual inputs like sketches and movement markers, improving the controllability of AI-generated video.
Generative-Native Video Editor — A dedicated editing software for Marey is under development, specifically aimed at improving project management and timeline oversight for users.
Talukdar elaborated, “We built this model with a focus on controllability, requiring significantly enhanced management over the output. This includes the ability to change characters and offers notable advancements like layer-based editing, specifically tailored for Hollywood production.”
Furthermore, Marey employs a hybrid diffusion-transformer architecture, integrating the strengths of both methodologies to boost controllability across the model.
Funding and Future Prospects
Moonvalley has recently disclosed a $70 million seed funding round, under the leadership of Bessemer Venture Partners, Khosla Ventures, and General Catalyst. Influential investors, including Hemant Taneja, Samir Kaul, and Byron Deeter, have expanded the company’s board as part of this round.
Talukdar acknowledged that this funding is significantly lower than some rivals, like Runway, which has reportedly amassed around $270 million, yet emphasized that Moonvalley has optimized its resources and assembled a skilled team of AI professionals.
Marey is currently in a limited access phase, allowing selected studios and filmmakers to experiment with the model, with plans for broader availability in the near future.
As Moonvalley and Asteria unveil Marey, they position themselves at the forefront of AI-assisted filmmaking, presenting an option that harmonizes technology with respect for creative integrity. However, the competitive landscape is intensifying as other AI video startups introduce their features, creating a dynamic and rapidly evolving field.
Source
venturebeat.com