Tech | Visa | Scholarship/School | Info Place

Former Snap AI CEO launches Higgsfield to challenge OpenAI’s Sora video generator

A few months ago, OpenAI attracted the attention of the tech world with Sora, a generative AI model that can transform scene descriptions into raw videos without the need for cameras or film crews. But Sora has so far been severely restricted, and the company appears to be targeting deep-pocketed creatives like Hollywood directors rather than necessarily amateurs or small-time marketers.

Alex Mashrabov, Snap’s former head of generative AI, senses an opportunity. So he launched Higgsfield AI, an AI-powered video creation and editing platform designed for more customized, personalized applications.

Higgsfield’s first app, Diffuse, is powered by a custom text-to-video model that can generate a video from scratch or take a selfie and generate a clip starring that person.

“Our target audience is all types of creators,” Mashrabov told TechCrunch, “from regular users who want to create interesting content with friends, to social content creators who want to try new content formats, to to social media marketers who want their brand to stand out.”

Mashrabov comes to Snap through his previous startup AI Factory, which Snap acquired in 2020 for $166 million. While at Snap, Mashrabov helped build products like AR effects and filters for Snapchat, including Cameos, as well as Snapchat’s controversial MyAI chabot

Higgsfield, which Mashrabov launched a few months ago with Yerzat Dulat, an AI researcher specializing in generated videos, offers a curated set of pre-generated clips, a tool for uploading reference media (i.e. images and videos), and a tip Editor that lets users describe the characters, actions, and scenes they want to depict. With Diffuse, users can insert themselves directly into AI-generated scenes or have their digital likeness mimic something captured in other videos, such as dance moves.

higgsfield

Image Source: higgsfield

“Our models support highly realistic movements and expressions,” Mashlabov said. “We are pioneering a ‘world model’ for consumers that will allow us to build best-in-class video generation and editing capabilities with a high degree of control.”

Higgsfield isn’t the only generative video startup going head-to-head with OpenAI. Runway was one of the first companies to get into this space, and its tools continue to improve. There’s also Haiper, which is backed by two DeepMind alumni and more than $13 million in venture capital cash.

Mashrabov believes Diffuse will stand out due to its mobile-first, social go-to-market strategy.

“By prioritizing iOS and Android apps over desktop workflows, we empower creators to create compelling social media content anytime, anywhere,” Mashrabov said. “In fact, by building on mobile devices, we are able to prioritize ease of use and consumer-friendly features from day one.”

Higgsfield also operates lean. Mashrabov said the generative model underpinning the platform was developed by a 16-person team in less than nine months and trained on 32 GPU clusters. (32 GPUs may sound like a lot, but considering OpenAI uses tens of thousands of GPUs, it’s not real.) and Higgsfield has raised just $8 million to date, mostly from a recent seed round led by Menlo Ventures.

higgsfield

Image Source: higgsfield

To stay one step ahead of competitors, Higgsfield plans to use the seed funding to build an improved video editor that lets users modify characters and objects in videos, and to train more powerful video generation models specifically for social media use cases. In fact, Marsh LaBeouf sees social media and social media marketing as a major money-making niche for Higgsfield.

While Diffuse is currently free to use, Mashrabov foresees that in the future marketers will need to pay some kind of fee or subscription to access advanced features, or to run batch or large-scale campaigns.

“We believe Higgsfield brings incredible realism and content production use cases to social media marketers,” he said. “We often hear from CMOs and creative directors that they need to optimize content production budgets and shorten timeframes while still delivering impactful content. Therefore, we believe video generation AI solutions will be the key to helping them achieve this goal core solution.”

Of course, Higgsfield is not immune to the broader challenges facing generative AI startups.

Generative AI models like Diffuse are known to “regurgitate” training data. Why does such a problem occur? Well, if models are trained on copyrighted content without permission or some kind of licensing agreement, then users of those models may inadvertently generate copyright-infringing works, exposing them to lawsuits.

higgsfield

Image Source: higgsfield

MashlaBeouf wouldn’t reveal the source of Higgsfield’s training data (other than saying it came from “multiple publicly available” places) or whether Higgsfield will retain user data to train future models, which May not be suitable for some business customers. He did note that Diffuse users can request deletion of their data at any time through the app.

As the wildfire spread of deepfakes on social media in recent months has shown, digital “clone” platforms like Higgsfield are also susceptible to abuse.

Likewise, Higgsfield could make it easier to steal creators’ content. For example, people can generate videos of themselves performing the same choreography simply by uploading a video of someone doing a choreography.

I asked Marsh LaBeouf what safeguards or protections Higgsfield might use to try to prevent abuse, and while he wouldn’t go into specifics, he claimed that the platform uses a mix of automated and manual moderation.

“We decided to first gradually roll out the product in selected markets and test it so that we can monitor for possible abuse and improve the product if necessary,” Mashlabov added.

We’ll have to wait and see how it plays out in practice.

#Snap #CEO #launches #Higgsfield #challenge #OpenAIs #Sora #video #generator

Leave a Reply

Your email address will not be published. Required fields are marked *