Real-time video AI developer Decart says it’s primed to rework video marketing with the discharge of Lucy 2, an modern new model that’s in a position to seamlessly edit longform live streams via natural language prompts.
The open-source model not only generates high-quality video, but provides the tools needed to dynamically edit that content on the fly without compromising its realism. It eliminates the necessity for AI video to be edited post-production, opening the door to new marketing uses in live streaming, virtual try-ons, personalised product placement and more, the corporate claims in a press release.
More coherent edits and prompt adherence
What sets Lucy 2 other than competing video generation models like Google’s Veo 3 and OpenAI’s Sora 3 is its ability to process incoming video instantaneously with sub-second latency. Users will give you the chance to use stylistic changes, manipulate objects and characters, modify the scenery and more – while the video is being generated.
Decart explained in a blog post that Lucy 2 integrates diffusion models which have been optimised for temporal consistency and low latency to enhance the general coherence of its videos. It can ingest live video from multiple sources, like a smartphone camera, web cam, or RTMP stream and apply AI processing to that footage in real-time to rework it in an infinite number of how. The model does this while running at 1080p and 30 fps, producing immediate results with none interruptions to the live video stream or lack of quality.
Creators will give you the chance to experiment with Lucy 2 in various ways, using easy prompts to “replace the background with New York City” and make it look as in the event that they’re livestreaming from a wholly different location. Or, else they may command Lucy 2 to rework the topic right into a cartoon character and completely change the looks of the person on film instantly.
According to Decart, Lucy 2’s real-time performance advantages from several tweaks the corporate has made, like optimising it to run on Nvidia’s most advanced GPUs and model distillation techniques.
Decart also talked concerning the architectural changes applied to Lucy 2, which make it work very in a different way from traditional video models. Notably, it integrates a diffusion pipeline that helps it to attain an unprecedented level of coherence in frames. It outputs video repeatedly on a frame-by-frame basis without time constraints to eliminate the weird artefacts and flickering effects that plagued earlier video models and achieve greater consistency in scenes.
The model also advantages from enhanced prompt adherence, which allows it to interpret the user’s instructions more accurately. This means creators will give you the chance to feed it with more complex descriptions. For instance, someone might say “add red, blue and green fire-respiratory dragons without affecting the unique lighting,” and Lucy 2 will achieve this without making mistakes.
A magic effect on marketing
Benchmark tests show that Lucy 2’s latency measures lower than 100 milliseconds per frame, making it viable for livestreaming application, and this shall be of huge interest to marketers already using AI-generated video.
For instance, it has implications for livestreaming collaborations with social media influencers and other content creators. Using Decart’s Delulu Stream tools, which are actually powered by Lucy 2, livestreamers on platforms like Twitch and TikTok can enhance their video streams in various ways, like by dynamically adjusting their appearance to seem like any character in any world.
Alternatively, streamers can apply thematic filters which might be more attentive to real-time chat commands. And, they’ll give you the chance to instantaneously insert brands’ products into their videos at any moment to showcase other ways and situations through which they’ll use them.
Instead of counting on recognised influencers, brands can use Lucy 2 to create a virtual ambassador that represents them exclusively. They’d give you the chance to construct entire “lives” for these digital ambassadors, showing them in various parts of the world, doing different activities to construct more emotional connections with consumers.
One advantage of virtual brand ambassadors is that they don’t have to sleep – operating across the clock. Brands may even livestream repeatedly, 24 hours a day, answering queries and providing tips about how you can use products with none fatigue.
Decart’s leadership also sees a variety of potential for Lucy 2 with regards to letting more realistic virtual try-ons for ecommerce. Some digital stores already employ augmented reality technology to assist users visualise how they may look while wearing a new jacket, or whether or not a new sofa goes to match the decor of their home, but Lucy 2 will bring the realism of those experiences to a different level.
Thanks to Lucy 2’s superior coherence, virtual try-ons shall be much less distinguishable from actually trying on a new shirt or pair of jeans in a physical store, and so they’ll be more dynamic too. A client couldn’t only try on a pair of new Levi’s, but in addition visualise themselves walking around in them in a park or through a shopping center. Lucy 2 will model the person’s motion accurately while synthesising how the material should react to movement to make it look as in the event that they’re really wearing it.
Another option is live product placement. Because Lucy 2 can edit livestreams on-the-fly and personalise them for every viewer, brands will give you the chance to insert different products in the identical video for tens of millions of various viewers, based on each audience member’s past engagement with product pages.
A new era for world models
Decart’s CEO Dean Leitersdorf has described the launch of Lucy 2 because the “GPT-3 moment for world models.” He said it’s the primary instance through which a world model can now run live, in real-time, with none compromise when it comes to video quality. “The shift doesn’t just improve video, it creates entirely new markets, from live media and entertainment to virtual try-on, gaming and robotics,” he said. “We’re confident these markets shall be measured in billions.”
As an additional advantage, Lucy 2 can be extremely flexible and inexpensive. Decart said the model has been optimised to run on Nvidia GPUs, and on Amazon and Google’s cloud infrastructure, giving customers multiple deployment options.
Energy-efficiency optimisations promise to cut back the fee of AI-generated video dramatically, with Decart saying sustained real-time video generation will cost around $3 per hour.
Read the complete article here
