Kunlun Wanwei launches SkyReels, the world's first AI short drama platform
2024-08-19
한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina
On August 19, Kunlun Wanwei released SkyReels, the world's first AI short drama platform that integrates video and 3D models. The SkyReels platform integrates script generation, character customization, storyboards, plots, dialogues/BGM, and film synthesis, allowing creators to easily create high-quality AI videos with one click.
Fantasy blockbusters, tragic romances, hilarious comedies... Just input a simple idea and SkyReels can complete the entire process from script to finished short play.
The SkyReels platform integrates Kunlun Wanwei's self-developed script model SkyScript, self-developed storyboard model StoryboardGen, self-developed 3D generation model Sky3DGen, and the industry's first innovative platform WorldEngine that deeply integrates AI 3D engine and video model.
SkyReels is ahead of large models such as GPT-4o in quality assessment of script quality, storyboard quality, character performance, etc.
SkyReels can generate complete scripts, storyboards, character dialogues and background music with one click through AI, support custom adjustments of character images, timbre and storyboards, and automatically convert content into 1080P 60-frame high-definition videos. It can generate up to 180 seconds of video at a time, which is a significant breakthrough compared to Sora's 60-second video and Keling's 10-second video. One-click integration of all creative achievements greatly improves the efficiency of video creation, reduces creation costs, and accelerates the arrival of the "one person, one play" era.
At the same time, WorldEngine combines the precise and controllable capabilities of the engine (such as lighting simulation, physics simulation, 3D space, real-time interaction, etc.) and the fantasy generation capabilities of AI video large models, providing a new online hybrid video creation mode, allowing video creation to move from fuzzy generation to more precise and controllable.
SkyReels highlights are as follows:
Fully automated script generation: Users input concepts or ideas, and AI will generate a script with a complete structure and rich plot in one click. It also supports automatic organization and polishing of existing scripts to improve creative efficiency.
High-quality plot design: The system generates high-quality storylines, dramatic conflicts, highlights and emotional expressions by analyzing a large number of script plots, ensuring the richness and attractiveness of the script.
Intelligent storyboard generation: Generate storyboard images and corresponding text scripts with one click based on the script content. Users can also freely adjust parameters such as storyboards, camera angles, and camera movement to ensure the accurate realization of creativity.
Emotional dialogue/BGM: Through emotional speech synthesis technology, the characters are given natural pauses between sentences, emotional expressions, and intonations, and AI BGM is automatically generated based on the video content, allowing the picture and music to blend perfectly.
Personalized customization: Supports intelligent generation and customized adjustment of character images, timbre and storyboards to meet the personalized needs of creators, and provides a variety of visual style options such as animation, cartoon, and real-life 3D to adapt to the preferences and content expression needs of different creators.
Content consistency and accuracy: Ensure the consistency of characters and scenes in different storyboards, and automatically deduce character performances based on the story description to achieve coordination between character expressions, movements, and lines, thereby improving the expressiveness of the story.
Efficient video generation: Automatically convert storyboards into continuous HD video, providing 1080P 60fps output to ensure a smooth viewing experience.
Convenient post-production synthesis: Integrate all creative results with one click, and quickly export and share them. It supports downloading in multiple formats for easy post-production editing.
SkyReels' AI script generation function is based on SkyScript, a large script model developed by Kunlun Wanwei. In order to make the scripts generated by AI have high-density and exciting plots of short plays, Kunlun Wanwei first built a high-quality short play structured data set SkyScript-100M with hundreds of millions of scale. This data set has high-quality annotations for the plot rhythm, exciting points, and emotional changes of a large number of wonderful short plays;
Secondly, the algorithm team designed a progressively generative script generation framework, which enabled the model to output highly structured and exciting scripts. Whether it is a novice author or an experienced playwright, it can inspire creativity and easily create fascinating short play scripts.
Storyboard generation, another important part of short play creation, is provided by StoryboardGen, a self-developed storyboard generation model developed by Kunlun Wanwei. StoryboardGen is a storyboard generation model based on DiT-MoE architecture (Mixed-of-Experts architecture based on Diffusion Transformer technology) pioneered by Kunlun Wanwei, which can achieve high-density storyboard information generation and greatly enhance visual narrative expression capabilities.
After inputting the script, the StoryboardGen model will break it down into two parts: the global part and the main part. The global part includes the overall description of the storyboard, such as the scene, the scene, and the composition. The main part includes the customization of the character image and the action character. Finally, it is gathered to the Storyboard agent to generate a complete storyboard. StoryboardGen's special multi-agent architecture can effectively ensure the controllability and consistency of characters and scenes, and create high-quality industrial-grade storyboards.
Kunlun Wanwei StoryboardGen large model technical schematic diagram
In the final image generation stage, WorldEngine is the industry's first innovative platform that deeply integrates AI 3D engines with large video models. This is an epoch-making technological innovation. WorldEngine uses layer fusion technology as a bridge to create an unprecedented 3D and video hybrid creation mode.
When users use SkyReels to generate AI videos, they can freely choose to use 3D AI engine generation or video large model generation for any element. This innovative model breaks through the anti-physical phenomena such as "penetration" and "anti-gravity" that often appear in current AI video large models, and achieves the perfect fusion of the precision of physical simulation and the fantasy creativity of video large models to the greatest extent. From now on, AI video generation large models will move from the "fuzzy control" era to the "precise control" era.
In terms of 3D AI engines, Kunlun Wanwei's self-developed Sky3DGen large model can generate a variety of 3D elements and scenes, from natural landscapes to future cities, from static objects to dynamic characters, and provides real-time 3D scene interaction capabilities, allowing users to instantly see the effects of adjustments and editing, speeding up the creation process. With the support of WorldEngine, users can also easily build a 3D virtual studio to achieve a dreamlike experience of virtual shooting.
Character performance is one of the core aspects of the presentation of skits. Therefore, in terms of AI character performance, Kunlun Wanwei SkyReels team has specially developed the ActorShow character generation model, which enables the AI characters generated by SkyReels to have stronger controllable generation capabilities of lip expressions and body movements, and the character presentation is more natural.
In response to the needs of lighting differences and style differences encountered in video post-synthesis, the Kunlun Wanwei SkyReels team has separately trained a special fusion generation model to ensure good consistency in the videos after layer fusion.
The launch of SkyReels, Kunlun Wanwei's AI short drama platform, marks a technological leap in global AI short drama creation. It will completely subvert the video content creation process, freeing creators from the tedious and complicated production process, focusing on creative realization and inspiration expression, making the dream of "one person, one drama" a reality.
SkyReels AI short drama platform not only provides powerful creation tools for professional content creators, but also greatly reduces the threshold for AI short drama creation, making it easy for non-professional users to get started. This revolutionary tool is expected to bring about explosive growth in AI short drama user-generated content (UGC) and professional user-generated content (PUGC), inject unprecedented vitality into the short drama content ecosystem, and promote further rapid growth in the short drama content creation and consumption market. An era of short drama creation and consumption driven by AI technology is coming.
Kunlun Wanwei has always been committed to providing more intelligent, efficient and personalized tools for global content creators through advanced AIGC technology. Driven by the "All in AGI and AIGC" strategy, Kunlun Wanwei has always taken "realizing general artificial intelligence and allowing everyone to better shape and express themselves" as its mission. In the future, Kunlun Wanwei will continue to innovate and provide users with high-quality AIGC products and services through in-depth development of the global market.
translate
search
copy
