Many AI short dramas were launched online with shocking scenes but still had some "flaws"｜New observations on short dramas⑤
2024-08-19
한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina
Cover News reporter Liu Ye
When the rapidly developing AI and short dramas with massive audiences embrace each other, short video users have new viewing options.
As early as March this year, CCTV Video and Tsinghua University jointly launched the AI short drama "Chinese Mythology", which made AI and short dramas publicly go hand in hand for the first time in China; on July 8, Bona Film Group and Douyin cooperated with AI short drama "Sanxingdui: Future Revelation" and went online, with a total dissemination of over 140 million, and was selected into the third batch of recommended catalogs of the "Travel with Micro Short Dramas" creation plan of the State Administration of Radio and Television; on July 13, Kuaishou followed closely and launched its own AI short drama "Shanhai Qijing: Breaking the Waves", which was played more than 52 million times in two weeks...
AI has used its unique charm since its inception to attract a large number of creators in the field of short dramas and also attracted a lot of attention from users. How does AI short drama perform technically? What are the "hard flaws" at present? Cover News interviewed industry insiders and watched two AI short dramas to find the answers for everyone.
Stills from Sanxingdui: Future Revelation
AI short plays are growing rapidly
Fantasy and science fiction are the preferred themes
In the introduction of CCTV Video's "Chinese Mythology", descriptions such as "AI full-process micro-short drama" and "art, storyboard, video, dubbing, and music are all completed by AI" undoubtedly cast a big stone on the lake of short dramas. This 6-episode short drama, which tells classic mythological stories such as "Nuwa mending the sky, Jingwei filling the sea, and Shennong tasting hundreds of herbs", has its name at the beginning of the book of domestic AI film and television creation. At that time, most ordinary users' understanding of AI was still limited to "writing text, generating pictures and a few seconds of video".
A few months later, "Sanxingdui: Future Revelation", which is set in Sanxingdui and the ancient Shu Kingdom, and "The Mirror of Mountains and Seas: Cutting Through the Waves", which is based on "Classic of Mountains and Seas" and tells the story of a young man saving his mother, won praise from the audience with their exquisite and atmospheric pictures. The opening special effects of "Sanxingdui: Future Revelation" brought the sense of science fiction to the maximum, allowing people to intuitively feel the rapid development of AI technology. Many netizens called it "comparable to a blockbuster."
In addition to the visual effects, the addition of professional film and television teams also gives the current AI short dramas a more complete storyline. If "Chinese Mythology" is a "re-presentation" of the story, then the last two short dramas are "secondary creations": based on the existing mythological or archaeological soil, new characters are created to grow new story branches.
But whether it is "Sanxingdui: Future Revelation", "Shanhai Qijing: Breaking the Waves", or "Chinese Mythology", they are all fantasy and science fiction themes, which may be related to the unique sense of technology of AI. At the screening of "Shanhai Qijing: Breaking the Waves" in early July, director Chen Kun said: "Why did we choose fantasy works as the first step? I think that in the process of catching up with traditional film and television production, fantasy and science fiction are the two tracks that are easiest to be seen and recognized by everyone."
In addition to the above works, many "all-AI-made" short films such as "Fantasy Store", "White Fox" and "Beautiful Legend of Sicily" have been launched one after another. According to the film producers, follow-up works of "Sanxingdui: Future Revelation" and "Shanhai Wonder Mirror: Breaking the Waves" are also being created. The film and television industry is riding on the rapid development of AI to explore new artistic expressions.
Xia Jiayuan's AI-drawn portraits have very natural details in static scenes.
Defects in sound, picture, and character expressions are unavoidable
There is still room for improvement in AI film and television details
Although AI short dramas make use of technology to the extreme, each episode is only two or three minutes long, and the plot is still not as attractive as ordinary short dramas. The reason is the strong "AI feel" of the short films: the dubbing is bland, the characters' expressions are slightly stiff, and there are generally problems such as poor matching between sound and lip movements.
"Chinese Mythology" provides the audience with many exquisite scenes, but the screen connection and the performance of character movements and expressions are slightly insufficient. The large amount of narration reduces the audience's memory of the protagonist. Netizens commented that "it is beautiful, but not much different from the game CG."
After several months of development, AI films and TV series can flexibly apply physical laws by training large models. Actions such as running and tumbling can be almost indistinguishable from the real thing. Li Kangkang, technical director of Sanxingdui: Future Revelation, previously introduced to reporters at a film screening, "We have made a lot of technical reserves in the early stage, including the generation of pure virtual people, and then trained exclusive models to complete the consistent control of the characters. However, overly complex movements still require the assistance of traditional film and television production methods."
The movements are smooth and natural, but the details are still distorted. In several AI short dramas that have entered the public eye, the faces of the characters are slightly stiff during conversations, which is still some distance away from real "performance". This problem could have been solved by shooting the characters in real life and then converting them into virtual ones, but for filmmakers who take the "pure AI production" route, this is not advisable.
“Tumbling and fighting can be achieved through image processing and physical simulation, but character expressions are rather special. AI needs to accurately synchronize the voice with every movement of the mouth and generate micro-expressions consistent with the content of the speech. These details are difficult to simulate perfectly, so it will lead to unnatural effects.” AI painting blogger Xia Jiayuan told reporters that he had previously participated in the production of the promotional short film "Sanxingdui·Future Revelation".
Due to this limitation, the story of the AI short play cannot be fully developed. After the creation of "The Mirror of Mountains and Seas: Breaking the Waves" was completed, Chen Kun said frankly, "In fact, I think that under the support of current technical conditions, the consistency of characters, scenes, character performances, including action interactions, may not be suitable for presenting very complex stories under the premise that the technology has not yet reached a high level of control and precision, which makes it easy for people to not understand."
Overall, the current AI short plays are no less impressive than blockbusters in terms of scenes and special effects, and the details of the pictures are also constantly improving. With the iteration of technology, what other shocks will AI short plays bring us? Will the audience pay for them? Perhaps the answer will be revealed soon.
(Pictures in this article are from the interviewee)
