five major chaos of ai-generated photos ②｜business in “pornographic photos”: face-changing and stripping, repeated bans but difficult to stop
2024-09-27
한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina
in this era where ai is everywhere and barriers to entry are almost unlimited, if you haven’t tried using ai to generate pictures or videos (hereinafter collectively referred to as ai-generated pictures), it is indeed not “trendy” enough.
like other new technologies, ai imaging technology has been full of controversy since its birth, with flowers and thorns accompanying it along the way. supporters see it as a disruptive technological revolution, while opponents see it as a scourge. especially in the past two years, large ai data models represented by chatgpt and sora have emerged. ai image generation has become a popular trend and is sought after by more and more netizens. while it brings great convenience, surprise, and comfort to people, it also inevitably exposes many negative problems such as being absurd, annoying, and even illegal.
ai is innocent, the key lies in how to use it. only by adhering to the bottom line of ethics and the rule of law can technology better benefit mankind. today, upstream news launches a special report on "five chaos in ai image generating", focusing on the various chaos exposed in the field of ai image generation in recent years, tracing the source, analyzing the cause and effect, and exploring solutions, in order to contribute to the healthy development of the ai industry. benefit.
in february 2024, mr. shang (pseudonym), who lives in yubei district, chongqing, suddenly received a text message containing a black and white nude photo. in the picture, mr. shang is holding a young woman in his arms, lying on a bed in a hotel, with the postscript "i have all your image data, please contact me quickly" and provided a phone number. because he usually keeps himself clean and is familiar with common sense such as internet fraud and ai face-changing, mr. shang knew that he was being defrauded by criminals using ai to generate photos, but ignored him. he also told his wife about his experience and shared it with his wechat friend group. , received a lot of good-natured ridicule. in this regard, mr. shang told a reporter from upstream news (report email: [email protected]), “i know that the picture was synthesized by ai, using the face on my id card to create a naked body. but i i’m curious, where did these scammers find my id photo and mobile phone number?” an investigation by an upstream news reporter found that according to reports from many media and police, similar ai face-changing scams have occurred in recent years. many people believed it to be true, resulting in huge amounts of assets being defrauded and even family breakdown. many criminals who produced and disseminated pictures and videos and committed fraud have also been severely punished by the law.
victims: some were defrauded of money, and some had their faces changed.
although mr. shang from chongqing received blackmail text messages from criminals, he was lucky to escape. in contrast, mr. wang from weihai, shandong province also encountered the same scam. he was asked for 680,000 yuan and was eventually forced to transfer 200,000 yuan.
the content of the text message mr. wang received was almost exactly the same as that of mr. shang. without thinking much, he dialed the number in the text message. the other party accurately stated his name and workplace, and told mr. wang that he could buy out the videos by transferring 680,000 yuan. if he didn't pay, the videos would be made public. under the threat of the other party, mr. wang believed it. subsequently, mr. wang raised funds from multiple parties and transferred 200,000 yuan. however, this 200,000 yuan cannot satisfy the other party's appetite at all. later, the police contacted mr. wang.
after careful observation, the police investigating the case found that in the photo in the text message, although the face was similar to mr. wang, the face shape and body shape were completely different from his. subsequently, the police quickly carried out emergency payment stop operations on the two bank accounts transferred by mr. wang. after research and analysis of the collection account, the police locked the identity information of the criminal suspects zheng and zhong. in january this year, weihai police arrested two suspects and brought them to justice.
mr. shang and mr. wang encountered text message fraud, and their avatars were made into nude photos by ai.
according to weihai police, in this case, the suspect first collected videos, photos and related information of the victim through the internet and other channels. the so-called "indecent videos" or "pornographic photos" are then synthesized using ai face-changing technology, and then accurately sent to the selected target through sms, email and other channels and leaving contact information. finally, they want to report, disseminate and inform their families. and other means to threaten the selected targets and require the victims to transfer money to their designated bank accounts. the police reminded that if you encounter such a situation, you must pay attention to retaining evidence and call the police in time.
compared with men, young and beautiful women, especially well-known female internet celebrities, are more likely to be affected by ai face-changing and ai "one-click clothes removal". photos and videos are even made into paid content for sale. in july last year, a female internet celebrity blogger with more than 4 million fans discovered that many fans left messages saying that they had seen her "pornographic videos." "i didn't think much about it at the time. i thought that the innocent will self-examine and the rumors will eventually pass. "it wasn't until one day a friend forwarded a pornographic video in which the heroine looked like her that she realized that she had been maliciously "face-changed" and that she had been made into a pornographic video.
according to media reports at the time, in the seller's price list, hundreds of internet celebrities were divided according to the level of fans, ranging from less than 500,000 fans to more than 10 million fans, and the corresponding face-swap porn videos were sold for 30 yuan to 50 yuan. yuan does not vary. in addition, there are many female stars in the table. one seller said that he owns more than 80 celebrities and more than 15,000 pictures. if you buy them all, the price is only 60 yuan, and 23 videos will be given as a gift. the seller also said that if you buy a face-swapping picture of a single celebrity, you will be charged according to the quantity. you can buy 300 photos for 5 yuan.
in addition, many photos of ordinary women in daily scenes such as schools, shopping malls, offices, subways, etc. have also become the targets of ai's face stealing. they have had their faces changed or their clothes taken off by ai. there are thousands or even tens of thousands of related pictures. of views.
a photo of a young woman in guangzhou was “undressed” with one click while riding the subway.
producer: forged nearly 7,000 nude photos, priced at 1.5 yuan each
as producers of pornographic and obscene content, many authors hidden online have been severely punished by law.
in july this year, cctv news reported an illegal case in which nearly 7,000 pornographic images were forged using ai technology. the criminal suspect bai moumou lives in beijing and was originally a technician for an internet company. he used ai "one-click clothing removal" technology to deepfake nearly 7,000 obscene pictures and sold them at a price of 1.5 yuan each. the pictures involve a large number of women, including students, teachers, and colleagues. the procuratorial organ believes that the defendant bai moumou produced and sold obscene materials for the purpose of making profits. the circumstances were serious and his behavior violated the relevant provisions of the criminal law of the people's republic of china. he should be held criminally responsible for the crime of producing and selling obscene materials for profit. .
a man in beijing produced nearly 7,000 obscene pictures and sold them for 1.5 yuan each.
in april last year, according to the hangzhou municipal people's procuratorate, yu, a man born in the 1980s from anhui province, was prosecuted on suspicion of making and distributing obscene materials for profit. yu became interested in this technology as early as 2020, when "ai face-changing" just started to emerge, and he also mastered this technology through self-study. later, yu registered an account on a foreign social platform and created a group. if other people wanted to join the "internal group", they had to pay a "membership fee" ranging from 198 yuan to 498 yuan. after joining the group, you can see a lot of people using ai. pornographic videos produced using face-swapping technology can also be "personally customized" using different images of female celebrities according to the needs of customers. in just a few months, yu earned tens of thousands of yuan.
in the public group established by yu, the public security organs examined more than 1,200 pornographic videos and more than 1,600 pictures. after identification, the above were all pornographic materials. in november last year, the xiaoshan district people's court of hangzhou city found the defendant yu guilty of producing and distributing obscene materials for profit. he was sentenced to seven years and three months in prison and fined 60,000 yuan.
young and beautiful women are prone to become victims of ai face-changing, and some businesses even offer "customized services."
survey: software is difficult to purchase, tieba has been blocked
according to previous media reports, in 2023, related software with the "one-click clothes removal" function will be available on many platforms. many netizens also purchase and exchange their experience on related software and products online.
for example, a certain product is clearly marked, "this software can realize the function of converting real people into cartoons, changing heads and changing clothes." this software sells for only 16.66 yuan, with monthly sales reaching more than 200 orders. someone asked in the comment area of the store, "can you really take off your clothes?" many people replied, "yes."
at that time, there were more than 9,000 posts related to ai face-changing in the post bar called "deepfake", which contained a large number of sales information about celebrity face-changing videos. a user named "synthetic alliance" openly sells face-swapping pornographic videos in the group. there are more than 150 videos in the list, and he said that "the price is only 58 yuan."
an upstream news reporter searched the deepfake post bar and found that the post bar had been blocked. the reporter searched for the keywords "ai face-changing" and "one-click clothes removal" on multiple e-commerce platforms, but none of them were displayed. in some stores, there are many software products with functions such as "intelligent erasure" and "intelligent image expansion", but the store owners replied that they do not have the function of "one-click clothes removal". however, the reporter noticed that a seller in zhejiang sold a software called "clothing changing artifact" for 18.80 yuan, claiming that "whether changing clothes or repainting, it can be easily handled."
the reporter contacted the seller via private message. regarding the inquiry about "can i remove my clothes?", the seller was very wary, "don't say it so bluntly, we are monitoring it!" then he said vaguely, "you can go to y (clothes)." after the reporter paid as required, the merchant issued here is a network disk download address and usage tutorial.
the reporter followed the merchant's instructions and found that the app can indeed realize the function of "one-click clothes removal".
on a certain second-hand trading platform, there are still merchants secretly selling software with the "one-click clothes removal" function.
lawyer’s statement: if the circumstances are particularly serious, the maximum sentence is life imprisonment
the reporter found that on some forums, there are still netizens secretly looking for free "one-click clothes removal" software. so, are merchants who publish or sell related software suspected of breaking the law or even committing a crime? if netizens use software to generate and publish relevant photos and videos, is it a violation of the law if they are only used to save them for “enjoyment”? what kind of punishment will you face if you share it?
fu jian, a well-known criminal defense lawyer and director of henan zejin law firm, told reporters that activities such as making pictures and videos and selling "one-click clothing removal" software involve infringement of personal privacy and portrait rights, are suspected of being illegal or even criminal, and may face criminal penalties. first of all, merchants who sell related software are suspected of producing, selling, and disseminating obscene materials. according to the provisions of the criminal law, those who produce, copy, publish, sell, or disseminate obscene materials for the purpose of making profits shall be sentenced to fixed-term imprisonment of not more than three years, criminal detention or surveillance, and a fine; if the circumstances are serious, he shall be sentenced to fixed-term imprisonment of not less than three years and not more than ten years, and shall also be fined; if the circumstances are particularly serious, he shall be sentenced to fixed-term imprisonment of not less than ten years or life imprisonment, and shall also be fined or his property shall be confiscated.
secondly, the right of portrait of a natural person is protected by law. without the consent of the natural person, the portrait of another person cannot be directly used. if netizens use software to generate relevant photos and videos and save them for "enjoyment", they are also suspected of infringing on other people's portrait rights. if it is shared on wechat groups or social platforms after production, it is suspected of infringing on other people's portrait rights and reputation rights, and you need to bear certain civil liability. his behavior is suspected of violating the "public security management punishment law", and in serious cases, he may be suspected of spreading obscene materials. depending on the seriousness of the circumstances, you may face penalties such as fixed-term imprisonment, criminal detention, and fines.
in response to phenomena such as ai face-changing and clothing removal, li liang, a professor at the law school of minzu university of china, said that first of all, it is necessary to clearly divide the infringing subjects of illegal applications of artificial intelligence; secondly, a mechanism for sharing liability (consequences) for illegal applications of artificial intelligence must be constructed; thirdly, scientific legislation should be carried out based on comprehensive consideration of legal principles and social interests, establish and improve the legal system related to artificial intelligence applications, promote technological research and scientific development, and strengthen supervision and regulatory mechanisms.
upstream news reporter jin xin partially synthesized information from cctv news, cctv, beijing news, people's daily, etc.
