news

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

the recent "n room 2.0" incident in south korea has once again brought the old topic of deepfake to the forefront.

the perpetrators gathered on telegram and used ai to synthesize photos of women into nude photos, indicating that the circle of deepfake has long expanded from entertainment stars and politicians to ordinary people like you and me.

in this era when ai has become a popular science, we want to understand how this technology, which seems to be not new but has become increasingly popular in recent years, has affected our daily lives.

how far has deepfake evolved? what harm will it cause? how can we use technology to fight deepfake? how can ordinary people prevent deepfake?

we talked about these issues with dr. chen peng, an algorithm scientist at ruilai intelligence. ruilai intelligence was founded in 2018 and incubated by the institute of artificial intelligence of tsinghua university. it has been deeply engaged in ai counterfeiting for many years.

chen peng told us that ordinary people have failed miserably in identifying deepfakes, and anti-deepfakes still depend on ai.

one picture, a few seconds, deepfake is getting easier

deepfake first emerged on reddit, the "american version of tieba" in 2017. its main form was to replace the faces of celebrities with the protagonists of pornographic videos or to spoof political figures.

today, spreading rumors and making pornographic stories are still the mainstream uses of deepfake, but it has become easier.

chen peng explained that collecting one photo is enough for face replacement. of course, the more data collected, the better the details of the face such as moles and facial features will be modeled, and the more realistic the face replacement effect will be.

in april this year, a performance art project by two german artists was a living example.

they designed an ai camera nuca. the camera body is 3d printed and has a built-in 37mm wide-angle lens. the photos taken will be transmitted to the cloud, and ai will "take off the clothes" and the "film" can be produced in less than 10 seconds.

nuca actually doesn’t know what you look like naked. it just analyzes your gender, face, age, body shape, etc. to present your naked body as seen by ai.

crude? perhaps it doesn’t matter. in a few seconds, you have been exposed to the ai, and others may even believe it is you.

similar details were also exposed in south korea's "n room 2.0": a telegram chat room with 227,000 people, with a built-in robot that can synthesize women's photos into nude photos and adjust breasts, generating deepfake content in 5 to 7 seconds.

▲screenshot of the chat room, illustrating how to use deepfake

changing faces and stripping off clothes are just one application of deepfake.

the synthesis or forgery of realistic content, including text, images, audio, and video, through generative ai models (gan, vae, diffusion models, etc.) can be called deepfake.

among them, audio deepfakes are also quite common.

in early 2023, technology journalist joseph cox called the bank's automated service hotline and played the ai ​​voice "my voice is my password" that he cloned using elevenlabs, asking to check his balance. unexpectedly, the voice verification was successful.

chen peng said this is not surprising. before, it took several minutes or tens of minutes to capture our voiceprint information, but now it may only take half a minute or tens of seconds to capture a rough idea. if we receive a few more harassing calls, our voices may be leaked.

of course, if you want to clone more accurately and copy speaking styles such as intonation, for example, let guo degang speak english crosstalk or taylor swift speak chinese, you still need more corpus.

even text is a field that has been deepfakeed. ai-generated texts have long been seen everywhere, used by students to cheat and do homework, giving teachers headaches, but we may not yet realize the risks behind this.

false news and rumors are the hardest hit areas of text deepfakes, chen peng said. in the past, humans had to write the text themselves, but now ai can generate various comments for a certain event and then automatically post them on social media.

deepfake is faster and simpler. in chen peng's opinion, there are three main reasons.

first, there have been breakthroughs in generative ai technologies such as image and video. second, computing power has become more and more popular, and consumer-grade graphics cards can already run generative ai models.

another very important point is that the deepfake technology has been optimized into various tools with lower barriers to entry.

taking face-changing as an example, there are many open source projects for deepfake, such as deepfacelive and deep-live-cam on github. users can download the code from the website and configure the running environment locally.

▲ai musk live broadcast, using deep-live-cam

if a newbie who doesn't understand the technology still finds it difficult, there are professionals who can directly feed the food to their mouths. they will package the model and write it into simple and easy-to-use software for players to download for free and earn some advertising fees themselves, including many apps that allow players to undress with one click.

as for audio deepfakes, there are already mature commercial companies that allow users to easily use services in the form of sdk (development kit) or api (application programming interface).

users don't even need a device with a graphics card to deploy the program, but instead upload audio and other content to the website, wait for the results to be generated, and then download it.

therefore, complex technical principles are hidden behind the scenes, and what is presented to users are "out-of-the-box" interfaces that allow even teenagers to easily create false information.

in a nutshell, chen peng's conclusion is:

deepfakes have become readily available to ordinary people.

humans may have failed miserably in identifying deepfakes with the naked eye

when a technology "enters the homes of ordinary people", the people most likely to be affected are ordinary people.

fraud is one of the most common ways deepfakes are used.

earlier this year, a multinational company’s hong kong branch was defrauded of $25 million due to ai. the victim participated in a video conference, and the other people were all scammers who had undergone “ai face-changing” and “ai voice-changing”.

▲the police demonstrated how to use deepfake to fake a multi-person video conference

now that things have come to this, what can we do to protect ourselves?

if someone uses deepfake to deceive you, taking advantage of ai loopholes is one way, but it has a shelf life.

for example, when we are on a video call, if we suspect that the other person has used ai to change their face, we can guide them to do some specific actions, such as putting their hands in front of them and swiping them quickly a few times, or turning their head widely.

if the model behind ai face-swapping is not specifically optimized for hand occlusion, then it will be exposed and the face may appear on the back of the hand or suddenly become distorted.

the same principle applies to turning the head. if the other party does not deliberately collect materials with the head turned more than 45 degrees during the data collection stage, the facial fit will appear unnatural.

but in the future, these visible flaws will definitely decrease gradually.

▲northwestern university’s “find the difference” test: ai-generated or real?

chen peng joked that if the scammers think you are a fat sheep waiting to be slaughtered and are determined to defraud you, stripping off your social media information and spending several days optimizing your model, then these methods are not guaranteed to be useful.

if a video does not have these defects, does it mean it is a real video? this is not true.

as for whether it is effective or not, it is definitely not completely or 100% effective, it is just effective to a certain extent.

to put it more professionally, human visual perception performs well at the semantic level, such as being able to easily distinguish the meaning of objects or scenes, but its perception ability is not as good as that of ai models when dealing with pixel-level, low-level nuances.

from this perspective, chen peng believes that ordinary people have failed miserably in identifying deepfakes, but experts may still have a chance because they have seen too much and have more comprehensive analytical abilities, and can tell that something is not in line with the rules.

we are not leeuwenhoek, nor do we have a sharp eye, but human nature is immutable. therefore, we can also put up a traditional psychological defense line that has nothing to do with technology - it is better to be safe than sorry.

fraud often follows the same principle: stealing privacy, using fear, greed, and emotional values ​​to make up stories, pretending to be an acquaintance or packaging oneself to gain trust, and the ultimate goal is to make money.

▲realbelieve, a product of ruilai wisdom, will issue an early warning during a video call

keep this in mind, and then be more vigilant, don't click on unfamiliar links, don't give verification codes casually, try not to over-expose personal biometric information such as face, voice, fingerprints, etc. on the internet, receive suspicious calls, be more cautious when talking about money, and verify the other party's identity in multiple ways, such as asking things that only each other knows.

as the old saying goes, it is best to conquer the heart. once we realize that we may be deceived, we may not be deceived.

magic vs. magic, ai beats ai

raising awareness of fraud prevention is not enough. the south korean "nth room 2.0" incident showed another form of deepfake.

victims of fake nude photos may encounter "revenge porn" - the perpetrators threaten to spread deepfake materials, blackmail and harass the victims, causing more serious secondary harm.

but this sickle may also be raised over our heads: imagine that a fraud gang gets your photo from somewhere, combines it into a vulgar video, and sends you text messages threatening you that if you don’t transfer money, the video will be exposed on the entire internet. how would you prove yourself?

ruilai wisdom, where chen peng works, has indeed encountered this type of personal business. the other party said that his face was replaced in a video and asked if they could clear his name.

of course there are ways: magic versus magic, ai versus ai.

chen peng introduced that there are two main technical routes for ai counterfeit identification: active defense and passive detection.

let’s talk about active defense first. when we post photos on social media and don’t want them to be used by others, we can embed some visually imperceptible noise into them.

if someone uses our photos to train a model, due to this invisible interference, the ai ​​will not be able to extract the visual representation well, and the final result may be distorted or blurred. this is called "adversarial sample attack."

"semi-fragile watermarking" is another active defense method. after adding a watermark, if someone edits our photo, the watermark will be destroyed, and we will know that the image has been processed and is not credible.

watermarks cannot directly prevent images from being deepfake, but they can detect and authenticate the authenticity of images.

▲similar explorations have also been conducted overseas. adobe has initiated the c2pa standard, which uses metadata parameters as a way to determine the provenance of images.

of course, the threshold for active defense is high, and we need to take precautions and process the images in advance.

more often, we cannot predict the future, and when we receive our own "nude photos", it is the first time we are "honestly" with ourselves. at this time, passive detection should be used.

ruilai intelligence has a series of ai products for counterfeiting detection, including the generative ai content detection platform deepreal, the facial ai security firewall realguard, and so on.

▲DeapReal

simply put, using ai to identify ai is divided into two steps: first, extract a large number of forgery features, and then build a model based on these samples to let ai learn the rules of forgery identification.

color distortion, unreasonable texture, unnatural expression, asynchrony between sound and picture, irregular iris shape, and inconsistency in highlights of the two pupils are all learning materials for ai.

among them, the accuracy of video authentication may be higher than that of image authentication, because video consists of a series of continuous images, which provides more information that can be used for authentication compared to single images, such as the continuity of a person's movements between different frames.

in essence, ai counterfeit detection is somewhat like humans finding faults with the naked eye, and it also exploits the flaws of the ai ​​model itself.

▲ researchers from the chinese academy of sciences open-sourced the ai ​​model for detecting deepfakes to the world

but the defects will certainly improve gradually, so a key question arises: did forgery come first, and then forgery detection? if so, won’t forgery detection always lag behind forgery?

chen peng replied that the generated technology may be slightly ahead of the anti-counterfeiting technology, but they have an internal red-blue confrontation attack and defense laboratory that simulates deepfakes while defending against deepfakes, and continuously improves the deepfake detection capabilities.

if any new deepfake technology comes out, they can reproduce it quickly and then verify it on the detection products. "when new technology comes out, even if i haven't seen it before, i can still detect it to a certain extent."

moreover, the model itself has a certain degree of generalization ability. as it has seen more deepfake content, it can accurately identify and detect content that it has never seen before to a certain extent.

▲ platforms such as station b will label ai face-changing entertainment content

in short, ai forgery and authentication is a long-term "cat and mouse game" of confrontation and mutual competition.

this is why chen peng has been researching ai counterfeit detection algorithms:

the anti-deepfake resistance is too strong and requires long-term investment. unlike many ai products, you don’t have to worry about them after they are done.

despite this, he remains optimistic: "with laws and regulations, platforms managing content, the industry providing technology and tools, and the media making more people aware of the risks, there will definitely be some easing when all aspects are managed to a certain extent."

in the future, when we surf the internet, we may fall into a somewhat absurd scenario: the verification code asks you to prove "i am a human", and deepfake asks you to prove "i am not myself".

technology cannot completely detect all malicious intent, but humans do not need to be overly anxious. the success of deepfake is only the final result, but prevention of deepfake can start at any time.

as chen peng said, even a very simple ai product is a very systematic project.

we are part of a larger system. only by giving the injured a voice, punishing the perpetrators, intervening with technology to prevent evil, and raising social values ​​a little bit, can we work together towards a future where technology is not feared but used reasonably.