news

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

An EEG cap with 18 electrodes.

Recently, the Ministry of Industry and Information Technology announced the shortlist of candidates for the Future Industry Innovation Tasks in 2023. Lingyisi, a brain-computer interface technology company from Shanghai's "Big Zero Bay", joined hands with Shanghai Jiaotong University, Shanghai Mental Health Center, Peking University Sixth Hospital, and Hangzhou Dianzi University to successfully take the lead.

At the end of August last year, the Ministry of Industry and Information Technology released a task list for future industry innovation tasks, mainly targeting the two frontier fields of future manufacturing and future information, focusing on four key directions: metaverse, humanoid robots, brain-computer interfaces, and general artificial intelligence. It systematically arranged 52 specific tasks, and promoted the innovative development of my country's future industries at the pace of planning a batch, deploying a batch, and implementing a batch.

In the brain-computer interface industry, Zero One has chosen the unpopular route of "non-invasive + emotion". At present, the company's independently developed 7 multimodal depression assessment prototypes and Zero One data collection system have been deployed in the psychiatric departments of three tertiary hospitals, achieving more than 4,500 patient depression assessments and multimodal data collection of healthy subjects, which will accelerate the implementation of emotional brain-computer interface technology.

Take a 10-minute photo of your mood

"Emotion X-ray Machine" test scene.

Normally, whether it is a fracture or a lung infection, doctors will ask patients to take an X-ray to check the lesion before making a diagnosis. Can the diagnosis of mental illness also be achieved through a similar "filming"? Three years ago, Zero One Thought, which was born at Shanghai Jiao Tong University, is working hard to turn this idea into reality.

Put on the EEG cap, sit in the seat like driving a racing game, and answer questions while watching the emotion-inducing materials displayed on the computer. At Zero One Thought, the reporter witnessed the diagnosis process of the "emotion X-ray machine" with his own eyes.

"You will see a set of images or videos. Please select the emotion that the picture expresses." As the tester gave the instruction, an 18th-century oil painting "The Drunken Priestess" appeared on the screen. The protagonist showed an elegant and friendly smile, as if she was slightly tipsy. At this time, although the subject's face showed no expression, the brain wave curve on the computer screen next to her immediately fluctuated, and her mental activities were "drawn" in real time. The whole process lasted about 10 minutes.

After being processed by the algorithm, a depression status report was completed in 2 minutes. The report clearly marked the quantitative parameters of the subject's risk of depression and anxiety. The radar chart showed several dimensions, which were the main basis for judging depression.

Lv Baoliang, professor of computer science and engineering at Shanghai Jiao Tong University and chief scientist of Zero One Thought, explained to reporters that the emotional feedback of patients with depression is different from that of ordinary people. For example, ordinary people feel happy and peaceful when they see the "priestess", but patients with depression generally cannot feel happy. This is because patients with depression are more inclined to negative choices. In the past, when measured by a scale, the subjects may conceal their true feelings, but it is difficult for them to escape the "eagle eyes" of the "emotional X-ray machine".

The desktop eye tracker is also what makes this device different from general emotion testing equipment. Lv Baoliang said that compared with facial expressions, eye movement signals are more closely related to emotions. The eye tracker set below the screen can collect more than 100 eye movement features such as pupil diameter, blinking frequency and fixation point, thus truly reflecting people's emotional state.

The first globally recognized standard sentiment dataset

How are emotions captured? The secret lies in the EEG cap. Its 18 electrodes touch the subject's scalp like little claws, covering and monitoring all areas of the brain.

The EEG cap is a sensor device that can collect an EEG signal with just one "claw". Lv Baoliang said that in choosing the key technical route of how to collect EEG signals, Lingyisi finally chose a non-invasive method. Although the EEG signals collected by non-invasive devices are more difficult to process, they are safer and can be used by a wider range of people.

In addition to continuously improving the wearing comfort of hardware devices such as EEG caps, building a standard dataset for emotional brain-computer interfaces is also an important topic in the field of emotional brain-computer interfaces. Due to the niche research field, expensive collection equipment, and complex data annotation, globally recognized standard emotional EEG datasets are extremely rare.

In 2014, Lv Baoliang's team proposed the framework of a multimodal emotional brain-computer interface that integrates EEG signals with eye movement signals for the first time in the world, and established a data set for it. Today, this data set, called SEED, has become the world's largest and most diverse emotional EEG data set, and is also one of the two most commonly used standard emotional EEG data sets in this field. Since its public release in October 2015, about 2,000 universities and research institutions from 81 countries and regions around the world have applied for use, with more than 4,880 applications and more than 1,400 papers published.

Taking the three-category emotional EEG dataset SEED as an example, Lv Baoliang's team selected 15 movie clips of three categories of emotions as emotion-inducing materials, and collected and recorded EEG according to international standards. The dataset contains 62-lead EEG and eye movement data of 15 subjects, 3 times each. The results show that eye movement signals are very good emotion recognition signals. If EEG signals or eye movement signals are used alone, their respective recognition rates are about 78%. If these two signals are combined through classic ensemble learning, the recognition rate of the three categories of emotions is increased to 88%.

Based on a series of original research in the fields of emotional intelligence and emotional brain-computer interface, Lv Baoliang was selected into Elsevier's 2023 "China Highly Cited Scholars" list in March this year. This is the fourth consecutive year that he has been selected for the list since 2020.

Minority innovation "Bo Le" continues

In the past two years, the popularity of brain-computer interfaces has risen sharply. In March this year, Neuralink, an American company, showed the first video of a patient with a brain-computer interface implant playing chess and games with his mind. At about the same time, Professor Hong Bo's team at Tsinghua University School of Medicine used a semi-invasive brain-computer interface to help a high-level paraplegic patient drink water autonomously under brain control.

Facing the "trend" of future industries, many investors are looking for brain-computer interface technologies and products that represent the future direction in the market. When investors find Lingyisi, they always ask Lu Baoliang a question: "Can your 'emotion X-ray machine' obtain a Class III medical device certificate?"

Based on the way sensors acquire brain signals, brain-computer interfaces are divided into invasive and non-invasive. In the "hot" brain-computer interface track, many people have not yet seen clearly how non-invasive brain-computer interface technology can be applied on a large scale. Based on the purpose and function, brain-computer interfaces can be divided into two directions: motion and emotion. Zero One Thought also chose the relatively unpopular field of emotion.

Fortunately, there are many "mentors" on this not-so-busy road of innovation. In 2019, Lv Baoliang's research on emotional brain-computer interface became part of the first major cross-disciplinary research project between medicine and engineering launched by Shanghai Jiaotong University. The project was jointly participated by 7 professors, including Director Sun Bomin of the Department of Functional Neurosurgery of Ruijin Hospital and Professor Fang Yiru of Shanghai Mental Health Center. In 2021, Lv Baoliang accepted investment from miHoYo to establish Lingyisi, dedicated to the research and development of "emotional X-ray machine".

The selection of this project for the Future Industry Innovation Task has allowed Lv Baoliang to see the spring of "cold research" again. Data shows that as of 2021, the number of psychiatrists in my country is less than 1.5% of the total number of physicians in the country. The disparity in the doctor-patient ratio in the field of mental illness highlights the market application prospects of the "emotion X-ray machine".

At present, Lingyisi is making every effort to promote the application for the Class III medical device certificate for the "Emotion X-ray Machine", and looks forward to the emotional brain-computer interface technology benefiting the public as soon as possible.

Author: Shen Qiusha

Text: Shen Qiusha Photos: Shen Qiusha and interviewees provided photos Editor: Fu Lu

Please indicate the source when reprinting this article.