news

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

Key Points

Tencent Technology News, August 16, according to foreign media reports, in an experiment that exceeded expectations, doctors implanted a brain-computer interface device in the brain of ALS patient Harel, successfully capturing the language he was trying to express. Then, with the help of advanced artificial intelligence technology, an output that was extremely close to Harel's original voice was generated. This achievement has brought new hope to patients with speech communication disorders.

Brain implants offer hope for patients with ALS who have lost their ability to speak

ALS, also known as amyotrophic lateral sclerosis, affects the nerve cells that control movement throughout the body. The disease causes patients to gradually lose the ability to stand, walk and use their limbs. Sometimes, it also causes patients to lose control of the muscles used for speaking, thus losing the ability to communicate verbally.

When Casey Harrell was diagnosed with ALS four years ago, he gradually lost the ability to interact closely with his wife and daughter - from walking with his wife, holding his daughter in his arms to simply turning the pages of a book. His wife, Levana Saxon, figuratively likened the disease to a "thief in the night" that silently deprived him of everything.

The most heartbreaking thing for Harrell, then 46, was the loss of his speech, after he sang one last bedtime rhyme to his daughter, belted out one last Whitney Houston tune at karaoke and gave one last solo Zoom presentation as a climate activist.

But last July, a breakthrough came. A team of doctors at the University of California, Davis, surgically implanted electrodes in Harrell's brain to decode his thoughts and restore his ability to communicate. This move made him the latest test subject in a frontier scientific exploration that is attracting a number of well-funded companies, including Elon Musk's Neuralink, which are working together to achieve a direct connection between the human brain and computers.

The researchers' recent report in the New England Journal of Medicine is encouraging, noting that Harrell's experimental results far exceeded expectations, setting a new standard for implantable speech decoders and highlighting the great potential of such devices for people with speech disorders.

Edward Chang, a neurosurgeon at the University of California, San Francisco, was not directly involved in Harrell's treatment, but as a pioneer in the field of voice implants, he said: "This is undoubtedly an exciting development." He emphasized that a few years ago this type of technology was only a science fiction imagination, but now it has been "rapidly iterating and improving."

Can recognize 6,000 words with an accuracy rate of 97.5%

The team implanted four electrode arrays in Harrell's cerebral cortex, which look like miniature spikes. The number is twice as many as those implanted in the language area of ​​another recent study of ALS patients. Whenever Harrell tries to make a sound through the subtle movements of his lips, jaw and tongue, the 64 tips on each array accurately capture the electrical signal pulses from neurons to support the decoding of his intentions.

Three weeks after the surgery, the team of scientists gathered in Harrell's living room in Oakland, California, for an unprecedented "switching on ceremony" - seamlessly connecting the delicate implant to a series of high-tech computers, with cables cleverly connected to two metal struts extending from Harrell's skull.

After a brief computer training session, the system quickly learned Harrell’s speech patterns, and the implant excelled in a 50-word test, recording and interpreting his intent with 99.6 percent accuracy.

This amazing effect forced scientists to re-examine their research path and even eliminated a preset stage from the initial analysis process. When Harrell first tried to ask the question "What is this useful for?", his voice trembled with excitement and his smile could not hide the tears. This scene became a milestone moment.

To the audience, the words "how" and "good" from Harrell's mouth may still be vague and difficult to distinguish, but to the electrodes that are deeply connected to the neurons in his brain, these words are very clear. On the screen, every word accurately reflects the words he always wants to express in his heart.

The device cleverly bypasses Harrell's facial muscles, which are limited by his disease, and instead directly uses his motor cortex, which is where language commands originate from. "The key breakthrough is to precisely implant more sophisticated arrays into the language center of the brain, achieving unprecedented precision," said Sergey Stavisky, a neuroscience pioneer at the University of California, Davis.

The next day, the system demonstrated its amazing ability to recognize more than 125,000 words with an accuracy rate of up to 90%, and successfully constructed sentences that Harrell was thinking and expressing completely independently for the first time. Even more amazing is that these sentences are conveyed in a voice that is almost like Harrell's own, thanks to the research team's in-depth simulation and restoration of his pre-illness voice characteristics using podcast interviews and old recordings.

“I’m looking for a cheetah,” Harrell said during a clinical trial network co-led by Brown University and Veterans Affairs neurologist Dr. Leigh Hochberg. This unique and unexpected expression was the second time he had uttered it spontaneously, and its uniqueness even prompted the research team to review the video and ultimately determine it was a subtle deviation in the decoding process.

However, the unexpected “I’m looking for a cheetah” quietly sent an important signal to the doctor in the room: the implant had captured Harrell’s most personal language fragments - it turned out that his daughter Aya had just returned home wearing a cheetah jumpsuit, inspiring her father’s desire to participate in her childlike fantasy. Then Harrell continued: “My lovely daughter, I have waited too long for this moment.”

As scientists continue to train and optimize the system, its ability to recognize Harel's speech becomes more and more outstanding. The study shows that over a period of eight months, Harel successfully spoke nearly 6,000 unique words, and the device maintained an accuracy rate of 97.5%, which has surpassed the performance of many smartphone voice transcription applications and achieved a qualitative leap compared to previous studies. In previous studies, the recognition accuracy of the implant was only about 75%, which means that one in every four words may be misunderstood.

ChatGPT helps you speak more formally

Although Neuralink and other devices can assist with cursor control, Harrell's implant has led him to a broader and more complex level of language communication. Dr. David Brandman, a neurosurgeon who performed the operation on Harrell, said: "This technology has evolved from a scientific demonstration to an everyday tool, allowing Casey (Harrell) to communicate fluently with his family and friends every day."

This leap is due in part toChatGPTWith the help of advanced artificial intelligence language models such as , Harrell's implant can instantly capture neuronal activity and accurately convert discharge patterns into vowel and consonant factors. The computer then cleverly aggregates these sound fragments into words and then constructs sentences to ensure that the output content is closest to Harrell's true intentions.

In a recent interview, Harrell demonstrated this process in person: during long pauses, the computer weaved his thoughts into sentences in the silence, and he fine-tuned the words through the screen, and finally guided himself to make sounds. However, Harrell's decoded voice appeared more formal, which was due to the system's tendency to generate complete sentences. The research team is working to optimize the artificial intelligence algorithm to more accurately capture and present Harrell's unique and non-common phrases, making communication more natural and smooth.

This new role seemed to unlock Harrell's long-hidden self, and he and his partner regained laughter and the joy of sharing life. Harrell said with deep emotion that just as learning a new language can stimulate hidden personality traits, his voice decoder also allowed him to regain those personality elements that had gradually faded away due to illness, even if they were slightly distorted during digital conversion, the authenticity remained.

He mentioned that the device occasionally captured and reproduced his past tone, and familiar words such as "What up?" flowed out of his old voice, which made him laugh and exclaimed "I love this scene."

In addition, the brain implant has broadened Harel's ability to express himself in communication and has also affected the way people around him respond to his words. Now, he can freely express his love to his five-year-old daughter, Aya, and this transmission of love has also prompted the latter to open up to him more.

Harrell also revealed that in the past, medical staff who visited him often misunderstood his language barrier as an intellectual or hearing defect, although the actual situation is far from that. Now, they have not only restored their normal volume communication, but also shown more delicate and respectful touch. In this regard, although Harrell felt helpless about the need for brain surgery in exchange for this change, "but I have chosen to let it go."

The implant has also inspired Harrell to return to a social life, eager to reconnect with old friends who have drifted apart due to concerns and shame. This time, Harrell said, he will "reunite with them in a way that they can accept" to overcome the long-standing barriers caused by the inability to speak. He said: "This has taught me to forgive. I look forward to telling them that everything is fine and now it's time to make amends."

In the future, it may be suitable for patients with severe paralysis

There are questions about whether the implant would work for people with more severe paralysis. After all, while Harrell's speech ability has deteriorated, it has not been completely lost, which complicates the study.

In addition, despite the significant practicality of this technology, it has not effectively alleviated the financial pressure on ALS patients and their families. Harrell's wife revealed that insurance companies are only willing to cover the cost of care if the patient receives hospice care or stops working and meets Medicaid conditions, which undoubtedly exacerbates the frustration of other patients giving up life-extending treatment.

Melanie Fried-Oken, a professor of neurology at Oregon Health & Science University, pointed out that it is this economic mechanism that invisibly exacerbates the trend of poverty among people with disabilities and limits their opportunities to access and benefit from medical technologies such as cutting-edge implants.

For Harrell, living in a world where technology can connect computers and brains but is powerless to solve our most pressing financial problems is a mixed bag of emotions. “I feel lucky, but I’m also angry!” he says.

In the interview, Harrell talked about the significant improvement in work efficiency and independence after surgery, which not only made him proud, but also strengthened his determination to help promote implants. Before starting the device every morning, Harrell always wants to wake up the system with a special test phrase, usually a lyric buried deep in his heart.

Scientists are working hard to turn this dream into reality. Before that, Harrell tried to sing the implant to wake up with a melody every day. Recently, he chose a classic song by a Chicago band. He particularly liked one line of the lyrics and often recited it to his wife affectionately: "If you leave me now, you will take away the most precious part of my life." (Compiled by Jinlu)