news

it has been almost two years since google's medpalm 2 model passed the u.s. medical licensing examination. is artificial intelligence (ai) qualified to become the "imaginary enemy" of clinicians?

in july this year, a paper published in nature medicine showed that even the most advanced large language model (llm) cannot make accurate diagnoses for all patients, and its diagnostic accuracy (73%) is significantly worse than that of human doctors (89%); in extreme cases (cholecystitis diagnosis), the accuracy of llm is only 13%.

but for pi jingtao, a doctor at the neurological center of beijing tsinghua chang gung hospital, he just learned a lesson from ai this year. in late august, the lingxi medical cerebrovascular disease specialty model, in which pi jingtao participated, was officially released. the model was developed by professor wu jian's team at the neurological center of his hospital and is the first medical artificial intelligence model based on special diseases in china.

"the capabilities of the big model are both strong and weak. we cannot let it play freely and unrestrainedly." pi jingtao said that the biggest difference between the big model for specific diseases and chatgpt is that the seemingly plausible diagnosis and treatment suggestions fabricated by ai should be nipped in the bud to avoid catastrophic consequences.

in late august, lingxi medical's cerebrovascular disease model was officially released. the model was developed by professor wu jian's team from the neurological center of beijing tsinghua chang gung hospital. it is the first medical artificial intelligence model based on a specific disease in china. the picture shows the lingxi medical model technology exchange meeting. photo provided by the interviewee

benchmarking specialists and disease experts

as one of the four major chronic diseases in china, about two-thirds of the first-time patients with cerebrovascular disease are elderly people over 60 years old. it has the characteristics of "high incidence, high prevalence, high mortality, and high recurrence rate". as of 2021, the elderly population in my country aged 65 and above has exceeded 200 million, and the shortage of cerebrovascular disease doctors and uneven levels of expertise are particularly prominent.

"there is no shortage of general practitioners in primary hospitals, but there is a shortage of specialists or disease experts. this is the problem that the big model aims to solve." pi jingtao introduced that the big model for cerebrovascular disease is a product of cooperation between medical institutions and technology companies. specifically, h3c group provides technical personnel, ai algorithms and computing power, and beijing tsinghua chang gung memorial hospital and tsinghua university provide big data and clinical needs to jointly create an auxiliary diagnosis tool for clinicians.

professor wu jian once said that medical artificial intelligence has great potential and advantages in alleviating the shortage of medical resources and improving the level of medical services. its core lies in the deep mining and intelligent analysis of massive health data, which can greatly improve the accuracy and efficiency of clinical diagnosis and treatment.

currently, the core function of this large model is to analyze and extract key information from medical records, match it with the clinical knowledge base, and ultimately provide standardized treatment recommendations that comply with clinical guidelines.

on the one hand, clinicians can input desensitized (excluding patient personal information) clinical course information, and the big model will generate the final treatment plan. during this period, if the big model detects omissions in the course information, it will remind the doctor to supplement it in time to ensure the standardization of medical records.

on the other hand, clinicians can also input simple information such as the patient's chief complaint (such as main symptoms and duration). the big model will guide the direction of the consultation through selective interaction, and gradually improve the clinical diagnosis and treatment process based on the options clicked by the doctor, thereby enhancing the doctor's evidence-based ability.

professionalism is reflected in two aspects

in pi jingtao's view, compared with general large models such as chatgpt, the professionalism of the disease-specific large model is reflected in the two aspects of the thinking chain and the knowledge base. taking the cerebrovascular disease-specific large model as an example, its data sources include two parts: one part is the desensitized clinical data, which involves comprehensive information such as the specific characteristics of the disease, the incidence, and the diagnosis and treatment process. the other part is publicly available clinical guidelines, a large number of neurology and neuroscience textbooks and reference books, which constitute the core content of the database. it is worth noting that the large model cannot directly receive this knowledge, but needs to be reorganized by clinicians and engineering teams. the framework and key content of the clinical guidelines are converted into a language and process that the computer can understand, and then input into the large model.

"if it is fed directly without restriction, the big model will diverge to other aspects and generate some new understandings. however, the clinical guidelines are already the highest level of standardized diagnosis and treatment recommendations. any modifications on this basis are wrong and are not what we want." pi jingtao told reporters that in order to make the big model "obedient", in addition to feeding it knowledge that the computer can "understand", the key is to teach it a set of clinicians' "thinking chains" and rely on this ability to infer the clinical data of different patients.

for example, the diagnosis and treatment process of a cerebrovascular disease doctor generally includes asking about the medical history, conducting a physical examination, considering auxiliary examinations, and making an accurate diagnosis after comprehensive analysis. based on the diagnosis results, the doctor will consider the patient's specific cause and other underlying diseases, and formulate a standardized treatment plan in combination with standardized diagnosis and treatment recommendations.

in this process, different patients' complaints point to different inquiry directions. but the problem is that although the thinking of the big model is "divergent", it does not have the ability to independently build a chain of thinking, so the engineering team needs to have a deep understanding of the clinical diagnosis and treatment scenarios and convert the doctor's clinical thinking into thinking that the machine can understand. this process allows both clinicians and technical teams to experience cross-learning.

"the gap is mainly a language barrier. for example, we don't understand the chain of thought, and they don't understand the relationship between different diseases. but this will not have a substantial impact on our research. as long as we understand the basic knowledge of the other party's field, we can clear the difficulties." said pi jingtao.

clinicians are still responsible

pi jingtao revealed that the cerebrovascular disease model is currently undergoing clinical verification at the beijing tsinghua chang gung hospital neurological center. previously, he and his colleagues conducted internal tests on the model using real cases or simulated complex clinical scenarios. this test is based on the previous simple test and aims to evaluate the model's ability to understand medical records of different complexity, different language styles, and different levels of doctors.

among them, the most troubling question for pi jingtao is, if there is an error in the big model, how to ensure that the clinicians are not disturbed? in other words, how should the clinicians deal with the relationship between themselves and the big model?

this contradiction appeared during the first internal test. at that time, pi jingtao simulated a complex clinical scenario, and the treatment plan given by the big model was different from the expected "standard answer". subsequently, the technicians intervened and traced back the reasoning process of the big model, trying to find possible errors, but to no avail. when pi jingtao used the "standard answer" to match the clinical guidelines, he was impressed by the result: there was a blind spot in his own diagnosis and treatment ideas.

"when i first designed this model, i had no idea whether it could be used in clinical practice and whether it could enhance and improve (diagnosis and treatment efficiency). but after this incident, i was very confident." pi jingtao summed up this experience and found that the patient's clinical symptoms were a comprehensive result. for example, a patient may be hospitalized due to neurological problems, but the cardiovascular, kidney, and liver conditions are also changing. although doctors have received standardized specialist training, there may still be blind spots in their diagnosis and treatment thinking, and they cannot guarantee that they can provide a comprehensive diagnosis and treatment plan every time.

in addition, clinical guidelines are constantly updated, and not all doctors can update their knowledge in a timely manner. when doctors lag behind in updating their knowledge, big models can help fill these gaps.

however, reading the guidelines and making judgments are still basic skills that clinicians cannot abandon. pi jingtao said that if doctors find that the diagnosis and treatment plan provided by the big model does not match their own judgment, they should not simply accept or reject it, but should explore the reasons in depth. this thinking process helps doctors improve their clinical diagnosis and treatment capabilities, which is the ultimate vision of the big model as a clinical auxiliary diagnostic tool. after all, although the big model can provide conclusions, the clinicians themselves are still responsible for the diagnosis and treatment process.

calling for more people to join

in july 2023, med-palm m, the world's first general practice medical model jointly created by google research and deepmind, was officially released. according to the data, this large model has the ability to understand clinical language, imaging and genomics, and its clinical use is just around the corner.

according to the 2023 healthcare ai big model industry research report, as of october 2023, the cumulative number of publicly available big models in china has reached 238, of which nearly 50 are medical big models, covering multiple fields such as patient consultation, doctor assistant, drug development, and health science popularization. according to pi jingtao's observation, the development of many medical big models starts from "general practice", trying to directly build a large general practice model covering all specialties for users to consult on various diseases.

however, after participating in the construction of a large model for cerebrovascular disease, professor wu jian's team expressed doubts about this model. they found that it was quite difficult to sort out the clinical guidelines for a disease, and it was also time-consuming and labor-intensive to create a corresponding chain of thinking. it was difficult to train a general practice model using the same method in the short term. in other words, the current general practice model can achieve medical popularization, but it is difficult to provide effective clinical guidance in specific specialty areas. therefore, the research and development idea of ​​professor wu jian's team is to start with specialty diseases and then move towards general practice.

"if the disease-specific big model for cerebrovascular disease is successful, we can replicate the successful experience to other neurological diseases; if the neurological diseases are fully covered, it will become a specialized big model; and if the experience of neurology is replicated to other departments, it will form a real general medical big model." pi jingtao said that in professor wu jian's view, the more people suffer from the disease in china and the greater the disease burden, the greater the clinical demand and research and development space for its disease-specific big model. for example, the four chronic diseases with the highest incidence in my country - hypertension, diabetes, coronary heart disease, and cerebrovascular disease - all have a very broad space for big model development.

at the launch of the cerebrovascular disease model, professor wu jian called on experts in the field of neurological diseases and professionals in the field of artificial intelligence to work together to grasp the reform possibilities of innovative technologies and improve the current status of clinical diagnosis and treatment. he said that if academic barriers can be broken, the repetitive work of medical models can be completely avoided.

"this is not something we can do alone," said pi jingtao.

daily economic news