Photo credit: www.sciencedaily.com
An interdisciplinary research team from the LKS Faculty of Medicine at the University of Hong Kong (HKUMed), the InnoHK Laboratory of Data Discovery for Health (InnoHK D24H), and the London School of Hygiene & Tropical Medicine (LSHTM) has introduced the first artificial intelligence (AI) model specifically designed to accurately classify the stage and risk category of thyroid cancer, achieving an impressive accuracy rate of over 90%. This groundbreaking model from HKUMed is expected to significantly reduce the time clinicians need for pre-consultation preparations by approximately 50%, in line with the HKSAR Government’s goal to integrate AI technology into healthcare. The results of this research are detailed in the journal npj Digital Medicine.
Thyroid cancer ranks among the most common types of cancer in both Hong Kong and worldwide. Effective management of this disease typically depends on two critical systems: (1) the 8th edition of the American Joint Committee on Cancer (AJCC) staging system for determining cancer stages, and (2) the risk classification system established by the American Thyroid Association (ATA) to evaluate cancer risk. These systems play a vital role in predicting patient survival rates and aiding treatment decisions. However, the traditional method of manually integrating complex clinical data into these frameworks can be both time-consuming and inefficient.
The research team has developed an AI assistant that utilizes advanced language models (LLMs) such as ChatGPT and DeepSeek, which are tailored to comprehend and process human language, in order to analyze clinical documents and enhance the precision and efficiency of thyroid cancer staging and risk assessments.
This innovative model harnesses four offline open-source LLMs — Mistral (Mistral AI), Llama (Meta), Gemma (Google), and Qwen (Alibaba) — for the examination of free-text clinical documents. It was trained using a U.S.-based open-access dataset that includes pathology reports from 50 thyroid cancer patients sourced from The Cancer Genome Atlas Programme (TCGA), followed by validation against reports from 289 TCGA patients along with 35 pseudo cases generated by endocrine surgeons.
By synthesizing outputs from all four LLMs, the team has greatly enhanced the model’s performance, achieving accuracy rates ranging from 88.5% to 100% for ATA risk classifications and between 92.9% and 98.1% for AJCC cancer staging. This represents a significant improvement over conventional manual document reviews and is anticipated to halve the time required by clinicians for pre-consultation tasks.
Professor Joseph T Wu, Sir Kotewall Professor in Public Health and Managing Director of InnoHK D24H at HKUMed, highlighted the impressive capabilities of the model. “Our model achieves more than 90% accuracy in classifying AJCC cancer stages and ATA risk categories,” he noted. “One of the primary advantages of this model is its ability to function offline, allowing for local deployment without the need to share sensitive patient data, thereby ensuring maximum privacy for patients.”
In light of the recent introduction of DeepSeek, further comparative assessments were conducted using a “zero-shot approach” against the latest versions of DeepSeek — R1 and V3 — as well as GPT-4o. “We were pleased to discover that our model delivers comparable performance to these advanced online LLMs,” added Professor Wu.
Dr Matrix Fung Man-him, Clinical Assistant Professor and Chief of Endocrine Surgery at HKUMed, remarked, “Beyond providing high accuracy in extracting and analyzing data from intricate pathology reports, surgical records, and clinical notes, our AI model substantially minimizes preparation time for medical professionals — nearly halving it compared to human interpretation. It simultaneously offers cancer staging and clinical risk stratification based on two internationally recognized clinical systems.”
“The AI model is flexible and can be seamlessly integrated into various environments across both the public and private sectors, as well as in both local and international healthcare and research institutions,” Dr Fung commented. “We are optimistic that implementing this AI solution in real-world settings could enhance clinicians’ operational efficiency and improve patient care quality, allowing more time for direct patient interactions.”
“In alignment with the government’s robust support for AI implementation in healthcare, demonstrated by the recent launch of an LLM-based medical report writing system in the Hospital Authority, our next step involves evaluating the performance of this AI assistant using a substantial amount of real-world patient data. Once validated, the AI model can be effectively deployed within clinical environments to assist clinicians in improving treatment efficiency,” explained Dr Carlos Wong, Honorary Associate Professor in the Department of Family Medicine and Primary Care at HKUMed.
This study was led by Professor Joseph Wu Tsz-kei, Sir Robert Kotewall Professor in Public Health at the School of Public Health and Managing Director & Lead Scientist of InnoHK D24H; Dr Matrix Fung Man-him, Clinical Assistant Professor and Chief of Endocrine Surgery in the Department of Surgery; and Dr Carlos Wong King-ho, Honorary Associate Professor in the Department of Family Medicine and Primary Care and Senior Research Director at InnoHK D24H, all affiliated with HKUMed. The leading authors included Dr Eric Tang Ho-man and Dr Tingting Wu from InnoHK D24H.
Source
www.sciencedaily.com