Photo credit: www.sciencedaily.com
Researchers at the University at Buffalo have developed an advanced clinical artificial intelligence tool that has shown exceptional performance on all segments of the United States Medical Licensing Exam (USMLE), as detailed in a study published recently in JAMA Network Open.
This tool, Semantic Clinical Artificial Intelligence (SCAI, pronounced “Sky”), has outperformed both most human physicians and other AI systems in achieving high scores on the USMLE. According to lead researcher Peter L. Elkin, MD, who serves as chair of the Department of Biomedical Informatics at UB, SCAI promises to be a valuable ally in medical practice.
Elkin notes that SCAI stands out as the most precise clinical AI tool to date, achieving an impressive score of 95.2% on the final segment of the USMLE, as compared to a score of 90.5% by the GPT4 Omni tool.
“As healthcare professionals, we’re accustomed to using computers as instrumental aids,” Elkin states. “However, SCAI transcends this role; it actively contributes to clinical decision-making based on its reasoning capabilities.”
Available for inquiries from medical professionals and the public alike, SCAI can be engaged with through this link.
The researchers evaluated SCAI’s performance based on the USMLE, a fundamental requirement for medical licensing across the United States, which assesses candidates’ abilities to apply essential medical knowledge and patient-centered skills. Notably, questions with visual elements were omitted from the test.
Elkin observes that while many AI systems analyze statistical correlations in online data to generate responses, SCAI represents a leap forward. “These systems are often labeled generative AI,” he remarks. “Some critics argue that they merely replicate existing internet content. However, models like SCAI are evolving to function as collaborative partners in clinical environments, going beyond serving as basic tools.”
“SCAI can tackle intricate queries and engage in deep semantic reasoning,” he elaborates. “We’ve constructed knowledge bases that mirror the reasoning processes cultivated during medical training.”
The development of SCAI was built upon a previous natural language processing tool, enhanced with expansive, authoritative clinical knowledge sourced from a variety of reputable outlets, including contemporary medical literature, clinical guidelines, genomic data, and insights into drug interactions—all while excluding potentially biased information such as clinical notes.
13 million medical facts
Containing an extensive repository of 13 million clinical facts, SCAI also accounts for the myriad interactions among these facts. The research team formed semantic networks using relational data (semantic triples such as “Penicillin treats pneumococcal pneumonia”) to enable logical deductions from these networks.
“We have equipped large language models with the ability to execute semantic reasoning,” Elkin explains.
Techniques utilized in SCAI’s design also include knowledge graphs to uncover new correlations in medical data and retrieval-augmented generation, which allows real-time access to external knowledge bases, mitigating the risk of “confabulation,” where AI systems may respond inaccurately due to insufficient information.
Elkin emphasizes that leveraging formal semantics enriches the context, allowing SCAI to provide precise responses to specific inquiries.
‘It can have a conversation with you’
“What sets SCAI apart from other large language models is its capacity for interactive dialogue; it fosters a human-computer partnership that enhances decision-making through its reasoning capabilities,” Elkin states.
Given its extensive access to medical data, SCAI has the potential to bolster patient safety, expand care accessibility, and “democratize specialty healthcare,” Elkin states, enabling primary care providers and patients to access information regarding various medical specialties more readily.
Despite SCAI’s robust capabilities, Elkin emphasizes its intended role to support rather than replace healthcare providers. “AI will not supplant physicians,” he asserts, “but a physician who embraces AI may outpace one who does not.”
Co-authors on the study from the University at Buffalo include Guresh Mehta, Frank LeHouillier, Melissa Resnick, PhD, Crystal Tomlin, PhD, Skyler Resendez, PhD, and Jiaxing Liu. Additional contributions came from Sarah Mullin, PhD, of Roswell Park Comprehensive Cancer Center, as well as Jonathan R. Nebeker, MD, and Steven H. Brown, MD, from the Department of Veterans Affairs.
This research was supported by funding from the National Institutes of Health and the Department of Veterans Affairs.
Source
www.sciencedaily.com