Large language models (LLMs) are approaching expert-level knowledge and reasoning skills in ophthalmology, according to a study published online April 17 in PLOS Digital Health.
Arun James Thirunavukarasu, M.B., B.Chir., from University of Oxford in the United Kingdom, and colleagues evaluated the clinical potential of state-of-the-art LLMs in ophthalmology. Responses to 87 questions were compared for GPT-3.5, GPT-4, PaLM 2, LLaMA, expert ophthalmologists, and doctors in training.
The researchers found that the performance of GPT-4 (69 percent) was superior to performance of GPT-3.5 (48 percent), LLaMA (32 percent), and PaLM 2 (56 percent) and compared favorably with expert ophthalmologists (median, 76 percent), ophthalmology trainees (median, 59 percent), and unspecialized junior doctors (median, 43 percent). Low agreement between LLMs and doctors was due to idiosyncratic differences in knowledge and reasoning, with overall consistency across individuals and type. Grading ophthalmologists preferred GPT-4 responses over GPT-3.5 due to higher accuracy and relevance.
“LLMs are approaching expert-level ophthalmological knowledge and reasoning, and may be useful for providing eye-related advice where access to health care professionals is limited,” the authors write. “Further research is required to explore potential avenues of clinical deployment.”
One author disclosed a patent on a deep learning system to detect retinal disease.
More information:
Arun James Thirunavukarasu et al, Large language models approach expert-level clinical knowledge and reasoning in ophthalmology: A head-to-head cross-sectional study, PLOS Digital Health (2024). DOI: 10.1371/journal.pdig.0000341
Copyright © 2024 HealthDay. All rights reserved.
Citation:
AI shows good clinical knowledge, reasoning for eye issues (2024, April 29)
retrieved 29 April 2024
from https://medicalxpress.com/news/2024-04-ai-good-clinical-knowledge-eye.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
part may be reproduced without the written permission. The content is provided for information purposes only.
>>> Read full article>>>
Copyright for syndicated content belongs to the linked Source : Medical Xpress – https://medicalxpress.com/news/2024-04-ai-good-clinical-knowledge-eye.html