Clinical Report: LLM Explanations Improve Diagnostic Accuracy in Radiology

Overview

A study involving 101 radiologists demonstrated that chain-of-thought explanations from large language models (LLMs) significantly improved diagnostic accuracy compared to standard outputs and no assistance. The findings suggest that structured reasoning enhances clinician decision-making in radiology.

Background

The integration of artificial intelligence in radiology is rapidly evolving, with large language models (LLMs) showing promise in enhancing diagnostic accuracy. Understanding how different prompting techniques affect clinician performance is crucial for optimizing AI tools in clinical settings. This study provides insights into the effectiveness of structured explanations in improving diagnostic outcomes.

Data Highlights

Key Findings

• Chain-of-thought explanations improved diagnostic accuracy by 12 percentage points compared to the control group.
• LLM GPT-4 achieved 80% accuracy with chain-of-thought prompting.
• Differential-diagnosis support did not significantly enhance accuracy compared to no LLM assistance.
• Radiologists were more likely to override incorrect LLM recommendations when using chain-of-thought explanations.
• Findings were consistent across various levels of radiologist experience and subspecialty expertise.

Clinical Implications

The study highlights the importance of explanation format in AI-assisted diagnostic processes. Clinicians may benefit from using LLMs that provide structured reasoning, as this can lead to improved diagnostic accuracy and better decision-making in radiology.

Conclusion

Chain-of-thought explanations from LLMs represent a meaningful advancement in AI-assisted radiology, enhancing diagnostic performance and supporting clinician reasoning. Further research is needed to evaluate these findings in routine clinical practice.

References

1. Author(s)/Org, npj Digital Medicine, 2023 -- The effect of medical explanations from large language models on diagnostic accuracy in radiology
2. European Radiology, 2023 -- The Current Landscape and Future Perspectives of Cardiac Radiology in Europe
3. European Radiology, 2026 -- Simplifying radiology reports with large language models: privacy-compliant open- versus closed-weight models
4. npj Digital Medicine, 2025 -- Enhancing Diagnostic Accuracy in Radiology: Insights from a Large Reasoning Model Compared to Traditional Approaches
5. ACR, 2026 -- ACR Approves First Practice Parameter for Imaging Artificial Intelligence
6. asco ai in oncology — AI Simplifies Patients' Comprehension of CT Reports—But Errors Are Possible
7. The effect of medical explanations from large language models on diagnostic accuracy in radiology | npj Digital Medicine
8. ACR Approves First Practice Parameter for Imaging Artificial Intelligence
9. Retrieval-augmented generation elevates local LLM quality in radiology contrast media consultation | npj Digital Medicine