Objective:

To evaluate the safety and efficacy of an AI workflow for drafting hospital course summaries in discharge documentation, focusing on potential harm and efficiency.

Key Findings:

• 88% of reviewed summaries were rated as having no potential for harm, indicating a positive safety profile.
• One summary was rated as likely to cause moderate harm but was later deemed safe after independent review.
• Omissions were reported in 25% of summaries, inaccuracies in 20%, and hallucinations in 2%, underscoring areas for improvement.
• Physician burnout scores decreased after implementation, suggesting potential benefits for clinician well-being.
• 67% of physicians reported perceived time savings, with 32% estimating over 15 minutes saved per summary, indicating efficiency gains.

Interpretation:

The AI system effectively reduced the risk of harm in discharge summaries, though addressing significant omissions remains a critical concern.

Limitations:

• The single-unit academic setting limits generalizability of findings.
• The small number of participating physicians and voluntary feedback may introduce bias.
• Lack of subgroup analyses and a contemporaneous control group limits the robustness of conclusions.
• Absence of systematic assessment of error rates in physician-authored summaries complicates the evaluation of the AI system's impact.

Conclusion:

While the AI workflow mitigated hallucinations, addressing omissions will require better integration with structured EHR data and alignment with physician judgment to enhance safety and efficacy.