A new framework argues that hospitals’ rapid deployment of large language models is outpacing safety guardrails, with benchmarking that emphasizes aggregate accuracy potentially masking critical clinical risks. The work highlights how medical language models can fail in subtle, high-impact ways when drafting documentation and supporting time-critical decision-making. The report’s thrust is practical: it calls for evaluation approaches that expose hidden failure modes in medical language workflows—an issue now central to how healthcare systems operationalize AI tools, credentialing, and risk management.