An audit of an AI oral health screening prototype found that benchmark performance did not translate to safe deployment. While benchmark duplication did not change aggregate scores, label errors, software defects, and pipeline failures distorted what users actually saw during real-world testing. The report underscores a gap between offline evaluation and operational behavior in clinical tools, where data pipelines and human-facing outputs can introduce failure modes even when model metrics appear stable. For product teams, the audit highlights the need to validate end-to-end systems—including labeling provenance, software releases, and inference flows—rather than relying only on dataset-level benchmarks.