Trust, But Verify!

The question is no longer whether AI can create clinical notes. The question is whether healthcare organizations are effectively monitoring the accuracy, completeness, and compliance of those notes.

New Technology, Familiar Compliance Risks

AI does not create new compliance principles—it creates new avenues for existing compliance risks.

What Could Go Wrong?

• Inaccurate Clinical Facts
• Missing Clinical Context
• Hallucinated Content
• Note Bloat

Hallucinated Content

• Examination findings not performed
• ROS elements not discussed
• Diagnoses not addressed

Example: The Hallucinated Physical Exam

Scenario

A provider uses ambient documentation during a routine follow-up visit. The AI-generated note documents:

“Cardiovascular exam reveals regular rate and rhythm. No lower extremity edema noted.”

During provider discussion, no cardiovascular examination was discussed or performed.

Audit Finding

The note contains examination findings unsupported by the encounter.

Compliance Risk

• Documentation integrity concerns
• Potential support for a higher level of service than warranted
• Provider attestation to inaccurate information

Auditor Question

Would the provider still have documented these findings if AI had not generated them?

Note Bloat

There’s a big difference between comprehensive and excessive documentation.

• Longer notes
• More narrative
• Additional wording

Example: The Contradictory Narrative

Scenario

The HPI states:

“Patient reports worsening shortness of breath over the last two weeks.”

The generated assessment states:

“Symptoms improving.”

Audit Finding

Contradictory information exists within the same note.

Compliance Risk

• Documentation reliability concerns
• Potential patient safety implications
• Increased audit vulnerability

Auditor Question

Was the note reviewed in its entirety prior to signature?

What Should Organizations Audit?

Audit Area #1

Documentation Accuracy

Questions:

• Did the encounter occur as documented?
• Is information clinically accurate?
• Does the note reflect the actual discussion?

Audit Area #2

Documentation Integrity

Questions:

• Does documentation appear cloned?
• Are repetitive patterns present?
• Are findings individualized?

Audit Area #3

Coding Support

Questions:

• Does MDM support code selection?
• Are diagnoses assessed and addressed?
• Is medical necessity evident?

Audit Area #4

Provider Oversight

Questions:

• Were edits performed?
• Is evidence of provider review present?
• Are providers signing notes without modification?

Red Flag Dashboard

Five AI Documentation Audit Red Flags

1. Notes substantially longer than historical patterns
2. Identical language across encounters
3. Findings documented but not discussed
4. Inconsistent documentation within the note
5. Increased high-level E/M coding distribution

Ambient documentation has the potential to improve efficiency, reduce administrative burden, and enhance the provider experience. However, successful implementation requires more than deploying new technology. It requires governance, oversight, and ongoing monitoring. As healthcare organizations increasingly rely on AI-generated documentation, compliance programs must evolve accordingly.

The goal is not to audit the technology itself, but to ensure that the medical record remains accurate, trustworthy, and capable of supporting the care delivered and services billed. In the end, the most effective AI governance strategy may be the same principle that has guided healthcare documentation for decades: trust, but verify.

Stay tuned – just as physicians and other professionals are embracing AI in their day-to-day operations, so is our government. We will dive into the new CMS Fraud initiative, CRUSH, in our next article. Coming soon.