Clinical consulting · Clinical AI
Clinical Review of AI Documentation and Workflows
Clinical AI should be judged by the medical work it supports. Dr. Gill reviews whether an AI-assisted note preserves the meaning of the encounter, and whether clinicians can catch and correct a problem before it affects care.
- Why AI-drafted notes are hard to review
- Two lanes of work
- Why a physician who uses ambient AI every clinic day
- Failure modes the review looks for
- What a signature on an AI-drafted note means
- Recording and patient communications, as context
- Key questions
- Materials typically examined
- What the client receives
- What a small scenario test cannot establish
- Scope
- Fictional illustrations
- How this connects to other services
Why AI-drafted notes are hard to review
An ambient-AI documentation tool listens to a visit, drafts a note, and hands it to the clinician to edit and sign. The finished note looks like any other. In many records it is all that survives: the audio may not be retained, the draft may be overwritten by the signed version, and the product version and configuration in use that day may be unknown to anyone reading the chart later.
The errors that matter are seldom conspicuous. A note can be readable, concise and wrong in a way that changes the patient's story: a possible diagnosis stated as established, a caregiver's medication placed on the patient's list, an option discussed for later recorded as treatment started. Much of the vocabulary is correct; the clinical meaning has shifted.
The opposite mistake is also common. An accurate summary need not reproduce the encounter word for word, and a clinician may properly add examination findings, prior results or judgment that were never spoken aloud. A transcript can itself omit or misattribute speech. The review therefore compares the full authorized source material with the final record and the clinician's reasoning, rather than treating every difference as an error.
Two lanes of work
Record review of AI-assisted notes
For counsel on either side, a review of disputed notes in an actual chart, examining whether specific assertions are supported and what any discrepancy meant clinically. Dr. Gill considers these assignments as part of his clinical documentation review.
Organizational evaluations
For healthcare organizations, developers or counsel, scenario sets, annotated comparisons and workflow reviews. Each is scoped individually around a defined question, source packet and work product.
Lane one: questions about a disputed chart
The question is usually narrow and concrete. Did the patient actually decline imaging, or did the draft compress a longer conversation into a refusal? Was the examination finding performed, or supplied by the draft? When the clinician corrected the plan in the note, did the problem list and medication list follow? The review identifies what the record can and cannot establish about how the note was produced, and what further material, such as retained drafts, audio, edit histories or the organization's workflow documentation, could resolve the question.
Lane two: evaluations before or after deployment
An agreed evaluation compares a defined source packet, the generated output, the clinician's changes and the final record, using authorized synthetic scenarios or lawfully provided material. Each material discrepancy is described with its supporting evidence, potential clinical significance, correction pathway and remaining uncertainty. The report states the known product version, configuration, review date, use setting and any unavailable information. A workflow review can examine whether clinicians can see the material needed to verify an assertion, how corrections are made before and after signature, whether a correction reaches linked lists and orders, and what happens during downtime.
Why a physician who uses ambient AI every clinic day
Dr. Gill uses an enterprise EHR with ambient-AI drafting in his daily clinical documentation. He reviews, edits and signs AI-drafted notes as part of ordinary patient care, so he has first-hand experience, with the product and configuration he uses, of where drafts tend to be reliable and where they drift. He has also published long-form analysis of the subject on this site: Who Owns the Decision? and the Healthcare AI dossier, which includes his analysis of ambient AI documentation. Published analysis is identified as such; it informs his questions, and it does not substitute for engineering expertise or for the evidence in a particular record.
Failure modes the review looks for
| Failure mode | What it looks like | Why it matters clinically |
|---|---|---|
| Certainty inflation | "Possible asthma" becomes "asthma" | Enters the problem list and shapes later treatment and coverage |
| Attribution error | A caregiver's symptoms or medication recorded as the patient's | Wrong history, wrong medication list |
| Negation loss or reversal | "No chest pain" lost, or "not discussed" written as "denies" | Alters how later readers judge risk |
| Plan-to-action conversion | "Start antibiotics if fever returns" becomes "antibiotics started" | Later clinicians assume treatment occurred |
| Omitted condition | A recommendation recorded without the condition attached to it | Changes when and whether follow-up happens |
| Unsupported examination finding | Examination elements documented that were not performed | False reassurance about findings never checked |
| Medication status error | A stopped drug listed as active, or a dose change lost | Continuity and reconciliation errors downstream |
| Correction not propagating | The note is fixed, but the problem list, order or summary is not | The error persists where others will rely on it |
What a signature on an AI-drafted note means
A signature may attest that the clinician reviewed the draft, or may be applied in a workflow that encourages rapid approval. The review examines the attestation language, the organization's rules for AI-assisted documentation, the time available for review where it can be determined, and whether the clinician's edits show engagement with the content. California enactments concern authorship and alteration of physician patient notes (AB 1637, Stats. 2026, ch. 536, approved September 27, 2026, adding Business and Professions Code §2266.1, effective January 1, 2027; not specific to AI) and, as reported in October 2026, clinical decision-support AI and independent clinical judgment (AB 1979, reported effective January 1, 2027). Their text and application to a given event should be confirmed by counsel.
Recording and patient communications, as context
Ambient tools record conversations. California Penal Code §632 addresses consent of all parties to recording a confidential communication, so how consent was obtained and documented may matter. Separately, Health and Safety Code §1339.75 (AB 3030, effective January 1, 2025) requires a disclaimer and instructions for reaching a human when generative AI produces patient communications about clinical information, unless a licensed provider reads and reviews them; it concerns patient communications, such as portal replies, rather than clinician notes generally. Dr. Gill can describe the clinical workflow and what the record shows. Whether a legal requirement was met is for counsel.
Key questions
Is the assertion supported?
What does the final record claim, which source supports it, and how strong is that support?
Did meaning change?
Did certainty, attribution, negation, timing or treatment intent shift between source and note?
Did it matter?
What clinical decision could depend on the difference?
Could the clinician see it?
Did the workflow give the clinician what was needed to verify the draft before signing?
Did the correction travel?
When an error was fixed, did linked lists, orders and summaries change too?
What is unknowable?
Which questions cannot be answered because audio, drafts or configuration were not retained?
Materials typically examined
- The final signed note, with any retained drafts, edit histories and addenda
- Audio or transcripts, where retained and lawfully produced
- Problem, medication and order records linked to the encounter
- The organization's AI-documentation policies, consent workflow and attestation language
- Product version and configuration information, where available
- For evaluations: the agreed source packet, generated outputs and clinician edits
What the client receives
Annotated comparison
Each clinically relevant difference, classified and explained, with its support and significance.
Record-review memorandum
For a disputed chart, what the record establishes about a contested assertion and what remains unknown.
Scenario set
Realistic fictional encounters built to probe specific failure modes in a supervised evaluation.
Workflow findings
Observed problems, the examples supporting them, and practical questions for implementation or further testing.
What a small scenario test cannot establish
Limits stated in every evaluation
A scenario exercise reports what happened in its own cases. It cannot establish an overall error rate, certify that a product is safe, or predict performance in another setting or version. Broader claims would need a suitable sampling design, defined comparison criteria, sufficient cases and quantitative expertise. NIST's Generative AI Profile (NIST AI 600-1) is a useful voluntary framework for risks such as confabulation and human-AI configuration; it is not a certification standard, and citing it does not validate a test.
Scope
Within Dr. Gill's scope
- Clinical accuracy, support and meaning of AI-assisted notes
- Clinical significance of discrepancies between source, draft and final record
- Clinician review, correction and attestation workflows
- Design of clinically realistic fictional scenarios
- Clinical questions for implementation and procurement
Complemented by another discipline
- Software architecture, model training and defect analysis
- Audit-log, metadata and audio forensics
- Quantitative validation and population-level error rates
- Regulatory classification and cybersecurity
- Legal sufficiency of consent and disclosures
Fictional illustrations
Fictional illustration
A refusal the record supported. Suppose a plaintiff alleges that an AI draft invented "patient declined CT." Documented: the retained draft contained the phrase; the clinician's edit added the reason given; a portal message from the patient the next day said she preferred to wait. A review would report that the assertion is independently corroborated and that the draft, in this instance, preserved the meaning of the encounter.
Fictional illustration
Treatment that never started. A note read "started amoxicillin." Documented: no prescription, no pharmacy fill, and a patient message asking when to begin "if it gets worse." Unknown: the original audio, which was not retained. A memorandum would identify a probable plan-to-action conversion, label it as an inference, and show that a later clinician relied on the note.
Fictional illustration
A twelve-scenario evaluation. Suppose a clinic tests twelve fictional encounters. Two drafts lose a negation and one misattributes a caregiver's statement; clinicians catch two of the three before signing. A report would describe those findings, state that twelve cases cannot establish a rate, and could suggest a change to how corrections reach the problem list.
How this connects to other services
- Clinical Documentation and Record Consistency, the broader record-review service
- Clinical Governance and Care Delivery Review, for decision authority and technology transitions
- Further reading: A Medical Note Can Change Meaning Without Changing Much Text; a fictional AI documentation comparison
Sources and scope
- NIST AI 600-1, Generative AI Profile (July 2024): a voluntary, cross-sector framework, not a finding about any product or a certification method.
- Penal Code §632: consent of all parties to recording a confidential communication.
- Health and Safety Code §1339.75 (AB 3030, effective January 1, 2025): generative-AI patient communications about clinical information.
- California Health Care Foundation, October 2026: secondary-source report of AB 1979 (2026). Confirm enacted text.
- AB 1637 (Stats. 2026, ch. 536), adding Business and Professions Code §2266.1 on authorship and alteration of physician patient notes; approved September 27, 2026; no urgency clause, so it takes effect January 1, 2027 (chaptered text, California Legislative Information).
- ONC SAFER Guides (2016 edition for events before 2025; 2025 edition thereafter): EHR safety practices, including contingency planning and clinician communication.
Questions counsel often ask
The note was drafted by an AI scribe. Who is responsible for its content?
Clinically, the signing clinician attests to the note under the organization's workflow, and the review examines what that attestation meant in practice. How legal responsibility is allocated among the clinician, the organization and any developer is a question for counsel and the tribunal. California enactments reported in 2026 may bear on later events; counsel should confirm their text and effective dates.
Can you test the product that drafted the note in our case?
Usually not in a way that reproduces the original output. Products change, configurations vary, and the original audio and draft may not have been retained. Dr. Gill can review the record that exists for clinical meaning and support. A separate scenario evaluation can be scoped, but its results describe the tested cases, not the historical encounter.
Was the patient's consent to recording required?
That is a legal question. California Penal Code §632 addresses consent of all parties to recording a confidential communication, which makes consent relevant to ambient recording. Dr. Gill can describe how consent was documented in the clinical workflow and what the record shows, but not whether it satisfied the law.
You use ambient AI in your own practice. Does that bias you?
It gives him working familiarity with how drafts are generated, edited and signed, which is the subject of the review. He has also published long-form analysis of AI documentation. Neither substitutes for the evidence in a particular record. Relevant relationships are assessed and disclosed for each engagement, and conclusions may support, limit or challenge a party's position.
Do you consult for AI developers or vendors?
Dr. Gill considers organizational assignments, including from developers, when the question is clinical and within his scope. Commercial relationships are assessed and disclosed for each engagement, and a consulting assignment is not an endorsement. His findings report what the reviewed cases showed, favorable or not.
Have a matter that may fit?
Send a short, non-clinical inquiry naming the parties, forum, dates and question. Conflicts are screened before any records are sent, and every party is offered the same method and terms.
This page describes professional consulting services for attorneys and healthcare organizations. It is not legal advice or medical advice. An inquiry does not create an engagement, an attorney–client relationship or a physician–patient relationship. Opinions are formed only after review of the evidence and may support, limit or contradict the position of the party that requests them. Fees are never contingent on the content of an opinion or the outcome of a matter.