KPSGILL policy proposal · model legislation and model regulation
Model AI Medical Record Integrity Standard
A physician should not have to attest that she wrote words a model generated — and when the model is wrong, the record should be able to say so. Two provisions, drafted.
The problem
A physician signs a note attesting that it reflects her evaluation. Increasingly, a model wrote most of the words: the history, the assessment wording, the discharge instructions, the referral summary, sometimes the message that goes to the patient. The attestation is unchanged from the era when she typed it herself, so the record now asserts something that is not true — and asserts it in the one place where traceability matters most.
The consequence is not abstract. When a note is later read by another clinician, a board, or a court, nothing in it separates what the physician decided from what a model generated. If the model was wrong, she is the only actor the record can identify. If the vendor changed the model, no one can tell which notes were affected. If she edited the draft heavily, that work is invisible.
Documentary baseline: the FDA’s lifecycle framing and predetermined change-control plans for AI-enabled devices; NHS England’s ambient-scribe programme at regional scale; the TGA’s intended-purpose regulation and its treatment of scope creep; and the pending California bill that would require a clinical decision support inventory. Each is a Layer 1 record on this site with its own primary source.
The recommendation
Two moves, and no more than two. Make provenance a property of the record rather than a claim by the clinician; and relocate physician responsibility from authorship to clinically material adoption.
Provenance
AI-generated text carries machine-readable provenance at the point of generation, including system and version identity, recoverable for the life of the record.
Attestation
No clinician is required to attest that she personally composed text generated by software.
Responsibility
Attaches to clinically material content she adopted — the decision she actually made.
Traceability
A withdrawn or corrected model version can be traced to every record it touched.
Notification
A material AI safety change triggers vendor notification to deploying institutions.
Retention
Technical audit logs are retained separately from the clinical record, under their own rule.
The drafted language
§ 1. Definitions. (a) “Generative clinical software” means software that composes, drafts or summarises clinical content for inclusion in a patient record. (b) “Clinically material” means content that a reasonable clinician would rely upon in evaluating, diagnosing, treating or communicating with the patient.
§ 2. Provenance. A health facility or provider organisation that deploys generative clinical software shall ensure that content it produces is recorded with machine-readable provenance identifying (1) that the content was machine-generated, (2) the system and version that generated it, and (3) the date and time of generation. Provenance shall be recoverable for the retention period of the record.
§ 3. Edit integrity. Where a licensee materially edits machine-generated content before adoption, the record shall preserve the fact and time of the edit. Nothing in this section requires the retention of a clinician’s draft reasoning.
§ 4. Attestation. No health facility, provider organisation, payer or certification body shall require a licensee to attest that she personally composed content generated by software. A licensee’s attestation shall be understood to affirm review and adoption of clinically material content.
§ 5. Traceability of model change. A vendor of generative clinical software shall notify each deploying institution of a material change affecting clinical safety, and shall provide the information necessary to identify records generated by the affected version.
§ 6. Separation of logs. Technical audit logs maintained under this article are not part of the clinical record for the purposes of patient access, and are retained under the schedule prescribed by the department.
§ 7. Construction. This article does not require the use of generative clinical software, does not authorise its use for any function otherwise prohibited, and does not diminish a licensee’s existing duty of care.
The companion regulatory provision would place §§ 2 and 5 in certification criteria rather than in a facility duty, so the obligation falls on the party that can discharge it. Both forms are drafted because the right vehicle depends on the jurisdiction.
Who bears what
Physician
Attestation becomes truthful and narrower. Exposure shifts from “you signed it” to “you adopted it” — which is the decision she made.
Patient
A later clinician can tell which parts of the record were reasoned and which were generated.
Payer
Coding derived from generated text becomes auditable to its source.
Competition
Provenance is a standard, not a feature. Making it a condition of certification prevents it from becoming a premium-tier product.
Equity
Under-resourced practices automate most and negotiate vendor terms least. A statutory floor is worth more to them than to anyone else.
Burden
Written by software, not by clinicians. It falls on vendors and certification, deliberately.
The strongest arguments against
- Provenance metadata will be used against physicians in discovery.
- Vendors will claim trade-secret protection over model identity.
- EHR certification is federal; a state standard invites a preemption fight.
- “Clinically material” is indeterminate.
- This will slow adoption of tools that reduce burnout.
Answers
- The alternative is worse. Without provenance the physician is the only traceable actor, so every failure lands on her by default. § 3 is deliberately narrow for this reason: the fact of an edit, not the content of her thinking.
- Model and version identity is not a trade secret in any sense the law protects. It is a serial number.
- Federal certification sets a floor, not a ceiling, and state law has always regulated medical-record content. § 2 is drafted as a record-content duty for exactly that reason.
- It is the ordinary standard of clinical judgement, and no more indeterminate here than in informed consent, where it has worked for decades.
- Nothing here restricts a function or requires a review step that is not already implied by the attestation. It removes a false attestation and adds metadata that software writes.
Metrics, sunset, and what is still unresolved
Metrics. Proportion of notes carrying machine-readable provenance; time to identify all records affected by a withdrawn model version; number of board cases in which authorship rather than judgement was the contested issue.
Sunset. Five-year review against certification practice. The article sunsets if federal certification adopts an equivalent or stronger provenance rule — the point is the standard, not the statute.
Open questions. Should provenance be visible to the patient in the portal, or recoverable only on audit? Does the standard reach autogenerated patient messages sent without clinician review — and if it does, is provenance sufficient, or is review required?
Related: the risk-tiered responsibility framework this standard sits inside · liability follows control · the two California AI bills in the documentary record.