KPSGILL Institute for AI & the Practice of Medicine · flagship programme
AI and the practice of medicine
Four questions decide everything practical: what is the software actually doing, what remains nondelegable, who answers when it is wrong, and what does the record show a year later. This is the framework, and the drafted language to implement it.
Why this is the flagship
State boards are being asked to police AI use with a disciplinary vocabulary built for human conduct, while a federation-level workgroup develops model guidance and payment and liability rules move independently of both. The first contested cases will otherwise set the standard by accident.
The question that decides every practical dispute is not whether a tool is cleared. It is what remains nondelegable, and who answers when the machine is wrong. Everything below is organised around those two questions.
A risk-tiered physician responsibility framework
One review duty for all AI is either so heavy it is ignored or so light it is meaningless. Responsibility should scale with what the system is doing to the clinical decision, not with what the vendor calls it.
What it does. A reference lookup, a calculator, a literature summary. The physician is the only decision-maker.
What the physician owes. Ordinary professional care in using a reference. No separate review duty arises.
What it does. A risk score, a flag, a suggested differential presented for consideration.
What the physician owes. The physician must be able to state why she agreed or disagreed. Documentation of the reasoning, not of the tool.
What it does. Ambient scribes, generated assessments, discharge instructions, referral summaries, patient messages.
What the physician owes. Review of clinically material content before adoption, with provenance carried by the system rather than attested by the clinician.
What it does. Autonomous triage, autogenerated messages sent without review, automated coding or problem-list changes, closed-loop adjustment.
What the physician owes. A named accountable physician, prospective governance, and an auditable record of every action taken without contemporaneous review. Some functions should not be delegable at all.
The tier is set by function, not by marketing category or regulatory pathway. A cleared device can sit in Tier 2 and an uncleared workflow feature in Tier 3 — which is precisely the gap a physician is currently left to absorb.
The generated record
This is the area that will grow fastest and is least governed: ambient scribes, generated histories, autogenerated assessments and plans, machine differentials, discharge instructions, automated coding, problem-list updates, patient messages, medication reconciliation, referral summaries. In most systems today all of it enters the chart as the physician’s own words.
The proposal is narrow and does two things: it makes provenance a property of the record, and it moves physician responsibility from authorship to clinically material adoption. Read the model standard, with the drafted language.
Liability should follow control
KPSGILL REFORM POSITIONThe default allocation puts the entire loss on the physician, who controls the least of the system. She did not design the model, choose the training data, write the warnings, negotiate the contract or configure the workflow. Everyone with more control has less exposure. That is not a moral observation; it is a design failure that also makes the system less safe, because the parties who could prevent the failure carry none of its cost.
| Party | What it actually controls | What it should answer for |
|---|---|---|
| Developer | Architecture, training data, known limitations, model updates | Failures of design, of representativeness, of undisclosed limitation, and of silent update |
| Vendor | Commercialisation, performance claims, warnings, monitoring, support | Failures of claim, of warning, of post-market monitoring, and of notification when the model changes |
| Health system | Procurement, integration, workflow, configuration, governance | Failures of deployment: no governance, no fallback, workflow that makes review impossible in the time allowed |
| Physician | The clinical decision actually adopted, within realistic conditions of practice | Failure to exercise judgement on clinically material content she adopted |
| Payer | Coverage rules and utilisation criteria that shape what the tool is used for | Failures where an algorithmic coverage rule, not clinical judgement, determined the outcome |
The strongest arguments against
- Apportionment invites endless third-party litigation.
- Physicians will over-rely on tools if their exposure narrows.
- Vendors will exit the market or price the risk into contracts.
Answers
- Apportionment already happens in product liability; medicine is the outlier in refusing it.
- Exposure that no available action can avoid deters nothing — it only allocates loss after the fact.
- A vendor unwilling to stand behind its own performance claims has told the buyer something useful.
Model board guidance: when AI use becomes a disciplinary question
MODEL MEDICAL BOARD GUIDANCEBoards should publish this before enforcement, not in response to their first case. Guidance that names the duty rather than the technology survives the technology.
1. Scope. This guidance addresses a licensee’s use of software that generates, drafts or recommends clinical content. It does not endorse or discourage any product.
2. Nondelegable functions. A licensee shall not delegate to software the final determination of a diagnosis, the selection of a treatment, or the decision to withhold treatment. Software may inform each of those determinations.
3. Review proportionate to function. The review a licensee owes is proportionate to the software’s function, as follows … [tiers 1–4 above].
4. Adoption, not authorship. A licensee is responsible for clinically material content she adopts into the record. A licensee shall not be required to attest that she personally composed text generated by software, and shall not be subject to discipline for the absence of such an attestation.
5. Documentation. Where software drafted clinical content, the record shall identify that fact and the system and version that produced it. The licensee’s obligation is to review; the institution’s obligation is to make that fact recoverable.
6. Institutional conditions. In any proceeding, the board shall consider whether the licensee had a realistic opportunity to review the content in the conditions of practice imposed by her institution.
7. What this guidance does not do. It does not create a duty to use software, and non-use of an available tool is not, of itself, a departure from the standard of care.
The strongest arguments against
- Guidance issued early will be obsolete within a year.
- Boards should not endorse or discourage particular technologies.
- Section 6 imports employment disputes into licensing.
Answers
- Guidance that names duties rather than products survives product cycles; sections 2 and 4 would have been true five years ago.
- This names no product. It names functions.
- Section 6 does not adjudicate the employment relationship; it prevents a board from punishing a physician for a constraint she did not set.
Related record: two California AI bills are unresolved at the end of session, and who controls clinical judgment sets out the documentary baseline.
Open questions this institute has not answered
- Should provenance be visible to the patient in the portal, or recoverable only on audit?
- Does a Tier 4 function ever become acceptable in primary care, and on what evidence?
- Should failure to use an available tool ever be a disciplinary question — and if so, who decides which tools are “available”?
- What counts as realistic conditions of practice, and who establishes them: the board, the institution, or an expert?
These are listed because they are unresolved. A position paper that has no open questions has stopped thinking.