LLMs, zero data retention, and HIPAA

Zero data retention (ZDR) is a configuration in which an AI provider does not persist your prompts and outputs after serving the request. It is a genuine risk-reduction control, and vendors frequently require it before they will handle regulated data, but it is not HIPAA compliance, it is not a substitute for a business associate agreement, and in at least one major provider’s current terms it is not even uniformly compatible with the services their BAA covers. This page explains the framework. Whether a specific AI deployment is permissible with your PHI is a question for privacy counsel, and the answer depends on your configuration, your contracts, and your state. Vendor terms in particular change frequently — verify against the vendor’s current documentation and your own executed agreement, not against this page. Checked August 2026.

​

What ZDR actually is

By default, most AI providers retain API inputs and outputs for a period, commonly around 30 days, for abuse monitoring, trust and safety review, and debugging. ZDR turns that logging off: the data exists transiently while the request is processed and is not written to durable storage afterward. That is the whole of it. It is a statement about one vendor’s storage behavior, on one configuration, for the services it covers.

​

Three things ZDR is constantly confused with

Also distinct, and worth checking separately: data residency (where processing happens), subprocessors (who else touches it), and retention of derived artifacts like embeddings, caches, and safety classifications.

​

Why vendors treat it as a precondition

The chain is straightforward. If PHI goes to an AI provider, that provider is creating, receiving, maintaining, or transmitting PHI on your behalf, which makes them a business associate under 45 C.F.R. § 160.103, and requires a BAA.¹ Providers generally do not want PHI sitting in general-purpose abuse-monitoring logs, because that puts regulated data into systems designed for a different purpose, reviewed by staff with a different mandate. Requiring ZDR shrinks the surface: less PHI at rest, a smaller breach blast radius, fewer systems in scope for a records request, and a simpler subprocessor story. So in practice, ZDR frequently shows up as a condition the vendor imposes before it will sign a BAA or let you send regulated data. That is a real and sensible pattern, and it is the sense in which ZDR is a prerequisite.

​

But the relationship is messier than “prerequisite”

Here is where the neat version breaks down, and it is worth knowing before you architect around an assumption. BAA coverage is scoped per service, not per vendor. Providers publish which of their products and endpoints the BAA covers, and the exclusions are frequently substantial. Anthropic, for example, publicly documents that its BAA covers the Messages API and specified Claude Enterprise features, while explicitly excluding its Batch API, Files API, Skills API, Code Execution, Computer Use, and Web Fetch.² And ZDR can point in the opposite direction. In the same documentation, Anthropic states that certain covered models require 30-day data retention and are not available with ZDR enabled, while a different covered service is only covered under the BAA when ZDR is enabled.²

​

Why ZDR is not HIPAA compliance

Even with ZDR on, a signed BAA, and covered services, HIPAA obligations remain, because most of them are obligations on you, not on the vendor.

​

1. Transmission is a disclosure

Sending PHI to a third party is a disclosure under the Privacy Rule regardless of what happens to it afterward. It must be permitted — for treatment, payment, or health care operations, or under an authorization — and it must satisfy minimum necessary. ZDR does not make an impermissible disclosure permissible. It only means the impermissible disclosure wasn’t stored.

​

2. Minimum necessary still applies to the prompt

Pasting an entire chart into a prompt because “it’s ZDR anyway” fails minimum necessary. ZDR governs what happens after the request; minimum necessary governs what you send. Prompt construction is a compliance surface.

​

3. The Security Rule applies to your systems

Access controls, audit logging, workforce training, and the required security risk analysis under 45 C.F.R. § 164.308(a)(1)(ii)(A) all cover your AI integration as an asset that handles ePHI. A vendor’s retention setting does not discharge any of them. OCR proposed the first major update to the HIPAA Security Rule in roughly two decades in January 2025, including removing the required/addressable distinction and tightening expectations around risk management and encryption. Confirm its current status before relying on either the old or the proposed framework — this is exactly the kind of rule whose status changes.³

​

4. Business associates are directly liable

Since the HITECH Act and the 2013 Omnibus Rule, business associates have independent HIPAA obligations and are subject to enforcement in their own right. In an MSO-PC group, that means the MSO, and the AI provider as the MSO’s subcontractor.

​

5. Patient rights don’t disappear

If an AI-drafted note becomes part of the designated record set, it is subject to patient access and amendment rights. The record lives in your EHR; the vendor’s retention setting is irrelevant to it.

​

6. Breach notification still attaches

An impermissible disclosure of unsecured PHI is presumed to be a breach unless you document a low probability of compromise. Disclosing PHI to a vendor with no BAA is potentially reportable whether or not the vendor stored it.

​

7. Stricter regimes sit on top

42 C.F.R. Part 2 imposes stricter confidentiality on substance use disorder records than HIPAA. State health privacy laws can be stricter. And a growing set of states now regulate AI in health care directly — Washington, for instance, has a law providing that AI may not be the sole means used to deny, delay, or modify health care services, with human review required for medical-necessity denials.⁴ Separately, HHS has addressed nondiscrimination in patient care decision support tools.⁵

​

Where the PHI actually persists: your side

This is the gap that matters most in practice, and it is almost always larger than the vendor’s. Teams secure ZDR from the provider and then log the full prompt to their observability stack. Prompt caching deserves specific attention. It is a retention mechanism whose entire purpose is persisting request content, it is frequently enabled by default for cost reasons, and it can operate at the vendor, at a gateway, or in your own application. Confirm where cached content lives, for how long, and whether it is inside your BAA’s scope. Every one of these belongs in your BAA inventory and your security risk analysis. See Build a minimum viable HIPAA program.

​

The MSO-PC chain

The structure adds two links most guidance doesn’t contemplate. Practical consequences:

• One BAA per PC. Each professional entity is a separate covered entity. A ten-PC group needs ten MSO-PC BAAs, and those BAAs need to actually contemplate the AI use.
• The MSO needs a subcontractor BAA with the AI provider, and must ensure it flows the same restrictions down.
• Records must stay segregated by PC. A shared AI tool spanning ten PCs must not let one PC’s context leak into another’s — each PC owns its own patients’ records.
• The MSO is directly liable, not merely contractually exposed.

The concrete case this wiki keeps flagging: 835 remittance files are PHI. Running them through a model for denial root-cause classification puts PHI in a prompt. Teams build that as a finance project and miss the BAA, the encryption, and the risk analysis entirely. See The 835.

​

The CPOM overlay, the part unique to this structure

Privacy is not the only exposure. An MSO deploying AI that influences clinical decisions may be exercising a function the corporate practice doctrine reserves to the PC. Recall what the newest statutes actually enumerate. California’s SB 351 bars management entities from participating in billing, coding, equipment selection, and clinical staff oversight, and bars private equity groups and hedge funds from determining what diagnostic tests are appropriate or determining the need for referrals. Oregon’s SB 951 reaches MSO control over scheduling, clinician compensation, coding, billing, and payer contract terms.⁶ Now map AI onto that list: The test is the same one this wiki applies everywhere: who decides, and who merely implements? AI that surfaces options for a clinician who genuinely decides is support. AI that decides and informs the clinician is control, regardless of what the MSA says, and regardless of whether a human nominally clicks approve on a queue of a thousand items. If you build clinical or coding AI in the MSO, document that the PC approved the tool, that clinicians can reject its output, and that rejection actually happens at a rate consistent with genuine review. See What an MSO can and can’t do and Run a CPOM self-audit. There is also a records question: if AI output becomes part of the clinical record, the PC owns it, not the MSO that built the tool.

​

De-identification: the other route

If data is properly de-identified under 45 C.F.R. § 164.514 — either Safe Harbor (removing the 18 specified identifiers with no actual knowledge of re-identifiability) or Expert Determination — it is not PHI, and HIPAA does not apply to it.⁷ That is a genuine path, and for many analytics use cases it is the right one. De-identifying free-text clinical narrative is harder than it looks. Safe Harbor’s identifier list is straightforward for structured fields and unreliable for prose, where identifying detail hides in context — an unusual diagnosis, a referring physician’s name in a sentence, a date embedded in a narrative, a description of an employer. Automated de-identification of notes has a meaningful error rate. If you rely on de-identification for anything sensitive, use Expert Determination and document it. Do not assume a regex pass over notes produces de-identified data.

​

Shadow AI is the bigger real-world exposure

For most groups, the largest actual risk is not the architected integration. It is a staff member pasting a patient’s chart into a consumer chatbot to draft an appeal letter. That is an impermissible disclosure to a vendor with no BAA, and no amount of enterprise architecture prevents it. The controls that work are an acceptable-use policy, explicit workforce training that names the behavior, and a sanctioned tool good enough that people don’t route around it. Train on the specific sentence: never paste patient information into an AI tool that isn’t on the approved list.

​

What good looks like

​

The one-sentence version

ZDR is a useful control that reduces how much PHI sits at a vendor. HIPAA is a set of obligations on you that ZDR does not touch, and the two are configured independently, sometimes in opposite directions.

​

Sources

1. 45 C.F.R. § 160.103 (definitions of business associate and covered entity); business associate contract requirements at 45 C.F.R. § 164.504(e). eCFR.
2. Anthropic, Is Anthropic willing to sign a BAA? (checked August 2026). Cited because the per-service coverage matrix and the ZDR interaction are documented publicly and explicitly; other major providers publish their own. Not an endorsement, and subject to change — read your vendor’s current terms.
3. HHS OCR proposed HIPAA Security Rule update, January 2025. Confirm current status. See HIPAA Journal, HIPAA Updates and HIPAA Changes.
4. State AI-in-healthcare laws, including Washington’s provision that AI not be the sole means of denying, delaying, or modifying care. See Live Compliance, AI in Healthcare: The Regulatory Landscape. Verify current text and effective date against the state’s own statute.
5. HHS OCR, “Ensuring Nondiscrimination Through the Use of Artificial Intelligence and Other Emerging Technologies,” addressing patient care decision support tools. See Reed Smith, HHS Recent Guidance on AI Use in Health Care.
6. Cal. S.B. 351 (2025); Or. S.B. 951 (2025). See the legislation tracker for citations and effective dates.
7. 45 C.F.R. § 164.514(a)–(b). HHS, Guidance Regarding Methods for De-identification of PHI.