> ## Documentation Index
> Fetch the complete documentation index at: https://mso.getlemma.com/llms.txt
> Use this file to discover all available pages before exploring further.

# LLMs, zero data retention, and HIPAA

> Zero data retention is a storage control, not a compliance status. Why vendors treat it as a precondition for handling PHI, why it is not the same as HIPAA, and where the PHI actually persists.

**Zero data retention (ZDR)** is a configuration in which an AI provider does not persist your prompts and outputs after serving the request. It is a genuine risk-reduction control, and vendors frequently require it before they will handle regulated data, but it is **not HIPAA compliance**, it is not a substitute for a business associate agreement, and in at least one major provider's current terms it is not even uniformly compatible with the services their BAA covers.

This page explains the framework. Whether a specific AI deployment is permissible with your PHI is a question for privacy counsel, and the answer depends on your configuration, your contracts, and your state. Vendor terms in particular change frequently — **verify against the vendor's current documentation and your own executed agreement**, not against this page. **Checked August 2026.**

## What ZDR actually is

By default, most AI providers retain API inputs and outputs for a period, commonly around 30 days, for abuse monitoring, trust and safety review, and debugging. **ZDR turns that logging off**: the data exists transiently while the request is processed and is not written to durable storage afterward.

That is the whole of it. It is a statement about **one vendor's storage behavior**, on **one configuration**, for **the services it covers**.

### Three things ZDR is constantly confused with

| Not the same as                   | Why it's different                                                                                                                                                               |
| --------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **"They don't train on my data"** | A provider can decline to train on your inputs while still retaining them for 30 days. No-training is now standard on enterprise and API tiers; it says nothing about retention. |
| **Encryption**                    | Encryption protects data in transit and at rest. ZDR is about whether it is written at rest at all. You want both.                                                               |
| **A BAA**                         | A contract allocating HIPAA obligations. ZDR is a technical setting. **Neither implies the other.**                                                                              |

Also distinct, and worth checking separately: **data residency** (where processing happens), **subprocessors** (who else touches it), and **retention of derived artifacts** like embeddings, caches, and safety classifications.

## Why vendors treat it as a precondition

The chain is straightforward. If PHI goes to an AI provider, that provider is creating, receiving, maintaining, or transmitting PHI on your behalf, which makes them a **business associate** under 45 C.F.R. § 160.103, and requires a BAA.<sup>1</sup>

Providers generally do not want PHI sitting in general-purpose abuse-monitoring logs, because that puts regulated data into systems designed for a different purpose, reviewed by staff with a different mandate. Requiring ZDR shrinks the surface: less PHI at rest, a smaller breach blast radius, fewer systems in scope for a records request, and a simpler subprocessor story.

So in practice, ZDR frequently shows up as **a condition the vendor imposes before it will sign a BAA or let you send regulated data**. That is a real and sensible pattern, and it is the sense in which ZDR is a prerequisite.

## But the relationship is messier than "prerequisite"

Here is where the neat version breaks down, and it is worth knowing before you architect around an assumption.

**BAA coverage is scoped per service, not per vendor.** Providers publish which of their products and endpoints the BAA covers, and the exclusions are frequently substantial. Anthropic, for example, publicly documents that its BAA covers the Messages API and specified Claude Enterprise features, while **explicitly excluding** its Batch API, Files API, Skills API, Code Execution, Computer Use, and Web Fetch.<sup>2</sup>

**And ZDR can point in the opposite direction.** In the same documentation, Anthropic states that certain covered models **require 30-day data retention and are not available with ZDR enabled**, while a different covered service is **only** covered under the BAA **when ZDR is enabled**.<sup>2</sup>

<Warning>
  Read that twice, because it inverts the intuition: **for some HIPAA-covered services the vendor requires you to keep retention on, and for others it requires you to turn it off.**

  ZDR and BAA coverage are **orthogonal settings that a vendor combines differently per service**, not a ladder where ZDR is step one and the BAA is step two. Architecting on "we turned on ZDR, so we're covered" can leave you outside BAA coverage entirely.

  We cite Anthropic here because it documents this publicly and unusually explicitly, not because it is unique — the major providers offering BAAs (Anthropic, OpenAI and Azure OpenAI, Google Vertex AI, AWS Bedrock) all publish per-service coverage matrices with their own exclusions. **Read your vendor's current matrix and your executed agreement.**
</Warning>

## Why ZDR is not HIPAA compliance

Even with ZDR on, a signed BAA, and covered services, HIPAA obligations remain, because **most of them are obligations on you, not on the vendor.**

### 1. Transmission is a disclosure

Sending PHI to a third party is a disclosure under the Privacy Rule regardless of what happens to it afterward. It must be permitted — for treatment, payment, or health care operations, or under an authorization — and it must satisfy **minimum necessary**.

ZDR does not make an impermissible disclosure permissible. It only means the impermissible disclosure wasn't stored.

### 2. Minimum necessary still applies to the prompt

Pasting an entire chart into a prompt because "it's ZDR anyway" fails minimum necessary. ZDR governs what happens *after* the request; minimum necessary governs *what you send*. Prompt construction is a compliance surface.

### 3. The Security Rule applies to your systems

Access controls, audit logging, workforce training, and the required **security risk analysis** under 45 C.F.R. § 164.308(a)(1)(ii)(A) all cover your AI integration as an asset that handles ePHI. A vendor's retention setting does not discharge any of them.

OCR proposed the first major update to the HIPAA Security Rule in roughly two decades in **January 2025**, including removing the required/addressable distinction and tightening expectations around risk management and encryption. **Confirm its current status** before relying on either the old or the proposed framework — this is exactly the kind of rule whose status changes.<sup>3</sup>

### 4. Business associates are directly liable

Since the HITECH Act and the 2013 Omnibus Rule, business associates have independent HIPAA obligations and are subject to enforcement in their own right. **In an MSO-PC group, that means the MSO**, and the AI provider as the MSO's subcontractor.

### 5. Patient rights don't disappear

If an AI-drafted note becomes part of the designated record set, it is subject to patient access and amendment rights. The record lives in your EHR; the vendor's retention setting is irrelevant to it.

### 6. Breach notification still attaches

An impermissible disclosure of unsecured PHI is **presumed to be a breach** unless you document a low probability of compromise. Disclosing PHI to a vendor with no BAA is potentially reportable whether or not the vendor stored it.

### 7. Stricter regimes sit on top

**42 C.F.R. Part 2** imposes stricter confidentiality on substance use disorder records than HIPAA. State health privacy laws can be stricter. And a growing set of states now regulate AI in health care directly — Washington, for instance, has a law providing that AI may not be the sole means used to deny, delay, or modify health care services, with human review required for medical-necessity denials.<sup>4</sup> Separately, HHS has addressed nondiscrimination in patient care decision support tools.<sup>5</sup>

## Where the PHI actually persists: your side

This is the gap that matters most in practice, and it is almost always larger than the vendor's.

Teams secure ZDR from the provider and then log the full prompt to their observability stack.

| Your system                                                    | Holds PHI?                                                    |
| -------------------------------------------------------------- | ------------------------------------------------------------- |
| **Application logs**, Datadog, Sentry, CloudWatch, and similar | **Yes**, if prompts or outputs are logged                     |
| **Error monitoring and stack traces**                          | **Yes**, frequently including request payloads                |
| **Conversation or chat history** in your database              | Yes                                                           |
| **Vector stores and embeddings**                               | **Yes**, embeddings derived from PHI should be treated as PHI |
| **Evaluation and test datasets**                               | **Yes**, a "golden set" of real prompts is a PHI store        |
| **Prompt caching**                                             | **Yes, by definition**, caching is retention                  |
| **Model gateway or proxy logs**                                | Yes                                                           |
| **Support tickets with screenshots**                           | Yes, and frequently overlooked                                |
| **Analytics on prompt content**                                | Yes                                                           |

**Prompt caching deserves specific attention.** It is a retention mechanism whose entire purpose is persisting request content, it is frequently enabled by default for cost reasons, and it can operate at the vendor, at a gateway, or in your own application. Confirm where cached content lives, for how long, and whether it is inside your BAA's scope.

Every one of these belongs in your BAA inventory and your security risk analysis. See [Build a minimum viable HIPAA program](/guides/compliance/build-a-hipaa-program).

## The MSO-PC chain

The structure adds two links most guidance doesn't contemplate.

```mermaid theme={null}
graph LR
    PC["PC<br/>covered entity<br/>(one per state)"] -->|BAA| MSO["MSO<br/>business associate"]
    MSO -->|subcontractor BAA| LLM["AI provider<br/>business associate,<br/>directly liable"]
    LLM -->|its own agreements| CLOUD["Cloud infrastructure<br/>and subprocessors"]
```

Practical consequences:

* **One BAA per PC.** Each professional entity is a separate covered entity. A ten-PC group needs ten MSO-PC BAAs, and those BAAs need to actually contemplate the AI use.
* **The MSO needs a subcontractor BAA with the AI provider**, and must ensure it flows the same restrictions down.
* **Records must stay segregated by PC.** A shared AI tool spanning ten PCs must not let one PC's context leak into another's — each PC owns its own patients' records.
* **The MSO is directly liable**, not merely contractually exposed.

The concrete case this wiki keeps flagging: **835 remittance files are PHI.** Running them through a model for denial root-cause classification puts PHI in a prompt. Teams build that as a finance project and miss the BAA, the encryption, and the risk analysis entirely. See [The 835](/concepts/payments/understanding-835s).

## The CPOM overlay, the part unique to this structure

Privacy is not the only exposure. **An MSO deploying AI that influences clinical decisions may be exercising a function the corporate practice doctrine reserves to the PC.**

Recall what the newest statutes actually enumerate. **California's SB 351** bars management entities from participating in **billing, coding, equipment selection, and clinical staff oversight**, and bars private equity groups and hedge funds from **determining what diagnostic tests are appropriate** or **determining the need for referrals**. **Oregon's SB 951** reaches MSO control over **scheduling, clinician compensation, coding, billing, and payer contract terms**.<sup>6</sup>

Now map AI onto that list:

| MSO-deployed AI tool                                                                | Risk                                                  |
| ----------------------------------------------------------------------------------- | ----------------------------------------------------- |
| **Auto-assigns diagnosis or procedure codes** with no meaningful clinician override | The MSO is determining coding, named in both statutes |
| **Recommends diagnostic tests**                                                     | Named in SB 351                                       |
| **Flags referral necessity**                                                        | Named in SB 351                                       |
| **Drives scheduling density or visit length**                                       | Named in SB 951                                       |
| **Screens or ranks clinical staff**                                                 | Clinical staff oversight                              |
| Surfaces coding *suggestions* a clinician reviews and can reject                    | Defensible, this is support                           |
| Drafts documentation the clinician edits and signs                                  | Defensible, the clinician authors                     |

**The test is the same one this wiki applies everywhere: who decides, and who merely implements?**

AI that surfaces options for a clinician who genuinely decides is **support**. AI that decides and informs the clinician is **control**, regardless of what the MSA says, and regardless of whether a human nominally clicks approve on a queue of a thousand items.

If you build clinical or coding AI in the MSO, document that the **PC approved the tool**, that clinicians can reject its output, and that rejection actually happens at a rate consistent with genuine review. See [What an MSO can and can't do](/concepts/model/what-msos-can-and-cant-do) and [Run a CPOM self-audit](/guides/compliance/run-a-cpom-self-audit).

There is also a records question: if AI output becomes part of the clinical record, **the PC owns it**, not the MSO that built the tool.

## De-identification: the other route

If data is properly de-identified under 45 C.F.R. § 164.514 — either **Safe Harbor** (removing the 18 specified identifiers with no actual knowledge of re-identifiability) or **Expert Determination** — it is not PHI, and HIPAA does not apply to it.<sup>7</sup>

That is a genuine path, and for many analytics use cases it is the right one.

**De-identifying free-text clinical narrative is harder than it looks.** Safe Harbor's identifier list is straightforward for structured fields and unreliable for prose, where identifying detail hides in context — an unusual diagnosis, a referring physician's name in a sentence, a date embedded in a narrative, a description of an employer. Automated de-identification of notes has a meaningful error rate.

If you rely on de-identification for anything sensitive, use Expert Determination and document it. Do not assume a regex pass over notes produces de-identified data.

## Shadow AI is the bigger real-world exposure

For most groups, the largest actual risk is not the architected integration. It is a staff member pasting a patient's chart into a consumer chatbot to draft an appeal letter.

That is an impermissible disclosure to a vendor with no BAA, and no amount of enterprise architecture prevents it. The controls that work are **an acceptable-use policy, explicit workforce training that names the behavior, and a sanctioned tool good enough that people don't route around it.**

Train on the specific sentence: *never paste patient information into an AI tool that isn't on the approved list.*

## What good looks like

<Steps>
  <Step title="Ask whether PHI needs to go to the model at all">
    De-identified data, structured fields only, or a locally hosted model may serve the use case. This is the cheapest control available.
  </Step>

  <Step title="Get the BAA before any PHI flows">
    And confirm **which services and endpoints it covers**. Exclusions are frequently substantial.
  </Step>

  <Step title="Confirm your retention configuration is what the BAA requires">
    Not what you assume. It may require ZDR on, or off, depending on the service.
  </Step>

  <Step title="Map the whole chain">
    You → MSO → provider → its cloud and subprocessors. Every link handling PHI needs an agreement.
  </Step>

  <Step title="Audit your own side">
    Application logs, error monitoring, chat history, vector stores, eval datasets, prompt caches, gateway logs, support tickets.
  </Step>

  <Step title="Apply minimum necessary to prompt construction">
    Send the fields the task needs, not the record.
  </Step>

  <Step title="Add it to the security risk analysis and BAA inventory">
    As an asset that handles ePHI.
  </Step>

  <Step title="Add access controls and audit logging to the AI feature itself">
    Who used it, on which patient, when.
  </Step>

  <Step title="Check the CPOM line on anything clinical or coding-related">
    Document PC approval and genuine clinician override.
  </Step>

  <Step title="Check 42 CFR Part 2, state privacy law, and state AI-in-healthcare law" />

  <Step title="Train the workforce on shadow AI, specifically" />
</Steps>

## The one-sentence version

**ZDR is a useful control that reduces how much PHI sits at a vendor. HIPAA is a set of obligations on you that ZDR does not touch, and the two are configured independently, sometimes in opposite directions.**

## Sources

1. 45 C.F.R. § 160.103 (definitions of business associate and covered entity); business associate contract requirements at 45 C.F.R. § 164.504(e). [eCFR](https://www.ecfr.gov/current/title-45/subtitle-A/subchapter-C/part-160/subpart-A/section-160.103).
2. Anthropic, [Is Anthropic willing to sign a BAA?](https://privacy.claude.com/en/articles/8114513-is-anthropic-willing-to-sign-a-baa) (checked August 2026). Cited because the per-service coverage matrix and the ZDR interaction are documented publicly and explicitly; other major providers publish their own. Not an endorsement, and subject to change — read your vendor's current terms.
3. HHS OCR proposed HIPAA Security Rule update, January 2025. **Confirm current status.** See HIPAA Journal, [HIPAA Updates and HIPAA Changes](https://www.hipaajournal.com/hipaa-updates-hipaa-changes/).
4. State AI-in-healthcare laws, including Washington's provision that AI not be the sole means of denying, delaying, or modifying care. See Live Compliance, [AI in Healthcare: The Regulatory Landscape](https://livecompliance.com/learn/ai-healthcare-regulations/). **Verify current text and effective date against the state's own statute.**
5. HHS OCR, "Ensuring Nondiscrimination Through the Use of Artificial Intelligence and Other Emerging Technologies," addressing patient care decision support tools. See Reed Smith, [HHS Recent Guidance on AI Use in Health Care](https://www.reedsmith.com/our-insights/blogs/health-industry-washington-watch/102k29k/hhs-recent-guidance-on-ai-use-in-health-care/).
6. Cal. S.B. 351 (2025); Or. S.B. 951 (2025). See the [legislation tracker](/reference/legal/cpom-legislation-tracker) for citations and effective dates.
7. 45 C.F.R. § 164.514(a)–(b). HHS, [Guidance Regarding Methods for De-identification of PHI](https://www.hhs.gov/hipaa/for-professionals/privacy/special-topics/de-identification/index.html).


## Related topics

- [HIPAA for MSO-PC operators](/concepts/compliance/hipaa-fundamentals.md)
- [What an MSO can and can't do](/concepts/model/what-msos-can-and-cant-do.md)
- [Billing compliance: the lines you never cross](/concepts/compliance/billing-compliance-basics.md)
- [The 835: how payers answer](/concepts/payments/understanding-835s.md)
- [Build a minimum viable HIPAA program](/guides/compliance/build-a-hipaa-program.md)
- [Put a BAA in place between MSO and PC](/guides/agreements/draft-a-baa.md)
- [Run a CPOM self-audit](/guides/compliance/run-a-cpom-self-audit.md)
- [CPOM & MSO legislation tracker](/reference/legal/cpom-legislation-tracker.md)
- [Glossary](/reference/appendix/glossary.md)