Healthcare leaders are asking the wrong question about AI.
For the past two years, the executive conversation has centered on one decision: which tool to buy. Which ambient scribe. Which prior authorization engine. Which care coordination copilot. The diligence is real. The budgets are real. And the results keep landing in the same disappointing place.
MIT’s 2025 research, published through its NANDA initiative as “The State of AI in Business,” found that roughly 95% of generative AI pilots never translate into measurable production impact. McKinsey’s global surveys point the same direction, and KLAS Research has repeatedly documented a stubborn gap between AI adoption and any clinical or operational payoff. The pattern is consistent enough to deserve a name. We can call it “pilot purgatory”.
Here is the uncomfortable part. The model is almost never why these projects fail.
An AI agent is only as capable as the data it can reach, trust, and act on. Point a sophisticated model at fragmented, inconsistent, inaccessible data, and you do not get intelligence. You get confident chaos wrapped under the garb of a sophisticated model.
Gartner has long estimated that poor data quality is among the leading reasons for AI initiatives to collapse. In healthcare, that problem is multiplied by the number of systems a single patient touches: the EHR, the lab system, the pharmacy, imaging, the billing platform, the payer. Each speaks its own dialect. Few were designed to answer questions on demand. So, the real question is not which model is smartest. It is whether your data layer can support any model at all.
You Are Evaluating in the Wrong Order
Most organizations evaluate AI platforms first and data readiness second, if at all. That sequence is backwards. A data layer assessment should come before a single vendor demo, because the data layer sets the ceiling on everything built above it.
The good news is that readiness is more diagnosable than it looks. You do not need a six-month maturity study to get an honest signal. You need three questions.
The Three-question Test
If a healthcare organization wants a blunt assessment of AI readiness, these are the three questions that matter most.
Question 1: Can an AI agent pull a patient’s complete clinical and claims context in a single request, without a human stitching together five separate system logins?
This is the accessibility test. A “yes” means your data is reachable through APIs, ideally FHIR, and that an agent can assemble a full picture the way a good clinician would. A “no” reveals that your context lives in silos, and that every “intelligent” workflow secretly depends on a person doing manual retrieval. Agents cannot reason about data they cannot see.
Question 2: When data is wrong, stale, or missing, does anyone find out before a clinician or a patient does?
This is the observability test. Most organizations discover bad data at the point of care, when a clinician mutters “that’s not right” and fixes it by hand. Those quiet corrections are invisible to every downstream system, including any AI you deploy. Mature organizations instrument their data flows. They can tell you, today, whether eligibility feeds are current and whether problem lists are complete. Everyone else is flying blind and calling it stable.
Question 3: Has your organization shipped an automated workflow that writes structured data back into the EHR? Not a PDF. Not a note. Actual FHIR resources.
This is the maturity test, and it is the most revealing of the three. Reading data is the easy half. Real automation requires writing data back into the systems of record, safely and in structured form. Most organizations have never done it, because write access is governed, scoped, and politically sensitive. An organization that has closed that loop even once has solved problems that read-only shops have not yet discovered.
What Most Healthcare Organizations Get Wrong About AI Readiness
The most dangerous phrase in healthcare AI is “we’re already ready.” It usually rests on one of four assumptions, and all four are traps.
- “We’re on EPIC, so we’re FHIR-ready.” Having a major EHR is not the same as having clean, accessible, agent-ready data. Vendor FHIR endpoints vary widely in what they actually return versus what the specification promises.
- “We built a data lake.” A data lake without governance is a data swamp. Storage is not the same as usability.
- “Our vendor handles that.” Vendors handle their slice. The integration layer between systems, the place agents actually operate, is still yours to own.
- “We passed our HIPAA audit.” This is the costliest confusion of all. Compliance readiness and AI readiness are entirely different assessments. One proves you can protect data. The other proves you can use it. Passing the first tells you almost nothing about the second.
Organizations that believe they are 80% ready are often closer to 40%. The last-mile data problems are the hardest, and nobody finds them until an agent hits them in production.
7 Signs Your Data Layer is Not Ready
These are the symptoms that surface in real environments, not on architecture diagrams.
- Patient matching fails across systems. The same person carries different identifiers, and no agent can be sure who it is looking at.
- Clinicians routinely correct data at the point of care, which means the data feeding any model is already wrong.
- There is no FHIR R4 bulk export. If you cannot move data at scale through a standard, automation stays manual.
- Coverage and eligibility data is stale by the time it is used.
- “We’d need a data extract request for that” is a common answer to routine data questions.
- Terminology is inconsistent: SNOMED in one system, local codes in another, free text in a third.
- There is no audit log of who accessed or changed what.
The Dimensions That Actually Matter
Data accessibility: Good looks like live APIs returning complete records. The warning sign is routine extract requests. The implication is simple: inaccessible data caps every use case you will ever attempt.
Data quality: Good means complete, current, deduplicated records. The warning sign is point-of-care corrections. Poor quality does not stay contained; AI broadcasts it.
Interoperability and terminology consistency: Good means standardized codes mapped across systems. The warning sign is the same concept represented three different ways. Without a shared vocabulary, agents misinterpret, and misinterpretation in clinical settings is not a rounding error.
Governance and security: Good means fine-grained, auditable access. The warning sign is all-or-nothing permissions, which force a choice between too much access and none.
Workflow integration and automation readiness: Good means structured write-back into systems of record. The warning sign is output that lands as unstructured text no downstream system can read. This is the dimension that separates demos from durable value.
What Actually Breaks Agents in Production
When agents fail after go-live, the autopsy rarely blames the model. It blames the environment around it. No normalized patient identity, so the agent cannot reliably know who it is looking at. Read access without write scopes, so the agent can see everything and do nothing. No orchestration layer, so workflow state cannot persist across a multi-day prior authorization cycle. An incomplete context window, where the agent sees the chart but not the claims, or the reverse. Legacy systems that hold the critical data behind no API at all.
But the single biggest barrier is not on that list, because it is not technical. It is political. In most organizations, nobody owns the data layer across systems, and nobody wants to. The EHR team owns the EHR. The revenue team owns billing. The analytics team owns the warehouse. The space between them, where agents live, is unclaimed territory.
The Same Shortfall, Different Disguises
The data layer gap does not announce itself the same way twice. It hides inside each use case and waits.
Take AI scribes, the most popular entry point. The minimum bar is not a microphone. It is structured write-back. When a scribe’s output lands as a blob of unstructured text, no downstream system can read it, and the promised efficiency evaporates. The success signal is concrete: diagnoses captured by the scribe show up as coded Condition resources, not free text someone has to re-enter.
Clinical decision support fails differently. It needs a complete, current problem list and medication list at the point of care. Feed it stale or partial data and it fires irrelevant alerts, clinicians learn to dismiss everything, and acceptance rates collapse into the single digits. The organizations that get this right measure alert acceptance and treat a low rate as a data problem, not a clinician problem.
Revenue cycle automation lives or dies on payer data. If denial reason codes sit in a billing system with no API, an agent cannot learn the patterns that would prevent the next denial. Where eligibility is current and denials are captured in structured form, first-pass approval rates start improving within a single billing cycle.
Prior authorization, the use case regulators are now forcing into the open, is the clearest test of all. It needs accessible FHIR clinical data, payer criteria that are mapped rather than buried, and approved write scopes. The common failure is silent: local, unmapped service-type codes cause submissions to fail without a clear error. When the foundation holds, the authorization queue shrinks and staff handle only the exceptions.
How to Become More AI-ready in 90 Days
Quick wins, Days 1 to 30: Audit which FHIR R4 endpoints are actually live and what they return versus what the spec claims. Map every workflow that still depends on fax, PDF, or manual re-entry; those are your automation targets. Identify one rule-based routing task a human does today that could be automated now.
Medium effort, Days 30 to 60: Resolve patient matching in your highest-volume data source. Stand up structured logging on at least one data flow, because you cannot improve what you cannot measure. Begin FHIR write-scope approvals with your EHR vendor now; that clock often runs two to four weeks at minimum.
Longer-term investments, Days 60 and beyond: Build a terminology normalization layer. Stand up a governed API registry that documents what tools exist, who can call them, and what they return. And name an owner: a person or team accountable for data quality in each domain. The last one is the hardest and most important.
The Compliance Forcing Function
Regulation is about to make this urgent. The CMS Interoperability and Prior Authorization final rule, CMS-0057-F, requires impacted payers to stand up a set of FHIR-based APIs, with operational provisions generally beginning in 2026 and API requirements generally due by January 1, 2027. CMS has estimated the rule could save roughly $15 billion over 10 years by moving prior authorization off fax machines and into standardized exchange.
The same FHIR plumbing that satisfies CMS-0057-F is the plumbing agents’ needs. The Da Vinci implementation guides for prior authorization, the CRD, DTR, and PAS specifications, double as a reference architecture for automation. TEFCA and QHIN participation signal longitudinal data access. HIMSS maturity models such as EMRAM and INFRAM, and a careful read of your own HL7 FHIR Capability Statement, tell you what your systems can really do.
One caution. None of this is legal advice, and meeting a regulatory deadline does not certify you as AI-ready. Compliance and readiness remain in different tests. But the deadline is a gift: it funds the foundation anyway.
What a Good Data Layer Does Not Fix
None of this makes the model reliable by itself. A clean, accessible, governed data layer removes bad inputs as an excuse, but a model can still generate a fluent, confident answer that is wrong even when the underlying data was right. Hallucination is a model behavior, not only a data problem, and it needs its own controls: grounding responses in the retrieved record rather than the model’s general knowledge, human review at defined checkpoints for anything clinical or financial, and ongoing measurement of output accuracy after launch, not just at go-live.
Data also drifts. A data layer that passes every test on day one degrades as source systems change, vendors update their FHIR implementations, new fields appear, and terminology mappings fall out of date. Readiness is not a one-time certification. It requires the same observability described in question two, run continuously, not just during the initial assessment.
Conclusion
The prior authorization burden alone shows what is at stake. The AMA’s 2024 survey found that physicians and their staff spend about 13 hours a week on prior authorizations, with the large majority of physicians saying the process worsens burnout and drives up overall utilization. This is exactly the kind of structured, repetitive, data-dependent work that agents can absorb, but only on a data layer that lets them read context, trust it, and write back.
Micky Tripathi, now Mayo Clinic’s chief AI implementation officer and previously the nation’s top health IT official, has described certified EHRs as the “digital floor” beneath modern healthcare, arguing that the benefits we expect from AI would be unreachable without that foundation already in place. The agent era extends the same logic. The floor now has to be queryable, governed, and writable.
The organizations that win with AI will not necessarily have the best models. They will have the best data foundations. Everyone has access to the same models. Almost no one has done the unglamorous work underneath them. That work, not the next demo, is the strategy.
Find out if your organization is ready for the next phase of healthcare interoperability. Take our CMS-0057-F FHIR Readiness Assessment to benchmark your FHIR capabilities, identify critical gaps, and prepare your data layer for AI-driven automation.