Building a Patient-Facing AI Chatbot for Medical Records

How thirteen iterations turned a basic RAG prototype into a healthcare AI system patients can actually trust.

The Problem.

Patients have a right to understand their own medical records. But medical data is dense, full of codes, abbreviations, and clinical jargon that was never designed to be read by a layperson. A lab result that says “Hemoglobin A1c/Hemoglobin.total in Blood: 7.2%” means something very specific — but most patients would simply like to know: “What’s my A1C?”

ChartChat was built to bridge that gap. It’s an AI-powered chatbot that sits on top of a patient’s medical records — structured as HL7 FHIR R4 resources — and lets them ask questions in plain language. Lab results, medications, allergies, visit history, immunizations, vital signs, and more — all accessible through a natural conversation.

The first version worked for simple queries. A patient could ask “what are my lab results?” and get a response. But the distance between “it works” and “it works correctly, safely, and fast enough for a real patient” turned out to be enormous. What follows is the story of thirteen iterations that closed that gap.

Privacy as a Foundation.

Before a single line of application code was written, the team made a foundational decision: all AI models would run inside the same secure environment as the patient data. No patient information — lab results, diagnoses, medications, clinical notes — would ever leave the network during any AI operation.

The default approach for most AI projects is to send data to a cloud-hosted service and pay per query. For healthcare data, that creates a data boundary many organizations cannot or will not cross. Compliance requirements, audit obligations, and patient trust all pointed the same direction: the models needed to be co-located with the data.

This added real operational complexity — specialized hardware to manage, models to update, inference performance to tune. But it eliminated an entire class of privacy concerns and meant patient data could flow through the AI pipeline without requiring additional legal review on every query.

The text understanding model was chosen specifically for its built-in strength with medical vocabulary. It recognized clinical abbreviations like A1C, eGFR, BUN, and LOINC display names without any custom training — a decision that would pay off significantly when it came to retrieval quality.

Safety & Guardrails: The Hardest Problem.

When the team started building ChartChat, they assumed safety guardrails would be the straightforward part. They were wrong. Getting a healthcare AI to reliably stay within its boundaries required three complete rewrites of the approach, and every failure taught something essential about how language models behave in clinical settings.


Why a Medical AI Made Things Harder

The first instinct was natural: if you’re building a healthcare application, use a healthcaretrained model. The initial model understood clinical terminology, could parse lab values fluently, and knew how to reason about conditions and treatments.

But there was a fundamental tension. A model trained to be maximally helpful with medical questions also resists not answering them. When a patient asked “should I go to the ER for this?” — a question ChartChat absolutely cannot answer — the model would try to help anyway. It would look at the retrieved lab values, reason about them, and hedge while still engaging. The training objective and the safety objective were in direct conflict.


Attempt 1: Rules-Based Prompting

The team’s first approach was detailed system prompt rules: do not diagnose, do not triage, do not advise on medication dosing, direct emergencies to 911.

This worked most of the time. But “most of the time” is not acceptable in healthcare. The failure modes were subtle — not hallucinated diagnoses, but things like answering a triage question by framing the answer as “what the records show” while still implying safety. Responding to “is my A1C dangerous?” with a technically factual answer a patient could read as medical clearance. Engaging with off-topic medical questions using general knowledge instead of staying scoped to the patient’s actual records.

The line between “explaining what a record says” and “giving medical advice” is conceptually clear but linguistically very blurry.


Attempt 2: Identity Over Rules

The breakthrough was a shift from writing rules to writing an identity. Instead of “do not give medical advice,” the system became: you are a records reader, not a clinician. Your only permitted action is to read what is written. When asked anything beyond that, return a specific scripted sentence.

Scripted responses turned out to be far more reliable than instructed refusals. When you tell an AI “don’t do X,” it still has to reason about X to know it shouldn’t. When you give it an exact sentence to return, it doesn’t need to reason — it just returns the sentence.


Attempt 3: A Layered Architecture

Even the tightest prompt instructions are probabilistic — they work until they don’t. For the highest-risk categories, the team needed a guardrail layer that could never be bypassed, regardless of how a message was phrased.


  • Layer 1 — Deterministic upstream check: handles the highest-risk cases and common courtesies. Fast, certain,
  • Layer 2 — LLM system prompt: LLM handles the nuanced cases that require language understanding — off-topic questions, triage deflection, medication dosing refusals, scope boundaries. The model handles thes.

Making Retrieval Work for Healthcare.

With safety guardrails in place, the next challenge was retrieval — getting the right medical data to the AI in the right order so it could answer accurately. This turned out to be where most of the hard engineering lived.


Intent Classification

A single search across all patient data doesn't work well in healthcare. "What is my A1C?" and "What medications am I taking?" can look similar to a search algorithm, causing medications to surface in lab results and vice versa.

The team introduced an intent classification step — a lightweight LLM analysis that categorizes every question into one of thirteen types before any retrieval begins: lab queries, medication queries, visit history, allergies, radiology, conditions, immunizations, care plans, vital signs, procedures, documents, appointments, and general questions. Each category routes to its own specialized retrieval path with its own strategy, thresholds, and data views.

Classification happens in a single fast call before any data is fetched. The result determines not just where to look, but how — structured SQL for discrete records, vector search for narrative content, or a hybrid of both. A question can carry multiple intents simultaneously; the system runs all matching retrieval paths and merges the results.


Hybrid Search: Structure + Similarity

This is where the team moved from pure similarity-based search to a hybrid approach.

Structured database queries guarantee completeness — a patient's full medication list, every allergy, all conditions. Similarity-based search provides relevance — surfacing the right result first when there are hundreds. These two signals alone still leave a gap: semantic embeddings sometimes deprioritize exact terminology matches. A patient asking "what is my creatinine?" might score lower than a semantically adjacent but less relevant result, simply because the embedding model generalized too broadly.


BM25 + Reciprocal Rank Fusion

BM25 is implemented directly in PostgreSQL using native tsvector full-text search and ts_rank_cd scoring — no separate search engine required. One deliberate design decision: queries use OR logic across individually stemmed terms rather than the default AND behavior of standard full-text search. Medical records are short documents — a single lab result contains one test name. Requiring all query terms to appear in the same document would silently drop results when a patient asks about multiple tests in a single question.

The vector results and BM25 results are merged using Reciprocal Rank Fusion. Each document receives a score of 1 / (60 + rank) from each list, and the scores are summed. A document appearing near the top of both lists outranks one that dominates only one signal. The constant 60 is the standard RRF dampening factor — it prevents a single top-ranked result from dominating when it's absent from the other list entirely.

The final ranked list reflects both exact terminology matches and semantic understanding. Neither signal overrides the other.


Trusting Medical Vocabulary Fusion

The language model had strong built-in knowledge of clinical terminology, but the team initially undermined it with overly conservative similarity thresholds.

A patient asking "what is my A1C?" was getting that result at position eleven, buried behind unrelated lab data. The abbreviation "A1C" was scoring poorly against the formal LOINC name "Hemoglobin A1c/Hemoglobin.total in Blood" stored in the database. The semantic relationship was there — the threshold was cutting it off before it could surface.

The fix was to let the model's domain expertise work as designed. A1C immediately surfaced at position one. Glucose appeared first for "blood sugar." Cholesterol panels appeared first for "cholesterol." No synonym tables, no manual mappings, no ongoing maintenance.

The lesson is general: trust your AI model's domain knowledge. Tune the sensitivity to let it work rather than working around it with manual rules. Measure on real queries and adjust empirically.


Cross-Section Enrichment

Healthcare data doesn't exist in silos, but retrieval systems often treat it that way.

A patient asking about a lab result in the Labs section gets more than just the lab value. The system automatically pulls their active conditions as background context before the LLM responds. Medication queries pull in allergy data. Care plan queries inject both conditions and current medications. This enrichment happens silently — the patient asks one question and the model answers with the full clinical picture.

The enrichment runs only when the primary retrieval actually found data. A patient with no lab records doesn't trigger a conditions fetch. The cross-section context is clearly labeled in the prompt so the LLM treats it as background rather than the primary answer subject.


Guarding Against Patient Self-Reporting

Patients often state values in their questions: "my A1C is 7.5, is that high?" If the record shows a different value, the model would reason from the patient's stated number rather than the actual one — confidently answering a question built on a false premise.

The system detects numeric claims in the message, finds the matching test name in the retrieved context, and prepends a correction to the context block before the LLM ever sees the question. The model sees both the actual record value and the patient's stated value, and answers based on what the record shows. No special prompting required — the correction is structural, not instructional.


Three Retrieval Strategies

As the team built retrieval for each data type, they recognized that one strategy doesn’t fit all clinical data. Three distinct patterns emerged:

Similarity-first with structured fallback — for labs, vital signs, care plans, clinical notes, and radiology. The patient’s question is semantically meaningful, so similarity search surfaces the right data. Structured queries serve as a safety net.

Complete list with relevance re-ranking — for medications, conditions, allergies, and immunizations. These are lists that can never be truncated. A patient’s full medication list must be present in every response — missing one medication because it scored below a threshold is a clinical safety issue. The system retrieves everything, then orders by relevance.

Pin and re-rank — for visits. “When was my last visit?” requires the most recent encounter at position one regardless of any score. “Did I have a cardiology visit?” requires relevant older encounters to surface. The most recent visit is always pinned first; the rest are ordered by relevance.


Making the Data Actually Accurate.

Allergies: From a Text Blob to Clinical Records

For months, the allergy answer was a simple semicolon-separated string: “peanuts; penicillin; Lipitor.” No severity, no reaction type, no criticality, no onset date. The data couldn’t be linked back to its source, and it wasn’t individually searchable.


Before
Allergies: peanuts; penicillin; Lipitor


After
Allergy: Lipitor. Category: medication. Criticality: unable-to-assess. Status: active. Reaction: hives. Onset: 2023-01-28


Real-World FHIR Data Surprises

Working with real patient data from multiple electronic health record systems surfaced inconsistencies that were invisible in test environments. FHIR compliance does not guarantee data consistency across sources.


Blood pressure ordering. The FHIR standard stores blood pressure as two components (systolic and diastolic). The initial implementation assumed a fixed order. In practice, some systems emit the values reversed — a patient’s blood pressure was displaying as 74/119 instead of 119/74. The fix: identify each component by its label, not its position.

Immunization status. Health record systems log vaccines that were ordered but declined or deferred. Without status context, the system listed declined vaccines as if they’d been administered. The fix: include status for every immunization record and filter out erroneous entries entirely.

Drug name formatting. Some systems use a safety convention that capitalizes portions of drug names to reduce confusion between similar-looking medications (e.g., “amLODIPine,” “predniSONE”). The AI relayed these unusual capitalizations verbatim to patients. The fix: normalize all medication names to standard formatting.

Context-Aware Conversations.

The team built three interaction modes that match how patients actually use a healthcare application:


Open conversation — the patient types a free-form question from anywhere in the app. The system classifies intent, retrieves relevant data, and generates an answer. Full pipeline.

Category-scoped conversation — the patient is browsing a specific section (Labs, Medications, Allergies). The system already knows the context, skips classification, and retrieves only within that category. Faster and more focused.

Record-specific conversation — the patient is looking at a specific record and asks a question about it.. The AI answers about that one item.

Automated Quality Assurance.

At a certain point, the system was complex enough that a change to one retriever could break another, and a fix for one edge case could regress three others. The team needed an objective way to know whether the system was getting better or worse after every change.

Page 8 They built an automated evaluation framework: a suite of test cases covering all thirteen intent categories, multiple patient profiles, safety guardrails, cross-category redirects, retrieval edge cases, and negative cases. Each test specifies the question, expected keywords that must appear, and forbidden keywords that must not. The full suite runs in under three minutes — practical to execute after every meaningful change.

Key discoveries from the evaluation framework: a guardrail keyword was incorrectly triggering on a valid clinical phrase. A performance threshold was being exceeded under concurrent load. Several retrieval fixes that worked in manual testing failed for edge-case patient profiles. The eval caught all of these before they reached production.