Clinical-Grade Patient Education Agent with LangGraph and LangSmith

lubu_labs · Healthcare · 2026 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Clinical patient-education agents for regulated healthcare

Lubu Labs needed a patient-education agent that could explain camera-based health scan results in plain language while avoiding unsafe diagnosis, hallucination, missing audit trails, and unreviewed low-confidence answers.

Constraints:

• Healthcare hallucinations can harm patients, not merely degrade UX.
• Answers must be grounded in approved medical knowledge rather than model training data.
• The system must route uncertain or risky cases to clinicians using explicit logic.
• Regulated environments require traceability of what was said, why, and with which knowledge/model versions.

Architecture

The agent uses LangGraph as an explicit state machine carrying medical findings, confidence scores, review flags, and thread IDs; RAG over an approved version-controlled medical knowledge base returns content and evidence-quality confidence; threshold routing sends low-confidence or clinically risky cases to human review; LangSmith traces every LLM call, retrieval result, confidence score, route, and metadata; PostgreSQL checkpointing makes patient sessions resumable.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  patient-conversation-ui["patient conversation UI"] --> langgraph-state-machine["LangGraph state machine"]
  camera-health-scan-results["camera health scan results"] --> langgraph-state-machine
  langgraph-state-machine --> medical-rag-retriever["medical RAG retriever"]
  medical-knowledge-base["medical knowledge base"] --> medical-rag-retriever
  medical-rag-retriever --> confidence-router["confidence router"]
  confidence-router --> llm-education-responder["LLM education responder"]
  confidence-router --> human-clinician-review["human clinician review"]
  llm-education-responder --> patient-conversation-ui
  human-clinician-review --> patient-conversation-ui
  langgraph-state-machine --> postgresql-checkpoint-store["PostgreSQL checkpoint store"]
  langgraph-state-machine --> langsmith-tracing["LangSmith tracing"]
  llm-education-responder --> langsmith-tracing
  langsmith-tracing --> evaluation-dataset["evaluation dataset"]
  langsmith-tracing --> audit-export["audit export"]

Components

Data flow

A patient asks about scan results; LangGraph state carries findings and session metadata; the system retrieves approved medical explanations with confidence scores; high-confidence educational answers go to the LLM response node, while confidence below threshold or review triggers go to a clinician-review node; LangSmith records retrievals, model calls, routing, metadata, and audit trail for later compliance and evaluation.

Why this architecture

Lubu chose explicit control flow and audit-grade observability because healthcare needs safe routing, grounded answers, resumability, and reconstructable decisions; prompts alone cannot enforce the education-vs-diagnosis boundary or prove what knowledge the AI used.

Models used: LLM patient education responder · retrieval confidence scoring model or heuristic

Scale

• The digital health platform measures 30+ vital signs from face scans.
  “measuring 30+ vital signs from face scans.”article, stated
• The article references ChatGPT Health reaching 230 million weekly users as healthcare AI mainstream context.
  “ChatGPT Health](https://openai.com/index/introducing-chatgpt-health/) with medical record integrations for 230 million weekly users.”article, stated
• Low confidence is defined as retrieval confidence below 0.6 for safe-answer refusal.
  “if retrieval confidence < 0.6, explicitly return "I don't have enough information to answer safely" and route to human review.”article, stated
• Operational routing used a 0.7 threshold, tuned down from 0.8.
  “Threshold-based routing (< 0.7 → human review, ≥ 0.7 → proceed)”article, stated

  “False positive rate (unnecessary reviews) decreased 40% after threshold tuning from initial 0.8 → 0.7”article, stated
• About 15% of conversations triggered human review, and around 80% of those routes were appropriate.
  “~15% of conversations triggered human review (mostly edge cases or off-topic questions)”article, stated

  “Of those, ~80% were appropriately routed (validated via clinician feedback loops)”article, stated

Alternatives considered (roads not taken)

Rely on better prompts or model choice

Rejected because: The article says shippability came from explicit control flow and audit observability, not just better prompts or model.

Let the LLM decide when to route to a clinician

Rejected because: The route is explicitly threshold-based, not an LLM decision, because uncertainty handling is safety-critical.

Let the model fill weak retrieval gaps from training data

Rejected because: Testing found the model bridged weak retrieval with training data, which is unsafe in healthcare.

Log only final messages

Rejected because: Final-message logs could not reconstruct what was retrieved, routed, or scored three weeks earlier.

Patterns

rag-retrieval (grounding)

The agent retrieves from an approved, version-controlled medical knowledge base and passes only high-confidence documents to the model.

confidence-thresholding (routing)

Combined confidence determines whether the agent proceeds or routes to clinician review.

human-in-the-loop-gate (safety)

Low-confidence, symptom, medication, diagnosis, and review-triggered conversations route to human clinicians instead of forcing an answer.

observability-tracing (compliance)

LangSmith traces capture model calls, retrievals, confidence, routing, metadata, and version tags for audit and debugging.

durable-execution (state-management)

LangGraph checkpointing with PostgreSQL persists conversation state so patients can resume sessions hours or days later.

eval-driven-development (clinical-quality)

The team used LangSmith evaluation datasets, known-unknown tests, weekly audit sampling, and continuous monitoring for clinical accuracy.

Tradeoffs

Route low-confidence cases to clinicians.

Chose Safety-first human review at confidence thresholds over Maximizing automated answer rate.

Rationale: In healthcare, a safe refusal or review is better than a confident but unsupported answer.

Tune the review threshold from 0.8 to 0.7.

Chose A lower threshold after clinician feedback over Keeping the initial more conservative threshold.

Rationale: Tuning reduced unnecessary reviews while maintaining appropriate routing based on feedback loops.

Use trace-level redaction plus full audit export.

Chose Redacted monitoring traces and full audit retention over Either exposing PII in monitoring or losing audit detail.

Rationale: The system needs privacy-preserving observability and regulatory reconstructability at the same time.

Gotchas (what bit them)

The model can confidently bridge weak retrieval gaps with training data.

Root cause: Low-confidence retrieval was not reflected in the generated answer, so the model filled missing approved knowledge with latent training information.

Resolution: Return a safe refusal and route to human review when retrieval confidence is below threshold; only pass high-confidence documents to context.

A missing audit trail blocks regulated deployment.

Root cause: Early logs captured final messages but not the full retrieval, routing, confidence, and reasoning chain.

Resolution: Tag every LangSmith trace with compliance metadata, version information, redaction, and audit export support.

Prompt-only boundaries are unsafe in clinical settings.

Root cause: The agent must distinguish education from diagnosis and know when not to answer, which cannot be left to a model's judgment alone.

Resolution: Enforce routing and boundaries in LangGraph code.

Evaluation

Lubu evaluates clinical safety through confidence-routing metrics, clinician feedback loops, LangSmith known-unknown datasets, weekly audit sampling, and continuous trace monitoring.

Metrics: 15% conversations triggering human review · 80% appropriate routing validated by clinicians · 40% false-positive review reduction · very low inappropriate clinical advice rate · weekly audit sampling · retrieval confidence thresholds · known-unknown evaluation cases

Outcomes

• The system shipped to production for a digital health platform while preserving clinical review routes.
• Roughly 15% of conversations triggered human review, with about 80% appropriate routing according to clinician feedback.
• Threshold tuning reduced unnecessary reviews by 40%, and weekly audits found very low inappropriate clinical advice.

The deployment handles patient education for scans measuring 30+ vital signs, uses 0.6 and 0.7 confidence thresholds, routes about 15% of conversations to review, and reports 80% appropriate routing with 40% fewer false-positive reviews after tuning.