Enterprise-Scale Healthcare LLM System for Unified Patient Journeys

john_snow_labs · Healthcare · 2024 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Healthcare patient-journey analytics

John Snow Labs needed to turn fragmented multimodal medical data into unified patient journeys that clinicians, nurses, administrators, and researchers can query in natural language while preserving PHI security, clinical accuracy, consistency, explainability, and enterprise scale.

Constraints:

• Patient data spans structured EHR tables, FHIR messages, unstructured notes, PDFs, imaging, claims, prescriptions, lab reports, and years of history.
• The system must run inside the customer's security perimeter without external LLM calls or data exfiltration.
• Organizations may serve tens of millions of patients and billions of documents, making long-context approaches unsuitable.
• Healthcare query answers must be accurate, consistent, explainable, and optimized against very large relational tables.
• Medical terminology, dates, duplicate mentions, uncertainty, and stakeholder-specific definitions must be normalized and auditable.

Architecture

John Snow Labs built an on-prem or customer-cloud platform that ingests patient data from APIs and files, uses healthcare-specific LLMs to extract clinical entities, assertion status, relations, codes, dates, and summaries, normalizes terms through SNOMED and other terminology systems, deduplicates repeated mentions into a patient knowledge graph, and stores traceable results in an OMOP-based relational database. Natural-language queries are handled by an agent that maps the request to optimized query templates, resolves terminology, adjusts SQL/schema, executes efficient queries, and explains the logic and provenance.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  patient-data-sources["Patient data sources"] --> healthcare-llm-extraction["Healthcare LLM extraction"]
  healthcare-llm-extraction --> terminology-service["Terminology service"]
  healthcare-llm-extraction --> temporal-normalization["Temporal normalization"]
  terminology-service --> deduplication-and-reasoning["Deduplication and reasoning"]
  temporal-normalization --> deduplication-and-reasoning
  deduplication-and-reasoning --> omop-relational-database["OMOP relational database"]
  omop-relational-database --> query-agent["Query agent"]
  query-agent --> rag-query-template-tool["RAG query-template tool"]
  query-agent --> terminology-service
  query-agent --> sql-and-schema-adjustment["SQL and schema adjustment"]
  query-agent --> result-explanation-ui["Result explanation UI"]
  rag-query-template-tool --> sql-and-schema-adjustment
  sql-and-schema-adjustment --> omop-relational-database
  sql-and-schema-adjustment --> result-explanation-ui

Components

Data flow

Raw patient documents and structured resources enter the system, where medical LLMs extract entities and semantic filters. Terminology services map codes, temporal logic resolves relative dates, reasoning merges and deduplicates repeated findings, and the resulting patient knowledge graph is stored in an OMOP relational database. A query agent then maps natural language to optimized query templates, resolves clinical terms, adjusts SQL, runs the query, aggregates results, and explains the business logic.

Why this architecture

The architecture rejects a simple long-context or text-to-SQL design because healthcare data is too large, messy, terminology-heavy, and trust-sensitive. Customer-local deployment protects PHI; OMOP gives an open operational data model; medical LLMs handle extraction and reasoning; and the query agent constrains natural-language requests to optimized, explainable query paths.

Models used: Healthcare-specific LLMs · John Snow Labs medical LLMs · SNOMED resolvers · FHIR · OMOP · RAG · SQL-generation/query agents

Scale

• The platform targets organizations serving 10 million to 30 million patients per year.
  “we're working with organizations that serve 10 million patients 20 million patients 30 million patients per year”transcript, stated
• A 10-million-patient organization can have around half a million data points per patient over five years.
  “if you have 10 million patients you easily going to have you know half a million data points about them over you know even a 5year period”transcript, stated
• The system is designed for millions of patients and billions of documents.
  “because we're going to deal here with you know millions of patients billions of documents and we need to be able to update the data in an ongoing basis”transcript, stated
• The extraction layer covers more than 400 entity types.
  “there are more than 400 types of entities we know to automatically extract from free text”transcript, stated
• Example query storage scale is 10 million patients and 300 million extracted rows.
  “I have this database say have 10 million patients 300 million you know rows of different things are extracted from different modalities”transcript, stated
• The speaker says their medical LLM wins almost 2:1 over GPT-4 in blind physician evaluation.
  “we ask medical doctors basically blind pick who they prefer right so their options are you know prefer model number one model number two”transcript, stated

Alternatives considered (roads not taken)

Use a proprietary knowledge base or database that does everything.

Rejected because: Enterprise data infrastructure needed an open, scalable, familiar model with tooling around it, so they chose OMOP.

Use long-context LLMs over patient histories.

Rejected because: The system operates at billions-of-documents scale, beyond any practical context-window framing.

Treat clinical querying as direct text-to-SQL.

Rejected because: Healthcare queries require terminology lookup, schema adjustment, optimized query templates, and explanation beyond what direct SQL generation can reliably do.

Allow arbitrary generated SQL against massive tables.

Rejected because: Unoptimized queries could hit 200-300 million-row tables and crash shared infrastructure.

Patterns

privacy-preserving-processing (security)

The platform runs inside the customer's infrastructure and does not send PHI to external LLMs, John Snow Labs, cloud providers, or OpenAI.

fine-tuning-adaptation (model adaptation)

John Snow Labs uses its own healthcare-specific LLMs fine-tuned and evaluated for clinical summarization, extraction, question answering, and query mapping.

tool-use-orchestration (agent architecture)

The query agent calls separate tools for RAG query-template matching, terminology resolution, schema adjustment, SQL execution, and presentation.

rag-retrieval (retrieval)

RAG maps a user's natural-language query to pre-built optimized query templates that the system knows how to run.

data-quality-loop (data processing)

The system normalizes terminology and dates, deduplicates repeated clinical mentions, and preserves traceability/confidence before storing patient facts.

Tradeoffs

Use open OMOP relational infrastructure.

Chose An open OMOP-based relational model over A proprietary data store or knowledge base that did everything.

Rationale: The platform is intended to become enterprise data infrastructure, so it needs open tools, scalability, and operational familiarity.

Deploy entirely inside customer infrastructure.

Chose On-prem/customer-cloud execution with no external LLM calls over Calling hosted external LLM APIs for PHI processing.

Rationale: Healthcare PHI and enterprise security requirements require all data and processing to stay inside the customer's perimeter.

Use a multi-step query agent.

Chose RAG template matching, terminology lookup, schema adjustment, SQL execution, and explanation over One-shot text-to-SQL against the patient database.

Rationale: Healthcare queries require reasoning over optimized queries, terminologies, schemas, and user explanations.

Gotchas (what bit them)

Long-context LLMs are the wrong abstraction for patient-journey scale.

Root cause: The input universe is millions of patients and billions of documents, not a document set that fits into context.

Resolution: Use extraction, normalization, relational storage, and query agents over a patient knowledge graph.

Repeated clinical mentions can swamp the true patient state.

Root cause: Hospital stays produce many repeated progress notes with symptoms at different specificity levels and temporary relevance.

Resolution: Deduplicate and reason field-by-field to produce the actual diagnoses, measurements, and uncertainty state.

Inconsistent answers destroy clinical trust.

Root cause: LLMs can answer the same question differently, which is unacceptable for doctors asking factual yes/no patient questions.

Resolution: Constrain querying through optimized templates, deterministic data stores, consistency requirements, and explanations.

Clinical text-to-SQL is too complex for generic SQL generation.

Root cause: Even simple healthcare queries require multi-page SQL, joins across clinical tables, and terminology hierarchy reasoning.

Resolution: Use a multi-step agent with template retrieval, terminology service, schema adjustment, optimized SQL, and explanation.

Evaluation

Evaluation combines blind physician preference testing of medical LLMs, clinical factuality/relevance/conciseness checks, consistency requirements, query-performance optimization, and explainability/provenance review.

Metrics: Blind physician preference against GPT-4 · Factuality · Clinical relevancy · Conciseness · Query consistency · Query speed and optimized execution · Traceability and explanation quality

Outcomes

• The platform can unify patient histories across modalities into an OMOP-backed patient knowledge graph.
• Medical LLMs reportedly outperformed GPT-4 in blind randomized physician evaluation.
• Clinical users can ask natural-language cohort and patient-history questions without directly knowing SQL or BI tools.

The system is positioned for 10-30 million patient/year organizations, hundreds of millions of extracted rows, and billions of documents.