Building Production-Ready Healthcare AI That Scales With Model Progress

anterior · Healthcare · 2026 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Healthcare prior authorization and clinical review AI

Anterior needed to build production LLM products for healthcare administration while foundation models were rapidly improving, so systems built around current limitations could become obsolete with the next model release.

Constraints:

• Prior authorization cases require reasoning over clinical guidelines, patient records, and policy documents.
• Early context windows could not fit entire medical-review cases at once.
• Healthcare workflows require accuracy, explainability, and customer-specific clinical nuance.
• Domain expertise is often tacit and not present in general model pretraining.
• Manual review by a single domain expert did not scale.
• Engineering teams still had to ship useful products while model capabilities changed every few months.

Architecture

Anterior's durable production pattern combines dynamic domain-knowledge injection with expert-review infrastructure. Clinical guidelines, patient records, and policy documents are processed by an LLM review workflow; a context-aware library of clinical tips is injected at inference time; clinicians review outputs in a dashboard; review data produces performance metrics, failure-mode classifications, suggested domain-knowledge injections, and in-context examples. Organizationally, the team uses a Sour Lesson scorecard, a 70/30 optimization-versus-experimentation split, monthly Anterior Labs hackathons, and modular code so obsolete techniques can be replaced quickly.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  prior-authorization-case[Prior authorization case] --> clinical-guidelines[Clinical guidelines]
  prior-authorization-case --> patient-medical-records[Patient medical records]
  prior-authorization-case --> llm-medical-review-workflow[LLM medical review workflow]
  clinical-guidelines --> llm-medical-review-workflow
  patient-medical-records --> llm-medical-review-workflow
  clinical-knowledge-library[Clinical knowledge library] --> domain-knowledge-selector[Domain knowledge selector]
  domain-knowledge-selector --> llm-medical-review-workflow
  llm-medical-review-workflow --> expert-review-dashboard[Expert review dashboard]
  clinician-reviewers[Clinician reviewers] --> expert-review-dashboard
  expert-review-dashboard --> review-data-pipeline[Review data pipeline]
  review-data-pipeline --> performance-reporting[Performance reporting]
  review-data-pipeline --> failure-mode-classifier[Failure mode classifier]
  review-data-pipeline --> clinical-knowledge-library
  review-data-pipeline --> in-context-example-store[In-context example store]
  in-context-example-store --> llm-medical-review-workflow
  sour-lesson-scorecard[Sour Lesson scorecard] --> experimentation-workflow[Experimentation workflow]
  experimentation-workflow --> anterior-labs-hackathon[Anterior Labs hackathon]
  experimentation-workflow --> modular-product-architecture[Modular product architecture]

Components

Data flow

A prior-authorization case brings together clinical guidelines, patient records, and policy context. The LLM workflow receives relevant clinical tips selected from the domain-knowledge library, produces a review output, and sends it to clinicians through the expert review dashboard. Review data is transformed into customer performance reports, failure-mode taxonomies, suggested domain-knowledge injections, and in-context examples, which feed back into future model runs. Separately, the Sour Lesson scorecard and experimentation workflow decide which technical approaches to keep, replace, or intentionally build as short-lived bridges.

Why this architecture

The enduring parts of Anterior's system focus on limitations that are not inherent to the LLM itself: customer-specific context, workflow-specific nuance, expert feedback, and product learning loops. These components compound with model progress because better models can use more injected context and richer expert-review data, while brittle workarounds for context-window or clinical-reasoning limits became obsolete as frontier models improved.

Models used: GPT-3.5-Turbo-16k · GPT-4 Turbo · GPT-4 · Claude 3 · Frontier models

Scale

• The author describes more than three years of LLM-powered product work.
  “3+ years building LLM-powered products at the frontier”article, stated
• Clinical guidelines often span 20-50 pages.
  “Clinical guidelines (often 20-50 pages, and with complex nested logic”article, stated
• Patient records can exceed 100 pages.
  “Patient medical records (can be 100+ pages)”article, stated
• The early product used GPT-3.5-Turbo-16k with a 16K context limit.
  “powered by GPT-3.5-Turbo-16k.”article, stated

  “16K tokens wasn’t enough to reason over all of this at once.”article, stated
• GPT-4 Turbo's 128K context window made whole-case reasoning feasible in November 2023.
  “In November 2023, GPT-4 Turbo launched with a 128K context window.”article, stated
• Hierarchical query reasoning took 1-2 months, won the first two enterprise customers, and was replaced within six months.
  “The whole thing took 1-2 months and landed us our first 2 enterprise customers”article, stated

  “within 6 months, we’d moved on to a different approach entirely.”article, stated
• Fine-tuning lost its advantage within 12-18 months.
  “Within 12-18 months, general models (GPT-4, Claude 3, etc.) became so good at clinical reasoning”article, stated
• Domain knowledge injection stayed in production for more than two years.
  “This approach is still in production 2+ years later”article, stated
• The author advocates a rough 70%/30% split between current-product optimization and experimentation.
  “I’ve advocated for a rough 70%:30% split at organizations where I’ve worked”article, stated

Alternatives considered (roads not taken)

Hierarchical query reasoning for 16K-context models

Rejected because: Long-context models made whole-case reasoning simpler, faster, and often more accurate.

Finetune specialized models for clinical reasoning

Rejected because: General frontier models became strong enough at medical reasoning that specialized finetunes no longer provided meaningful advantage.

Manual review by one medical doctor

Rejected because: A single expert could check outputs and improve the product early, but that review process did not scale.

Optimize only the current product

Rejected because: The author argues teams need workflow-level experimentation to avoid anchoring on today's state of play.

Maintain superseded code because it took effort to build

Rejected because: The real sunk cost is maintaining obsolete approaches after better model capabilities emerge.

Patterns

rag-retrieval (context-engineering)

Anterior dynamically injects context-specific clinical knowledge at inference time rather than relying only on model pretraining or fixed prompts.

human-in-the-loop-gate (review)

Clinicians perform structured expert reviews of AI outputs, and that review data drives metrics, failure-mode analysis, knowledge injections, and examples.

eval-driven-development (quality)

Expert-review data provides performance metrics and failure-mode classifications that prioritize improvement work and accelerate AI iteration cycles.

data-quality-loop (data)

Clinician reviews become a technical moat: more expert-reviewed data improves differentiation, in-context examples, and domain-knowledge selection.

fine-tuning-adaptation (training)

Anterior tried fine-tuning for clinical reasoning and output formatting, but later deprioritized it once general models became strong enough.

Tradeoffs

Ship hierarchical reasoning despite obsolescence risk

Chose A tree-structured workaround for 16K context over Waiting for long-context models.

Rationale: The approach became obsolete within six months, but it served customers and helped win the first two enterprise customers.

Inject domain knowledge at inference time

Chose Dynamic clinical tips selected by context over Depending only on the model's general clinical reasoning.

Rationale: Customer-specific and workflow-specific nuance is not inherent to general models, and larger context windows make the approach stronger over time.

Build expert-review infrastructure

Chose A clinician review dashboard and downstream data system over Ad hoc manual review by the founder/doctor.

Rationale: Structured reviews scale expert judgment, create performance metrics, and produce reusable product data.

Allocate resources across present and future

Chose Rough 70% current-product optimization and 30% new-approach testing over All-in current optimization or detached experimentation.

Rationale: The split acknowledges the need to ship now while continuously testing whether new model capabilities change the right product approach.

Make experimentation cheap

Chose Modular architecture with swappable components over Highly coupled systems that preserve old approaches.

Rationale: Low coupling lets teams test new model capabilities and replace entire subsystems before obsolete code becomes a maintenance burden.

Use monthly disruption rituals

Chose Anterior Labs hackathons over Passive awareness of new model releases.

Rationale: A recurring internal practice forces teams to test new models, capabilities, and tools against the current product and has produced ideas that entered the main product.

Gotchas (what bit them)

Workarounds for temporary context limits can become obsolete quickly.

Root cause: Hierarchical query reasoning solved the 16K context limit, but 128K models made whole-case reasoning possible.

Resolution: Use the workaround when it serves customers, but make the hypothesis explicit and be ready to replace it.

Domain-specific fine-tuning can lose to frontier model progress.

Root cause: General models absorbed enough medical literature and clinical data that clinical reasoning was no longer the limiting factor.

Resolution: Try prompting/context engineering first in specialized verticals unless fine-tuning solves a durable limitation.

Building too specifically around current limitations creates technical debt.

Root cause: Model capabilities change, and product architecture can be made redundant by the next model release.

Resolution: Prioritize approaches that compound with model progress and score approaches against plausible model-evolution scenarios.

Building too abstractly for future capabilities prevents shipping.

Root cause: Teams still need to serve customers today using current model capabilities.

Resolution: Accept some short-lived approaches when they materially help customers or company trajectory, but keep the tradeoff explicit.

A separate experimental team can anchor away from real product needs.

Root cause: If experimentation is isolated by engineer rather than workflow, experimental engineers may still carry today's assumptions or disconnect from product realities.

Resolution: Have all engineers mix optimization and experimentation at the workflow level.

Evaluation

Anterior's durable evaluation loop is expert-review driven: clinicians review AI outputs, the system converts reviews into performance metrics and failure modes, and those data guide domain-knowledge injections, in-context examples, customer reporting, and iteration priorities. The Sour Lesson scorecard also evaluates approaches against future model-capability scenarios before allocating effort.

Metrics: Clinician review outcomes · Performance metrics from expert reviews · Failure mode classifications · Suggested domain knowledge injections · In-context example generation · Customer performance reporting · Approach relevance under context-window expansion · Approach relevance under improved domain reasoning · Approach relevance under multimodal capability · Approach relevance under workflow/last-mile improvements

Outcomes

• Hierarchical query reasoning helped land the first two enterprise customers but was replaced within six months after GPT-4 Turbo's 128K context window.
• Clinical fine-tuning became less useful within 12-18 months as general frontier models improved at medical reasoning.
• Dynamic domain knowledge injection remained in production for more than two years.
• The expert review system became a core product component for customer reporting, AI iteration, and performance improvement.
• Review data became part of Anterior's technical moat.
• Anterior Labs monthly hackathons regularly produced ideas that entered the main product.
• The organization adopted a mindset of explicit hypotheses, modular architecture, and willingness to delete superseded code.

The source includes unusually specific model-progress and workflow numbers: 20-50 page guidelines, 100+ page records, 16K to 128K context expansion, 1-2 month build, first two enterprise customers, six-month replacement window, 12-18 month fine-tuning obsolescence, 2+ years production durability, and a rough 70/30 allocation split.