Production AI Deployment: Lessons from Real-World Agentic AI Systems

databricks_/_various · Healthcare · 2026 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Healthcare generative AI deployment

Databricks and Flo Health needed to move generative AI from demos into production healthcare workflows where quality, privacy, cost, safety, and operational debugging all matter.

Constraints:

• Flo Health serves a large global user base with health data.
• Healthcare recommendations require strong accuracy, safety, privacy, and security.
• Production systems must be observable enough to debug quality and latency regressions.
• Many enterprises have regulatory, regional, and budget constraints that shape architecture.

Architecture

The Databricks approach treats agentic AI as modular software engineering. For Flo Health, Agent Bricks ingests data into a vector store, fine-tunes open-source models such as Llama, generates synthetic data, evaluates with LLM judges, and serves the resulting model. Around that core, teams define constraints, prefer deterministic steps over unconstrained autonomy, use MLflow judges and traces, track cost/latency/tool health, and build governance before broad production rollout.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  flo-health-app["Flo Health app"] --> unified-databricks-platform["Unified Databricks platform"]
  unified-databricks-platform["Unified Databricks platform"] --> agent-bricks-pipeline["Agent Bricks pipeline"]
  agent-bricks-pipeline["Agent Bricks pipeline"] --> vector-store["Vector store"]
  agent-bricks-pipeline["Agent Bricks pipeline"] --> open-source-model-tuning["Open-source model tuning"]
  agent-bricks-pipeline["Agent Bricks pipeline"] --> synthetic-data-generator["Synthetic data generator"]
  agent-bricks-pipeline["Agent Bricks pipeline"] --> llm-judge-evaluation["LLM judge evaluation"]
  agent-bricks-pipeline["Agent Bricks pipeline"] --> serving-endpoint["Serving endpoint"]
  open-source-model-tuning["Open-source model tuning"] --> serving-endpoint["Serving endpoint"]
  synthetic-data-generator["Synthetic data generator"] --> llm-judge-evaluation["LLM judge evaluation"]
  llm-judge-evaluation["LLM judge evaluation"] --> open-source-model-tuning["Open-source model tuning"]
  serving-endpoint["Serving endpoint"] --> flo-health-app["Flo Health app"]
  mlflow-tracing["MLflow tracing"] --> agent-bricks-pipeline["Agent Bricks pipeline"]
  mlflow-tracing["MLflow tracing"] --> evaluation-and-monitoring-loop["Evaluation and monitoring loop"]
  evaluation-and-monitoring-loop["Evaluation and monitoring loop"] --> llm-judge-evaluation["LLM judge evaluation"]

Components

Data flow

Flo connects health data into a unified Databricks platform. Agent Bricks ingests that data into a vector store, uses it to fine-tune an open-source model, generates synthetic data, measures behavior with LLM judges, and exposes the model through a serving endpoint. In production, tracing breaks a RAG-style application into retrieval, model, embedding, and other steps so teams can diagnose quality and latency problems.

Why this architecture

The design favors deterministic, modular systems because many production use cases do not need full agent autonomy. Agent Bricks automates the repetitive GenAI pipeline while MLflow judges/traces give teams a way to measure quality, diagnose component regressions, and keep costs and latency visible.

Models used: Llama · open-source models · LLM judges

Scale

• Flo Health reports more than 80 million monthly active users.
  “We have more than 80 millions of monthly active users.”transcript, stated
• The speaker gives a client constraint example of 20,000 daily users.
  “the current client that I'm working with expects 20,000 daily users, which is a lot.”transcript, stated
• Flo says Agent Bricks doubled medical accuracy and safety.
  “We using agent bricks to double our medical accuracy and safety and basically we were able to get higher quality of the models”transcript, stated
• A quality regression example involved a 10% drop after introducing a new component.
  “when I introduced a new component and out of the sudden the quality dropped by 10%.”transcript, stated
• The tracing example uses a 30-second latency diagnosis scenario.
  “imagine you build your system, it has 30 seconds latency, but you don't know where the latency comes from.”transcript, stated

Alternatives considered (roads not taken)

Full autonomous agents with free tool choice

Rejected because: The speaker says most use cases can be sequenced into deterministic steps instead.

Rely only on latest frontier APIs

Rejected because: Flo used open-source fine-tuning and smaller cheaper models to improve accuracy and cost.

Ship without observability

Rejected because: The presenter treats tracing, tool stats, cost tracking, and latency breakdowns as deployment requirements.

Patterns

fine-tuning-adaptation (modeling)

Flo fine-tunes open-source models such as Llama on health data through Agent Bricks.

synthetic-data-generation (data)

Agent Bricks generates synthetic data as part of the automated model-improvement pipeline.

llm-as-judge (evaluation)

LLM judges score outputs and can be customized for specific measurement criteria.

eval-driven-development (evaluation)

The framework starts with defining success and measurement before model or feature work.

observability-tracing (operations)

MLflow traces expose step inputs, outputs, retrieved documents, and latency breakdowns.

cost-aware-routing (operations)

The guidance includes cost tracking and provisioned throughput for high-traffic LLM applications.

rag-retrieval (architecture)

The production architecture is described around retrieval data preparation, vector stores, and RAG traces.

Tradeoffs

Autonomy level

Chose Deterministic sequenced steps over Free-choice autonomous tool use.

Rationale: Most enterprise use cases do not require full autonomy and benefit from guided system paths.

Model strategy

Chose Fine-tuned smaller open-source models over Always using large expensive frontier models.

Rationale: The Flo case argues for higher accuracy and cheaper solutions through Agent Bricks automation.

Observability stack

Chose MLflow tracing and judges over Opaque production calls.

Rationale: Traces identify whether latency comes from the LLM, retriever, embedding model, or another step.

Gotchas (what bit them)

Teams over-trust the LLM and under-engineer the system.

Root cause: Agentic AI is treated as autonomy rather than a set of engineered steps.

Resolution: Define components and deterministic steps that guide input to output.

Quality drops can be hard to debug after component changes.

Root cause: Many moving pieces make it unclear which step caused the regression.

Resolution: Use traces and per-step measurement to localize the failure.

Constraint blindness kills production projects.

Root cause: Teams start from use cases without budgets, traffic expectations, region limits, or security constraints.

Resolution: Ask constraint questions, including expected users and infrastructure cost, before launching the project.

Evaluation

Evaluation is framed as defining success, using MLflow/LLM judges and custom judges, tracing each component, and monitoring quality, safety, latency, and cost continuously.

Metrics: medical accuracy · safety · privacy · security · latency · tool success/failure · cost · quality score

Outcomes

• Flo Health unified fragmented data and used Agent Bricks to improve open-source model quality.
• The pipeline doubled medical accuracy and safety, according to the customer video.
• The framework produced lower-cost delivery for a larger audience.

Flo reports over 80 million monthly active users and doubled medical accuracy/safety; the talk also uses 20,000 daily users as a production capacity example.