Scaling Healthcare AI Agents from Prototype to 100 Million Conversations

hyro · Healthcare · 2026 · source Agentic: yes · promotional bias: med · summary-only (no fetched source) · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: healthcare conversational agents

Hyro needed to scale healthcare AI agents from custom deployments to a platform serving major health systems while preserving deterministic reliability for mission-critical conversations.

Constraints:

• Healthcare users needed appointment and contact-center tasks to be reliable, not merely conversational.
• The company had to scale from dozens of agents to thousands across roughly 50 healthcare systems.
• Voice conversations imposed tight latency budgets, with every speech and model hop consuming response time.

Architecture

Hyro evolved from deterministic linguistic parsing and knowledge/capabilities graphs into a modular healthcare-agent platform. The platform uses skills, building blocks, integrations, a computational graph, speech-to-text/text-to-speech layers, selected LLM calls, provider fallbacks, and multi-region deployment so customer-specific agents can be assembled quickly without losing control.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  A["Voice and web channels"] --> B["Speech-to-text layer"]
  B --> C["Computational graph"]
  A --> D["Skills platform"]
  D --> C
  C --> E["Knowledge graph"]
  D --> F["Customer integrations"]
  F --> E
  C --> G["Language model decision points"]
  G --> H["Provider fallback layer"]
  H --> I["Text-to-speech layer"]
  I --> A

Components

Data flow

User speech or web input enters Hyro's stack, speech-to-text converts voice inputs, the computational graph and skills decide the conversation path, integrations and knowledge graph data fill needed slots, selected language models support decisions and response generation, and text-to-speech returns voice output. Provider and region fallbacks keep the same flow available during failures.

Why this architecture

Hyro chose a hybrid controlled stack because healthcare requires reliability, explainability, and fallback control, while modern LLMs add conversational understanding only at selected points.

Models used: speech-to-text models · text-to-speech models · OpenAI models · Gemini · Hyro-trained linguistic/data-science models

Scale

• Hyro serves roughly 50 major healthcare systems.
  “They serve approximately 50 major healthcare systems covering around 100 million people”summary, stated
• The platform covers around 100 million people.
  “They serve approximately 50 major healthcare systems covering around 100 million people”summary, stated
• Hyro processed roughly 100 million calls over two years in production.
  “processing roughly 100 million calls over the past two years in production.”summary, stated
• The platform shift reduced deployments from months to days.
  “The result was transformative: time-to-deployment dropped from months to days”summary, stated

Alternatives considered (roads not taken)

End-to-end speech-to-speech model

Rejected because: It seemed too risky for healthcare because Hyro would lose deterministic control over mission-critical tasks.

Generic multi-industry agents

Rejected because: Hyro narrowed to healthcare so the product could become more accurate and relevant.

Single model managing appointment scheduling

Rejected because: Hyro found that one model could hallucinate unavailable appointment times, so it used specialized models inside controlled logic.

Patterns

tool-use-orchestration (agent-architecture)

Hyro decomposes healthcare conversations into skills, building blocks, and integrations that the platform assembles into customer agents.

fallback-degradation (reliability)

Speech, model, and regional providers have fallback paths so healthcare conversations continue during provider or infrastructure failures.

model-routing (model-operations)

Hyro uses multiple specialized models at specific points inside the computational graph rather than one model controlling the whole conversation.

latency-optimization (production-operations)

Voice-agent architecture treats latency as an ongoing development constraint because every model and speech hop consumes the user's patience budget.

Tradeoffs

LLM integration architecture

Chose layered computational graph with selected LLM calls over end-to-end speech-to-speech model.

Rationale: The layered design preserved control, explainability, and fallback reliability for healthcare workflows.

Market focus

Chose healthcare-only specialization over serving many verticals with generic agents.

Rationale: Specialization increased product relevance even though it meant abandoning customers and production code.

Voice and web product design

Chose separate voice and web offerings over one shared product across channels.

Rationale: The split increased complexity but let Hyro optimize for the different interaction patterns of voice and web.

Gotchas (what bit them)

Conversational LLMs could hallucinate appointment slots.

Root cause: A single model-generated flow could invent times not present in the healthcare scheduling system.

Resolution: Hyro kept deterministic flow control in production and later reintroduced LLMs as five specialized model roles inside the architecture.

Platform quality depended on configuration quality.

Root cause: Skills and building blocks only work well if solutions architects connect and configure them correctly.

Resolution: Hyro exposed events and analytics so customers and internal teams could see what users wanted and how agents performed.

Latency improvements could be erased by a new heavy feature.

Root cause: Every additional capability in the layered architecture adds response-time cost.

Resolution: Hyro reframed latency as a continuous mindset rather than a one-time project.

Evaluation

Hyro evaluates through production reliability, explainability, deployment speed, customer analytics, conversation events, and healthcare task success such as appointment scheduling.

Metrics: 100% reliability target · deployment time from months to days · calls processed · customers served · latency · appointment scheduling success

Outcomes

• Hyro reduced deployment time from months to days.
• Hyro supports approximately 50 major healthcare systems and around 100 million people.
• The platform processes roughly 100 million production calls over two years.
• The architecture supports conversational appointment scheduling while preserving deterministic explainability.

This extraction is summary-only because the YouTube transcript fetch remained retryable, so source-backed review should discount it relative to fetched cases.