Production Evolution of an AI-Powered Medical Consultation Assistant

doctolib · Healthcare · 2023 · source Agentic: no · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Healthcare consultation documentation

Doctolib needed to evolve an AI Consultation Assistant from MVP to reliable clinical workflow support, converting speech into structured medical summaries and ICD-10-like codification while collecting enough feedback to improve safely.

Constraints:

Architecture

Doctolib’s assistant combines ASR, medical summarization, and ontology codification. The team bootstrapped evaluation with simulated consultations and fake patient personas, then collected explicit ratings/comments plus implicit usage signals such as edits and validations. Quality evaluation uses LLM-as-judge for summaries, F1 for codification, medical-expert go/no-go review, A/B tests for product changes, and two improvement loops: prompt engineering over days/weeks and data/model fine-tuning over weeks/months.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  consultation-audio-recorder["Consultation audio recorder"] --> asr-transcription["ASR transcription"]
  asr-transcription["ASR transcription"] --> summarization-model["Summarization model"]
  summarization-model["Summarization model"] --> medical-codification-model["Medical codification model"]
  summarization-model["Summarization model"] --> clinician-review-surface["Clinician review surface"]
  summarization-model["Summarization model"] --> llm-as-judge-evaluator["LLM-as-judge evaluator"]
  medical-codification-model["Medical codification model"] --> f1-codification-evaluator["F1 codification evaluator"]
  clinician-review-surface["Clinician review surface"] --> feedback-collection-system["Feedback collection system"]
  feedback-collection-system["Feedback collection system"] --> online-metrics-dashboard["Online metrics dashboard"]
  feedback-collection-system["Feedback collection system"] --> short-term-prompt-loop["Short-term prompt loop"]
  feedback-collection-system["Feedback collection system"] --> long-term-fine-tuning-loop["Long-term fine-tuning loop"]
  online-metrics-dashboard["Online metrics dashboard"] --> a-b-testing-platform["A/B testing platform"]
  llm-as-judge-evaluator["LLM-as-judge evaluator"] --> short-term-prompt-loop["Short-term prompt loop"]
  f1-codification-evaluator["F1 codification evaluator"] --> long-term-fine-tuning-loop["Long-term fine-tuning loop"]
  medical-expert-review["Medical expert review"] --> short-term-prompt-loop["Short-term prompt loop"]
  medical-expert-review["Medical expert review"] --> a-b-testing-platform["A/B testing platform"]
Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component Role Interacts with
Consultation audio recorder Captures simulated or real consultation audio. ASR transcription
ASR transcription Converts consultation audio into text. Consultation audio recorder, Summarization model
Summarization model Produces structured medical consultation summaries. ASR transcription, Medical codification model, Clinician review surface, LLM-as-judge evaluator
Medical codification model Maps summary content to medical ontology concepts such as ICD-10. Summarization model, F1 codification evaluator
Clinician review surface Lets practitioners validate, edit, delete, rate, and comment on assistant outputs. Summarization model, Feedback collection system
Feedback collection system Collects explicit and implicit feedback signals from usage. Clinician review surface, Online metrics dashboard, Short-term prompt loop, Long-term fine-tuning loop
Online metrics dashboard Tracks adoption, activation, acceptance, ratings, bad experience rate, and bias patterns. Feedback collection system, A/B testing platform
LLM-as-judge evaluator Automated evaluator for summary hallucination and recall/completeness. Summarization model, Short-term prompt loop
F1 codification evaluator Measures medical content codification accuracy. Medical codification model, Long-term fine-tuning loop
Medical expert review Human safety/usability gate for candidates before exposure to real beta testers or broad rollout. Short-term prompt loop, A/B testing platform
A/B testing platform Randomly assigns consultations to test/control groups for hypothesis and do-not-harm tests. Online metrics dashboard, Medical expert review
Short-term prompt loop Days-to-weeks prompt/refinement loop for immediate user frustrations. Feedback collection system, LLM-as-judge evaluator, Medical expert review
Long-term fine-tuning loop Weeks-to-months data/model improvement loop using weak labeling, annotation, and fine-tuning. Feedback collection system, F1 codification evaluator

Data flow

A consultation is recorded and transcribed by ASR. The transcript is summarized into a structured medical note and mapped to medical ontologies. Before broad beta, simulated consultations supply initial data. In production, explicit ratings/comments and implicit edits/deletions/validations feed online metrics. Candidate prompt or model changes pass medical expert review, backtesting, A/B testing, and do-not-harm checks before rollout; longer-term feedback enriches annotation and fine-tuning data.

3 supporting quotes
“The system combines three core components: automatic speech recognition (ASR) to convert consultation audio into text, summarization to transform transcripts into structured medical consultation summaries, and medical content codification”summary, stated
“Their explicit feedback mechanism solicits direct input from practitioners through ratings and verbatim comments.”summary, stated
“Their implicit feedback mechanism captures usage signals without requiring any additional user action.”summary, stated

Why this architecture

The architecture separates feedback, evaluation, and improvement loops because clinical quality must be measured from actual practitioner behavior, not only offline metrics. Simulated consultations solved the initial data cold start; explicit/implicit feedback provides production signal; A/B testing protects against misleading backtests and second-order infrastructure changes.

3 supporting quotes
“effective AI systems require user feedback for improvement, but meaningful feedback only comes from users engaging with a reasonably functional system.”summary, stated
“To generate initial training and evaluation data without real users, the team conducted "fake consultations" by leveraging their internal workforce.”summary, stated
“They use A/B testing for two distinct purposes: Hypothesis Testing to validate and quantify the positive impact of promising new versions”summary, stated

Models used: ASR model · summarization model · medical codification model · LLM-as-judge evaluator

Scale

Alternatives considered (roads not taken)

Use real clinicians immediately for initial training data

Rejected because: The product needed a safe, usable MVP before meaningful real-user feedback could be collected.

“The team's solution was pragmatic—they accepted that initial performance would be imperfect and focused on building a "safe and usable" MVP.”summary, stated

Adopt ROUGE/BLEU/BERTScore as summary-quality metrics

Rejected because: Those metrics did not correlate well with actual summary quality for this medical pipeline.

“commonly used academic metrics like ROUGE, BLEU, and BERTScore proved less effective than expected at correlating with actual summary quality”summary, stated

Rely only on explicit clinician ratings

Rejected because: Explicit feedback is high quality but optional and sparse; implicit behavioral signals add scale without disrupting workflow.

“This continuous stream of behavioral data complements the targeted insights from explicit feedback and provides much larger sample sizes for analysis.”summary, stated

Only backtest changes offline

Rejected because: A/B do-not-harm tests catch infrastructure or pseudonymization changes that may hurt user experience despite neutral backtests.

“"Do Not Harm" Testing to ensure that infrastructure changes, pseudonymization methods, or other under-the-hood modifications don't negatively affect user experience”summary, stated

Patterns

synthetic-data-generation (data)

Fake consultations and patient personas bootstrap realistic initial evaluation/training data.

“Team members role-played with actual practitioners and created fake patient personas to simulate plausible consultation scenarios.”summary, stated

llm-as-judge (evaluation)

Doctolib uses LLM-as-judge for automated summary quality dimensions such as hallucination and recall.

“For summary quality assessment, they employed an LLM-as-judge approach for automated evaluation. This allowed them to focus on critical quality dimensions such as hallucination rate (for factuality) and recall (for completeness).”summary, stated

human-in-the-loop-gate (clinical safety)

Medical experts perform safety/usability review before real beta exposure and validate prompt candidates.

“Before deployment to real beta testers, internal medical experts conducted thorough safety and usability reviews, establishing a formal go/no-go decision process.”summary, stated

eval-driven-development (evaluation)

A/B testing and metric validation drive changes rather than assuming offline metrics transfer.

“To mitigate these biases and isolate the true impact of changes, the team employs A/B testing as their "gold standard" for evaluation. They randomly assign consultations to test or control groups and compare KPIs between groups after a defined period.”summary, stated

data-quality-loop (improvement)

Explicit and implicit feedback feed annotation, weak labeling, and fine-tuning loops.

“This includes refining data annotation processes by incorporating user feedback and addressing identified model weaknesses, adapting sampling strategies, annotation tools, guidelines, and review processes.”summary, stated

prompt-versioning (prompt operations)

The short-term loop benchmarks and refines prompts before medical validation and A/B rollout.

“benchmarking and refining prompts through prompt engineering, subjecting promising candidates to medical expert validation”summary, stated

Tradeoffs

Initial data strategy

Chose Simulated consultations over Waiting for real-user production feedback.

Rationale: Simulation allowed safe early evaluation without exposing patients or burdening clinicians.

“This approach provided crucial data for initial system evaluation without exposing real patients or burdening actual practitioners”summary, stated

Feedback collection

Chose Explicit plus implicit feedback over Mandatory ratings only.

Rationale: Explicit comments are interpretable but sparse; implicit signals scale without disrupting clinical work.

“The team designed a dual-mode feedback collection system that balances thoroughness with minimal disruption to busy healthcare professionals. This is a crucial consideration in healthcare settings where practitioner time is precious and workflow interruption must be minimized.”summary, stated

Metric choice

Chose LLM-as-judge for summaries and F1 for codification over One generic metric set for all outputs.

Rationale: Different pipeline stages need metrics that correlate with their actual quality target.

“For medical content codification, they used the standard F1 score to measure their ability to detect and correctly code medical concepts.”summary, stated

Improvement horizon

Chose Short-term prompt loop plus long-term fine-tuning loop over A single optimization cadence.

Rationale: Immediate frustrations and foundational model/data improvements operate on different time scales.

“The team implemented a multi-tiered optimization strategy with distinct time-frame loops, recognizing that different types of improvements require different iteration speeds.”summary, stated

Gotchas (what bit them)

User-feedback cold start blocked improvement.

Root cause: Useful feedback requires users to engage with a reasonably functional assistant first.

Resolution: Create simulated consultations with fake patient personas and a safe MVP.

“effective AI systems require user feedback for improvement, but meaningful feedback only comes from users engaging with a reasonably functional system.”summary, stated

Academic metrics did not match production medical summary quality.

Root cause: ROUGE, BLEU, and BERTScore did not correlate with the qualities Doctolib needed.

Resolution: Validate metric relevance and use LLM-as-judge plus F1 for codification.

“established metrics don't always transfer well to specific production contexts, and teams should validate metric relevance empirically.”summary, stated

Feedback data contains bias.

Root cause: New User Bias, Memory Effect, and Weekend/Holiday Effect distort raw ratings.

Resolution: Use A/B testing and bias-aware online metric interpretation.

“They identify three specific bias patterns: New User Bias, where new users tend to provide more positive ratings due to initial enthusiasm; Memory Effect”summary, stated

Prompt engineering has diminishing returns.

Root cause: Easy prompt fixes get exhausted while deeper quality issues require data/model changes.

Resolution: Run a longer-term loop for weak labeling, annotation process refinement, and fine-tuning.

“the explicit acknowledgment that prompt engineering has diminishing returns and that long-term improvements require investment in data quality and model training”summary, stated

Evaluation

Doctolib combines simulated consultations, LLM-as-judge summary evaluation, F1 codification scoring, medical expert go/no-go review, explicit/implicit online feedback metrics, bias-aware interpretation, and A/B testing.

Metrics: hallucination rate · recall · F1 score · Power User Rate · Activation Rate · Suggestion Acceptance Rate · Average Rating · Bad Experience Rate

“For adoption tracking, they monitor Power User Rate (proportion of beta testers using the assistant for all consultations) and Activation Rate”summary, stated
“For quality tracking, they monitor Suggestion Acceptance Rate (reflecting ASR accuracy and utility), Average Rating”summary, stated

Outcomes

The strongest scale details are over one year of beta/improvement work, days-to-weeks prompt loops, weeks-to-months model loops, and tracked adoption/quality metrics rather than exact adoption counts.

“The product launched to widespread adoption and positive feedback after over a year of continuous improvement with beta testers and medical experts.”summary, stated
“While they report "widespread adoption and overwhelmingly positive feedback," concrete metrics on improvement magnitudes, error rates, or clinical outcomes are not provided. This is common in company blog posts but limits independent verification of the claimed success.”summary, stated