AI-Powered Medical Content Review and Revision at Scale

flo_health · Healthcare · 2026 · source Agentic: no · promotional bias: med · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Medical content review and revision

Flo Health needed to keep a large library of women's-health articles medically accurate as guidelines changed, while reducing manual expert workload and preserving editorial quality.

Constraints:

• medical content affects health decisions and requires high accuracy
• manual review of every article is slow, costly, and inconsistent
• guidelines and content may arrive as PDFs, TXT files, JSON files, or pasted text
• medical experts need interfaces for review, manual adjustment, and final validation
• long medical articles require chunking so unrelated text is not revised accidentally
• regulatory, ethical, and regional healthcare safeguards are required

Architecture

Flo Health and AWS built MACROS, an AI-assisted medical content review system around Amazon Bedrock and AWS orchestration. Content and guidelines enter through a Streamlit frontend on ECS, direct S3 uploads, or API-style integration. Step Functions orchestrate Lambda-hosted preprocessing, rule optimization, content review, revision, and statistics computation. Textract extracts PDF text. The review pipeline optionally filters irrelevant rules, chunks source text, reviews each section against rules, revises only non-adherent sections, and post-processes revisions back into the document. CloudWatch monitors the system, while medical experts review outputs in the UI and provide feedback.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  streamlit-frontend["streamlit-frontend"] --> api-gateway-auth["api-gateway-auth"]
  streamlit-frontend --> s3-content-store["s3-content-store"]
  streamlit-frontend --> step-functions-workflow["step-functions-workflow"]
  step-functions-workflow --> preprocessing-lambda["preprocessing-lambda"]
  preprocessing-lambda --> textract-parser["textract-parser"]
  preprocessing-lambda --> s3-content-store
  step-functions-workflow --> rule-optimizer["rule-optimizer"]
  step-functions-workflow --> content-review["content-review"]
  step-functions-workflow --> content-revision["content-revision"]
  rule-optimizer --> bedrock-models["bedrock-models"]
  content-review --> bedrock-models
  content-review --> content-revision
  content-revision --> bedrock-models
  content-revision --> post-processing["post-processing"]
  post-processing --> s3-content-store
  s3-content-store --> compute-stats["compute-stats"]
  compute-stats --> streamlit-frontend

Components

Data flow

A user prepares medical content and guidelines, then uploads PDF, TXT, JSON, or pasted text through the ECS-hosted Streamlit UI or directly to S3. Step Functions starts preprocessing in Lambda; Textract extracts PDF text when needed. Rule Optimizer and Content Review/Revision Lambda functions call Bedrock to extract rules, review chunks, and propose revisions. The system stores intermediate and final outputs in S3. A Compute Stats Lambda reads and combines review/revision results, and medical experts use the UI to inspect statistics, adjust changes, and validate output.

Why this architecture

MACROS is split into workflow steps because medical review needs traceable, bounded changes rather than free-form rewriting. Optional filtering avoids unnecessary calls when rules are irrelevant. Chunking helps review long articles section by section and prevents unintended edits elsewhere. Multi-call mode trades more model calls for higher detection accuracy by evaluating one rule per call. Model tiers separate simple chunking from complex review and revision to balance cost and quality. The UI keeps experts in control for validation and adjustment.

Models used: Amazon Bedrock foundation models · Claude Haiku models family · Claude Sonnet models family · Claude Opus models family · large language models

Scale

• Flo creates thousands of medical articles each year for millions of users.
  “Flo Health creates thousands of medical articles every year, providing millions of users worldwide with medically credible information on women’s health.”article, stated
• The target success metric included 90% content-piece recall.
  “Success Metrics: * Accuracy: Content piece recall of 90% * Efficiency: Reduce detection time from hours to minutes per guideline”article, stated
• The proof of concept aimed to reduce detection time from hours to minutes per guideline.
  “Efficiency: Reduce detection time from hours to minutes per guideline * Cost Reduction: Reduce expert review workload”article, stated
• The project target included a 10x speedup over manual review.
  “Speed: 10x faster than manual review process”article, stated
• The preliminary PoC result reported 80% accuracy and over 90% recall.
  “The solution exceeded target processing speed improvements while maintaining 80% accuracy and over 90% recall in identifying content requiring updates.”article, stated

Alternatives considered (roads not taken)

Manual review of every article

Rejected because: Manual review was time-consuming and prone to human error for a large, changing medical content library.

One Bedrock call assessing an article against all rules

Rejected because: Multi-call mode can improve non-adherent section detection by using one independent call per rule.

LLM filtering for every relevance check

Rejected because: The source notes non-LLM alternatives such as embedding similarity or keyword overlap can support filtering.

Use the same model size for every step

Rejected because: The architecture assigns simpler work to smaller models and complex reasoning/revision to larger models for cost efficiency.

Patterns

prompt-chaining (orchestration)

MACROS decomposes content review into filtering, chunking, review, revision, and post-processing stages, with Bedrock calls for different subtasks.

structured-output-contract (integration)

The review step emits XML with section text, adherence flags, rule names, and non-adherence reasons so outputs can be parsed reliably.

model-routing (model-operations)

The solution chooses smaller models for simpler steps such as chunking and larger Claude families for review and revision.

human-in-the-loop-gate (governance)

Medical experts remain in the loop through the UI, manual editing, review sessions, and final validation of AI suggestions.

eval-driven-development (evaluation)

The PoC used explicit success metrics and regular expert feedback to guide system refinement.

Tradeoffs

Use AI as augmentation, not final authority.

Chose human-AI collaboration with expert validation over full automation of medical content updates.

Rationale: AI can streamline review, but medical experts remain necessary for final validation and clinical precision.

Revise only non-adherent paragraphs.

Chose targeted revision after review over rewriting the whole article.

Rationale: Targeted revision preserves adherent sections and reduces the chance of introducing unrelated changes.

Offer multi-call mode for higher detection accuracy.

Chose one independent Bedrock call per rule over one call assessing the whole article against all rules.

Rationale: Separate calls improve attention to each rule but increase call volume and cost.

Use model tiers by task complexity.

Chose smaller models for simple steps and larger models for complex review/revision over one high-end model for every step.

Rationale: The tiered approach balances performance, accuracy, and cost efficiency.

Gotchas (what bit them)

Medical content review needs standardized inputs before AI review works well.

Root cause: Content and guidelines can arrive in inconsistent formats and structures.

Resolution: Standardize input formats and document structures, and create diverse test sets across medical topics.

Long medical articles are hard for LLMs to assess safely in one piece.

Root cause: The model can miss sections or alter unrelated text if context is not segmented.

Resolution: Chunk source text into paragraphs or semantically complete sections and revise only flagged non-adherent sections.

Medical AI needs strict compliance and expert oversight.

Root cause: The content is sensitive and subject to medical, ethical, and regional requirements.

Resolution: Use robust documentation, strict guideline version control, regional regulatory consideration, and continuous medical expert oversight.

Cost management quickly becomes important for multi-step review workflows.

Root cause: Review, revision, filtering, and multi-call modes can multiply token usage.

Resolution: Track token usage and optimize prompt design, batch processing, and model selection.

Evaluation

Flo defined PoC success metrics for recall, efficiency, workload reduction, quality, and speed; medical experts and content teams reviewed the system with AWS specialists through regular sessions.

Metrics: 90% content-piece recall target · hours-to-minutes detection time target · 10x faster than manual review target · 80% accuracy preliminary result · over 90% recall preliminary result · expert workload reduction

Outcomes

• the PoC achieved 80% accuracy and over 90% recall for identifying content needing updates
• the system reduced detection time from hours to minutes per guideline
• medical guidelines were applied more consistently than in manual reviews
• expert review burden was reduced while retaining expert final validation

Flo creates thousands of medical articles each year for millions of users; MACROS targets hours-to-minutes guideline review and 10x speed improvement over manual review.