Document Metadata Extraction at Scale Using Generative AI for Healthcare and Financial Services

aarete · Consulting · 2024 · source Agentic: no · promotional bias: high · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Healthcare and financial-services document metadata extraction

AArete needed to extract structured metadata from provider and vendor contracts at a scale and accuracy that manual processing and rules-based extraction could not deliver.

Constraints:

Healthcare payer and financial-services documents contain payment terms, obligations, SLAs, KPIs, and other business-critical fields.
Manual extraction required institutional knowledge and only processed about 100 documents per week per person.
Rules-based OCR extraction failed on document variation and only reached about 50-60% accuracy.
Accuracy matters because extracted contract metadata drives claims, reimbursement, and savings analysis.
The production target required hundreds of thousands of documents per week.

Architecture

AArete built Doczy.ai, a generative-AI metadata extraction and interpretation system. Documents are processed through OCR/text extraction and then API calls to AWS Bedrock using Nova Pro and Anthropic Sonnet models. The system interprets provider and vendor contracts like a human analyst, extracts structured metadata into repositories for downstream analytics, and combines generative AI with AArete's healthcare ontology/domain expertise.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  provider-and-vendor-contracts["Provider and vendor contracts"] --> doczy-ai-pipeline["Doczy.ai pipeline"]
  doczy-ai-pipeline --> ocr-text-extraction["OCR/text extraction"]
  doczy-ai-pipeline --> aws-bedrock-model-calls["AWS Bedrock model calls"]
  doczy-ai-pipeline --> structured-metadata-repository["Structured metadata repository"]
  structured-metadata-repository --> claims-and-savings-analytics["Claims and savings analytics"]

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Provider and vendor contracts`	Unstructured documents containing reimbursement, obligations, SLA, KPI, and vendor terms.	`Doczy.ai pipeline`
`Doczy.ai pipeline`	Generative-AI document metadata extraction and interpretation workflow.	`OCR/text extraction`, `AWS Bedrock model calls`, `Structured metadata repository`
`OCR/text extraction`	Converts PDFs/documents into text before interpretation.	`Doczy.ai pipeline`
`AWS Bedrock model calls`	Nova Pro and Anthropic Sonnet API calls for generative extraction.	`Doczy.ai pipeline`
`Structured metadata repository`	Stores extracted contract fields for downstream use.	`Claims and savings analytics`
`Claims and savings analytics`	Compares contract reimbursement information to claims and identifies savings.	`Structured metadata repository`

Data flow

Provider or vendor contracts are written/signed in a healthcare payer environment, processed through Doczy.ai, converted/interpreted by Bedrock-hosted models, and stored as structured metadata for downstream reimbursement, claims-overpayment, and contract analysis.

“it looks at the document, understands it like a human would extract the information from it into a structured repository that is then usable for downstream needs.”transcript, stated

“each provider contract or vendor contract when they are written, when they're signed it process through Doczy, gets the metadata out of it”transcript, stated

Why this architecture

AArete moved from manual and rules-based extraction to generative AI because contracts varied too much for page/line/character rules and manual processing could not scale. Bedrock-hosted models provided flexible interpretation while the team's ontology and domain expertise anchored the extraction.

“If it followed a certain template, the results would be consistent. If the documents had varied level of information at varied locations, we were out of luck.”transcript, stated

“we could not have otherwise done if not for the technology, obviously along with the ontology and the domain expertise that we incorporated into it.”transcript, stated

Models used: AWS Bedrock · Amazon Nova Pro · Anthropic Sonnet 3.5 · Anthropic Sonnet 3.7 · Anthropic Sonnet 4.0 · AWS Textract

Scale

Manual extraction processed about 100 documents per week per person.
“we were only processing a hundred documents per week per person using that manual approach.”transcript, stated
Rules-based extraction produced about 50-60% accurate results.
“it was giving us essentially, you know, 50% to 60% accurate results.”transcript, stated
Doczy.ai processed about 500,000 documents per week.
“500,000 documents per week. This is actually a low ball number”transcript, stated
The generative-AI approach reached 99% accuracy.
“we were able to get it at 99% accuracy level”transcript, stated
The system reduced manual effort by 97% and enabled $330 million in client savings.
“So we got a 97% reduction in the manual effort, all in all enabled a lot of savings for our clients. You know, we were able to get a $330 million of savings”transcript, stated

Alternatives considered (roads not taken)

Continue manual consultant extraction

Rejected because: It was mundane, low-scale work that processed only about 100 documents per week per person.

“it's not really ideal to just put human resources on this problem, right?”transcript, stated

Rules-based OCR extraction

Rejected because: Rules broke when fields moved or document templates varied.

“If the documents had varied level of information at varied locations, we were out of luck.”transcript, stated

Scale without accuracy

Rejected because: The team emphasizes that high throughput is not useful unless extraction is accurate.

“But then who would want a solution or product that is not accurate, right? It doesn't matter if you are able to do it at that scale, you need to be accurate.”transcript, stated

Patterns

`structured-output-contract` (extraction)

Doczy.ai extracts document information into a structured repository / JSON-like metadata for downstream analysis.

“grab these characters and for this field, put it into this JSON structure.”transcript, stated

`batch-offline-processing` (operations)

The system processes contract documents at high weekly volume rather than one analyst at a time.

“500,000 documents per week.”transcript, stated

`model-routing` (model-ops)

AArete used multiple Bedrock models including Nova Pro and Anthropic Sonnet versions for extraction.

“So we use Nova Pro Anthropic Sonnet models, 3.5, 3.7, and even 4.0.”transcript, stated

`eval-driven-development` (evaluation)

Accuracy and manual-effort reduction were treated as core product success metrics.

“we were able to get it at 99% accuracy level”transcript, stated

Tradeoffs

Move from rules to generative extraction

Chose Bedrock-hosted generative AI over OCR plus hand-written rules.

Rationale: Rules were brittle to document variation and produced only 50-60% accuracy.

“If it followed a certain template, the results would be consistent. If the documents had varied level of information at varied locations, we were out of luck.”transcript, stated

Automate analyst work

Chose Doczy.ai metadata extraction over Adding more manual consultants.

Rationale: Manual work was mundane, slow, and pulled consultants away from strategic client work.

“our consultants were not too happy with it because it's mundane work, they wanna be able to doing something more effective that is more strategic with the clients.”transcript, stated

Use AWS Bedrock heavily

Chose Nova Pro and Anthropic Sonnet via Bedrock over Owning model hosting directly.

Rationale: The case emphasizes Bedrock as the production API layer for extracting outputs.

“API calls to AWS Bedrock and then the output gets extracted.”transcript, stated

Gotchas (what bit them)

Document templates defeat brittle extraction rules.

Root cause: Field positions vary across contracts and document layouts.

Resolution: Use generative interpretation over text rather than page/line/character rules.

“go to this page, go to this line, grab these characters and for this field, put it into this JSON structure.”transcript, stated

Manual document expertise does not scale.

Root cause: Consultants had institutional knowledge but could process only a small weekly volume.

Resolution: Capture ontology/domain expertise in an AI extraction tool.

“We had, you know, 17 years of ontology experience doing this manually.”transcript, stated

High throughput is useless if extraction accuracy is low.

Root cause: Downstream reimbursement and claims analytics depend on correct metadata.

Resolution: Treat 99% accuracy as a necessary companion to 500k-doc/week scale.

“It doesn't matter if you are able to do it at that scale, you need to be accurate.”transcript, stated

Evaluation

AArete compared manual, rules-based, and generative-AI extraction by throughput, accuracy, manual effort reduction, and downstream client savings.

Metrics: 100 documents/week/person manually · 50-60% rules-based accuracy · 500,000 documents/week with Doczy.ai · 99% accuracy · 97% manual effort reduction · $330 million savings

“it was giving us essentially, you know, 50% to 60% accurate results.”transcript, stated

“The production system achieved 99% accuracy while processing up to 500,000 documents per week, resulting in a 97% reduction in manual effort”summary, stated

Outcomes

Scaled document metadata extraction to about 500,000 documents per week.
Reached 99% extraction accuracy.
Reduced manual effort by 97%.
Enabled $330 million in client savings through contract and claims analysis.

The system targets healthcare payer and financial services documents, including provider/vendor contracts, and is described as one of AArete's heavy AWS Bedrock consumers.

“So we got a 97% reduction in the manual effort, all in all enabled a lot of savings for our clients.”transcript, stated

“we were also one of the top consumers of AWS Bedrock until recently.”transcript, stated