Building Robust Legal Document Processing Applications with LLMs

anzen · Insurance · 2023 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: High-stakes legal and insurance document processing

Anzen needed to use LLMs in document-processing workflows where hallucinated or malformed answers are unacceptable, while source documents have messy layouts, changing inputs, and model behavior can drift over time.

Constraints:

• Insurance and legal domains require correctness rather than creative plausibility.
• Prompting alone cannot reliably prevent hallucinated citations or wrong answers.
• Third-party models can change behavior under the hood after deployment.
• Large document contexts create token-limit and token-cost pressure.
• Naive OCR loses question/answer relationships in complex insurance forms.
• Production systems need explicit feedback and monitoring because models will misbehave sometimes.

Architecture

Anzen's robust document-processing architecture separates deterministic and generative work. Documents are first cleaned and structured: OCR extracts text, LayoutLM identifies questions versus answers, and the system reconstructs a text representation that preserves the original form's logic. A fine-tuned classifier trained on curated examples decides whether a document is an insurance application before the expensive LLM step. The LLM receives cleaner context and is constrained with structured outputs/function calls. User feedback, saved inputs/outputs, dashboards, latency/performance monitoring, and alerting close the loop so model drift or data changes can be detected and used for future fine-tuning.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  insurance-application-document[Insurance application document] --> ocr-extractor[OCR extractor]
  insurance-application-document --> layoutlm-document-parser[LayoutLM document parser]
  ocr-extractor --> structured-document-representation[Structured document representation]
  layoutlm-document-parser --> structured-document-representation
  structured-document-representation --> document-classifier[Document classifier]
  structured-document-representation --> llm-extraction-step[LLM extraction step]
  document-classifier --> llm-extraction-step
  llm-extraction-step --> function-call-schema[Function call schema]
  llm-extraction-step --> feedback-system[Feedback system]
  llm-extraction-step --> monitoring-system[Monitoring system]
  feedback-system --> monitoring-system
  feedback-system --> fine-tuning-dataset[Fine tuning dataset]
  fine-tuning-dataset --> document-classifier[Document classifier]

Components

Data flow

An insurance application enters the pipeline as a layout-heavy document. OCR alone is insufficient, so Anzen combines OCR with LayoutLM to identify questions and answers, reconstructing a cleaner text representation that preserves the form's structure. A fine-tuned classifier first determines whether the document is an insurance application. Only confirmed applications proceed to an LLM step, which receives cleaner context and returns structured data through function-call-style schemas. In production, user feedback, saved model inputs/outputs, dashboards, performance metrics, and alerting monitor whether the system is drifting or misbehaving.

Why this architecture

The system is built around reducing the LLM's uncertainty rather than assuming the model will stop hallucinating. Function calls constrain output shape; retrieval and multi-step processing limit context to relevant information; LayoutLM/OCR preprocessing improves input quality before the LLM sees the document; a classifier avoids spending generative-model effort on documents that should be rejected; and feedback plus monitoring detect drift from changing providers, changing inputs, or nondeterminism.

Models used: ChatGPT / OpenAI models · LayoutLM · Fine-tuned classification model

Scale

• The production use case processes many insurance applications.
  “we process a lot of insurance applications which look something like this”transcript, stated
• The classifier example was trained on a few thousand examples.
  “a classification model that was trained on a few thousand examples”transcript, stated
• Fine-tuned classifiers can work with a few hundred examples and become excellent with a few thousand.
  “you can get pretty good performance with just a few hundred examples um and you know if you scale that up to a few thousand you can start to get really excellent results”transcript, stated
• Anzen reports 99.9% accuracy for classifying documents across multiple types in one case.
  “one case at Onsen we have something which essentially has you know 99.9 uh accuracy across for classifying documents across a bunch of different types”transcript, stated
• The talk cites a rapid model-drift example where GPT-3.5 improved from 7.4 correct to 87 while GPT-4 worsened on the same math test.
  “gpt4 in particular went from getting it right almost all the time to almost never and interestingly during the same period GPT 3.5 got better from um only 7.4 correct to 87.”transcript, stated

Alternatives considered (roads not taken)

Prompt the model not to be wrong

Rejected because: Natural-language instructions still produced fabricated sources, so correctness needs architecture, data, constraints, and monitoring.

Specify JSON format through long prompts

Rejected because: Prompt-only formatting is tedious, long, and does not guarantee the exact schema an application needs.

Send the whole document or document set to the LLM for every question

Rejected because: Large contexts can exceed model limits or become expensive; retrieval and multi-step extraction reduce token load.

Use naive OCR text as the LLM input

Rejected because: Raw OCR loses relationships among checkboxes, questions, answers, and layout, making the text hard for even humans to reconstruct.

Use a general LLM for document classification and extraction in one step

Rejected because: A smaller fine-tuned classifier can cheaply decide whether a document belongs in the pipeline, constraining the LLM to confirmed cases.

Patterns

structured-output-contract (validation)

Anzen recommends function calls to define the exact output schema instead of relying on natural-language format instructions.

rag-retrieval (retrieval)

For large document contexts, the talk recommends embedding passages and queries to retrieve only the most relevant document sections before calling the model.

prompt-chaining (workflow)

The recommended document workflow extracts structured fields once, then feeds condensed information to a later model step for higher-level judgment.

data-quality-loop (data)

Anzen improves LLM accuracy by investing in OCR/LayoutLM preprocessing and by feeding user feedback into future training and fine-tuning.

fine-tuning-adaptation (modeling)

Anzen uses a fine-tuned classifier for document-type detection before generative extraction, reserving LLM work for cases that need it.

observability-tracing (operations)

The production architecture saves inputs and outputs, tracks latency/performance, and uses dashboards and alerting to detect degradation.

Tradeoffs

Constrain output with function calls

Chose Schema-defined function-call outputs over Long natural-language JSON formatting prompts.

Rationale: Function calls are more reliable and easier than spelling out every output field in a prompt.

Retrieve or condense context before generation

Chose Search and multi-step document processing over Sending all source text for every query.

Rationale: The model needs relevant information, but raw large contexts create context-window and cost problems.

Use LayoutLM before LLM processing

Chose Layout-aware preprocessing plus OCR over Plain OCR text.

Rationale: Layout-aware preprocessing preserves the question/answer structure that an LLM needs to reason correctly over insurance forms.

Gate LLM extraction with a classifier

Chose Fine-tuned document classifier first over Letting the LLM both identify and extract every document.

Rationale: A classifier narrows the problem space and lets the LLM operate only on confirmed insurance applications.

Build first-class feedback and monitoring

Chose Product feedback, dashboards, alerting, and saved inputs/outputs over Relying only on usage metrics or launch-time testing.

Rationale: Explicit correctness feedback is more valuable than rough usage proxies and supports rapid detection plus future model improvement.

Gotchas (what bit them)

Instruction-only anti-hallucination is not enough for high-stakes applications.

Root cause: The model can fabricate plausible sources or answers even when prompted to be certain.

Resolution: Use architecture-level controls: structured outputs, supplied context, preprocessing, monitoring, and feedback.

Provider model behavior can change after deployment.

Root cause: Third-party models may be updated under the hood, and nondeterministic outputs can vary even for the same input.

Resolution: Build feedback and monitoring loops that detect performance changes quickly.

Raw OCR can destroy the semantics of insurance applications.

Root cause: Checkboxes, tables, questions, and answers encode relationships in layout that plain text loses.

Resolution: Use a layout-aware model plus OCR to reconstruct a structured text representation before LLM processing.

Using LLMs for every task increases cost and widens the failure surface.

Root cause: Classification does not require generative capabilities and can be handled by smaller tuned models.

Resolution: Route classification to a fine-tuned model and pass only confirmed cases to the LLM.

Context stuffing can hit limits or become too expensive.

Root cause: Large document sets may exceed the context window or generate high third-party API token costs.

Resolution: Use retrieval or multi-step extraction to condense relevant information before higher-level reasoning.

Evaluation

The talk recommends evaluating production document AI with structured-output validation, classification accuracy, user-reported correctness feedback, saved inputs/outputs for debugging, latency/performance metrics, dashboards, alerts, and trend monitoring over time.

Metrics: Classification accuracy across document types · User-reported incorrect results · Feedback spike alerts · Model input/output logs for debugging · Performance and latency trends · Dashboarded changes over time

Outcomes

• Anzen reports 99.9% document-classification accuracy across multiple document types in one production case.
• The LayoutLM-plus-OCR preprocessing produced a more faithful text representation that the model understood better.
• The architecture reduces hallucination risk by supplying cleaner context, constraining outputs, and narrowing LLM tasks with a classifier.
• Feedback and monitoring create a path for detecting regressions and improving future models.

The talk frames Anzen's system as production insurance document processing, with a classifier trained on a few thousand examples, reported 99.9% classification accuracy, and monitoring designed for model and data drift over time.