AI-Powered Legal Document Review and Analysis Platform

lexbe · Legal · 2024 · source Agentic: no · promotional bias: high · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Legal eDiscovery document review

Lexbe needed to let legal teams query and analyze massive litigation document collections without missing critical evidence, while preserving legal confidentiality, scalability, source citations, and production recall quality.

Constraints:

• case document sets can range from 100,000 to more than one million documents
• missed evidence can cause unfavorable legal outcomes
• traditional keyword searches can return hundreds or thousands of results without synthesis
• legal users need grounded answers and source citations, not untraceable summaries
• production readiness depended on recall, because findings-of-fact reports require many source documents
• sensitive legal documents require encryption, role-based access, and controlled infrastructure

Architecture

Lexbe built Lexbe Pilot as an AI-powered Q&A and reporting assistant inside its eDiscovery platform using Amazon Bedrock Knowledge Bases. Users access the frontend through CloudFront; requests pass through an Application Load Balancer to Fargate backend services. Legal documents live in S3, Apache Tika extracts text into a separate S3 repository, Titan Text v2 creates embeddings, vectors are stored for retrieval, and Amazon OpenSearch indexes document text and metadata in both vector and text modes. Bedrock Knowledge Bases retrieves relevant context, Sonnet 3.5 generates grounded responses, and the frontend returns answers with citations. Lexbe iterated for eight months with AWS, using recall as the production gate and adding reranker technology to reach 90% recall.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  legal-user["legal-user"] --> web-frontend["web-frontend"]
  web-frontend --> cloudfront["cloudfront"]
  cloudfront --> application-load-balancer["application-load-balancer"]
  application-load-balancer --> fargate-backend["fargate-backend"]
  fargate-backend --> bedrock-knowledge-bases["bedrock-knowledge-bases"]
  fargate-backend --> s3-document-bucket["s3-document-bucket"]
  s3-document-bucket --> apache-tika-extractor["apache-tika-extractor"]
  apache-tika-extractor --> s3-text-bucket["s3-text-bucket"]
  s3-text-bucket --> titan-embedding-generator["titan-embedding-generator"]
  titan-embedding-generator --> vector-database["vector-database"]
  vector-database --> bedrock-knowledge-bases
  opensearch-index["opensearch-index"] --> bedrock-knowledge-bases
  bedrock-knowledge-bases --> reranker["reranker"]
  bedrock-knowledge-bases --> sonnet-35-llm["sonnet-35-llm"]
  sonnet-35-llm --> response-renderer["response-renderer"]
  response-renderer --> web-frontend
  recall-evaluation-loop["recall-evaluation-loop"] --> bedrock-knowledge-bases
  recall-evaluation-loop --> reranker

Components

Data flow

A legal user queries Lexbe Pilot in the web frontend. CloudFront routes the request through an Application Load Balancer to Fargate backend services. Documents stored in S3 are processed by Apache Tika, which extracts text into a separate S3 bucket used by Bedrock. Titan Text v2 generates embeddings, vector storage and OpenSearch provide semantic and keyword retrieval, and Bedrock Knowledge Bases fetches relevant context. Sonnet 3.5 uses the retrieved context to generate a response, findings-of-fact report, or inferential answer with links back to source documents. Recall measurements and reranker experiments guide production readiness.

"A user accesses the frontend application through a web browser.

2. Request routing: The request is routed through Amazon CloudFront, which connects to the backend through an [Application Load Balancer](https://aws.amazon.com/elasticloadbalancing/application-"

Why this architecture

Lexbe needed recall-oriented retrieval over huge document collections, not simple answer generation. Bedrock Knowledge Bases provided a managed RAG workflow, while OpenSearch kept both vector and text search available for legal retrieval. Fargate gave horizontally scalable backend processing without server management. Titan embeddings and token-size tuning were iterated against recall, and reranking was added once retrieval recall became the bottleneck. Source-linked reports and answers fit legal review because attorneys must trace generated claims back to documents.

Models used: Amazon Bedrock Sonnet 3.5 · Titan Text v2 embeddings · Amazon Titan embeddings · Cohere embeddings · reranker technology · Amazon Bedrock Knowledge Bases

Scale

• Legal case document sets range from 100,000 to more than one million documents.
  “case documents, which can range anywhere from 100,000 to over a million.”article, stated
• Keyword searches can return hundreds or thousands of documents as cases grow.
  “keyword searches that previously returned a handful of documents might now produce hundreds or even thousands.”article, stated
• Lexbe and AWS collaborated over an eight-month period.
  “Over an eight-month period, Lexbe worked hand-in-hand with the Amazon Bedrock Knowledge Bases team”article, stated
• The first RAG iteration had only 5% recall in January 2024.
  “January 2024. The initial system only had a 5% Recall Rate showing that much work was needed to get to production.”article, stated
• Recall improved to 36% in April 2024.
  “April 2024. New features were added to Amazon Bedrock Knowledge Bases leading to a noticeable boost in accuracy. We were now at 36% Recall Rate.”article, stated
• Token-size tuning brought recall to 60% in June 2024.
  “June 2024. Parameter adjustment, particularly around token size, led to another jump in performance. This brought Recall Rate to 60%.”article, stated
• Titan Embed text-v2 reached 66% recall in August 2024.
  “August 2024. A Recall Rate of 66% was achieved using Titan Embed text-v2 models.”article, stated
• Reranker technology enabled up to 90% recall by December 2024.
  “Fifth iteration: December 2024**.** Introduction of Reranker technology proved invaluable and enabled up to 90% Recall Rate.”article, stated
• The final reporting example produced a five-page findings-of-fact report.
  “It produced a polished, five-page report complete with clear section headings and hyperlinks back to every source document”article, stated
• Lexbe has provided eDiscovery software since January 2006.
  “Karsten Weber is the CTO and Co-founder of Lexbe, an eDiscovery provider, since January 2006.”article, stated

Alternatives considered (roads not taken)

Traditional keyword search

Rejected because: Keyword search can return hundreds or thousands of documents without the contextual synthesis needed to find hidden critical evidence.

Vector retrieval alone

Rejected because: Lexbe used both vector and text modes in OpenSearch, preserving semantic and keyword retrieval paths for large legal datasets.

Ship the first acceptable-looking RAG prototype

Rejected because: The initial 5% recall was far below production quality, so the team iterated for eight months until recall reached 90%.

Evaluate with generic answer satisfaction alone

Rejected because: Legal findings-of-fact reports require many source documents, so recall was the core production-readiness metric.

Manage backend servers directly

Rejected because: AWS Fargate gave Lexbe serverless container scaling for large-scale workloads without managing underlying infrastructure.

Patterns

rag-retrieval (retrieval)

Lexbe Pilot uses Amazon Bedrock Knowledge Bases to retrieve grounded context from entire legal document datasets before generating answers and reports.

hybrid-search (retrieval)

Amazon OpenSearch indexes document text and metadata in both vector and text modes so legal teams can use semantic and keyword retrieval.

retrieval-ranking (retrieval)

Reranker technology was added after earlier retrieval improvements and was the step that moved recall to 90%.

eval-driven-development (evaluation)

Lexbe used recall-rate acceptance criteria over an eight-month iteration cycle to decide when the feature was ready for production.

context-compaction (context)

Lexbe Pilot distills massive result sets and multilingual document collections into concise answers and multi-page findings-of-fact reports.

structured-output-contract (contracts)

The system generates structured findings-of-fact reports with section headings and hyperlinks to source documents.

cost-aware-routing (operations)

Lexbe uses ECS/Fargate and Linux Spot Market processing to control production processing costs.

safety-policy-enforcement (safety)

Sensitive legal documents are protected with AWS encryption and role-based access controls.

Tradeoffs

Use managed Bedrock Knowledge Bases

Chose fully managed RAG workflow over custom end-to-end retrieval and orchestration infrastructure.

Rationale: Knowledge Bases handled ingestion, semantic search, and contextual retrieval while Lexbe optimized recall and integration.

Optimize for recall

Chose recall-rate acceptance criteria over generic accuracy or answer fluency as the production gate.

Rationale: Legal reports need many supporting documents, and missing evidence is a greater risk than a less polished answer.

Use hybrid vector and text search

Chose OpenSearch vector and text modes over semantic-only or keyword-only retrieval.

Rationale: Legal review benefits from both semantic similarity and exact keyword/document lookup.

Add reranking after embedding and token-size tuning

Chose reranker technology in the final iteration over stopping at Titan Embed text-v2's 66% recall.

Rationale: The reranker was the intervention that moved retrieval from partial recall to the reported 90% production target.

Use serverless container deployment

Chose AWS Fargate and ECS over managing fixed backend servers.

Rationale: Legal workloads vary by case size and deadline, so horizontal scaling and less infrastructure management fit the workload.

Gotchas (what bit them)

The first RAG implementation was nowhere near production recall.

Root cause: Legal findings-of-fact require retrieving many relevant documents, and the initial configuration only reached 5% recall.

Resolution: Lexbe treated recall as the production gate and iterated through Knowledge Bases features, token-size tuning, Titan Embed text-v2, and reranking.

Keyword search becomes unmanageable at legal-document scale.

Root cause: As cases grow, keyword matches can return hundreds or thousands of documents without ranking the truly important evidence.

Resolution: Lexbe added semantic retrieval, LLM synthesis, and source-linked answer generation over full datasets.

Legal RAG must prioritize recall over answer elegance.

Root cause: A finding-of-facts report may require a large number of source documents; leaving out an important document changes the legal analysis.

Resolution: The team made recall the acceptance criterion instead of optimizing only generated-answer quality.

Vendor case-study outcomes should be read with context.

Root cause: The post is co-authored by AWS and emphasizes AWS services and benefits; it gives strong recall metrics but broad claims for speed and cost.

Resolution: Use the documented recall progression and architecture as evidence, while treating generalized efficiency claims as promotional.

Sensitive legal documents require production security controls.

Root cause: eDiscovery data is confidential and may include highly sensitive case material.

Resolution: The system uses AWS security controls, including encryption and role-based access controls.

Evaluation

Lexbe used recall rate as the primary production-readiness metric over an eight-month collaboration with AWS. The system improved from 5% recall in January 2024 to 36% in April, 60% in June after token-size tuning, 66% in August with Titan Embed text-v2, and up to 90% in December after adding reranking.

Metrics: recall rate · retrieval accuracy · token-size configuration impact · embedding model impact · reranker impact · findings-of-fact source coverage

Outcomes

• Lexbe Pilot became an AI-powered Q&A assistant integrated into the Lexbe eDiscovery platform.
• The RAG system can query entire legal datasets and return grounded, contextually relevant results.
• Recall improved from 5% in January 2024 to up to 90% after reranker introduction in December 2024.
• The system produced a five-page findings-of-fact report with section headings and source-document hyperlinks.
• The system inferred a family relationship across tens of thousands of documents by connecting email metadata and content.
• The platform handles multilingual document content, including English and Spanish.

The source reports case sets from 100,000 to over one million documents, keyword result sets of hundreds or thousands, an eight-month optimization period, recall milestones of 5%, 36%, 60%, 66%, and up to 90%, a five-page findings-of-fact report, tens of thousands of documents in an inference example, and English/Spanish multilingual handling.