Building and Evaluating Legal AI with Multi-Modal Evaluation Systems

unify · Legal · year n/a · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Legal AI product evaluation

Harvey needed a way to evaluate high-stakes legal AI products whose outputs are long, subjective, domain-specific, and dangerous to launch on shallow benchmark scores alone.

Constraints:

Architecture

Harvey built a layered evaluation operating system around legal AI products. Lawyers define tasks, datasets, rubrics, and review criteria with engineers; BigLawBench and product datasets test open-ended legal work; human side-by-side ratings remain the highest-quality launch signal; LLM-as-judge evaluations and step-wise workflow checks scale routine assessment; LangSmith and custom tools support dogfooding and repeated release decisions.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  legal-ai-products["Legal AI products"] --> evaluation-workflow["Evaluation workflow"]
  legal-ai-products --> citation-and-verification-layer["Citation and verification layer"]
  evaluation-workflow --> domain-expert-lawyers["Domain expert lawyers"]
  evaluation-workflow --> human-preference-evaluation["Human preference evaluation"]
  evaluation-workflow --> biglawbench["BigLawBench"]
  evaluation-workflow --> step-wise-workflow-evaluation["Step-wise workflow evaluation"]
  evaluation-workflow --> llm-as-judge-evaluation["LLM-as-judge evaluation"]
  evaluation-workflow --> dogfooding-feedback["Dogfooding feedback"]
  evaluation-workflow --> langsmith-and-custom-tools["LangSmith and custom tools"]
  domain-expert-lawyers --> biglawbench
  domain-expert-lawyers --> human-preference-evaluation
  biglawbench --> llm-as-judge-evaluation
  langsmith-and-custom-tools --> human-preference-evaluation
  langsmith-and-custom-tools --> step-wise-workflow-evaluation
Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component Role Interacts with
Legal AI products Draft, summarize, extract, search, and run legal workflows for customers. Evaluation workflow, Citation and verification layer
Evaluation workflow Coordinates model, prompt, algorithm, and UI assessment before launch. Legal AI products, Domain expert lawyers, Human preference evaluation, BigLawBench, Step-wise workflow evaluation, LLM-as-judge evaluation, Dogfooding feedback
Domain expert lawyers Define use cases, datasets, rubrics, feedback, and legal quality standards. Evaluation workflow, Human preference evaluation, BigLawBench
Human preference evaluation Collects side-by-side ratings, numerical scores, and qualitative attorney feedback. Evaluation workflow, Domain expert lawyers
BigLawBench Benchmarks complex open-ended legal tasks that mirror real legal work. Evaluation workflow, Domain expert lawyers, LLM-as-judge evaluation
LLM-as-judge evaluation Applies detailed domain-specific rubrics to evaluate complex outputs at scale. Evaluation workflow, BigLawBench
Step-wise workflow evaluation Breaks agentic or RAG workflows into evaluable intermediate steps. Evaluation workflow
LangSmith and custom tools Provide routine evaluation infrastructure and custom human-rater workflows. Evaluation workflow, Human preference evaluation, Step-wise workflow evaluation
Dogfooding feedback Captures qualitative internal product feedback before customer rollout. Evaluation workflow
Citation and verification layer Grounds generated legal statements in sources so users can verify outputs. Legal AI products

Data flow

For a new product or model, legal domain experts help define use cases and evaluation datasets. Engineers prototype the workflow, run BigLawBench and step-wise automated checks, collect side-by-side attorney preferences and qualitative feedback, dogfood internally, inspect regressions, and only then launch or adjust the model, prompt, algorithm, or UI.

3 supporting quotes
“lawyers provide initial context about what a document type is, how it's used, and when it comes up in daily work.”summary, stated
“Engineers build a prototype, and the team iterates through multiple cycles of reviewing outputs until the results meet expert standards.”summary, stated
“Despite the positive results, they ran additional tests on product-specific datasets to understand where the model worked well and where it had shortcomings.”summary, stated

Why this architecture

Harvey treated evaluation as a multi-modal product-development system because legal correctness is not reducible to a single automated score. Human lawyers supply domain taste and rubrics, automated judges increase coverage, and step-wise checks make long workflows tractable.

3 supporting quotes
“Human evaluation remains Harvey's highest-quality signal, and significant effort goes into improving the throughput and efficiency of collecting human preference data.”summary, stated
“For complex multi-step workflows and agents, Harvey breaks problems into components to evaluate each step separately, making the overall evaluation problem more tractable.”summary, stated
“all evaluation criteria are crafted by in-house domain experts and are distinct for each question-answer pair”summary, stated

Models used: GPT-4.1 · LLM-as-judge models

Scale

Alternatives considered (roads not taken)

Use academic legal benchmarks as the main launch signal

Rejected because: Benchmarks such as LegalBench did not represent complex, open-ended legal work over external documents.

“The speaker specifically critiques benchmarks like LegalBench, which tend to feature simple yes/no questions with no reference to external materials.”summary, stated
“Real legal work involves complex, open-ended questions with subjective answers that require analyzing extensive external documents.”summary, stated

Rely only on automated evaluations

Rejected because: Harvey kept human legal judgment as the highest-quality signal because quality is nuanced and subjective.

“A particularly important insight from the talk is that quality in legal AI is highly nuanced and subjective.”summary, stated
“Taste Matters Alongside Metrics: While rigorous, repeatable evaluations are critical, human judgment, qualitative feedback, and taste remain essential.”summary, stated

Evaluate only final end-to-end answers for complex workflows

Rejected because: Harvey decomposed agentic workflows into intermediate steps so each part could be assessed more tractably.

“This decomposition approach allows greater use of automated evaluations and is applied to complex workflows, citation accuracy, and agentic processes.”summary, stated

Commit to one evaluation platform

Rejected because: Harvey used LangSmith for some evaluations and custom tools for human-rater workflows because no single platform fit everything.

“However, they've also built their own tools for human rater-focused evaluations, suggesting that no single platform meets all their needs.”summary, stated

Patterns

eval-driven-development (evaluation)

Harvey uses evaluation data, dogfooding, domain rubrics, and human preference results to guide launches of models, prompts, products, and algorithms.

“These evaluations drive launch decisions for new models, prompts, or algorithms. Harvey has built custom tooling to scale these evaluations, allowing them to be run routinely across many different tasks.”summary, stated
“Better tooling made it easier to run evaluations, which led to more teams using them more frequently, improving iteration speed and product quality while increasing confidence in launches.”summary, stated

human-in-the-loop-gate (evaluation)

Lawyers participate throughout feature discovery, rubric design, side-by-side evaluation, UI iteration, and launch readiness.

“Lawyers are embedded at every stage of the product development process, working side-by-side with engineers, designers, and product managers.”summary, stated
“The workflow for building a new feature follows a collaborative pattern: lawyers provide initial context about what a document type is, how it's used, and when it comes up in daily work.”summary, stated

llm-as-judge (evaluation)

Harvey uses model-based evaluation with detailed legal rubrics for complex open-ended outputs where human-only review would not scale.

“For automated evaluation of these complex outputs, Harvey develops detailed rubrics broken into categories:”summary, stated
“Harvey seeks to leverage automated evaluations wherever possible. However, they acknowledge significant challenges with applying existing academic benchmarks to real-world legal work.”summary, stated

prompt-chaining (workflow)

Complex legal workflows are broken into step-wise sub-evaluations such as query rewriting, retrieval, answer generation, and citation creation.

“a typical pipeline includes query rewriting, chunk/document retrieval, answer generation, and citation creation. Each step can be evaluated independently.”summary, stated

rag-retrieval (retrieval)

Harvey's evaluation examples include retrieval, citation, and document-grounded legal workflows over extensive external legal materials.

“Real legal work involves complex, open-ended questions with subjective answers that require analyzing extensive external documents.”summary, stated
“The principle extends to any multi-step process where intermediate outputs can be assessed against clear criteria.”summary, implied

Tradeoffs

Human preference signal as launch authority

Chose Keep attorney side-by-side evaluation as the highest-quality signal over Rely on automated benchmark or judge scores alone.

Rationale: Legal quality includes subtle correctness, nuance, and taste that automated scores do not reliably capture.

“Human evaluation remains Harvey's highest-quality signal, and significant effort goes into improving the throughput and efficiency of collecting human preference data.”summary, stated
“Taste Matters Alongside Metrics: While rigorous, repeatable evaluations are critical, human judgment, qualitative feedback, and taste remain essential.”summary, stated

Custom legal benchmark investment

Chose Build BigLawBench with open-ended legal tasks over Use LegalBench-style simple yes/no benchmark tasks.

Rationale: Real legal work requires analyzing external documents and producing subjective long-form outputs.

“benchmarks like LegalBench, which tend to feature simple yes/no questions with no reference to external materials.”summary, stated
“Real legal work involves complex, open-ended questions with subjective answers that require analyzing extensive external documents.”summary, stated

Prototype-first product development

Chose Rapid prototyping across prompts, algorithms, and UI over Detailed product requirement documents before building.

Rationale: Making work tangible let lawyers and engineers iterate on quality faster.

“Prototype Over PRD: Rather than relying on detailed product requirement documents, Harvey emphasizes rapid prototyping and iteration.”summary, stated
“This approach makes work tangible and accelerates learning cycles.”summary, stated

Hybrid evaluation tooling

Chose Use LangSmith for some routine evaluations and custom tools for human-rater workflows over Standardize on one evaluation platform.

Rationale: Different evaluation modes needed different tooling surfaces.

“They leverage LangSmith extensively for a subset of their evaluations, particularly routine evaluations related to step-wise task breakdowns.”summary, stated
“However, they've also built their own tools for human rater-focused evaluations, suggesting that no single platform meets all their needs.”summary, stated

Gotchas (what bit them)

Standard legal benchmarks did not match real product quality needs.

Root cause: Academic benchmark tasks were too simple and lacked the external-document, open-ended, subjective nature of legal work.

Resolution: Harvey built BigLawBench with complex open-ended legal tasks and domain-specific rubrics.

“To address this gap, Harvey built their own benchmark called BigLawBench, which contains complex open-ended tasks with subjective answers that more closely mirror actual legal work.”summary, stated

Automated metrics missed product regressions.

Root cause: Some failures were qualitative, stylistic, or brand-specific rather than captured by benchmark scores.

Resolution: Harvey supplemented automated tests with product-specific datasets and internal dogfooding before rollout.

“This additional testing uncovered regressions that automated metrics wouldn't catch—for example, GPT-4.1 was much more likely to start every response with "Certainly!" which was off-brand for Harvey's product.”summary, stated

Human legal evaluation is high quality but expensive and slow.

Root cause: Legal quality requires trained attorneys and detailed domain-specific judgment.

Resolution: Harvey invested in tooling, automated judges, step-wise evaluation, and custom human-rater workflows to improve throughput.

“Given the cost and time requirements of human evaluation—especially when using domain experts like trained attorneys—Harvey seeks to leverage automated evaluations wherever possible.”summary, stated
“Evaluation is fundamentally an engineering problem. Investment in strong tooling, processes, and documentation pays back quickly—the speaker suggests 10-fold returns.”summary, stated

Legal answer quality is subjective even when both answers are factually correct.

Root cause: Lawyers care about nuance, detail, style, and domain fit beyond hallucination avoidance.

Resolution: Harvey collects preference judgments, numerical ratings, and qualitative comments from legal raters.

“both factually correct with no hallucinations—yet one was strongly preferred by in-house lawyers due to additional nuance and detail in the definitions.”summary, stated
“For each comparison, raters are asked to provide relative preferences, numerical ratings on a scale (typically 1-7, from "very bad" to "very good"), and qualitative feedback.”summary, stated

Evaluation

Harvey combines side-by-side human preferences, 1-7 ratings, qualitative attorney feedback, BigLawBench, LLM-as-judge rubrics, step-wise workflow checks, product-specific datasets, and internal dogfooding.

Metrics: Side-by-side preference · 1-7 human rating scale · Rubric categories for structure, style, substance, and accuracy · Product-specific dataset performance · Internal dogfooding regressions

3 supporting quotes
“Human evaluation remains Harvey's highest-quality signal, and significant effort goes into improving the throughput and efficiency of collecting human preference data.”summary, stated
“For automated evaluation of these complex outputs, Harvey develops detailed rubrics broken into categories:”summary, stated
“Despite the positive results, they ran additional tests on product-specific datasets to understand where the model worked well and where it had shortcomings. They also conducted extensive internal dogfooding to collect qualitative feedback.”summary, stated

Outcomes

Harvey applied this evaluation system in a legal AI business serving nearly 400 customers globally, including one-third of the largest 100 US law firms and eight of the top ten largest firms.

3 supporting quotes
“Comparing their baseline system against the new 4.1-based system, they found the new system's responses skewed significantly toward higher ratings on the 1-7 scale.”summary, stated
“This additional testing uncovered regressions that automated metrics wouldn't catch—for example, GPT-4.1 was much more likely to start every response with "Certainly!" which was off-brand for Harvey's product. These issues had to be addressed before rollout.”summary, stated
“Harvey includes citation features to ground all AI statements in verifiable sources, allowing users to verify that summaries and generated content are correct and acceptable.”summary, stated