Hybrid Agent Architecture with Open-Source Workers and Frontier Advisors for Legal AI

harvey · Legal · 2026 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: legal AI agent execution and cost-efficient frontier-quality inference

Harvey and Fireworks needed a way to reach frontier-level quality on complex legal-agent tasks without sending every query through an expensive frontier model, while preserving legal-domain evaluation rigor, citation discipline, controllability, and a deployable training-to-serving path.

Constraints:

• legal tasks involve long reasoning chains, multiple documents, and strict citation discipline
• all-pass production readiness requires every rubric criterion to pass, not just a high average score
• frontier models remain expensive on full legal-agent trajectories
• open-source models are competitive on mean score but leave gaps on strict all-pass completion
• the system must expose when and why expensive frontier assistance is invoked
• post-trained models must move from training to production without numeric drift or a separate deployment pipeline
• reported results come from a 100-task LAB slice rather than the full 1,250-task benchmark

Architecture

Fireworks and Harvey evaluated two complementary system-level approaches on Harvey's Legal Agent Benchmark. The first is a hybrid agent harness where GLM 5.1 acts as the open-source worker responsible for most reasoning, drafting, and tool use, while Claude Opus 4.7 is exposed as a callable advisor tool that the worker invokes only when it needs help with retrieval, drafting, validation, or another difficult subtask. There is no separate router: the worker decides when to request the frontier advisor, producing sparse advisor usage and longer downstream worker trajectories. The second approach is Fireworks-native post-training of Kimi K2.6, using supervised fine-tuning on rubric-passing LAB trajectories and reinforcement fine-tuning against LAB evaluators, with the same Fireworks infrastructure handling training, evaluation, serving, and bit-for-bit handoff to production endpoints.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  Legal_Agent_Benchmark["Legal Agent Benchmark"] --> Baseline_model_runners["Baseline model runners"]
  Legal_Agent_Benchmark --> Hybrid_harness["Hybrid harness"]
  Legal_Agent_Benchmark --> Post_training_pipeline["Post-training pipeline"]
  Hybrid_harness --> GLM_5_1_worker["GLM 5.1 worker"]
  GLM_5_1_worker --> Legal_task_tools["Legal task tools"]
  GLM_5_1_worker --> Claude_Opus_4_7_advisor["Claude Opus 4.7 advisor"]
  Post_training_pipeline --> Kimi_K2_6_model["Kimi K2.6 model"]
  Kimi_K2_6_model --> Fireworks_production_endpoint["Fireworks production endpoint"]
  Post_training_pipeline --> Fireworks_production_endpoint
  Legal_Agent_Benchmark --> Cost_and_quality_dashboard["Cost and quality dashboard"]
  Hybrid_harness --> Cost_and_quality_dashboard
  Post_training_pipeline --> Cost_and_quality_dashboard

Components

Data flow

A 100-task distribution-mirrored LAB slice is run through candidate open-source and frontier models to establish mean-score, all-pass, and cost baselines. In the hybrid harness, GLM 5.1 executes the legal task, calls tools, drafts outputs, and invokes Claude Opus 4.7 as an advisor only when it reaches uncertain retrieval, drafting, or validation steps; the advisor response then informs further worker turns. In the post-training path, Kimi K2.6 is run through LAB, rubric-passing trajectories become SFT data, RFT uses per-criterion LAB rewards, and the resulting checkpoint is served from the same Fireworks endpoint used for production traffic.

Why this architecture

The architecture separates ordinary legal-agent work from the expensive moments that benefit from frontier intelligence. Open weights provide the controllable, lower-cost core; Claude Opus is used as a sparse callable advisor; and LAB makes both mean quality and strict all-pass readiness visible. The post-training path addresses a different lever by making the open-source model more domain-specific while keeping training and serving on the same infrastructure so improvements are deployable without conversion drift.

Models used: GLM 5.1 · Claude Opus 4.7 · Kimi K2.6 · DeepSeek V4 Pro · GPT-5.5 · open-source worker models · frontier advisor model

Scale

• The evaluation uses a 100-task slice mirrored from a 1,250-task LAB release.
  “The 100-task slice is a distribution-mirrored subset of the 1,250-task LAB release, preserving the practice-area mix of the full benchmark.”article, stated
• LAB spans 1,250 tasks, 24 legal practice areas, and more than 75,000 expert-written rubric criteria.
  “a suite of 1,250 tasks across 24 practice areas with 75,000+ expert-written rubric criteria.”article, stated
• The hybrid harness achieved 18/100 all-pass at $368 across the 100-task slice.
  “The harness moves GLM 5.1 from 12 / 100 all-pass to 18 / 100 — higher than Claude Opus 4.7’s 14 / 100 — at $368 across the 100 tasks”article, stated
• The advisor was invoked 0.83 times per task on average.
  “Across the run, the advisor is invoked just 0.83 times per task on average — sparse-but-targeted use.”article, stated
• SFT moved Kimi K2.6 from 11/100 to 15/100 all-pass while inference cost stayed around $84.
  “All-pass moved from 11 / 100 to 15 / 100 and mean score from 0.863 to 0.876, at essentially unchanged inference cost ($84 vs. $75 across the 100-task run).”article, stated
• RFT ran 46 rollout steps and reached a final smoothed mean score of 0.886.
  “Across 46 rollout steps the smoothed mean score climbs from 0.82 at the start to 0.886 at the final step”article, stated

Alternatives considered (roads not taken)

Claude Opus 4.7 end-to-end on every legal task

Rejected because: It achieved 14/100 all-pass at $954, while the hybrid GLM+Opus advisor harness achieved 18/100 at $368.

GLM 5.1 alone

Rejected because: It was cheap and competitive on mean score but the advisor added six more fully passing tasks, closing the production-readiness gap.

Prompting only without post-training

Rejected because: The SFT section says passing trajectories were clean enough to produce a measurable lift with no reward model, relabeling, or architecture changes.

Separate research and production deployment stacks

Rejected because: The post-training workflow emphasizes same-platform training and serving to avoid numeric drift and a research-to-production gap.

Patterns

multi-agent-orchestration (agent-architecture)

The hybrid harness coordinates an open-source worker and a frontier advisor, with the worker deciding when to call the advisor as a tool.

tool-use-orchestration (agent-architecture)

Claude Opus 4.7 is exposed as a callable advisor tool rather than the primary model for every task.

cost-aware-routing (operations)

The harness makes frontier-model calls sparse and tunable, dialing advisor calls up for complex matters and down for routine ones.

eval-driven-development (quality)

LAB provides both mean-score and all-pass metrics, guiding harness and post-training experiments against domain-specific legal rubrics.

fine-tuning-adaptation (model-adaptation)

Kimi K2.6 is adapted through SFT on rubric-passing trajectories and RFT against per-criterion LAB rewards.

observability-tracing (operations)

Trace analysis is used to understand advisor invocation, worker turn-count expansion, and the advisor's steering role.

Tradeoffs

Use a frontier model as advisor rather than primary dependency

Chose GLM 5.1 worker with callable Claude Opus 4.7 advisor over Claude Opus 4.7 end-to-end for every task.

Rationale: The hybrid harness beat Opus all-pass performance while costing a much smaller fraction of the standalone Opus run.

Let the worker self-trigger advisor calls

Chose worker-controlled advisor invocation over external router or orchestrator deciding when to invoke Opus.

Rationale: The design gives the worker adaptive access to second opinions during retrieval, drafting, or validation while keeping orchestration inspectable.

Use all-pass as production-readiness metric

Chose strict all-pass alongside mean score over average rubric score alone.

Rationale: Average scores can look strong while leaving long-tail criteria incomplete, which is unacceptable for legal work.

Start post-training with a simple SFT recipe

Chose rubric-passing trajectories from LAB as SFT data over reward model, human relabeling, or architecture changes for the first lift.

Rationale: The team wanted a clean baseline and found that the LAB trajectory data was already good enough to improve all-pass.

Run post-training and serving on one platform

Chose Fireworks training, evaluation, and serving on the same endpoint over training in one environment and deploying through a second production stack.

Rationale: The same-platform loop is meant to avoid numeric drift and make the model-development loop tractable.

Gotchas (what bit them)

Mean score can hide production failure in legal tasks.

Root cause: Legal deliverables can be unusable when any required criterion is missed, even if most rubric items pass.

Resolution: Use all-pass alongside mean score to measure whether a task fully satisfies every criterion.

The advisor pattern depends on the worker knowing when it needs help.

Root cause: There is no external router, so the worker model itself must recognize uncertainty and invoke the frontier advisor appropriately.

Resolution: Monitor traces, advisor usage, and worker turn-count behavior to tune the harness and validate where the pattern holds.

Research-to-production handoff can silently degrade trained models.

Root cause: Models trained in one environment often need conversion or redeployment elsewhere, creating opportunities for numeric drift or optimization loss.

Resolution: Use Fireworks-native training and serving so the checkpoint that trains against LAB is the same model served in production.

Benchmark-slice results may not generalize to the full legal workload.

Root cause: The experiment uses a 100-task slice rather than the complete 1,250-task LAB benchmark and excludes undisclosed production cost factors such as training cost.

Resolution: Treat results as a strong system-design signal but validate on broader practice areas, full benchmarks, and real client work before deployment decisions.

Evaluation

The team evaluates on a 100-task distribution-mirrored slice of Harvey's Legal Agent Benchmark using mean score, all-pass, standard error estimates, advisor invocation rates, and total inference cost. Post-training runs are evaluated continuously during RFT against LAB mean score, and cost comparisons use Fireworks serverless rates plus published frontier-model API pricing.

Metrics: 100-task LAB slice from 1,250 tasks and 24 practice areas · 75,000+ expert-written rubric criteria in LAB · GLM 5.1 mean score 0.8921 at $121 · Claude Opus 4.7 all-pass 14/100 at $954 · GLM 5.1 + Opus advisor all-pass 18/100 at $368 · advisor invoked 0.83 times per task on average · Kimi K2.6 SFT all-pass 15/100 at $84 · RFT final smoothed mean score 0.886 after 46 rollout steps

Outcomes

• The hybrid harness reached 18/100 all-pass, exceeding Claude Opus 4.7's standalone 14/100 while costing $368 instead of $954.
• The advisor added six fully passing tasks over GLM 5.1 alone for an additional $246.
• SFT lifted Kimi K2.6 from 11/100 to 15/100 all-pass and from 0.863 to 0.876 mean score.
• RFT improved Kimi K2.6's smoothed mean score to 0.886 by rollout step 46.
• The article argues that open weights at the core plus frontier advisor calls can close much of the frontier gap while improving cost efficiency and controllability.

The results are strong but bounded: they are based on a 100-task slice, estimated inference costs, and a Fireworks/Harvey-authored case study. The most useful outcome is the production architecture pattern: sparse frontier advising plus domain-specific post-training, both evaluated against strict legal rubrics.