Systematic Prompt Optimization for Production Relevance Judges Using DSPy

dropbox · Tech · 2026 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Workspace search relevance evaluation

Dropbox Dash needed to run an LLM relevance judge across ranking, training-data generation, and offline evaluation, but manually tuned prompts for expensive models did not transfer cleanly to cheaper models and every model swap risked regressions.

Constraints:

The relevance judge scores query-document pairs from 1-5.
The judge feeds multiple production pipelines.
Manual prompt tuning took too long for model swaps.
Cheaper models had to preserve both score quality and parseable JSON.
Human relevance judgments were needed as optimization targets.

Architecture

Dropbox used DSPy as a systematic prompt-optimization loop for the Dash relevance judge. Human-annotated relevance data, NMSE metrics, and malformed-output checks fed DSPy optimizers that adapted prompts from expensive o3-style models to cheaper models such as gpt-oss-120b while preserving task constraints.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  query_document_pair_input["Query-document pair input"] --> llm_relevance_judge["LLM relevance judge"]
  human_annotation_pipeline["Human annotation pipeline"] --> evaluation_infrastructure["Evaluation infrastructure"]
  llm_relevance_judge --> evaluation_infrastructure
  evaluation_infrastructure --> dspy_optimizer["DSPy optimizer"]
  dspy_optimizer --> llm_relevance_judge
  llm_relevance_judge --> production_pipelines["Production pipelines"]

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Query-document pair input`	Search candidate and user query scored for relevance.	`LLM relevance judge`
`LLM relevance judge`	Scores query-document relevance from 1-5 and emits structured JSON.	`Query-document pair input`, `Evaluation infrastructure`, `Production pipelines`
`Human annotation pipeline`	Provides relevance judgments and explanations for optimization.	`Evaluation infrastructure`
`Evaluation infrastructure`	Computes NMSE, format failures, and regression metrics.	`LLM relevance judge`, `Human annotation pipeline`, `DSPy optimizer`
`DSPy optimizer`	Searches prompt variants against metrics and task constraints.	`Evaluation infrastructure`, `LLM relevance judge`
`Production pipelines`	Ranking, training data generation, and offline evaluation consumers of judge scores.	`LLM relevance judge`

Data flow

Query-document pairs and human relevance judgments enter an evaluation harness. The LLM relevance judge scores the pair and emits JSON. Metrics such as NMSE and malformed-output rate are computed, DSPy optimizes the prompt against those targets, and the optimized judge is used by ranking, training data generation, and offline evaluation pipelines.

4 supporting quotes

“Dropbox Dash needed to scale their LLM-based relevance judge, which scores query-document pairs from 1-5, across multiple production pipelines including ranking, training data generation, and offline evaluation.”summary, stated

“Evaluation infrastructure: A system for comparing model outputs to human annotations, computing NMSE, tracking formatting failures, and providing these metrics to DSPy optimizers.”summary, stated

“Human annotation pipeline: A process for collecting human relevance judgments with explanations on internal, non-sensitive datasets”summary, stated

“This dual objective—minimize NMSE while maintaining valid output format—gave them a concrete target for DSPy optimization.”summary, stated

Why this architecture

Dropbox turned prompt adaptation into a measured optimization problem because manual prompt engineering plateaued, was slow for model swaps, and could not reliably protect structural output requirements.

3 supporting quotes

“manually-tuned prompts for expensive models like OpenAI's o3 didn't transfer cleanly to cheaper models, and every model swap risked quality regressions.”summary, stated

“Manual prompt retuning could eventually recover performance, but it required weeks of iteration and introduced regression risk in unexpected edge cases.”summary, stated

“Quality plateaued early, every model swap risked regressions, and there was no systematic way to measur”summary, stated

Models used: OpenAI o3 · gpt-oss-120b · DSPy

Scale

The relevance judge scores query-document pairs from 1-5.
“scores query-document pairs from 1-5, across multiple production pipelines including ranking, training data generation, and offline evaluation.”summary, stated
DSPy reduced NMSE by 45%.
“they reduced NMSE by 45% (from 8.83 to 4.86).”summary, stated
Model adaptation time dropped from 1-2 weeks to 1-2 days.
“Model adaptation time dropped from 1-2 weeks of manual iteration to 1-2 days”summary, stated
The cheaper judge enabled 10-100x more data labeling at the same cost.
“they could label 10-100 times more data at the same cost, increasing coverage and statistical power”summary, stated

Alternatives considered (roads not taken)

Continue using expensive o3-style model for every judgment

Rejected because: At production scale, using the expensive model for every judgment became economically unsustainable.

“Running the most expensive model for every judgment became economically unsustainable.”summary, stated

Manual prompt retuning for each cheaper model

Rejected because: It took weeks and still introduced hidden regression risk.

“Manual prompt retuning could eventually recover performance, but it required weeks of iteration and introduced regression risk in unexpected edge cases.”summary, stated

Patterns

`eval-driven-development` (quality)

DSPy optimization was driven by NMSE against human labels and malformed-output metrics.

“This dual objective—minimize NMSE while maintaining valid output format—gave them a concrete target for DSPy optimization. The clarity of this objective is worth emphasizing: many organizations struggle with LLM evaluation because they haven't defined what success looks like in measurable terms.”summary, stated

`context-engineering` (prompt optimization)

Dropbox optimized the judge prompt systematically for different target models instead of hand-tuning prompt text.

“The approach transformed prompt engineering from fragile manual iteration into a repeatable optimization loop measured against human-annotated relevance judgments.”summary, stated

Tradeoffs

Move relevance judging to cheaper models with optimized prompts.

Chose DSPy-optimized cheaper judge over Expensive o3 judge for every judgment.

Rationale: It reduced cost enough to label far more data while preserving measured quality.

“Because the optimized judge could run on a much cheaper model than o3, they could label 10-100 times more data at the same cost”summary, stated

Use systematic prompt optimization rather than manual iteration.

Chose DSPy optimization loop over Manual prompt engineering.

Rationale: It cut model-adaptation time and reduced regression risk.

“cut model adaptation time from 1-2 weeks to 1-2 days”summary, stated

Gotchas (what bit them)

Prompts tuned for one model may not transfer to a cheaper model.

Root cause: Different models respond differently to the same judge instructions and output constraints.

Resolution: Use DSPy to optimize prompts per model against explicit metrics.

“prompts carefully tuned for one model didn't transfer cleanly to others.”summary, stated

Malformed JSON can corrupt downstream evaluation and training data.

Root cause: LLM judges are part of production pipelines that expect parseable structured output.

Resolution: Optimize for both scoring quality and valid output format.

“In their evaluation framework, these formatting failures were treated as fully incorrect responses—a pragmatic choice reflecting production reality where unparseable outputs cause pipeline failures, dropped examples, and unreliable metrics.”summary, stated

Evaluation

Dropbox compared model outputs to human annotations with NMSE and tracked malformed JSON failures as structural reliability metrics.

Metrics: NMSE. · Malformed JSON rate. · Human relevance judgment agreement. · Model adaptation time. · Data labeling volume at fixed cost.

“reduced their normalized mean squared error (NMSE) by 45% when adapting to gpt-oss-120b”summary, stated

Outcomes

NMSE dropped 45%, from 8.83 to 4.86.
Adaptation time dropped from 1-2 weeks to 1-2 days.
Data labeling capacity increased 10-100x at the same cost.
Malformed JSON outputs dropped 97% on smaller models.

The judge supports multiple Dash production pipelines and scores query-document pairs from 1-5.

“The results were substantial: comparing the best DSPy-optimized prompt to the original manual prompt, they reduced NMSE by 45% (from 8.83 to 4.86). This meant the judge's scores tracked human ratings much more closely.”summary, stated

“improved structural reliability by reducing malformed JSON outputs by 97% on smaller models.”summary, stated