Model Routing

Definition

Model routing selects a model or model configuration for a request, step, or workload according to capability and operating constraints. The decision happens on the normal execution path: Sierra sends retrieval, reasoning, classification, and speech work to different model families, while LinkedIn uses frontier models where iteration matters and smaller fine-tuned models on its much larger candidate-evaluation path. Sierra LinkedIn

Corpus Support

The analysis counts 169 deduplicated agentic systems (17.1%) across 14 industries. Evidence includes 89 article, 148 transcript, and 30 summary quotes; 265 are stated and 2 implied. Corpus analysis

Problem It Solves

One model rarely dominates every task on reasoning, speed, context, modality, cost, and deployment constraints. Sword uses evaluations to determine whether a smaller or alternate model preserves quality; Parlance recommends a pragmatic path from vendor APIs toward open models only when business needs justify it. Sword Health Parlance Labs

Applicability Signals

• A compound system contains materially different inference tasks. Sierra
• Workload scale makes the strongest model uneconomic on every path. LinkedIn
• Context size, modality, or latency requirements vary by request. GoDaddy
• Evaluations can compare candidate models on the actual task. Sword Health

Architecture and Variants

• Task-specialized routing: assign embedding, reasoning, classification, and speech operations to suitable models. Sierra
• Scale-tier routing: use smaller adapted models for high-volume paths and frontier models for agile lower-volume work. LinkedIn
• Dynamic request routing: switch model class as context, reliability, or customer state changes. GoDaddy
• Model-agnostic adapter layer: normalize capability gaps behind a common agent loop. Block

Why Teams Choose It

Routing lets teams spend capability where it changes outcomes while keeping simpler work fast and affordable. It also creates a controlled migration path when models improve, provided task evaluations and adapters keep model-specific behavior visible. Sword Health Block

Tradeoffs

• More routes improve fit but multiply evaluation, prompt, and compatibility work. GoDaddy
• Small models reduce cost and latency but require task-specific evidence that quality holds. Sword Health
• A common adapter simplifies orchestration while hiding provider differences that still need explicit handling. Block

Gotchas and Resolutions

• Generic leaderboards do not predict applied performance; compare models on project-specific cases. Parlance Labs
• Prompts can regress when moved between model versions; version and evaluate model-prompt pairs. GoDaddy
• Provider capability gaps leak into the agent loop unless adapters expose them deliberately. Block

When Not to Use It

Do not add routing when one model meets the task's quality, latency, deployment, and cost envelope and the extra evaluation surface outweighs likely savings. Parlance's staged recommendation is to begin with what works and add open or specialized models when a demonstrated need appears, not to build a router for its own sake. Parlance Labs

Boundaries and Related Patterns

model-routing selects among models during normal operation. cost-aware-routing makes economics the primary decision dimension. latency-optimization includes parallelism, streaming, caching, and workflow changes beyond model choice. fallback-degradation activates after failure or unmet service conditions rather than choosing the preferred path. GoDaddy Block

Operational Checklist

• Define routing dimensions and precedence: task, quality, latency, context, modality, cost, and deployment boundary. Sierra
• Evaluate each model-route combination on real task cases. Sword Health
• Version prompts, adapters, and route policy with the selected model. GoDaddy
• Trace route decisions and compare quality by route. LinkedIn

Representative Implementations

Methodology and Limitations

This page samples 169 deduplicated systems. Reports often describe selected models without publishing route-level quality and cost data, so the corpus supports the architectural decision more strongly than any universal routing policy. Corpus methodology