Multi-Model Routing for Cost-Efficient AI Code Generation

cognition · Tech · 2026 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: AI code generation cost optimization

Cognition needed to lower AI coding-agent cost without losing frontier-level code quality, because naïve model routers performed poorly on real mergeable code despite benchmark scores.

Constraints:

Using expensive frontier models for every coding task is unsustainable.
Simple prompt-level routing lacks enough information to judge task difficulty.
Tool-based advisor calls can create costly cache misses.
Some coding subtasks require frontier judgment while others are mechanical.

Architecture

Devin Fusion uses a sidekick architecture: a frontier main agent retains planning, ambiguous requirement interpretation, and final review while delegating well-defined subtasks to a cheaper sidekick model. Both agents keep persistent cached context, and routing can change mid-session.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  main-frontier-agent["Main Frontier Agent"]
  sidekick-agent["Sidekick Agent"]
  dynamic-router["Dynamic Router"]
  cached-contexts["Cached Contexts"]
  coding-tools["Coding Tools"]
  main-frontier-agent --> sidekick-agent
  main-frontier-agent --> coding-tools
  sidekick-agent --> main-frontier-agent
  sidekick-agent --> coding-tools
  dynamic-router --> main-frontier-agent
  dynamic-router --> sidekick-agent
  cached-contexts --> main-frontier-agent
  cached-contexts --> sidekick-agent
  coding-tools --> main-frontier-agent
  coding-tools --> sidekick-agent

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Main Frontier Agent`	Owns planning, ambiguous decisions, monitoring, and final review.	`Sidekick Agent`, `Coding Tools`
`Sidekick Agent`	Executes delegated mechanical or well-defined coding subtasks with a cheaper model.	`Main Frontier Agent`, `Coding Tools`
`Dynamic Router`	Moves work between models during a session based on task complexity and context.	`Main Frontier Agent`, `Sidekick Agent`
`Cached Contexts`	Maintain separate persistent context for each agent to reduce repeated token processing.	`Main Frontier Agent`, `Sidekick Agent`
`Coding Tools`	Provide repository inspection, editing, testing, and code execution.	`Main Frontier Agent`, `Sidekick Agent`

Data flow

A coding task starts with a frontier main agent. The main agent delegates mechanical or well-scoped work to the sidekick, monitors progress, keeps high-level decisions, and reviews final output. Separate cached contexts avoid resending full task context on every cross-model interaction.

“this system runs two parallel agents simultaneously: a frontier model serving as the "main agent" and a more cost-effective model serving as the "sidekick."”summary, stated

“the sidekick architecture maintains separate, persistent cached contexts for both the main and sidekick models”summary, stated

Why this architecture

The sidekick design preserves frontier judgment for hard parts while extracting cost savings from cheaper models on mechanical work. Persistent context avoids expensive full-context advisor calls.

“defaulting to delegation and monitoring while retaining responsibility for significant decisions including overall planning, interpretation of ambiguous requirements, and final review.”summary, stated

“By keeping a frontier model actively involved throughout task execution, the system continues to benefit from capabilities that may not be well-captured by benchmarks.”summary, stated

Models used: frontier coding model · cheaper sidekick model · Devin Fusion

Scale

Devin Fusion reduced costs by 35% versus frontier-only models.
“The solution achieved a 35% cost reduction compared to frontier models while maintaining equivalent performance on the FrontierCode benchmark”summary, stated
88% of internally merged pull requests were driven entirely by automated routing.
“with 88% of internally merged pull requests being driven entirely by the automated routing system.”summary, stated
A JavaScript modernization example achieved 62% cost reduction without quality impact.
“delegating the time-consuming test execution to the sidekick achieved a 62% cost reduction with no quality impact.”summary, stated
An OpenTracing removal example achieved 32% cost reduction.
“mechanically removing OpenTracing integration across many Go files—work requiring consistency but minimal judgment—was successfully delegated for a 32% cost reduction.”summary, stated

Alternatives considered (roads not taken)

One-time prompt-level model routing

Rejected because: Initial prompts often lack enough information and sessions can become harder later.

“Many routing systems make a one-time decision about which model to use based on the initial prompt, but prompts often lack sufficient information to accurately assess task difficulty”summary, stated

Tool-based smart friend or advisor calls

Rejected because: Each secondary-model call required sending full context and caused cache misses.

“each call to the secondary model required sending the full task context without being able to leverage caching, resulting in expensive reprocessing of tokens.”summary, stated

Patterns

`model-routing` (architecture)

A frontier main agent dynamically delegates appropriate coding work to a cheaper sidekick.

“a frontier model delegates appropriate tasks to a cheaper model while retaining decision-making authority”summary, stated

`context-engineering` (context)

Separate cached contexts let both models accumulate task context without repeated full-context sends.

“separate, persistent cached contexts for both the main and sidekick models, allowing both to build up context incrementally over the course of a session.”summary, stated

`tool-use-orchestration` (architecture)

Both main and sidekick agents are fully fledged coding agents with tools.

“Both agents are fully-fledged with their own toolsets and ability to gather and act on context independently.”summary, stated

Tradeoffs

Keep frontier model in the loop

Chose Main frontier agent supervising a cheaper sidekick over Routing whole tasks away from frontier models.

Rationale: This preserves frontier judgment while still reducing cost on mechanical work.

“capture the benefits of frontier model intelligence—creativity, nuanced judgment, and general problem-solving capability—while offloading mechanical and well-defined subtasks to cheaper models.”summary, stated

Use parallel sidekick agents

Chose Persistent sidekick context over Occasional secondary-model advice calls.

Rationale: Persistent context avoids repeated full-context token processing.

“The importance of cache efficiency permeates the design.”summary, stated

Gotchas (what bit them)

Routing can degrade quality when hard judgment is delegated.

Root cause: Some features require subtle product or architecture decisions that cheaper models mishandle.

Resolution: Keep the frontier model responsible for ambiguous decisions and final review.

“a complex React/Redux feature requiring subtle judgment calls saw degraded quality when critical decisions were delegated to the sidekick”summary, stated

Provider cache expiry complicates long-running multi-model sessions.

Root cause: Most cached inputs expire after a short window.

Resolution: Engineer around persistent contexts and caching constraints as first-class infrastructure.

“most cached inputs have only a 5-minute expiry, presenting a significant operational challenge for longer-running tasks.”summary, stated

Evaluation

Cognition evaluates with FrontierCode, internally merged pull requests, cost reduction examples, and qualitative failure analysis.

Metrics: cost reduction · FrontierCode performance · internally merged PR share

“maintaining equivalent performance on the FrontierCode benchmark”summary, stated

Outcomes

Devin Fusion reduced costs by 35% compared with frontier-only models.
It maintained equivalent FrontierCode benchmark performance.
88% of internally merged PRs were driven entirely by automated routing.

The case reports 35%, 62%, and 32% cost-reduction examples and 88% of internal merged PRs driven by routing.

“a multi-model routing system designed to reduce the cost of AI-powered code generation while maintaining frontier-level performance.”summary, stated