Evolution from Context Engineering to Harness Engineering: Philosophical and Practical Approaches to Building Production LLM Systems

boundary_/langchain/_humanlayer · Tech · 2026 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: production agent harness design

The discussion addresses how teams should build production LLM systems as models, tools, harnesses, and orchestration layers mature beyond simple context-engineered chat loops.

Constraints:

Teams need to decide when to use existing harnesses versus building custom orchestration.
Models are increasingly trained to use specific tool schemas well.
Custom harness work can become obsolete as frontier models improve.
Evaluation is required to know whether harness changes actually improve behavior.
Engineers still need to understand low-level tool-calling primitives even when abstraction layers improve.

Architecture

The participants frame a production agent as a model wrapped by a harness: deterministic code, tool definitions, tool execution, context-window shaping, permissioning, compaction, outer orchestration, and evaluation loops. The recommended approach is to fit the harness to tasks and production traces, use existing model-trained tool schemas where possible, and only build or tune bespoke harnesses when evals show the high-value workflow deserves that engineering investment.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  task-and-production-traces["Task and production traces"] --> evaluation-loop["Evaluation loop"]
  evaluation-loop --> harness-configuration["Harness configuration"]
  harness-configuration --> model-and-tool-loop["Model and tool loop"]
  model-and-tool-loop --> outer-orchestration-layer["Outer orchestration layer"]
  outer-orchestration-layer --> harness-configuration

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Task and production traces`	Concrete target behavior and examples used to define what the agent should do.	`Evaluation loop`
`Evaluation loop`	Checks whether harness, prompt, context, or tool changes improve the desired task behavior.	`Harness configuration`
`Harness configuration`	System prompts, skills, tool definitions, compaction, permissioning, and environment controls wrapped around the model.	`Model and tool loop`
`Model and tool loop`	The core model loop that selects tools, passes arguments, receives results, and eventually returns an answer.	`Outer orchestration layer`
`Outer orchestration layer`	Higher-level while loop or workflow that allocates memory, instructions, and environmental controls around the inner harness.	`Harness configuration`

Data flow

A task enters the harness, which supplies system prompts, tool definitions, context, environmental controls, and execution loops around the model. The model emits tool calls or answers, tools return context, outer orchestration may manage memory and permissions, and evals over production traces or task behavior determine whether context or harness edits improved outcomes.

“the harness was all of the deterministic code that would come in.”transcript, stated

“harness engineering is all about wrapping the model to do useful work on like some task that I care about.”transcript, stated

Why this architecture

The architecture emphasizes harnesses because models now gain capability from being trained to specific tool schemas and execution environments. But the speakers warn that custom work should be eval-driven: adapt prompts, tools, context flow, and harness primitives only when task-specific evidence shows the investment beats the frontier lab's default harness.

“the model inside the harness gets better at calling specific tools.”transcript, stated

“And you build an eval for it first, right?”transcript, stated

Models used: LLMs · Claude Code · Codex · GPT OSS 120B · MCP · Tool calling · Custom compaction · Production trace evals

Scale

The talk frames model releases as arriving roughly every 6 months.
“learn to use them faster than they can release another model every 6 months.”transcript, stated
The speakers describe potentially staying 5 to 10% ahead by learning to use models faster than models improve.
“you will always be 5 to 10% ahead.”transcript, stated

Alternatives considered (roads not taken)

Use only context engineering around a chat-completions loop.

Rejected because: The speakers argue harness-engineered systems can outperform plain context engineering when the model and tools are shaped together.

“build products that are better than what you could do with just context engineering and just agent engineering.”transcript, stated

Train a custom model for every tool set.

Rejected because: A cheaper alternative is to make your systems fit the tool schemas that frontier models are already trained to use well.

“make your thing fit into the tool set the model already is really freaking good at using.”transcript, stated

Let automated research optimize without looking at the data.

Rejected because: The speakers warn that teams can overfit evals or avoid understanding the actual production data.

“They never look at the data. They're just like Claude, figure”transcript, stated

Patterns

`tool-use-orchestration` (agent harness)

The harness wraps the model with tool definitions, executions, context insertion, and environmental controls.

“this harness will also engineer context into the context window.”transcript, stated

`eval-driven-development` (harness tuning)

Harness changes should be driven by evals over the behavior the agent must perform.

“Don't waste time trying to understand the system without building like some sort of like evaluation loop.”transcript, stated

`context-compaction` (harness context management)

The speakers discuss compaction and context recycling as parts of harness behavior that engineers may need to understand or customize.

“they're recycling the context windows and how they do the explore tool or the plan tool.”transcript, stated

Tradeoffs

Use existing harness primitives before building custom ones.

Chose Fit systems to tool schemas frontier models already use well over Training or hand-building a bespoke harness immediately.

Rationale: Existing harnesses can deliver value sooner and reserve custom work for high-value proven bottlenecks.

“go to the like the most like simple like harness, which is like the tool calling thing with a file system”transcript, stated

Invest custom effort only where the workflow is critical.

Chose Focused harness engineering for high-stakes parts over Optimizing every harness in the company.

Rationale: The speakers advise spending team energy only on the parts that are high-risk or uniquely valuable.

“this thing is the final thing that we use for filing taxes for our customers, and we can't”transcript, stated

Gotchas (what bit them)

Custom harness code can become tech debt when models improve.

Root cause: A model release may make hand-built capability scaffolding obsolete.

Resolution: Use evals to decide where custom harness work is worth keeping or replacing.

“you write a bunch of code around a model to make it better, and then the model gets better and all your code is irrelevant”transcript, stated

Models are not interchangeable across harnesses.

Root cause: Models can be trained or reinforced to prefer specific tool schemas and workflows.

Resolution: Choose a harness/model pairing intentionally instead of assuming any model works with any tool interface.

“if you try to use cloud code models in the Codex harness, it's complete trash. It does not work.”transcript, stated

Engineers can overfit the eval set.

Root cause: Automated harness optimization may pass existing tests without generalizing to new production behavior.

Resolution: Inspect the underlying data and keep evals grounded in real production traces.

“we've just like overfit to the entire eval set and this will like completely like not generalize”transcript, stated

Evaluation

The talk recommends building task-specific eval loops from production traces or target behaviors, then fitting prompts, skills, tool design, context engineering, and harness configuration to those evals while watching for overfit.

Metrics: Task behavior pass rate · Tool-call quality · Context-compaction quality · Production trace coverage · Overfit/generalization checks · Cost and latency when relevant

“production trades as like eval set and just like fit them and then like look at it as a human and like try to improve it”transcript, stated

Outcomes

The discussion clarifies harnesses as deterministic wrappers and orchestration layers around model/tool loops.
It argues for eval-driven harness tuning rather than blind context engineering.
It stresses that engineers need primitive-level understanding even when higher-level harnesses improve.

This is a philosophical/practical discussion rather than a deployed case with user metrics; the concrete scale references concern model-release cadence and small percentage advantages from better model use.

“they should understand uh the tool calling loop at the most fundamental level.”transcript, stated