Building and Evaluating Production AI Agents: From Function Calling to Complex Multi-Agent Systems

google_deepmind · Tech · 2025 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Production agent systems and evaluation

Google DeepMind product builders needed to adapt agent harnesses, tool orchestration, evaluation, and model selection as Gemini/Gemma capabilities rapidly evolved from simple function calling to complex multi-step agentic behavior.

Constraints:

Model capability changes can invalidate existing agent harness assumptions.
Complex agents require evaluation beyond single prompt-output checks.
Long context windows still suffer context rot and distractor effects.
Deployment spans cloud Gemini models and small Gemma models for constrained environments.

Architecture

The conversation frames production agents as a combination of model, tools, harnesses, evaluation files, context management, and model-size decisions. Builders can use standard Gemini function calling, custom agent flows, orchestrators with subagents, and Gemma variants for edge or specialized tasks, all guarded by repeatable evaluation harnesses.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  agent-harness[Agent Harness] --> gemini-model[Gemini Model]
  agent-harness --> gemma-model[Gemma Model]
  gemini-model --> tool-calls[Tool Calls]
  tool-calls --> agent-harness[Agent Harness]
  agent-harness --> evaluation-harness[Evaluation Harness]
  agent-harness --> context-manager[Context Manager]
  context-manager --> subagent-orchestrator[Subagent Orchestrator]
  subagent-orchestrator --> agent-harness[Agent Harness]
  gemma-model --> evaluation-harness[Evaluation Harness]

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Gemini Model`	General high-capability model for complex tool-using agents.	`Agent Harness`, `Tool Calls`
`Gemma Model`	Open-weight or smaller model option for local, edge, or specialized use cases.	`Agent Harness`, `Evaluation Harness`
`Agent Harness`	Coordinates model calls, tools, context, and product behavior.	`Gemini Model`, `Gemma Model`, `Tool Calls`, `Evaluation Harness`, `Context Manager`
`Tool Calls`	Function-calling interface for external actions and multimodal tools.	`Gemini Model`, `Agent Harness`
`Evaluation Harness`	Repeatable tests for model and agent behavior at increasing complexity.	`Agent Harness`, `Gemma Model`
`Context Manager`	Filters, compresses, and routes context to avoid context rot.	`Agent Harness`, `Subagent Orchestrator`
`Subagent Orchestrator`	Spins up subagents and moves context between them for complex tasks.	`Context Manager`, `Agent Harness`

Data flow

A product team chooses Gemini or Gemma, exposes tools through function calling or a custom harness, evaluates each step with repeatable test files, adds orchestration/subagents when the task exceeds single-flow complexity, and revises the harness as model capabilities improve.

“function calling/tool calling mode. And just one quick thing before I get back to you, Hugo. Tool calling and function calling are used very interchangeably.”transcript, stated

“evaluation harness is just a repeatable way to to understand your evolve results. So again, let me stick to a food analogy because I think it's fun.”transcript, stated

Why this architecture

The architecture is deliberately elastic because improved models reduce some orchestration needs while more ambitious products add complexity back. Evaluation harnesses become the stable control surface for deciding when harnesses, context management, or model choices should change.

“rewritten agentic harnesses like three times in the last three years because Gemini gets so good that I don't need to check for certain things or I can just defer entire entirely the model. But on the other side we keep doing more and more complex things. So I need to build”transcript, stated

“and why building robust eval and agent harnesses is one of the biggest competitive moes you can have right now. This is a high signal conversation on exactly what changes when you move from simple chat bots to complex agentic”transcript, stated

Models used: Gemini 3 · Gemini 1.5 · Gemini · Gemma · Gemma 270M · Gemma 27B

Scale

Gemini solved a Project Euler problem in six minutes while making multiple tool calls and fixing errors.
“Gemini spent six minutes and solved it right and in that time it was making multiple tool calls getting multiple results fixing multiple errors”transcript, stated
The speaker rewrote agentic harnesses about three times in three years.
“internally within Google like I've rewritten agentic harnesses like three times in the last three years because Gemini gets so good that I don't need to check for certain things or I can just defer entire entirely the model. But on the other side we keep doing more and more complex things.”transcript, stated
Gemma spans model sizes from 270M through 27B.
“but we go down to 12B, 4B, 1B, and the smallest now is 270M. So this is something that runs on your could run on your phone quite reliably and quite easily. So to answer this like agents question again, if you need your agent”transcript, stated

Alternatives considered (roads not taken)

Keep a fixed agent harness while models improve.

Rejected because: Model improvements changed what checks and orchestration were needed.

“three years because Gemini gets so good that I don't need to check for certain things or I can just defer entirely the model. But on the other side, we keep doing more and more complex things. So, I need to build new art harnesses for more more complex sets of behaviors than ever before.”transcript, stated

Evaluate agents like a single model call.

Rejected because: Agentic flows are more complex and require evaluating multiple functions, steps, and behaviors.

“showed earlier Agentic flows are just more inherently complex than the single prompt single generation flow. So if you take a can keep using the same example now that we're stuck on it is make me what make me a tell me the recipe for a Marmite sandwich or vegomite sandwich”transcript, stated

Assume long context solves retrieval and orchestration.

Rejected because: Large context windows still exhibit context rot and distractor problems.

“because of course we have really large context windows but we see there's context rot as well right so if you utilize even sub amounts of the available context we degradation in in in retrieval among other things. So something that people”transcript, stated

Use Gemini for every deployment.

Rejected because: Offline, edge, or specialized use cases may require Gemma.

“if you need your agent to work somewhere without internet where like you're off the grid, then you're going to have to use a Gemma model.”transcript, stated

Patterns

`tool-use-orchestration` (agent harness)

Function/tool calling lets Gemini-based systems perform multi-step tasks through external actions.

“we went from very simple tool calls like get weather and quickly we've gone to in 17 months gone to LMS”transcript, stated

`eval-driven-development` (evaluation)

Teams use repeatable evaluation harnesses as the core way to understand and evolve agents.

“evaluation harness is just a repeatable way to to understand your evolve”transcript, stated

`multi-agent-orchestration` (complex agents)

Orchestrators spin up subagents and move context between them for large tasks.

“agent architectures that use orchestrators to spin up sub agents and move context between them.”transcript, stated

`context-compaction` (context management)

The discussion calls out context rot and context filtering as reasons to curate high-signal sources.

“curating super high signal sources that are then getting put into Gemini's context.”transcript, stated

`fine-tuning-adaptation` (model customization)

Smaller Gemma variants can be fine-tuned for narrow specialized tasks.

“if you need really great one to five function calls for example, you can easily train a Gemma 270M to be really good at that one task”transcript, stated

Tradeoffs

Harness complexity

Chose Rebuild harnesses as model ability changes over Freeze an early orchestration layer.

Rationale: Better models can simplify some checks, but more complex products still need new harness design.

“I don't need to check for certain things or I can just defer entirely the model. But on the other side, we keep doing more and more complex things.”transcript, stated

Model deployment

Chose Gemini for general capability and Gemma for local/specialized needs over One model family for all constraints.

Rationale: Some environments cannot call cloud Gemini, while smaller Gemma models can be specialized.

“if you need your agent to work somewhere without internet where like you're off the grid, then you're going to have to use a Gemma model.”transcript, stated

Context strategy

Chose Curated high-signal context and orchestration over Relying on raw large context windows.

Rationale: Context rot makes needle-in-haystack success an incomplete proxy for agent reliability.

“because of course we have really large context windows but we see there's context rot as well right”transcript, stated

Gotchas (what bit them)

Agent harnesses become stale as model capabilities move.

Root cause: Improved models eliminate some needed checks while new product ambitions add complexity elsewhere.

Resolution: Continuously rebuild harnesses and keep evaluation harnesses close to product behavior.

“rewritten agentic harnesses like three times in the last three years because Gemini gets so good”transcript, stated

Long context windows can still rot.

Root cause: Distractors and excessive tokens can degrade retrieval and reasoning even with million-token contexts.

Resolution: Use high-signal source curation, filtering, compression, and orchestration with subagents.

“context rot does continue to like needle in a haststack does seem like a vanity”transcript, stated

Agent evaluation grows faster than model evaluation.

Root cause: Agents combine model behavior, tools, functions, routing, and multi-step flows.

Resolution: Break evaluation into repeatable files for function calling, tools, and complex scenarios.

“evaluation, it ends up being a little bit more complex than now than a py file. Um because agents are tend to be more complex”transcript, stated

Evaluation

The source recommends repeatable evaluation harnesses for products and agents, splitting simple function-calling tests from more complex multi-tool and multimodal scenarios.

Metrics: Function-calling ability · Tool-call correctness · Agent harness behavior · Context robustness · Model-family fit

“I'll have a Python file that's just test the function calling ability. Then maybe I'll have another Python file”transcript, stated

Outcomes

The discussion documents a shift from simple function calling to complex multihop agents.
Google teams repeatedly rewrote agentic harnesses as Gemini improved.
Gemma 270M is positioned as a small model for narrow function-calling tasks.

Concrete scale includes a six-minute Gemini tool-use task, 17 months from simple tool calls to complex multistep agents, three harness rewrites in three years, and Gemma sizes down to 270M.