Building Production Agentic Systems with Platform-Level LLMOps Features

anthropic · Tech · 2025 · source Agentic: yes · promotional bias: high · source-backed · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Platform-level LLMOps for production agentic systems

Anthropic needed platform primitives that let developers extract better production performance from Claude-based agents. The talk frames three core problems: exposing trained model capabilities through usable APIs, managing finite and expensive context windows, and giving agents secure compute infrastructure so they can write and run code autonomously across local, web, and mobile experiences.

Constraints:

Architecture

Anthropic's platform strategy exposes Claude's capabilities as API primitives and infrastructure. Extended thinking lets developers set a reasoning-token budget. Tool use APIs expose built-in and custom tools with schemas. MCP connects agents to external systems such as GitHub or Sentry. Memory stores context outside the window and retrieves it when relevant, while context editing clears stale tool results. Code execution lets Claude write and run code in secure sandboxed environments hosted by Anthropic, with platform-managed containers and security. Agent skills package scripts, instructions, and resources that Claude can pull into a sandbox when the task matches. Together these features support Claude Code and other agentic systems across local, web, and mobile settings.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  claude-api["Claude API"] --> extended-thinking-controller["Extended thinking controller"]
  claude-api --> tool-use-api["Tool use API"]
  claude-api --> context-manager["Context manager"]
  claude-api --> code-execution-tool["Code execution tool"]
  extended-thinking-controller --> claude-code-agent["Claude Code agent"]
  tool-use-api --> claude-code-agent
  tool-use-api --> mcp-connectors["MCP connectors"]
  mcp-connectors --> context-manager
  context-manager --> memory-tool["Memory tool"]
  context-manager --> context-editing["Context editing"]
  memory-tool --> context-manager
  context-editing --> context-manager
  context-manager --> claude-code-agent
  claude-code-agent --> code-execution-tool
  code-execution-tool --> sandbox-environment["Sandbox environment"]
  sandbox-environment --> session-persistence["Session persistence"]
  session-persistence --> claude-code-agent
  code-execution-tool --> agent-skills["Agent skills"]
  agent-skills --> mcp-connectors
  web-and-mobile-claude-code["Web and mobile Claude Code"] --> session-persistence
Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component Role Interacts with
Claude API Developer-facing API surface that exposes model capabilities and agent infrastructure. Extended thinking controller, Tool use API, Context manager, Code execution tool
Extended thinking controller Token-budget mechanism that lets developers choose longer reasoning or quicker answers. Claude Code agent
Tool use API Built-in and custom tool calling with name, description, and input schema. Claude Code agent, MCP connectors
MCP connectors Standardized access to external tools and context such as GitHub or Sentry. Context manager, Agent skills
Context manager Coordinates relevant information in the context window through MCP, memory, and context editing. Memory tool, Context editing, Claude Code agent
Memory tool Client-side file-system style memory for storing context outside the window and retrieving it only when relevant. Context manager
Context editing Mechanism for clearing irrelevant old tool results from the context window. Context manager
Claude Code agent Agentic coding product that reads/searches/writes files, reruns tests, and uses the platform features. Session persistence, Code execution tool
Code execution tool API primitive that lets Claude write and run code inside secure sandboxed environments. Sandbox environment, Agent skills
Sandbox environment Anthropic-hosted secure execution environment with container orchestration and security handled by the platform. Session persistence
Session persistence Persistent execution state for web/mobile sessions where users start work, walk away, and return later. Claude Code agent
Agent skills Folders of scripts, instructions, and resources that Claude can pull into context and execute in the sandbox. Code execution tool, MCP connectors
Web and mobile Claude Code Non-local client surfaces that require hosted compute, sandboxing, orchestration, and persistence. Session persistence

Data flow

A user asks Claude Code or another agent to complete a task. The platform may allocate extended-thinking tokens, assemble relevant context from MCP and memory, clear old tool results with context editing, and let Claude call tools such as file read/search/write/test operations. For web/mobile or code-execution workflows, Claude writes and runs code inside a secure sandboxed environment managed by Anthropic's container infrastructure. If the task matches an agent skill, Claude pulls scripts, instructions, and resources into context and executes the skill inside the sandbox. Session state persists so users can leave and return to completed work.

6 supporting quotes
“our models, we think about building a platform that helps you do three things. Um so first, the platform helps you harness Claude's capabilities. We're training Claude to get good at a lot of stuff and we need to give you the tools in our API to use the things that”transcript, stated
“you the tools in our API to use the things that Claude is actually getting good at. Next, we help you manage Claude's context window. Keeping the right context in the window at any given time is really really critical to getting the best outcomes from Claude.”transcript, stated
“outcomes from Claude. And third, we're really excited about this lately. We think you should just give Claude a computer and let it do its thing. So I'll talk about how we're we're evolving the platform to give you the infrastructure and otherwise that you need to actually”transcript, stated
“Claude Code takes advantage of this feature in our API to decide whether or not to have Claude think longer. Another basic example is tool use. Claude has gotten really good at reliably calling tools. Um, so we expose this in our API with both our own”transcript, stated
“um, and call those tools and pass the right arguments. So, this is relevant for cloud code. Cloud code has many, many, many tools and it's calling them all the time to do things like read files, search for files, write to files, um, and do stuff like rerun tests and otherwise. So, the”transcript, stated
“released our code execution tool in the API um which allows Claude to run write code and run that code in a secure sandboxed environment. Um, so our platform handles containers, it handles security, and you don't have to think about these things because they're running on our servers. Um, so you can”transcript, stated

Why this architecture

Anthropic argues that model capability is not enough: trained capabilities need productized API controls; context performance depends on active memory and pruning; and autonomous agents need managed compute. The architecture raises performance by exposing reasoning budgets, reliable tool calls, MCP, memory, context editing, sandboxed code execution, and skills as platform features rather than making each developer build the same LLMOps infrastructure.

4 supporting quotes
“we need to give you the tools in our API to use the things that Claude is actually getting good at. Next, we help you manage Claude's context window. Keeping the right context in the window at any given time is really really critical to getting the best”transcript, stated
“Getting the right context at the right time in the window is one of the most important things that you can do to maximize performance.”transcript, stated
“outputs for the things that you're doing just by giving Claude runway to go and do that. But the challenge for letting you do that is actually the infrastructure as well as stuff like expertise like how do you give cloud access to things that um when it's using a computer it will get”transcript, stated
“future of agents is letting the model work pretty autonomously within a sandbox environment and we're giving you the infrastructure to be able to do that. And this gets really powerful once you think about giving the model actual domain expertise in the things that you're trying”transcript, stated

Models used: Claude · Claude Code · Claude models with extended thinking · Claude models with million-token context windows

Scale

Alternatives considered (roads not taken)

Expose only raw model endpoints.

Rejected because: Anthropic argues model capabilities must be surfaced through API features such as extended thinking, tool use, memory, context editing, and code execution.

“we're getting Claude really good at a bunch of stuff and here are the ways that we expose that to you um in our API”transcript, stated
“good at a bunch of stuff and here are the ways that we expose that to you um in our API as ideally customizable features. So here's a first example um relatively basic. Claude got good at thinking um and Claude's performance on various tasks”transcript, stated

Always let Claude reason longer.

Rejected because: The extended-thinking budget lets developers trade off reasoning time against quick answers based on task complexity.

“do you want Claude to think longer for something more complex or do you want Claude to just give you a quick answer?”transcript, stated
“want Claude to just give you a quick answer? Um, we also expose this with a budget. Um, so you can tell Claude how many tokens to essentially spend on thinking. Um, and so for cloud code, um, pretty good example. Obviously, you're often debugging pretty complex systems with cloud”transcript, stated

Keep all potentially useful context in the prompt window.

Rejected because: Tool results and old context can consume space without helping future responses, so memory and context editing keep only relevant information in the window.

3 supporting quotes
“out old tool results. Um, and we did this because tool results can actually just be really large and take up a lot of space in the window. And we found that tool results from past calls are not necessarily super relevant to help claude get good responses later on in a session. And so you can think”transcript, stated
“tool results from past calls are not necessarily super relevant to help claude get good responses later on in a session.”transcript, stated
“evals. Um which was really really huge. And so it just kind of shows you the importance of keeping things in the window that are only relevant at any given time. And we're expanding on this by giving you larger context windows. So for some of our models, you can have a million token”transcript, stated

Run Claude Code only on the user's local machine.

Rejected because: Web and mobile Claude Code require hosted compute, secure execution, container orchestration, and persistent sessions when users walk away.

3 supporting quotes
“When you're running cloud code locally, cloud code is essentially using your machine as its computer. But if you're starting a session on the web or on mobile and then you're walking away, what's happening? Like where is that where is um cloud code running?”transcript, stated
“if you're starting a session on the web or on mobile and then you're walking away, what's happening? Like where is that where is um cloud code running? Where is it doing its work? Um and so we had some hard problems to solve. We needed a secure environment”transcript, stated
“work? Um and so we had some hard problems to solve. We needed a secure environment for cloud to be able to write and run code that's not necessarily like approved code by you. Um we needed to solve or container orchestration at scale. Um and we needed session persistence um because uh”transcript, stated

Use MCP alone for expertise.

Rejected because: MCP supplies tools and context, but skills package the scripts, instructions, and resources that teach Claude how to use them well.

“skill into context and go ahead and use it. And you can combine skills with tools like MCP. So MCP gives you access to tools and access to context. Um, and then skills give you the expertise to actually make use of those tools and make use of that context. Um, and so”transcript, stated
“skills give you the expertise to actually make use of those tools and make use of that context. Um, and so for cloud code, a good example is web design. Maybe whenever you launch a new product or a new feature, um, you build landing pages. And when you build”transcript, stated

Patterns

tool-use-orchestration (agent_tools)

Claude Code relies on built-in and custom tool calling for reading/searching/writing files and rerunning tests.

3 supporting quotes
“Claude has gotten really good at reliably calling tools. Um, so we expose this in our API with both our own built-in tools like our web search tool, um, as well as the ability to create your own custom tools.”transcript, stated
“both our own built-in tools like our web search tool, um, as well as the ability to create your own custom tools.”transcript, stated
“read files, search for files, write to files, um, and do stuff like rerun tests”transcript, stated

context-compaction (context_management)

Context editing clears old tool results that consume window space and may no longer help later reasoning.

3 supporting quotes
“context editing helps you clear stuff out that's not relevant right now and shouldn't be in the window.”transcript, stated
“our first iteration of our context editing is just clearing out old tool results.”transcript, stated
“tool results can actually just be really large and take up a lot of space in the window.”transcript, stated

agent-memory (memory)

The memory tool stores codebase patterns or workflow preferences outside the prompt and retrieves them only when relevant.

3 supporting quotes
“we introduced a memory tool to help you actually keep context outside of the window”transcript, stated
“Claude knows how to pull back into the window only when it actually needs it.”transcript, stated
“your patterns for your codebase or maybe your preferences for your git workflows”transcript, stated

rag-retrieval (retrieval)

MCP gives agents access to external systems and context outside the immediate prompt window.

3 supporting quotes
“MCP as a standardized way for agents to interact with external systems.”transcript, stated
“GitHub or Century. there are plenty of places kind of outside of the agent's context”transcript, stated
“So MCP gives you access to tools and access to context. Um, and then skills give you the expertise to actually make use of those tools and make use of that context.”transcript, stated

cost-aware-routing (performance)

Extended thinking uses a token budget so applications can spend more reasoning tokens on complex tasks and answer quickly for simpler ones.

3 supporting quotes
“Claude's performance on various tasks um scales with the amount of time you give it to reason”transcript, stated
“Um, we also expose this with a budget. Um, so you can tell Claude how many tokens to essentially spend on thinking.”transcript, stated
“you can tell Claude how many tokens to essentially spend on thinking.”transcript, stated

safety-policy-enforcement (security)

Code execution is run in secure sandboxed environments with platform-managed containers and security.

3 supporting quotes
“a secure environment for cloud to be able to write and run code that's not necessarily like approved code by you.”transcript, stated
“Um so we released our code execution tool in the API um which allows Claude to run write code and run that code in a secure sandboxed environment. Um, so our platform handles containers, it handles security, and you don't have to think about these things because they're running on our servers.”transcript, stated
“Um, so our platform handles containers, it handles security, and you don't have to think about these things because they're running on our servers.”transcript, stated

durable-execution (execution)

Web/mobile Claude Code sessions persist while users walk away and return to results after hosted execution continues.

3 supporting quotes
“Um and we needed session persistence um because uh we launched this and many of you were excited about it and started many many sessions and walked away and we had to make sure that um all of these things were ready to go when you came back and um wanted to see the results of what Claude did.”transcript, stated
“Um and we needed session persistence um because uh we launched this and many of you were excited about it and started many many sessions and walked away and we had to make sure that um all of these things were ready to go when you came back and um wanted to see the results of what Claude did.”transcript, stated
“ready to go when you came back and um wanted to see the results of what Claude did.”transcript, stated

eval-driven-development (evaluation)

Anthropic reports internal evals showing a 39% performance improvement from memory plus context editing.

“we saw a 39% bump in performance over o over the benchmark on our own internal evals.”transcript, stated
“it just kind of shows you the importance of keeping things in the window that are only relevant”transcript, stated

Tradeoffs

Expose extended thinking as a token-budgeted API feature.

Chose Developer-controlled reasoning budget over One fixed reasoning mode for every query.

Rationale: Developers can spend more reasoning on complex debugging and choose quicker responses for simpler tasks.

“do you want Claude to think longer for something more complex or do you want Claude to just give you a quick answer?”transcript, stated
“you can tell Claude how many tokens to essentially spend on thinking.”transcript, stated

Use memory plus context editing together.

Chose Store relevant context outside the window and clear stale content inside it over Only increasing the context window.

Rationale: Anthropic reports a 39% benchmark gain and argues performance depends on keeping only relevant context in the window.

3 supporting quotes
“memory helps you keep stuff outside the window and pull it back in when it makes sense”transcript, stated
“context editing helps you clear stuff out that's not relevant right now”transcript, stated
“And so um we found that if we combined our memory tool with context editing, we saw a 39% bump in performance over o over the benchmark on our own internal evals.”transcript, stated

Standardize external system access through MCP.

Chose MCP connectors for tools/context over Custom one-off integrations for every agent/system pair.

Rationale: MCP provides a standardized way for agents to interact with external systems and improves performance beyond prompt-window-only context.

“MCP as a standardized way for agents to interact with external systems.”transcript, stated
“this will obviously get you much better performance than an agent that only sees the things that are in its window”transcript, stated

Provide hosted sandboxed code execution.

Chose Anthropic-managed containers and security over Expect every developer to manage execution infrastructure.

Rationale: The platform abstracts secure execution and container management needed for autonomous agent work.

“our platform handles containers, it handles security, and you don't have to think about these things”transcript, stated
“future of agents is letting the model work pretty autonomously within a sandbox environment”transcript, stated

Combine skills with MCP.

Chose Skills for expertise plus MCP for context/tool access over Skills or MCP alone.

Rationale: MCP gives access, while skills encode domain-specific procedures and resources for Claude to apply.

3 supporting quotes
“And you can combine skills with tools like MCP. So MCP gives you access to tools and access to context. Um, and then skills give you the expertise to actually make use of those tools and make use of that context.”transcript, stated
“So MCP gives you access to tools and access to context. Um, and then skills give you the expertise to actually make use of those tools and make use of that context.”transcript, stated
“Um, and then skills give you the expertise to actually make use of those tools and make use of that context. Um, and so for cloud code, a good example is web design.”transcript, stated

Gotchas (what bit them)

Good models still need API-level capability controls.

Root cause: Capabilities trained into the model are not automatically accessible or controllable in production without interface design.

Resolution: Expose features such as extended thinking budgets and reliable tool-use schemas through the API.

3 supporting quotes
“we need to give you the tools in our API to use the things that Claude is actually getting good at.”transcript, stated
“Claude got good at thinking um and Claude's performance on various tasks um scales with the amount of time you give it to reason through those problems.”transcript, stated
“Claude has gotten really good at reliably calling tools. Um, so we expose this in our API with both our own built-in tools like our web search tool, um, as well as the ability to create your own custom tools.”transcript, stated

Context windows fill with stale tool results and competing information.

Root cause: Coding agents carry technical designs, whole codebases, instructions, and tool-call history, and old results can be large but no longer useful.

Resolution: Use MCP for retrieval, memory for relevant out-of-window storage, and context editing to clear irrelevant tool results.

3 supporting quotes
“You've got your technical designs, you've got your entire codebase.”transcript, stated
“Um you've got instructions, you've got tool calls. All these things might be in the window at any given time. And so how do you make sure the right set of those things are in the window?”transcript, stated
“tool results can actually just be really large and take up a lot of space in the window.”transcript, stated

Hosted web/mobile agents require infrastructure that local agents get for free.

Root cause: A local Claude Code session uses the user's machine, but hosted sessions need secure compute, orchestration, and persistence.

Resolution: Provide platform-managed secure code execution, containers, sandboxing, and session persistence.

3 supporting quotes
“When you're running cloud code locally, cloud code is essentially using your machine as its computer. But if you're starting a session on the web or on mobile and then you're walking away, what's happening?”transcript, stated
“We needed a secure environment for cloud to be able to write and run code”transcript, stated
“we needed to solve or container orchestration at scale. Um and we needed session persistence”transcript, stated

Large context windows do not remove the need for context management.

Root cause: Even million-token windows need the right information in the right place and may still run low on room.

Resolution: Combine larger windows with context editing and model awareness of remaining context space.

3 supporting quotes
“Combining that larger window with the tools to actually edit what's in your window maximizes your performance.”transcript, stated
“teaching Claude to get better and better at actually understanding what's in its context window.”transcript, stated
“maybe it has a lot of room to run, maybe it's almost out of space.”transcript, stated

Evaluation

The talk gives one concrete evaluation signal: Anthropic internal evals showed a 39% performance improvement from combining the memory tool with context editing. It also describes task-dependent reasoning budget control and context relevance as performance levers, but does not present external benchmark details, production monitoring, failure recovery, or cost-control metrics.

Metrics: 39% internal benchmark performance improvement · Extended-thinking token budget · Tool-call reliability · Context-window relevance · Million-token context availability for some models · Session persistence for many web/mobile sessions

4 supporting quotes
“we saw a 39% bump in performance over o over the benchmark on our own internal evals.”transcript, stated
“Claude's performance on various tasks um scales with the amount of time you give it to reason”transcript, stated
“Getting the right context at the right time in the window is one of the most important things”transcript, stated
“for some of our models, you can have a million token context window.”transcript, stated

Outcomes

The source reports Claude Code calling hundreds of tools, a 39% internal-eval improvement from memory plus context editing, million-token context windows for some models, and many web/mobile sessions that required persistence after users walked away. It does not provide production cost, latency, availability, or incident metrics.

5 supporting quotes
“Another basic example is tool use. Claude has gotten really good at reliably calling tools. Um, so we expose this in our API with both our own built-in tools like our web search tool, um, as well as the ability to create your own custom tools.”transcript, stated
“And so um we found that if we combined our memory tool with context editing, we saw a 39% bump in performance over o over the benchmark on our own internal evals.”transcript, stated
“So a fun story is we recently launched cloud code on web and mobile. Um and this was a fun project for our team because we had a lot of problems to solve.”transcript, stated
“allows Claude to run write code and run that code in a secure sandboxed environment.”transcript, stated
“Skills are basically just folders of scripts, instructions, and resources that Claude has access to”transcript, stated