Building and Deploying a Pokemon-Playing LLM Agent at Anthropic

anthropic · Tech · 2023 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Long-horizon agent evaluation

Anthropic needed a playful but revealing way to test whether Claude could maintain coherent behavior over long horizons, learn from feedback, and use tools over thousands of game actions.

Constraints:

The agent had to operate over hours of prompts and actions rather than a single-turn task.
Pokemon progress is visible enough to act as an informal benchmark.
The model needed persistent state because game knowledge and decisions accumulate over time.
The project was an evaluation/learning experiment rather than a production user workflow.

Architecture

Claude Plays Pokemon gives the model a prompt, button-press tools, screenshots after actions, a self-managed knowledge base inside its context, and lightweight navigation/world information. The agent presses buttons, observes the resulting screen, updates memory, and continues over long sessions.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  claude-agent[Claude Agent] --> button-tools[Button Tools]
  button-tools --> game-emulator[Game Emulator]
  game-emulator --> screenshot-observation[Screenshot Observation]
  screenshot-observation --> claude-agent[Claude Agent]
  claude-agent --> knowledge-base[Knowledge Base]
  navigation-helper[Navigation Helper] --> claude-agent[Claude Agent]

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Claude Agent`	LLM player making game decisions.	`Button Tools`, `Knowledge Base`, `Game Emulator`
`Button Tools`	Allowed controller actions such as A, B, start, select, and directions.	`Game Emulator`
`Game Emulator`	Pokemon environment that returns screen observations.	`Claude Agent`, `Screenshot Observation`
`Screenshot Observation`	Visual feedback after each action.	`Claude Agent`
`Knowledge Base`	Persistent in-context memory for game facts and navigation notes.	`Claude Agent`
`Navigation Helper`	Minor state/world information supporting movement.	`Claude Agent`

Data flow

Claude receives the Pokemon-playing prompt, chooses button-press actions, gets a screenshot and small state dump after each action, writes persistent notes into a knowledge base, and repeats the loop as it navigates the game world.

“essentially um it has like the ability to use like Press buttons it can press like AB start select up down left right you know like Press buttons um it has this like concept of a knowledge base that's actually like stuck in its prompt but it's just like bits of information”transcript, stated

“it actually sees the world is when it presses a button it gets a screenshot back like to see where where it is after the button press it also gets like a little dump of some stuff that like reads directly from the game like its current location and a few other things”transcript, stated

Why this architecture

The architecture is intentionally simple because the experiment tests long-horizon tool use and memory rather than game-specific training. Button tools and screenshots create a closed action-observation loop, while the knowledge base gives Claude a way to persist useful discoveries over long sessions.

“an interesting way to see how models handle these like long Horizon decisioning things all of these evals that people are used to like mlu and uh gqa like like there's all these evals that exist out there and most of them are like here's a prompt and I get a”transcript, stated

“bloats so the knowledge base is a way for it to like kind of track wander Horizon things yeah that makes sense well dude talk to me about some of the stuff that you're doing right now at anthropic I”transcript, stated

Models used: Claude · Claude Sonnet 3.5 · Claude Opus

Scale

The project started around June of the prior year.
“like the first time I tried this was in like June last year there's basically like two things that were true I was working I work with customers at anthropic I was working with a bunch of customers working on agents and like I wanted some place for”transcript, stated
The evaluation stresses hours of prompts and actions.
“over hours and hours and hours and prompts and prompts and prompts of like taking new information trying to learn update do stuff it became interesting enough for us to like understand what the models”transcript, stated
The agent must track information over long periods of time.
“that it stores to keep track of things over a long periods of time so it's like uh they electric is super effective against water the thing it could like decide to write there potentially but it like fully controls that so it can like”transcript, stated

Alternatives considered (roads not taken)

Use standard benchmark scores alone.

Rejected because: Pokemon exposed capability changes that were not obvious from similar benchmark scores.

“some of these benchmarks look kind of similar and then suddenly like we hook it up to Pokemon and it's like oh it's like night and day this model like can do it you know it's like whoa that's crazy that's so weird we got to show more”transcript, stated

Train specifically on Pokemon.

Rejected because: Part of the point was that the model was not trained on Pokemon.

“fun about this is that like it's not trained on Pokemon you know like part of the fun thing is like this is It's exploring this thing for the first time and and uh getting a feel for that and so I think we're going to stay in the”transcript, stated

Rely on prompt state alone.

Rejected because: The model needed a knowledge base to manage long-period information.

“this like concept of a knowledge base that's actually like stuck in its prompt but it's just like bits of information that it stores to keep track of things over a long periods of time so it's like uh they electric is super effective”transcript, stated

Patterns

`tool-use-orchestration` (game agent)

Claude controls the game through explicit button-press tools.

“give it access to three tools”transcript, stated

`agent-memory` (long-horizon state)

The knowledge base stores discoveries and navigation facts across long play sessions.

“information over longer periods of time”transcript, stated

`eval-driven-development` (capability evaluation)

Pokemon is used as an unconventional benchmark for long-horizon coherence.

“benchmarks wise is about as good as Opus”transcript, stated

Tradeoffs

Evaluation task

Chose Pokemon gameplay over Only static benchmarks.

Rationale: The game gives visible long-horizon progress and failures.

“Pokemon it's like I don't know you beat a gym leader like that's that's a thing”transcript, stated

Action space

Chose Button tools over Game-specific scripted actions.

Rationale: Button presses keep the interface generic and force the model to reason from observations.

“it presses buttons it sees what happens afterwards and it presses the next button”transcript, stated

Memory design

Chose Self-managed knowledge base over No persistent memory.

Rationale: Long gameplay requires recalling type advantages, locations, and prior discoveries.

“the knowledge base is one way that it can keep track of information”transcript, stated

Gotchas (what bit them)

Long-horizon agents bloat their own context.

Root cause: The summary and memory grow as the agent accumulates information.

Resolution: Use a knowledge base to externalize and organize long-running state.

“the summary it hopefully writes kind of bloats”transcript, stated

Benchmarks can miss practical agent capability shifts.

Root cause: Traditional scores may look similar while embodied long-horizon behavior changes sharply.

Resolution: Use interactive tasks like Pokemon as complementary evals.

“it's like night and day this model like can”transcript, stated

Evaluation

The project evaluates Claude by observing game progress, long-horizon coherence, prompt/action loops, knowledge-base use, and capability differences across model versions.

Metrics: Game progress · Long-horizon coherence · Prompt/action count · Knowledge retention · Model-version comparison

“taking new information trying to learn”transcript, stated

Outcomes

The agent became a public demonstration of Claude playing Pokemon.
The project surfaced long-horizon model improvements across versions.
It provided an informal benchmark for agent memory and tool-use behavior.

The transcript emphasizes hours of play, prompts and prompts of actions, a June 2024 origin, and long-period knowledge retention rather than production traffic.

“it got put out into the world this weekend”transcript, stated