Reducing Agent Costs and Eliminating Rate-Limit Failures with Sandboxed Code Execution

port_of_context · Tech · 2026 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Production GTM intelligence agents over data-heavy developer sources

Port of Context needed a scheduled GTM intelligence agent to search GitHub and Hacker News across multiple exact terms, aggregate large result sets, and post Slack digests without rate-limit failures or excessive model-context cost.

Constraints:

• Searches had to use specific key terms, not broad keywords.
• The agent needed to run multiple scheduled searches per day.
• Each run could return large JSON payloads from GitHub and Hacker News.
• The team wanted Slack digests that humans could scan in seconds.
• The architecture had to support multiple model families without changing the workload.
• The execution path needed to avoid per-minute rate caps and high input-token costs.

Architecture

Port of Context built a GTM agent on goose, a custom gtmdata MCP server, Arcade.dev runtime, and pctx Code Mode. The MCP server exposes typed tools for GitHub issue/PR/repository search, Hacker News search, and Slack posting. With Code Mode on, the agent writes one TypeScript program, runs parallel MCP calls inside a permissioned sandbox, keeps intermediate 100k-token payloads out of the model context, and returns only the final digest. The standard loop was evaluated as a baseline and failed more often due to repeated JSON payloads re-entering context.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  A["Scheduled GTM prompt"] --> B["goose agent harness"]
  B --> C["pctx Code Mode sandbox"]
  B --> D["gtmdata MCP server"]
  C --> D
  D --> E["Arcade.dev runtime"]
  E --> F["GitHub APIs"]
  D --> G["Hacker News search"]
  E --> H["Slack channel"]
  B --> I["Model provider"]

Components

Data flow

A scheduled prompt asks the agent to find exact-term mentions across sources and post a digest. In Code Mode, goose has the model emit a TypeScript program. pctx type-checks and runs it inside a Deno sandbox, calling gtmdata's typed MCP tools, often in parallel with Promise.all. GitHub and Hacker News result payloads stay inside the sandbox; the program reduces them into a Slack-ready digest and calls the Slack tool. Only the reduced final result returns to model context.

Why this architecture

Code Mode was chosen because the workload is data-heavy, multi-tool, and scheduled. Moving intermediate tool payloads into a typed sandbox avoids replaying every JSON result through the model context, which directly reduces tokens, cost, and rate-limit failures while preserving the same MCP tool surface.

Models used: Sonnet 4-5 · Opus 4-6 · GPT-5-4

Scale

• The GTM agent searches GitHub and Hacker News for mentions from the past seven days.
  “Use gtm agent to look up all mentions of the specific term “code mode” across sources over the past 7 days”article, stated
• The experiment ran three models, with Code Mode on and off, six runs per model.
  “Models: Sonnet 4-5, Opus 4-6, GPT-5-4”article, stated

  “Runs per model: 6 (3 ON, 3 OFF)”article, stated
• Code Mode delivered 100% task delivery versus 56% with the standard loop.
  “100% delivery with Code Mode ON. 56% with it off. Every rate-limit failure in the experiment was an OFF-mode run.”article, stated
• Average cost fell from $0.41 to $0.20 per run.
  “Average cost per run: $0.20 ON vs $0.41 OFF, roughly 2× cheaper.”article, stated
• Average input tokens fell from 93.6k to 31.8k per run.
  “Average input tokens per run: 31.8k vs 93.6k, roughly 3× more efficient.”article, stated
• The cheapest correct result cost $0.056.
  “Cheapest correct result in the dataset: $0.056, a Code Mode ON run.”article, stated

Alternatives considered (roads not taken)

Goose's standard tool-use loop

Rejected because: Each tool result re-entered the model context as a large JSON message, driving token growth and per-minute rate-limit failures.

Anthropic Programmatic Tool Calling

Rejected because: The experiment needed BYO-LLM comparison across Claude and GPT; Anthropic PTC is Claude-only.

Cloudflare Code Mode or Pydantic Monty

Rejected because: Those implementations are tied to specific platforms/agent abstractions, while pctx kept goose as the harness and made the model a config knob.

Patterns

tool-use-orchestration (architecture)

The agent calls MCP tools for GitHub issues, PRs, repositories, Hacker News, and Slack, either through a classic loop or through one sandboxed TypeScript program.

structured-output-contract (implementation)

Typed MCP tool schemas and TypeScript pre-flight checks let Code Mode chain calls as typed function calls and surface diagnostics before runtime.

cost-aware-routing (serving)

The execution mode choice routes data-heavy workloads into a sandbox to cut model input tokens and per-run cost.

context-engineering (context)

Port of Context positions itself as building the context engineering and execution layer, and Code Mode keeps large tool payloads out of model context by moving them into sandboxed execution.

latency-optimization (serving)

Parallel Promise.all execution and sandbox-side reduction avoid serial model/tool turns and remove the rate-limit cliff caused by repeated large context payloads.

Tradeoffs

Execution mode

Chose Code Mode sandboxed TypeScript over standard tool-use loop.

Rationale: The sandbox holds intermediate payloads and executes tools in parallel, making runs cheaper and more reliable.

Runtime portability

Chose pctx BYO-LLM in goose over Claude-only or platform-specific Code Mode.

Rationale: The team needed to compare model families and run the same workload without changing the harness.

Authorization/tool runtime

Chose Arcade.dev managed OAuth and policies over custom token management.

Rationale: Arcade handled GitHub and Slack scopes through per-request OAuth primitives without the team writing token-management code.

Gotchas (what bit them)

Classic tool loops hit rate limits because every large payload re-entered context.

Root cause: Four 15-30KB search results accumulated in the model's input context before the final Slack post.

Resolution: Code Mode kept intermediate payloads inside the sandbox and returned only a short final result.

Sonnet misread phrase-search intent and produced confident broad counts.

Root cause: It passed "code mode" unquoted to GitHub Search, which ANDed the words instead of searching the phrase.

Resolution: The experiment exposed this as a model-specific failure profile; Opus and GPT handled phrase quoting differently.

GPT got the API call right but mis-framed counts.

Root cause: Two GPT-5-4 Code Mode runs reported the safe-limit array length instead of the real total_count field.

Resolution: Typed fields help, but the evaluation surfaced that count framing still needs explicit validation.

Model families had different failure profiles on the same prompt.

Root cause: The same agent/tool setup behaved differently across Sonnet, Opus, and GPT.

Resolution: BYO-LLM execution allowed isolating the model as the only changing variable.

Evaluation

Port of Context ran the same prompt in fresh goose sessions across three models, with Code Mode on and off, holding the architecture, MCP server, upstream APIs, and Slack channel constant.

Metrics: 3 models tested: Sonnet 4-5, Opus 4-6, GPT-5-4 · 6 runs per model: 3 Code Mode on, 3 off · 100% delivery with Code Mode on versus 56% off · $0.20 average cost on versus $0.41 off · 31.8k input tokens on versus 93.6k off · $0.056 cheapest correct result

Outcomes

• Code Mode became the GTM Agent's production execution path.
• It delivered every task in the experiment, while the standard loop delivered 56%.
• It cut average per-run cost roughly in half and average input tokens by roughly 3x.
• The experiment exposed model-specific failure modes across Sonnet, Opus, and GPT.
• Port of Context planned more sources, more keyword sets, scheduled goose runs, and a public GTM agent.

The case is a controlled experiment over 18 runs, not a long-term production SLO report, but it uses real GitHub/Hacker News/Slack workloads and production-style tool/runtime constraints.