Building and Scaling a Production MCP Server for Developer Tooling

github · Tech · 2026 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Developer tooling infrastructure

GitHub needed to run a production MCP server that gives agents broad access to GitHub tools without overwhelming model context, weakening security, or making tool calls unreliable at large public adoption scale.

Constraints:

• GitHub's product surface spans repositories, issues, pull requests, actions, projects, and user-specific tools
• most users keep default configurations, so safety and context quality cannot depend on manual JSON tuning
• agents can hallucinate repository permissions and fail tool calls unless the server encodes intent and permissions
• MCP authentication must support wildly different security postures without normalizing long-lived plaintext tokens
• prompt-injection and data-exfiltration risks remain unsolved for agents that can read private data and communicate externally
• the remote server must scale horizontally without session affinity

Architecture

GitHub operates its MCP server as a stateless remote HTTP service. Each request creates a fresh SDK server instance, dynamically adds tools according to user configuration, token scopes, OAuth grants, policies, and server-token context, then calls GitHub APIs on the agent's behalf. Redis session storage tracks client identity rather than holding stateful MCP process state. The team reduced default tool context, grouped CRUD operations, trimmed tool outputs, encoded multi-call GitHub workflows behind higher-level tools, added evals that test tools against each other, and moved authentication toward OAuth 2.1 with PKCE, step-up OAuth, and permission-based tool filtering.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  mcp-clients[MCP clients] --> remote-mcp-service[Remote MCP service]
  mcp-clients --> oauth-flow[OAuth flow]
  oauth-flow --> permission-filter[Permission filter]
  remote-mcp-service --> request-server-instance[Request server instance]
  remote-mcp-service --> redis-session-storage[Redis session storage]
  remote-mcp-service --> observability-stack[Observability stack]
  request-server-instance --> tool-configuration-and-policies[Tool configuration and policies]
  tool-configuration-and-policies --> permission-filter
  permission-filter --> github-api-tools[GitHub API tools]
  request-server-instance --> github-api-tools
  github-api-tools --> github-apis[GitHub APIs]
  evaluation-pipeline[Evaluation pipeline] --> github-api-tools
  evaluation-pipeline --> observability-stack
  insiders-feature-flags[Insiders feature flags] --> remote-mcp-service
  insiders-feature-flags --> mcp-clients

Components

Data flow

A client connects to GitHub's remote MCP server using OAuth or a token. On each request, the service creates a new MCP server instance, filters the available tools according to user configuration, token scopes, policies, or server-token identity, and exposes only the allowed tool surface. The agent calls those tools; server-side implementations may perform multiple GitHub API calls to satisfy the intent robustly. Redis-backed session data records client identity, observability tracks tool usage and failures, and evals compare tools in the pool to tune descriptions, context footprint, and call behavior.

Why this architecture

GitHub's design pushes complexity into the MCP server because agents perform worse when every raw tool and edge case is dumped into context. Dynamic request-time tool construction lets defaults, permissions, and policies be applied automatically, matching the reality that users rarely configure tool sets. OAuth and step-up scopes make secure remote auth the easier path, while stateless per-request instances and Redis-backed client identity keep the service horizontally scalable. Evaluating tools against each other matches the actual agent decision problem: tools compete inside one context window.

Models used: agent client models · Claude Code tool search · OpenAI tool-search-like APIs

Scale

• The server had over 100 tools after roughly a month of feature additions.
  “After a month or so of new features, agents in some ways were getting worse at using GitHub and context windows were getting blown out quicker. And uh you know, we picked, I think, over 100 tools.”transcript, stated
• The initial default context reduction cut initial load by about 49%.
  “we cut the amount of context used by focusing the tools more specifically to the general case. And based on usage to like about 49% reduction of the initial load.”transcript, stated
• The default configuration was reduced to about 40 tools.
  “I think like I think you get about 40 tools if you use the default configuration. And then you can kind of expand or contract that based on your own preference.”transcript, stated
• A tailored list-pull-requests output reduced output tokens by more than 75%.
  “just by tailoring exactly what comes with the list pull requests, it's like actually lost more than 75% of the tokens used in the output.”transcript, stated
• The remote MCP server served around 7 million tool calls per week and was approaching 8 million.
  “we've been able to scale to this point we serve around 7 million tool calls a week. And we you know, we don't have session affinity.”transcript, stated

  “we're fast approaching 8 million tool calls a week.”transcript, stated
• The open-source project had over 11 million Docker downloads, 126 contributors, more than 2,300 issues and PRs, nearly 4,000 forks, and almost 30,000 stars.
  “it's actually got over 11 million Docker downloads of our standard IO server which is by not like by far not the most used version of it either. Um we've got 126 contributors now and over 2,300 issues and PRs”transcript, stated

  “we've got almost 4,000 forks which blows my mind. I kind of want to know like the weirder things that people have done that they haven't contributed back. Uh [snorts] yeah, nearly 30,000 stars”transcript, stated

Alternatives considered (roads not taken)

Expose every GitHub tool directly in the model context

Rejected because: Agents became confused and forgetful when over 100 tools and too much context were shoved directly into the window.

Rely on users to configure tool sets, dynamic tool selection, or semantic tool search

Rejected because: Those designs required user configuration, and most users simply kept defaults.

Expose low-level API sequences to the agent

Rejected because: GitHub moved robust multi-call operations into server-side tools to save round trips, context, time, and improve success.

Dynamic client registration for OAuth

Rejected because: GitHub judged it operationally risky because it caused app-database growth, rate-limit bucketing problems, and unreliable app identity.

Stateful singular MCP server processes

Rejected because: GitHub chose a stateless server setup with Redis and per-request server construction to scale without session affinity.

Patterns

tool-use-orchestration (agent-tooling)

GitHub wraps platform operations as MCP tools, dynamically exposes the right tools per request, and hides multi-call GitHub API workflows behind single agent-facing tools.

context-compaction (context-management)

The team reduced default tool context, grouped tools, filtered by permissions, and cut verbose outputs so agents see less irrelevant MCP context.

safety-policy-enforcement (security)

PAT scopes, OAuth scopes, server-token identity, and policies dynamically remove tools the user should not or cannot use.

eval-driven-development (quality)

GitHub evaluates tools in the context of the full tool pool, checking whether each tool is called at the right and wrong times rather than only optimizing isolated descriptions.

human-in-the-loop-gate (workflow)

Insiders MCP apps let users review and edit agent-drafted GitHub issues before posting them publicly.

Tradeoffs

Context strategy

Chose optimize and filter the default tool set over exposing the full GitHub tool universe by default.

Rationale: Broad access was valuable, but too much tool context degraded agent performance; the default needed to work for users who never configure anything.

Tool abstraction

Chose server-side intent-encoded tools over agent-managed sequences of raw GitHub API calls.

Rationale: Moving edge cases into server code reduces round trips, context use, time, and tool failures.

Authentication path

Chose remote OAuth with PKCE and step-up scopes over local runtimes and long-lived plaintext access tokens.

Rationale: OAuth made secure connection easier for clients and allowed incremental permission grants instead of broad token exposure.

OAuth client identity

Chose reject dynamic client registration over accepting unbounded dynamic client records.

Rationale: GitHub judged DCR too costly for application databases, rate-limit grouping, and reliable app identity.

Deployment model

Chose stateless per-request MCP server construction over stateful singular MCP server processes.

Rationale: A fresh request-scoped instance lets the service dynamically assemble tools and scale without session affinity.

Gotchas (what bit them)

Large MCP tool surfaces can make agents worse, not better.

Root cause: Over 100 tool descriptions and platform coverage were shoved into context, causing confusion and faster context-window exhaustion.

Resolution: Optimize the default tool surface, group CRUD tools, reduce outputs, and continue experimenting with better tool discovery approaches.

Elegant configuration options failed because users kept defaults.

Root cause: Tool sets, dynamic selection, and semantic discovery still depended on users or clients configuring the server.

Resolution: Treat the default experience as the product and optimize it from observed usage patterns.

Plaintext access tokens created a standing security hazard.

Root cause: MCP users often stored long-lived, over-privileged tokens where agents could access them.

Resolution: Move secure remote HTTP/OAuth to the path of least resistance and support OAuth 2.1, PKCE, token-scope filtering, and step-up scopes.

Some tool failures came from agents not knowing permissions.

Root cause: Agents may call repositories or user-specific tools they cannot access, and hallucination cannot be eliminated entirely.

Resolution: Filter available tools by token scopes, OAuth scopes, server-token identity, and user-specific context before the agent sees them.

Prompt-injection exfiltration remains structurally hard for useful agents.

Root cause: Useful agents combine private-data access, external instructions, and external communication, the same capabilities behind the lethal-trifecta attack.

Resolution: Support different user risk profiles and security postures while acknowledging the industry has not solved the underlying conflict.

Evaluation

GitHub evaluates the MCP server by looking at production usage patterns, tool failure data, token consumption, and tool-choice evals that test tools against each other in the same pool rather than optimizing descriptions in isolation.

Metrics: over 95% tool-call success rate · 49% initial context-load reduction · over 75% token reduction for list pull requests output · around 7 million tool calls per week · approaching 8 million tool calls per week

Outcomes

• GitHub's MCP server scaled to around 7 million and nearly 8 million weekly tool calls
• the project reached over 11 million Docker downloads, 126 contributors, almost 30,000 stars, and nearly 4,000 forks
• context and output-token reductions improved the default experience
• tool-call success reached roughly over 95%
• OAuth, PKCE, step-up scopes, and permission filtering made secure remote access easier

The talk frames the server as high-scale production infrastructure: per-request stateless MCP construction, no session affinity, millions of weekly tool calls, and open-source traffic over seven issues/PRs per day for more than a year.