Platform-Driven AI Agent Orchestration for Large-Scale Engineering

linkedin · Tech · 2025 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Enterprise engineering automation and AI agent platforms

LinkedIn needed to scale AI agents across a massive engineering organization without every team rebuilding fragile orchestration, tool access, context serving, safety, evaluation, and audit infrastructure. The platform had to let humans express intent while agents execute repeatable engineering work safely and consistently.

Constraints:

• LinkedIn operates at very large product and engineering scale, including 1.3 billion members, 7,000 deployables, and 10,000-plus repositories.
• AI agent tasks must be repeatable, high-coordination, and relatively easy to validate.
• Human intent is often ambiguous, so agents need structured specs, acceptance criteria, and guardrails.
• Agents must run in sandboxed environments with scoped identity, permissions, retries, and persisted state.
• Tool calls must be explicitly permissioned, schema-stable, auditable, and machine-oriented.
• Context must be scoped and served progressively rather than dumped into the model all at once.
• Humans must retain judgment over merges, deployments, approvals, and ambiguity resolution.

Architecture

LinkedIn built a shared AI-agent platform with an intent-to-execution pipeline: developers submit structured specs through chat or APIs; an orchestrator provisions sandboxed execution environments with identity and permissions; agents use approved MCP-style tools for code search, dependency analysis, logs, metrics, PR history, ownership, and system architecture; context services scope relevant code, dependencies, tests, ownership, infrastructure, operations, and prior PRs; memory systems pull task-relevant lessons and store validated outcomes; validation runs builds, tests, static analysis, evals, and safety checks; and outputs become reviewable artifacts such as PRs with full traces for human review.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  Human_intent_interface["Human intent interface"] --> Structured_spec["Structured spec"]
  Structured_spec --> Orchestrator["Orchestrator"]
  Orchestrator --> Sandbox_execution_environment["Sandbox execution environment"]
  Orchestrator --> Agent_worker["Agent worker"]
  Orchestrator --> Audit_trace["Audit trace"]
  Sandbox_execution_environment --> Agent_worker
  Sandbox_execution_environment --> Validation_layer["Validation layer"]
  Agent_worker --> Approved_tool_layer["Approved tool layer"]
  Agent_worker --> Context_service["Context service"]
  Agent_worker --> Memory_system["Memory system"]
  Agent_worker --> Validation_layer
  Approved_tool_layer --> MCP_servers["MCP servers"]
  Approved_tool_layer --> Audit_trace
  Context_service --> Memory_system
  Validation_layer --> Reviewable_artifact["Reviewable artifact"]
  Reviewable_artifact --> Human_review_gate["Human review gate"]
  Reviewable_artifact --> Audit_trace
  Human_review_gate --> Memory_system

Components

Data flow

Human intent is converted into a structured spec with steps, tool constraints, acceptance criteria, and guardrails. The orchestrator runs the task asynchronously, provisions a sandbox, persists state, and gives the agent scoped tools and task-specific context. The agent plans, edits, queries tools, validates changes, retries failures through the orchestrator, and emits responses or pull requests. Every plan, tool call, output, reasoning step, and code change is logged so humans can review and approve the result.

Why this architecture

LinkedIn chose platform-level foundations because agent workflows repeat the same needs across coding, operations, incidents, testing, and analytics: reliable orchestration, safe tools, scoped context, memory, evaluations, auditability, and human authority. Centralizing those abstractions prevents 100 fragile implementations and lets teams focus on their domain-specific agents.

Models used: LLMs · commercial cloud-hosted models · custom models · LLM-as-a-Judge · GitHub Copilot · Anthropic Model Context Protocol

Scale

• LinkedIn has 1.3 billion members.
  “We have 1.3 billion members.”article, stated
• LinkedIn makes 17,000 connections every minute.
  “We make 17,000 connections every minute on LinkedIn.”article, stated
• LinkedIn serves roughly 2 million feed updates every minute.
  “We have roughly close to 2 million feed updates being served every minute.”article, stated
• LinkedIn serves apps and site experiences in 36 languages.
  “We serve the site in 36 languages.”article, stated
• LinkedIn has about 7,000 deployables.
  “We have about 7,000 deployables.”article, stated
• LinkedIn serves about 3.2 million peak QPS.
  “We serve about 3.2 million peak QPS.”article, stated
• LinkedIn exchanges 45 trillion Kafka messages every day.
  “We exchange 45 trillion Kafka messages every day.”article, stated
• LinkedIn has more than 10,000 repositories.
  “We have 10,000-plus repositories.”article, stated
• LinkedIn submits more than one million PRs each year.
  “We submit over a million PRs every year.”article, stated
• The same orchestrated workflow is designed to scale from one repository to 100 repositories.
  “The same workflow runs whether it is one repo or 100.”article, stated

Alternatives considered (roads not taken)

Continue relying on scripts, self-serve UIs, and searchable documentation

Rejected because: Those systems lack cognition, leaving humans to perform repetitive reasoning and hands-on work whenever cognition loops appear.

Let every team build its own agent stack

Rejected because: Duplicating orchestration, tools, context, and safety across teams would create expensive repetition and inconsistent outcomes.

Expose raw APIs and human-oriented tools directly to agents

Rejected because: Raw APIs and dashboards create ambiguity; agents need stable schemas, clear permissions, error codes, and rich context.

Give agents all available context at once

Rejected because: The information universe is too large and noisy, so context must be scoped by task and execution step.

Train a custom model first

Rejected because: LinkedIn recommends exhausting commercial models, RAG, cache-augmented generation, and post-training options before taking on the operational overhead of pretraining.

Patterns

tool-use-orchestration (agent-architecture)

LinkedIn routes all agent work through explicitly exposed, permissioned, versioned tools with stable schemas rather than direct API access.

durable-execution (orchestration)

The orchestrator persists state, provisions repeatable sandboxes, handles retries/backoffs, and lets jobs pause or resume without losing progress.

async-message-orchestration (orchestration)

LinkedIn treats online, nearline, and batch invocations differently, with asynchronous execution and targeted human notifications for non-interactive work.

context-engineering (context)

The platform scopes, fetches, and serves code, dependencies, PRs, ownership, tests, infra, operations, and architecture context step by step so agents reason from reality.

agent-memory (memory)

Agents retrieve task-relevant memories before execution and store validated lessons, reviewer feedback, failure patterns, conventions, and best practices after execution.

eval-driven-development (evaluation)

LinkedIn treats evals as the trust mechanism for agents, using golden tasks, objective signals, subjective judges, regression detection, and traceability.

human-in-the-loop-gate (control)

Humans review artifacts, approve or reject merges and deployments, resolve ambiguity, and teach the system, while agents handle continuous execution.

safety-policy-enforcement (safety)

Sandboxes, explicit permissions, read/write boundaries, privilege escalation, out-of-scope guardrails, and forbidden irreversible actions limit what agents can do.

observability-tracing (operations)

LinkedIn logs every plan, tool call, output, reasoning step, code change, and execution trace so reviewers and platform operators can inspect agent behavior.

multi-agent-orchestration (agent-architecture)

Incident investigation is decomposed across an incident agent, insights agent, coding agent, evaluator agent, and optimizer agent that collaborate behind one user-facing facade.

Tradeoffs

Centralize agent infrastructure as a platform responsibility.

Chose shared platform abstractions over team-by-team agent stacks.

Rationale: Shared orchestration, tooling, context, safety, and evals reduce repetition and inconsistent outcomes while letting teams focus on domain problems.

Use structured specs instead of free-form goals.

Chose ordered steps, tool constraints, acceptance criteria, and guardrails over ambiguous prompts like do the usual thing.

Rationale: Agents lack human intuition, so explicit structure keeps execution deterministic and reviewable.

Adopt MCP and open standards for tool access.

Chose model-independent MCP tools over coupling tools to proprietary function-calling formats.

Rationale: Model velocity is high, so LinkedIn wanted tools that different models and agents can use without vendor lock-in.

Scope context progressively.

Chose task- and step-specific context retrieval over providing every available fact at once.

Rationale: Progressive scoping reduces irrelevant information and gives the agent the facts needed for the current step.

Give agents autonomy but not final authority.

Chose sandboxed execution with human approval gates over agents merging, deploying, or taking irreversible actions directly.

Rationale: Agents can do meaningful work faster, while humans retain accountability for decisions and authority.

Reuse and extend existing systems where possible.

Chose open-source software, open protocols, existing storage/RPC/queueing systems, and Copilot extensions over building everything from scratch.

Rationale: The AI tooling space moves quickly, so LinkedIn favors buying, extending, and adopting standards to keep pace.

Gotchas (what bit them)

Fragmented agent implementations create duplicated plumbing and inconsistent behavior.

Root cause: Without shared foundations, every team solves orchestration, tools, context, and safety separately.

Resolution: Make AI enablement a platform responsibility with shared orchestration, context engineering, tool calling, evals, and safety abstractions.

Ambiguous human intent leads agents to hallucinate, expand scope, or make unsafe changes.

Root cause: Humans rely on assumptions, shortcuts, and intuition that agents do not share.

Resolution: Convert messy goals into structured specs with ordered steps, explicit tool constraints, acceptance criteria, and guardrails.

Raw or noisy tool responses cause agents to guess.

Root cause: Human-facing tools and dashboards lack stable machine schemas, error codes, and concise context.

Resolution: Expose machine-oriented tools with structured inputs/outputs, versioning, permissions, error codes, and rich context.

Better models do not fix stale or missing facts.

Root cause: Many failures come from incomplete context rather than reasoning limitations.

Resolution: Invest in code, dependency, tests, ownership, infrastructure, operations, PR, and architecture context services.

Dumping all context into the model overloads the agent.

Root cause: The information universe is large and noisy, especially in a multi-repo organization.

Resolution: Scope context to the repos and services involved, then refine it by execution step.

Agent autonomy without authority boundaries creates safety risk.

Root cause: Agents need enough freedom to do meaningful engineering work, but deployments, main-branch merges, irreversible calls, and unrestricted internet access are too risky.

Resolution: Run agents in sandboxes with explicit pause points and reserve authority decisions for humans.

Evaluation

LinkedIn treats evals as tests for agent trust: curated golden datasets, objective signals where possible, LLM-as-a-Judge for subjective areas, regression detection, visualization tooling, audit traces, reasoning chains, model interactions, environmental awareness, and human-defined policy and scoring.

Metrics: Build pass/fail · Test pass/fail · Static analysis results · Safety checks · Golden task performance · Regression detection · Objective task signals · Subjective LLM-as-a-Judge scores · Tool-call traces · Reasoning and state traces · Environmental awareness for trigger-based agents

Outcomes

• LinkedIn built a shared platform for agent orchestration, tooling, context, memory, human review, and evaluation across engineering use cases.
• The platform supports development, operations, information retrieval, analytics, troubleshooting, incident investigation, UI QA, and background coding agents.
• The background coding agent executes specs asynchronously in isolated sandboxes and returns PRs or series of PRs with audit trails.
• Custom MCP servers augment GitHub Copilot with LinkedIn-specific context so generated code follows LinkedIn conventions.
• The UI QA agent replaces manual and expensive integration testing for server-driven UI regression checks across iOS, Android, and web.
• The analytics agent provides multimodal answers, charts, and visualizations for engineering, product, and business users.

The case is explicitly platform-scale: 1.3 billion members, 3.2 million peak QPS, 45 trillion Kafka messages daily, more than 10,000 repositories, and over one million PRs per year.