Building Production-Scale AI Agents with Extended GenAI Tech Stack

linkedin · Tech · 2025 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Production-scale multi-agent platform infrastructure

LinkedIn needed to extend its GenAI application stack from simple applications to autonomous and semi-autonomous agents that can think, plan, act, collaborate with users, run across web/mobile sessions, and remain observable, secure, and interoperable at LinkedIn scale.

Constraints:

Architecture

LinkedIn extended its existing GenAI stack instead of replacing it. Agents are defined as annotated gRPC services and registered into a skill registry; multi-agent coordination is built on the existing messaging platform with FIFO delivery, history lookup, retries, multiple message threads, and end-user synchronization; adapter libraries and a central agent lifecycle service translate between messaging and RPC endpoints; client libraries support push notifications, cross-device state synchronization, streaming, and fallbacks; pre-production observability uses LangSmith while production emits privacy-safe OpenTelemetry spans; traces feed offline evals and prompt tuning; experiential memory, context engineering, playground tooling, LangGraph providers, MCP, and A2A round out the platform.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  member-or-customer[Member or customer] --> web-and-mobile-clients[Web and mobile clients]
  web-and-mobile-clients --> client-integration-library[Client integration library]
  web-and-mobile-clients --> frontend-api-server[Frontend API server]
  frontend-api-server --> agent-lifecycle-service[Agent lifecycle service]
  agent-facade[Agent facade] --> agent-lifecycle-service
  agent-facade --> specialized-agents[Specialized agents]
  grpc-agent-definition[gRPC agent definition] --> build-plugin[Build plugin]
  build-plugin --> skill-registry[Skill registry]
  skill-registry --> agent-lifecycle-service
  grpc-agent-definition --> agent-rpc-endpoints[Agent RPC endpoints]
  agent-lifecycle-service --> messaging-platform[Messaging platform]
  messaging-platform --> message-history[Message history]
  agent-lifecycle-service --> agent-rpc-endpoints
  agent-rpc-endpoints --> specialized-agents
  specialized-agents --> experiential-memory[Experiential memory]
  offline-memory-jobs[Offline memory jobs] --> experiential-memory
  experiential-memory --> privacy-policy-interfaces[Privacy policy interfaces]
  specialized-agents --> mcp-tools[MCP tools]
  specialized-agents --> a2a-protocol[A2A protocol]
  langgraph-providers[LangGraph providers] --> messaging-platform
  langgraph-providers --> experiential-memory
  langgraph-providers --> specialized-agents
  playground[Playground] --> skill-registry
  playground --> specialized-agents
  playground --> langsmith[LangSmith]
  langsmith --> evaluation-platform[Evaluation platform]
  opentelemetry[OpenTelemetry] --> evaluation-platform
Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component Role Interacts with
Member or customer LinkedIn user interacting with an agent through web or mobile surfaces. Web and mobile clients
Web and mobile clients Client applications that present agent interactions, streaming updates, task completion notifications, and synchronized state. Member or customer, Client integration library, Frontend API server
Client integration library Library for server-to-client push, cross-device state synchronization, incremental streaming, error handling, and fallbacks. Web and mobile clients, Frontend API server
Frontend API server Companion server library exposing endpoints for message communication, response retrieval, and authorization. Web and mobile clients, Agent lifecycle service
Agent facade User-facing agent experience that may delegate to multiple specialized agentic applications. Agent lifecycle service, Specialized agents
Specialized agents Task-specific agentic applications exposed through standard service contracts. Agent facade, Agent RPC endpoints, Experiential memory, MCP tools, A2A protocol
gRPC agent definition Annotated service schema definition that describes agent metadata for registration and invocation. Build plugin, Agent RPC endpoints
Build plugin Registration mechanism that sends annotated agent definitions into the skill registry. gRPC agent definition, Skill registry
Skill registry Central service tracking available agents, their metadata, and invocation methods. Build plugin, Agent lifecycle service, Playground
Messaging platform Existing LinkedIn messaging infrastructure reused as the multi-agent orchestrator and delivery substrate. Agent lifecycle service, Message history, Specialized agents
Message history Persistent message lookup and sequencing state used for long-lived tasks and multi-agent execution. Messaging platform, Agent lifecycle service
Agent lifecycle service Central adapter layer that creates, updates, and retrieves messages while invoking agent RPC endpoints. Frontend API server, Agent facade, Skill registry, Messaging platform, Agent RPC endpoints
Agent RPC endpoints Service endpoints exposed by agents and invoked through lifecycle-service adapters. gRPC agent definition, Agent lifecycle service, Specialized agents
Experiential memory Memory layer for facts, preferences, learned information, and long-term personalized context. Specialized agents, Offline memory jobs, Privacy policy interfaces
Offline memory jobs Big-data jobs that curate and refine long-term agent memories. Experiential memory
LangSmith Pre-production tracing and evaluation layer for LangGraph/LangChain agent components. Evaluation platform, Playground
OpenTelemetry Production observability foundation for structured privacy-safe spans. Evaluation platform
Evaluation platform Offline evaluation, model regression, and prompt-tuning layer powered by persisted execution traces. LangSmith, OpenTelemetry
Playground Developer tooling for experimenting with agents, skills, memory, identity, and traces before product integration. Skill registry, LangSmith, Specialized agents
LangGraph providers Custom providers that adapt LangGraph to LinkedIn messaging and memory infrastructure. Specialized agents, Messaging platform, Experiential memory
MCP tools Standardized tool-based interfaces for agents to explore and interact with external capabilities. Specialized agents
A2A protocol Open protocol for agent-to-agent teamwork and distributed collaboration. Specialized agents
Privacy policy interfaces Explicit, authenticated, policy-driven interfaces governing data flow across memory, client data, playground, and lifecycle components. Experiential memory

Data flow

Developers define agents as annotated gRPC services and register them through a build plugin into the skill registry. When a member or customer interacts with an agent, client libraries and the frontend API server communicate with the agent lifecycle service, which uses the skill registry to identify callable agent RPC endpoints and uses the messaging platform to create, update, retrieve, and sequence messages. Agents can exchange single responses, synchronous streams, or asynchronous messages, use experiential memory and tools, and emit traces to LangSmith in pre-production or OpenTelemetry in production. Persisted traces then feed offline evaluations, model regression tests, and prompt-tuning experiments.

5 supporting quotes
“Developers simply annotate this definition with some platform defined proto3 options”article, stated
“register it via a build plugin into the skill registry (a central service that tracks available agents, their metadata, and how to invoke them) as described in our [previous blog”article, stated
“allows agents to remember facts, preferences, and any other things they have learned. However, the core of all agent interactions happens through the messaging platform and thus leverages both technology and existing operations expertise. Figure 2: Diagram showing the relationship between”article, stated
“building libraries that abstracted everything behind async calls to a central agent lifecycle service that did the necessary messaging <-> rpc adaptations by appropriately invoking the messaging platform to create/update/retrieve messages and invoke the”article, stated
“Execution traces are persisted and aggregated into datasets that power offline evaluations, model regression tests, and prompt tuning experiments.”article, stated

Why this architecture

LinkedIn reused infrastructure because production agents need service contracts, distributed orchestration, client delivery, observability, and privacy controls that the company already operates at scale. Messaging fit long-running multi-agent workflows better than a custom orchestrator because it already provided FIFO delivery, history lookup, retries, regional scaling, and end-user synchronization. The split between LangSmith and OpenTelemetry reflects different needs: rich pre-production introspection versus lean, privacy-safe production monitoring.

5 supporting quotes
“Rather than solve this from scratch, we looked to our existing production systems for inspiration and realized that our messaging system best emulated the characteristics we wanted in a multi-agent orchestrator. Long lived tasks could be broken down into a”article, stated
“could be broken down into a sequence of messages with guaranteed first in first out (FIFO) delivery and seamless message history lookup. Parallelization at scale could be handled via multiple message threads. We could also piggy back on existing resilience constructs built into the messaging system”article, stated
“persistent retries and eventual delivery in the face of issues like host outages or cross-region traffic shifts.”article, stated
“This high-fidelity view allows engineers to quickly identify failure modes, adjust agent behaviors, and iterate with confidence.”article, stated
“instrument key agent lifecycle events—such as LLM calls, tool invocation, and memory usage—into structured, privacy-safe OTel spans. This allows us to correlate agent behavior with upstream requests, downstream calls, and platform performance at scale.”article, stated

Models used: Large language models · LangGraph · LangChain · Claude · OpenAI · Azure-hosted models

Scale

Alternatives considered (roads not taken)

Build a new multi-agent orchestrator from scratch

Rejected because: LinkedIn's existing messaging system already provided the desired distributed-systems properties.

“Rather than solve this from scratch, we looked to our existing production systems for inspiration and realized that our messaging system best emulated the characteristics we wanted in a multi-agent orchestrator. Long lived tasks could be broken down into a”article, stated
“our messaging system best emulated the characteristics we wanted in a multi-agent orchestrator. Long lived tasks could be broken down into a sequence of messages with guaranteed first in first out (FIFO) delivery and seamless message history lookup.”article, stated

Expose messaging details directly to application developers

Rejected because: Developers needed adapter libraries and a lifecycle service to hide messaging-to-RPC complexity.

“avoid exposing the messy details of messaging to the application developers. We did this by building libraries that abstracted everything behind async calls to a central agent lifecycle service that did the necessary messaging <-> rpc adaptations by”article, stated
“building libraries that abstracted everything behind async calls to a central agent lifecycle service that did the necessary messaging <-> rpc adaptations by appropriately invoking the messaging platform to create/update/retrieve messages and invoke the”article, stated

Use only one observability stack for all stages

Rejected because: Pre-production needs full introspection, while production needs lean privacy-safe telemetry integrated with distributed infrastructure.

3 supporting quotes
“we’ve adopted a hybrid observability strategy tailored to the two distinct stages of agent development: pre-production and production.”article, stated
“engineers to quickly identify failure modes, adjust agent behaviors, and iterate with confidence. At this stage, we capture and retain full execution context. In production, the constraints change. Agents operate within our distributed infrastructure, interacting with many downstream”article, stated
“While the traces are leaner than in pre-production, they are optimized for production debugging, reliability monitoring, and compliance.”article, stated

Keep a proprietary skill registry as the long-term interoperability mechanism

Rejected because: LinkedIn is incrementally adopting MCP and A2A to avoid fragmentation and support open agent ecosystems.

“We are incrementally adopting these open protocols, moving away from a proprietary skill registry, paving the way for more intelligent, interoperable, and context-aware agent ecosystems. There is no one correct path towards building successfully with agents.”article, stated
“their data via standardized interfaces. We are incrementally adopting these open protocols, moving away from a proprietary skill registry, paving the way for more intelligent, interoperable, and context-aware agent ecosystems. There is no one correct path towards”article, stated

Use only asynchronous queue-backed delivery

Rejected because: Interactive experiences sometimes need faster predictable delivery, so LinkedIn added a sync delivery mode with different consistency tradeoffs.

“async Messaging based delivery for agent invocation, we have now enabled a new sync delivery mode, which bypasses the async queue and directly invokes the agent with sideways message creation. This significantly speeds up delivery with predictable times for user facing interactive agentic”article, stated
“agent developers now have the option of strong consistency with async delivery vs eventual consistency with sync delivery.”article, stated

Patterns

multi-agent-orchestration (agent-architecture)

LinkedIn models complex agent products as facades over multiple task-specific agentic applications rather than one monolithic agent.

“the agent is not a single monolithic application, but rather a facade over multiple such agentic applications.”article, stated
“So by defining our agents, we can then lay the groundwork to orchestrate multiple agents - increasing their capability and effectiveness in tackling a task.”article, stated

async-message-orchestration (agent-architecture)

The core orchestration substrate is LinkedIn's messaging platform, which sequences long-lived tasks, supports retries, and can stream or split responses.

“Long lived tasks could be broken down into a sequence of messages with guaranteed first in first out (FIFO) delivery”article, stated
“split their responses across multiple asynchronous messages (asynchronous streaming)”article, stated

human-in-the-loop-gate (control-and-safety)

Agents seek clarification, feedback, or approval at key decision points so autonomy stays paired with user control.

“agents seek clarification, get feedback, or request approvals at key decision points”article, stated
“What this means is that agents seek clarification, get feedback, or request approvals at key decision points, balancing autonomy with control. By combining LLM reasoning with task memory, tools, and human collaboration, Gen AI agents mark a shift from one-off prompts to continuous,”article, stated

observability-tracing (operations)

LinkedIn uses LangSmith for rich pre-production traces and OpenTelemetry for structured production spans, then reuses traces for evaluation.

“We’ve integrated with LangSmith for tracing and evaluation.”article, stated
“We instrument key agent lifecycle events—such as LLM calls, tool invocation, and memory usage—into structured, privacy-safe OTel spans.”article, stated

agent-memory (context-engineering)

Experiential memory lets agents retain facts, preferences, and learned information across interactions, with offline jobs refining long-term memories.

“experiential memory allows agents to remember facts, preferences, and any other things they have learned.”article, stated
“leveraging data intensive big data offline jobs to curate and refine long term agent memories.”article, stated

eval-driven-development (evaluation)

Execution traces are turned into datasets for offline evaluation, regression tests, and prompt tuning.

“traces become the raw material for understanding agent behavior over time, detecting regressions, and improving agent quality systematically.”article, stated
“datasets that power offline evaluations, model regression tests, and prompt tuning experiments.”article, stated

privacy-preserving-processing (security-and-privacy)

Memory stores and component domains are siloed, sharing happens through explicit policy-driven interfaces, and cross-component calls require authentication and authorization.

“user data is handled with strict boundaries to support privacy, security, and control.”article, stated
“Any sharing between these domains is designed to happen through explicit, policy-driven interfaces”article, stated

tool-use-orchestration (agent-architecture)

LinkedIn is adopting MCP so agents can access tool-based interfaces through standard protocols rather than proprietary integrations.

“MCP empowers agents to explore and interact with the world through tool-based interfaces”article, stated
“MCP makes it easier for companies to surface and activate their data via standardized interfaces.”article, stated

Tradeoffs

Reuse messaging as the orchestrator

Chose Existing LinkedIn messaging platform plus lifecycle adapters over A new purpose-built multi-agent orchestration system.

Rationale: Messaging already offered ordering, history, retry, regional resilience, and user delivery/sync mechanisms.

“our messaging system best emulated the characteristics we wanted in a multi-agent orchestrator.”article, stated
“build upon the existing end-user delivery and synchronization mechanisms used by LinkedIn messaging.”article, stated

Keep developers on gRPC-style contracts

Chose Annotated gRPC service definitions registered into a skill registry over A brand-new agent definition mechanism unrelated to LinkedIn service patterns.

Rationale: Agents run online in production, so LinkedIn reused the same service-to-service communication mechanism already familiar in the company.

“Since agents run online in production, we decided to reuse the same mechanism we use for service to service communication at LinkedIn”article, stated
“the same mechanism we use for service to service communication at LinkedIn, aka gRPC, and came up with a standard gRPC service schema definition. Developers simply annotate this definition with some platform defined [proto3”article, stated

Split observability by lifecycle stage

Chose LangSmith for pre-production and OpenTelemetry for production over One uniform trace system across development and production.

Rationale: Development needs rich full-context traces, while production requires scalable, privacy-safe, compliance-ready telemetry.

3 supporting quotes
“In pre-production, our focus is on rich introspection and iteration.”article, stated
“In production, the constraints change. Agents operate within our distributed infrastructure, interacting with many downstream systems and handling real user data.”article, stated
“we rely on a hardened observability foundation built on OpenTelemetry (OTel)”article, stated

Offer both async and sync agent invocation

Chose Async queue-backed delivery plus a faster sync mode over Only one invocation consistency/latency model.

Rationale: Background tasks benefit from messaging durability, while interactive user-facing experiences may need faster predictable delivery.

3 supporting quotes
“we have now enabled a new sync delivery mode, which bypasses the async queue”article, stated
“This significantly speeds up delivery with predictable times for user facing interactive agentic experiences.”article, stated
“strong consistency with async delivery vs eventual consistency with sync delivery.”article, stated

Adopt open protocols incrementally

Chose MCP and A2A adoption over Staying with proprietary skill-registry-only integrations.

Rationale: Open protocols reduce fragmentation and help agents discover tools and collaborate across frameworks and distributed environments.

“MCP and A2A have become foundational protocols for enabling dynamic agent discovery and collaboration”article, stated
“We are incrementally adopting these open protocols, moving away from a proprietary skill registry, paving the way for more intelligent, interoperable, and context-aware agent ecosystems.”article, stated

Preserve human control in autonomous workflows

Chose Human-in-the-loop clarification, feedback, and approval over Fully autonomous execution without user checkpoints.

Rationale: Current LLM cognitive limits and trust/safety needs require explicit user control points.

“Given the cognitive limitations of current LLMs and to ensure that human operators have control”article, stated
“What this means is that agents seek clarification, get feedback, or request approvals at key decision points, balancing autonomy with control. By combining LLM reasoning with task memory, tools, and human collaboration, Gen AI agents mark a shift from one-off prompts to continuous,”article, stated

Gotchas (what bit them)

Monolithic agents do not fit complex LinkedIn workflows.

Root cause: Member and customer tasks can be long-running, multi-step, parallel, and heterogeneous.

Resolution: Model an agent product as a facade over multiple specialized agents and orchestrate them through platform services.

“Having a single monolithic agent application isn’t tenable for many of the complex tasks and workflows”article, stated
“Each agent handles a specific task, making the system easier to build, test, and extend.”article, stated

Multi-agent systems inherit distributed-systems failures plus stochastic behavior.

Root cause: Agents run across regions, threads, services, and non-deterministic LLM workloads.

Resolution: Reuse messaging infrastructure with FIFO delivery, history lookup, retries, eventual delivery, and regional operating expertise.

“all the classic distributed systems problems of consistency, availability and partitioning return with an additional twist”article, stated
“a highly non-deterministic workload given the nature of GenAI agents.”article, stated

Messaging-based orchestration can leak complexity to app developers.

Root cause: Agents expose gRPC endpoints, but orchestration state lives in messages and histories.

Resolution: Use adapter libraries and a lifecycle service to hide messaging-to-RPC translation.

“The only missing piece was building an adapter between messaging and the gRPC contract”article, stated
“avoid exposing the messy details of messaging to the application developers.”article, stated

Agent UX can break when tasks outlive a session or move across devices.

Root cause: Long-running asynchronous tasks require completion notifications, response retrieval, authorization, and synchronized state.

Resolution: Build client and frontend-server libraries for push, state sync, incremental streaming, error handling, response retrieval, and authorization.

3 supporting quotes
“long running tasks that can span beyond a single user session with an application”article, stated
“Server-to-client push to enable agents to notify users when agents are done with long running tasks.”article, stated
“Cross-device state synchronization to maintain consistent application state across multiple devices.”article, stated

Traditional observability is insufficient for GenAI agents.

Root cause: Agent behavior combines distributed systems with dynamic stochastic LLM decisions.

Resolution: Use richer tracing during development and privacy-safe structured spans in production, then feed both into evaluation.

3 supporting quotes
“For GenAI agents, it's significantly harder. Unlike conventional software that follows deterministic logic, agents combine dynamic, stochastic behaviors with distributed systems patterns.”article, stated
“agents combine dynamic, stochastic behaviors with distributed systems patterns.”article, stated
“This layered approach—developer-friendly traces in pre-production, scalable structured observability in production”article, stated

Agent memory can become a privacy and data-boundary risk.

Root cause: Experiential, conversation, client, playground, and lifecycle data stores can expose member information if flows are not compartmentalized.

Resolution: Silo stores by design and require explicit policy interfaces, strong authentication, authorization, access logging, and auditability.

3 supporting quotes
“Experiential Memory, Conversation Memory, and other data stores are siloed by design”article, stated
“strong authentication and authorization checks enforced for every cross-component call”article, stated
“These safeguards ensure that permitted agents or services can access specific data, and all access is logged and auditable, keeping member information compartmentalized and secure.”article, stated

Evaluation

LinkedIn evaluates agents through trace-driven workflows. Pre-production traces in LangSmith support developer iteration and evaluation; production OpenTelemetry spans capture privacy-safe lifecycle events; and persisted traces are aggregated into datasets for offline evaluations, model regression tests, prompt tuning, and systematic improvement.

Metrics: LLM call traces · Tool invocation traces · Control-flow traces · Memory usage spans · Upstream request correlation · Downstream call correlation · Platform performance · Offline evaluation datasets · Model regression tests · Prompt-tuning experiments

4 supporting quotes
“developers can capture detailed execution traces, including LLM calls, tool usage, and control flow across chains and agents.”article, stated
“We instrument key agent lifecycle events—such as LLM calls, tool invocation, and memory usage”article, stated
“correlate agent behavior with upstream requests, downstream calls, and platform performance at scale.”article, stated
“datasets that power offline evaluations, model regression tests, and prompt tuning experiments.”article, stated

Outcomes

The article emphasizes global product availability, multi-region horizontal scaling, multiple message threads for parallelism, cross-device/session support, more than a hundred available agentic frameworks, and big-data offline jobs for long-term memories.

5 supporting quotes
“we’re excited to make it globally available in English to customers by the end of September.”article, stated
“This post will cover extensions to the GenAI application platform that have enabled us to build AI agents that think, plan, and act in collaboration with users, marking the next phase in LinkedIn’s GenAI evolution.”article, stated
“It allows agents to respond in a single chunk, incrementally across a single synchronous response”article, stated
“scalable structured observability in production, and unified trace-based evaluation across both—allows us to move fast without sacrificing visibility or reliability. ## Balancing experimentation and integration with developer tooling While there is intense focus on bringing agents to product”article, stated
“paving the way for more intelligent, interoperable, and context-aware agent ecosystems.”article, stated