Production Agent Platform Architecture for Multi-Agent Systems

linkedin · Tech · year n/a · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Enterprise production agent platform architecture

LinkedIn needed to scale production-grade agentic AI across many teams while supporting long-running recruiter workflows, multi-agent coordination, Python-first GenAI development, reusable skills, memory, and observability.

Constraints:

Architecture

LinkedIn built an agent platform around Python business logic, gRPC services, LangChain and LangGraph application modeling, supervisor/sub-agent coordination, messaging for long-running ambient flows, layered agent memory, a centralized skills registry, and custom observability. Hiring Assistant demonstrates the platform: a recruiter describes a role, the agent generates qualifications from the prompt and documents, works asynchronously, and later returns sourced candidates for review.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  recruiter-experience["Recruiter experience"] --> hiring-assistant-supervisor["Hiring Assistant supervisor"]
  hiring-assistant-supervisor["Hiring Assistant supervisor"] --> recruiter-experience["Recruiter experience"]
  hiring-assistant-supervisor["Hiring Assistant supervisor"] --> specialized-sub-agents["Specialized sub-agents"]
  specialized-sub-agents["Specialized sub-agents"] --> hiring-assistant-supervisor["Hiring Assistant supervisor"]
  hiring-assistant-supervisor["Hiring Assistant supervisor"] --> agent-messaging-service["Agent messaging service"]
  agent-messaging-service["Agent messaging service"] --> hiring-assistant-supervisor["Hiring Assistant supervisor"]
  hiring-assistant-supervisor["Hiring Assistant supervisor"] --> skills-registry["Skills registry"]
  skills-registry["Skills registry"] --> hiring-assistant-supervisor["Hiring Assistant supervisor"]
  hiring-assistant-supervisor["Hiring Assistant supervisor"] --> agent-memory["Agent memory"]
  agent-memory["Agent memory"] --> hiring-assistant-supervisor["Hiring Assistant supervisor"]
  python-grpc-application-framework["Python gRPC application framework"] --> hiring-assistant-supervisor["Hiring Assistant supervisor"]
  hiring-assistant-supervisor["Hiring Assistant supervisor"] --> python-grpc-application-framework["Python gRPC application framework"]
  python-grpc-application-framework["Python gRPC application framework"] --> internal-inferencing-stack["Internal inferencing stack"]
  internal-inferencing-stack["Internal inferencing stack"] --> python-grpc-application-framework["Python gRPC application framework"]
  agent-messaging-service["Agent messaging service"] --> specialized-sub-agents["Specialized sub-agents"]
  specialized-sub-agents["Specialized sub-agents"] --> agent-messaging-service["Agent messaging service"]
  specialized-sub-agents["Specialized sub-agents"] --> skills-registry["Skills registry"]
  skills-registry["Skills registry"] --> specialized-sub-agents["Specialized sub-agents"]
  specialized-sub-agents["Specialized sub-agents"] --> linkedin-services["LinkedIn services"]
  linkedin-services["LinkedIn services"] --> specialized-sub-agents["Specialized sub-agents"]
  agent-memory["Agent memory"] --> specialized-sub-agents["Specialized sub-agents"]
  specialized-sub-agents["Specialized sub-agents"] --> agent-memory["Agent memory"]
  skills-registry["Skills registry"] --> linkedin-services["LinkedIn services"]
  linkedin-services["LinkedIn services"] --> skills-registry["Skills registry"]
  custom-observability["Custom observability"] --> hiring-assistant-supervisor["Hiring Assistant supervisor"]
  hiring-assistant-supervisor["Hiring Assistant supervisor"] --> custom-observability["Custom observability"]
  custom-observability["Custom observability"] --> python-grpc-application-framework["Python gRPC application framework"]
  python-grpc-application-framework["Python gRPC application framework"] --> custom-observability["Custom observability"]
Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component Role Interacts with
Recruiter experience User-facing workflow where recruiters describe roles, attach documents, and review sourced candidates. Hiring Assistant supervisor
Hiring Assistant supervisor Supervisor agent coordinating sub-agents for role intake, qualification generation, sourcing, and candidate review workflow. Recruiter experience, Specialized sub-agents, Agent messaging service, Skills registry, Agent memory
Specialized sub-agents Agents that perform pieces of the hiring workflow and interact with LinkedIn services through skills. Hiring Assistant supervisor, Skills registry, LinkedIn services
Python gRPC application framework Production service framework using Python, gRPC, LangChain, and LangGraph for GenAI business logic. Hiring Assistant supervisor, Internal inferencing stack
Internal inferencing stack LinkedIn's model interface supporting Azure OpenAI and on-premises large language models. Python gRPC application framework
Agent messaging service Extension of LinkedIn messaging infrastructure for long-running user-agent and agent-agent communication with retry. Hiring Assistant supervisor, Specialized sub-agents
Agent memory Scoped and layered working, long-term, and collective memory used by agents across interactions. Hiring Assistant supervisor, Specialized sub-agents
Skills registry Central registry where services expose discoverable skills including RPC calls, database queries, prompts, and other agents. Hiring Assistant supervisor, Specialized sub-agents, LinkedIn services
LinkedIn services Existing internal systems accessed by sub-agents through skills. Specialized sub-agents, Skills registry
Custom observability Agent-specific observability layer for production debugging and reliability. Hiring Assistant supervisor, Python gRPC application framework

Data flow

A recruiter describes a job and attaches supporting documents. Hiring Assistant generates qualifications, informs the recruiter that it will work in the background, then a supervisor agent coordinates sub-agents that use LinkedIn services through skills. Long-running work is modeled as agent-to-agent and user-to-agent messages with queue-based retry. Agents read and write scoped memory, discover skills from the registry, invoke them synchronously or asynchronously, and produce outputs that are surfaced back to the recruiter.

3 supporting quotes
“the recruiter starts off the process by describing the kind of job that they'd like to fill.”transcript, stated
“the agent will automatically generate different qualifications based off of what the kind of recruiter user initially inputed”transcript, stated
“there's a supervisor agent that coordinates between different sub aents and each sub aent is allowed to interact with existing LinkedIn services systems”transcript, stated

Why this architecture

LinkedIn chose a Python-first framework because Java blocked rapid GenAI iteration with open-source tooling. It chose messaging, layered memory, and skills because production agents need asynchronous communication, ordered dependencies, and reusable access to internal services beyond local function calls.

3 supporting quotes
“this Java Python setup really wasn't working out.”transcript, stated
“we realized we had to use Python no matter what because at the end of the day, you know, we have to be on top of the latest industry trends”transcript, stated
“one agents can take a lot of time to process data. This leads into the whole ambient agent idea”transcript, stated

Models used: Azure OpenAI models · on-premises large language models · LangChain · LangGraph

Scale

Alternatives considered (roads not taken)

Continue building GenAI applications primarily in Java

Rejected because: Teams wanted Python for prompt engineering, evaluations, libraries, and fast-changing GenAI techniques.

“they wanted to experiment with Python. They want to use Python for prompt engineering for the evaluations but because of our stack they were forced to build in Java”transcript, stated
“if your stack is fundamentally non-Python. And as soon as there is that kind of language gap, then it becomes really difficult for teams honestly to innovate and iterate”transcript, stated

Let every team invent its own GenAI production patterns

Rejected because: LinkedIn wanted a default framework that removed guesswork and standardized production utilities.

“let's build a framework and make it the default um and specifically make a fra make a framework so that teams don't have to guess about what's the right way to do things.”transcript, stated

Model long-running agent flows only as synchronous calls

Rejected because: Ambient agents take time, can fail, and need retryable user-agent and agent-agent communication.

“model longrunning asynchronous flows. We modeled this as precisely a messaging problem.”transcript, stated
“if messages get failed um they'll be automatically picked up and retried uh uh via a queuing system.”transcript, stated

Treat skills as only local function calls

Rejected because: LinkedIn needed skills to cover RPC calls, database queries, prompts, and other agents, including asynchronous invocation.

“function calling is usually, you know, local to to your to your box. But here, skills can be really be anything.”transcript, stated
“It can be RPC calls, database queries, prompts, and more importantly, other agents.”transcript, stated

Patterns

multi-agent-orchestration (agent-architecture)

Hiring Assistant uses a supervisor agent to coordinate specialized sub-agents that interact with LinkedIn services.

“a pretty traditional supervisor multi- aent architecture where there's a supervisor agent that coordinates between different sub aents”transcript, stated
“each sub aent is allowed to interact with existing LinkedIn services systems”transcript, stated

async-message-orchestration (agent-architecture)

LinkedIn models ambient, long-running agent work as messaging, including user-to-agent and agent-to-agent messages with retry.

“model longrunning asynchronous flows. We modeled this as precisely a messaging problem.”transcript, stated
“agentto agent messaging, but also user to agent messaging as well.”transcript, stated

agent-memory (context-management)

The platform provides scoped/layered working, long-term, and collective memory for agents.

“our agentic memory, um, you'll probably see, uh, I think you've you've seen themes of this from from people outside with the booths as well.”transcript, stated
“there's working memory, long-term memory, collective memory, and these these each provide different functions that the agent can utilize”transcript, stated

tool-use-orchestration (agent-architecture)

LinkedIn extends tool calling into centralized skills that can represent RPC calls, database queries, prompts, and other agents.

“we've developed this notion of skills, which is is is like function calling in a lot of senses”transcript, stated
“skills can be really be anything. It can be RPC calls, database queries, prompts, and more importantly, other agents.”transcript, stated

model-routing (model-operations)

The chat model interface lets teams switch between Azure OpenAI and on-prem LLMs with a few lines of code.

“LinkedIn uses um, Azure OpenAI but also uses on-remise large language models as well.”transcript, stated
“teams if they want to switch between model providers they can do so with a couple lines of code”transcript, stated

observability-tracing (operations)

LinkedIn built custom observability because agentic execution requires observability beyond ordinary service metrics.

“we also build custom observability because agentic ways of of execution require particular observability solutions.”transcript, stated
“observability is is paramount. You can't fix what you can't observe.”transcript, stated

durable-execution (reliability)

The messaging layer includes retry behavior for failed messages through a queue, supporting long-running ambient flows.

“if messages get failed um they'll be automatically picked up and retried uh uh via a queuing system.”transcript, stated

Tradeoffs

Adopt Python as the GenAI production language.

Chose Python for business logic, engineering, evaluation, and production GenAI apps over Continuing Java-first GenAI production services.

Rationale: Python aligned LinkedIn teams with the fast-moving GenAI ecosystem and reduced friction between experimentation and production.

“we realized we had to use Python no matter what because at the end of the day, you know, we have to be on top of the latest industry trends”transcript, stated
“Let's use Python for the business logic, the engineering, the eval, and pretty much everything else”transcript, stated

Build a default framework instead of one-off team stacks.

Chose Standard Python/gRPC/LangChain/LangGraph framework over Each team deciding independently how to build production GenAI services.

Rationale: A default framework saved team time, removed guesswork, and standardized production utilities.

“make a framework so that teams don't have to guess about what's the right way to do things.”transcript, stated
“standard utilities for tool calling”transcript, stated

Use gRPC for the production framework.

Chose Python services with gRPC over A framework without built-in streaming, binary serialization, or cross-language support.

Rationale: Streaming, performance, and cross-language support mattered for LLM applications inside a polyglot LinkedIn environment.

“Namely, it's built-in streaming support was really awesome. Uh, binary serialization is a big performance boost.”transcript, stated
“just the native cross language features,”transcript, stated

Treat skills as synchronous and asynchronous enterprise capabilities.

Chose Central skills registry with sync/async invocation over Local synchronous function calls only.

Rationale: Ambient agents need to call services, prompts, databases, and other agents across team boundaries and over long-running flows.

“we let agents invokei skills synchronously but also asynchronously”transcript, stated
“The asynchronicity part is is absolutely critical.”transcript, stated

Gotchas (what bit them)

Java production defaults slowed GenAI iteration.

Root cause: The open-source GenAI ecosystem, prompt engineering, and evaluations were Python-centered while LinkedIn's production business logic was Java-centered.

Resolution: LinkedIn adopted Python for GenAI business logic, engineering, evaluations, and production application development.

“your stack is fundamentally non-Python. And as soon as there is that kind of language gap, then it becomes really difficult for teams honestly to innovate and iterate”transcript, stated
“Let's use Python for the business logic, the engineering, the eval, and pretty much everything else”transcript, stated

Long-running ambient agents cannot be modeled as simple request/response calls.

Root cause: Agents take time to process data and need to communicate with users and other agents asynchronously.

Resolution: LinkedIn extended its messaging infrastructure for user-agent and agent-agent communication, including queue-based retry.

“agents can take a lot of time to process data. This leads into the whole ambient agent idea”transcript, stated
“if messages get failed um they'll be automatically picked up and retried uh uh via a queuing system.”transcript, stated

Parallel agents need ordered coordination.

Root cause: Outputs from one agent may depend on outputs from another.

Resolution: LinkedIn built agent memory and messaging infrastructure to coordinate dependencies across agents.

“agents can execute in parallel and furthermore the outputs of one agent might depend on the outputs of another agent.”transcript, stated
“how can we make sure that things are done in the right order?”transcript, stated

Local-only function calling was too narrow for enterprise agents.

Root cause: Production agents needed access to existing services, database queries, prompts, and other agents across teams.

Resolution: LinkedIn centralized the skills concept in a registry so services can expose discoverable reusable skills.

“function calling is usually, you know, local to to your to your box. But here, skills can be really be anything.”transcript, stated
“team services could expose skills um and register them in a central place”transcript, stated

Evaluation

LinkedIn evaluates the platform by organizational adoption, service creation, time-to-build improvements, production readiness concerns such as availability/reliability/observability, and the ability to support Hiring Assistant's ambient multi-agent workflow.

Metrics: More than 20 teams using the framework · Over 30 GenAI services created · Non-trivial apps built in days instead of weeks · Production concerns: availability, reliability, observability, and nondeterministic workload handling

4 supporting quotes
“we over 20 teams use this framework and I think this honestly probably a lower estimate at this point.”transcript, stated
“over 30 services have been created to support Generi product experiences on LinkedIn.”transcript, stated
“we're able to build non-trivial apps in days down from weeks”transcript, stated
“we're still building production software. You should still consider the usual things, availability, val reliability”transcript, stated

Outcomes

The case reports more than 20 teams, over 30 GenAI services, a messaging system derived from infrastructure serving countless members daily, and a company-wide shift from Java-first to Python-first GenAI production development.

4 supporting quotes
“I'd like to start by showcasing LinkedIn's first ever production agent. Um, we built an agent called LinkedIn Hiring Assistant.”transcript, stated
“we over 20 teams use this framework and I think this honestly probably a lower estimate at this point.”transcript, stated
“over 30 services have been created to support Generi product experiences on LinkedIn.”transcript, stated
“we're able to build non-trivial apps in days down from weeks”transcript, stated