Multi-Agent AI Platform for Customer Experience at Scale

cisco · Tech · year n/a · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Enterprise customer-experience operations and lifecycle automation

Cisco Customer Experience needed agentic AI for renewals, adoption, support, delivery, sentiment, and lifecycle workflows at the scale of a 20,000-person organization and $26B+ recurring-revenue business, while preserving accuracy, deployment flexibility, regulatory constraints, and production operating discipline.

Constraints:

• Use cases must tie to immediate customer value, secure/reliable operations, or lifecycle visibility rather than generic AI novelty.
• The platform must support cloud, on-premises, and hybrid deployment requirements for regulated customers, including federal, healthcare, and European data-residency contexts.
• Risk recommendations and renewal workflows require predictive accuracy beyond what LLMs alone can provide.
• Production, limited-availability, experimentation, and evaluation work require different team metrics and governance.
• Structured enterprise data and SQL joins must be handled carefully because LLMs are unreliable for SQL-heavy operations.

Architecture

Cisco built an agentic Customer Experience platform using LangChain/LangGraph-style orchestration, supervisor agents, specialized renewals/adoption/delivery/sentiment/installed-base agents, LangSmith tracing, multiple LLMs, and traditional ML prediction models. The system runs across on-premises data centers, cloud, and hybrid environments without changing the agentic framework. LLM agents handle language, workflow coordination, context passing, and final reasoning, while deterministic ML models handle risk predictions. Cisco pairs the runtime architecture with use-case gating, limited-availability cohorts, independent evaluation teams, and separate experimentation/production teams.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  Channels["CX user channels"] --> Supervisor["Supervisor agent"]
  Supervisor --> Renewals["Renewals agent"]
  Supervisor --> Adoption["Adoption agent"]
  Supervisor --> Delivery["Delivery agent"]
  Supervisor --> Sentiment["Sentiment analysis agent"]
  Supervisor --> Installed["Installed base agent"]
  Renewals --> ML["Predictive ML models"]
  Sentiment --> ML
  Renewals --> Data["Enterprise data sets"]
  Adoption --> Data
  Delivery --> Data
  Installed --> Data
  Data --> Cortex["Snowflake Cortex metadata layer"]
  Supervisor --> Context["LangChain context layer"]
  Context --> Models["LLM provider layer"]
  Context --> Tracing["LangSmith tracing"]
  Tracing --> Eval["Independent evaluation team"]
  Cohorts["Limited availability cohorts"] --> Prod["Production team"]
  Cohorts --> Experiment["Experimentation team"]
  Prod --> Eval
  Experiment --> Eval

Components

Data flow

A CX user or customer asks a lifecycle question, such as a renewal-risk query. The supervisor receives the natural-language request, decomposes it, and routes the request to the relevant specialized agent. The renewals agent gathers customer/product/renewal context, calls adoption and delivery agents for current status, triggers sentiment analysis and installed-base checks as needed, sends risk questions to predictive ML models, carries context through LangChain, records traces in LangSmith, and returns a formatted answer. The same framework can run on-premises or in cloud deployments with provider/model choices configured behind the architecture.

Why this architecture

Cisco chose a supervisor-based multi-agent architecture because customer-lifecycle questions require customer identity, products, purchase dates, adoption status, sentiment, installed base, and risk signals that a monolithic agent could not handle at the target accuracy. The platform combines LLMs with deterministic ML because LLMs are useful for language and workflow reasoning but poor for prediction. It also abstracts deployment and model selection so the same agentic framework can run in regulated on-premises settings or cloud environments.

Models used: Mistral Large · Claude Sonnet 3.7 · GPT-4.1 · o3 · Custom predictive machine learning models · Fine-tuned LLMs for on-premises accuracy · LangChain and LangGraph-style agent framework · LangSmith tracing · Snowflake Cortex semantic context

Scale

• Cisco is described as a $56B+ company.
  “Cisco is a $56 billion plus company with half of the recurrent revenue of the company.”transcript, stated
• The customer-experience business carries more than $26B in recurring revenue.
  “we have more than $26 billion that runs on this. So talk with Lenchain like hey you're not joking here.”transcript, stated
• The CX organization is over 20,000 people.
  “I'm talking about 20,000 people organization north of this. So there's a lot of things that could be improved”transcript, stated
• Cisco filtered a customer's 412 proposed AI use cases down to five that actually contributed business value.
  “customer had 412 use cases for AI and when you talk with them ends up being five that actually collaborate to the business. So boy this is less than 10%.”transcript, stated
• The support automation target operates over roughly 1.6 to 1.8 million annual support cases, with 60% fully automated according to the summary and transcript context.
  “Processing 1.6-1.8 million support cases annually”summary, stated

  “60% of that is fully automated and beyond that integrating this directly on the product and sentiment analysis across the whole life cycle”transcript, stated
• The renewals limited-availability deployment reduced operational-task time by 20% in under three weeks.
  “the results for us was actually a reduction on 20% of the time used it in less than 3 weeks and the limited availability.”transcript, stated
• Risk recommendations targeted and achieved at least 95% accuracy.
  “we're going to go 95% accuracy and higher and we accomplished that.”transcript, stated

Alternatives considered (roads not taken)

Choose tools and techniques before defining use cases and metrics

Rejected because: Cisco's lesson is to define use-case criteria and business metrics first, then choose RAG, prompts, few-shot, supervisor, fine-tuning, or chain techniques only when they serve that use case.

Use an LLM alone for renewal-risk prediction

Rejected because: The speaker says LLMs are probabilistic and bad at prediction, so risk questions are routed to deterministic ML models.

Use a single agent for complex customer-lifecycle questions

Rejected because: A single agent did not meet the 95%+ accuracy target for questions requiring renewal, adoption, delivery, sentiment, installed-base, and risk signals.

Treat experimentation and production as the same team/process

Rejected because: Cisco separates production, limited availability, experimentation, and evaluation because each has different metrics, risk tolerance, and feedback loops.

Let the development team evaluate its own production system

Rejected because: Cisco uses a separate evaluation team with golden datasets so product teams cannot rationalize away missed performance, cost, or metric targets.

Have LLMs perform SQL joins directly

Rejected because: Cisco found LLM-SQL integration hard and advises normalizing data first while using Snowflake Cortex only for metadata reference.

Patterns

multi-agent-orchestration (architecture)

Cisco uses a supervisor to decompose lifecycle questions and coordinate renewals, adoption, delivery, sentiment, installed-base, and other agents.

tool-use-orchestration (workflow)

The agentic workflow uses LangChain/LangGraph-style context passing, tool/API/MCP entry points, and agents that call specialized workflows and predictive models.

eval-driven-development (quality)

Cisco keeps evaluation separate from development, uses golden datasets, and requires systems to meet performance, cost, and accuracy targets.

human-in-the-loop-gate (governance)

The renewal workflow still keeps humans in the loop and Cisco uses limited-availability cohorts/SMEs to validate real user questions before broader release.

observability-tracing (operations)

Cisco uses LangSmith traces to inspect supervisor-agent decomposition and agent interactions in production demos.

context-engineering (context)

The platform treats customer-specific context, agent context carrying, MCP-style context, semantic metadata, and future agent directories as explicit architecture concerns.

Tradeoffs

How to choose which AI work to fund.

Chose Use-case criteria tied to value, secure/reliable operations, and lifecycle insight. over Letting teams pursue every AI idea that appears interesting or technically novel..

Rationale: Cisco's experience showed that only a small fraction of proposed AI use cases contributed business value.

How to support regulated customers with different deployment constraints.

Chose A configurable model/deployment layer that can run the same agentic framework on-premises, in cloud, or hybrid. over A cloud-only architecture tied to one model provider..

Rationale: Federal, healthcare, European, B2B, and B2C customers impose different security, compliance, and residency requirements.

How to answer renewal-risk questions accurately.

Chose Supervisor-orchestrated agents plus deterministic ML prediction models. over A single LLM agent making end-to-end risk predictions..

Rationale: Lifecycle questions require many product, customer, status, sentiment, and risk signals, and Cisco wanted 95%+ accuracy.

When to involve end users.

Chose Talk to users and SMEs before building full AI features. over Building a cool system first and seeking customer feedback afterward..

Rationale: Cisco found that renewal, adoption, and other users ask different questions, so discovery must happen before implementation.

How to organize delivery teams.

Chose Separate production, limited availability, experimentation, and evaluation responsibilities. over One blended team owning every stage from prototype through evaluation..

Rationale: The source argues that each lane has different metrics and incentives, and evaluation needs independence.

Gotchas (what bit them)

Most proposed AI use cases did not create real business value.

Root cause: Teams pursued cool AI ideas before tying them to customer value, operations reliability, or lifecycle visibility.

Resolution: Cisco defined use-case criteria and metrics before selecting RAG, prompts, supervisor patterns, fine-tuning, chains, or other techniques.

LLMs alone were not suitable for enterprise risk prediction.

Root cause: The speaker describes LLMs as probabilistic and weak for prediction, while risk recommendations need deterministic, high-accuracy outputs.

Resolution: Cisco routes risk questions to predictive ML models and uses LLMs for language, workflow, and final reasoning around those predictions.

LLM-to-SQL integration failed when treated as direct SQL reasoning.

Root cause: The model was not reliable for joins and structured enterprise SQL operations.

Resolution: Cisco normalizes data first and uses Snowflake Cortex semantic context for metadata reference rather than asking LLMs to do joins.

Production teams and experimentation teams optimize for different outcomes.

Root cause: Production work values stability, cost, latency, and business metrics, while experimentation requires freedom to fail fast.

Resolution: Cisco runs production, limited availability, and experimentation in parallel, ideally with different teams.

Developer-owned evaluation can rationalize missed quality targets.

Root cause: The same team building a model may blend or justify its own results.

Resolution: Cisco separates the evaluation team and gives it golden datasets and authority over performance, cost, and metric checks.

Evaluation

Cisco evaluates the platform through business metrics, limited-availability user cohorts, LangSmith traces, 95%+ risk-recommendation accuracy targets, independent golden-dataset evaluation, and production metrics for stability, latency, cost, performance, and business impact.

Metrics: 20% reduction in operational-task time in less than three weeks · 95%+ risk-recommendation accuracy · 60% fully automated support cases · 1.6-1.8 million annual support cases · $26B+ recurring revenue context · 20,000+ people in the CX organization · 412 proposed AI use cases narrowed to five value-contributing cases

Outcomes

• Cisco deployed multiple CX agents in production at scale for renewals, support, adoption, sentiment, and lifecycle workflows.
• The renewals deployment reduced operational-task time by 20% in less than three weeks of limited availability.
• Risk recommendations reached the speaker's stated 95%+ accuracy target.
• Support automation handles about 60% of a large annual case volume without human touch.
• The same agentic framework can run on-premises and in the cloud without code changes.

The source places the platform inside a $56B+ company, $26B+ recurring-revenue CX business, 20,000+ person organization, 1.6-1.8M annual support-case workload, and deployment models spanning on-premises data centers, cloud, and hybrid environments.