AI-Powered Contact Center Copilot: From Research to Enterprise-Scale Production

cresta_/_openai · Tech · 2025 · source Agentic: no · promotional bias: med · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Enterprise contact-center agent assistance

Cresta needed to turn academic NLP and RL research into a production AI copilot that improved contact-center outcomes enough for enterprise buyers to trust and pay for it.

Constraints:

Contact-center systems are business-critical and cannot tolerate unreliable service
Customers require measurable ROI rather than chatbot-era hype
Each enterprise customer has different conversation data, products, and expert behaviors

Architecture

Cresta built a real-time contact-center copilot as a thin UI layer over existing agent desktops, backed by conversation-data pipelines, customer/vertical-specific model training, continuous evaluation, and ROI A/B tests. The stack evolved from LSTMs and transformers to foundation models fine-tuned or mid-trained on customer and domain conversation data, while enterprise deployments kept tenant/data isolation where required.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  agent-desktop-plugin["agent-desktop-plugin"]
  conversation-data-ingestion["conversation-data-ingestion"]
  labeling-and-benchmarking["labeling-and-benchmarking"]
  domain-training-repository["domain-training-repository"]
  mlops-training-pipeline["mlops-training-pipeline"]
  real-time-suggestion-models["real-time-suggestion-models"]
  usage-and-roi-telemetry["usage-and-roi-telemetry"]
  conversation-data-ingestion --> labeling-and-benchmarking
  conversation-data-ingestion --> domain-training-repository
  labeling-and-benchmarking --> mlops-training-pipeline
  domain-training-repository --> mlops-training-pipeline
  mlops-training-pipeline --> real-time-suggestion-models
  real-time-suggestion-models --> agent-desktop-plugin
  agent-desktop-plugin --> usage-and-roi-telemetry

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`agent-desktop-plugin`	Small UI surface, often a browser plugin, that inserts suggestions into the contact-center agent's existing workflow.	`real-time-suggestion-models`, `usage-and-roi-telemetry`
`conversation-data-ingestion`	Ingests millions of customer conversations and customer-specific interaction logs for labeling, training, and evaluation.	`labeling-and-benchmarking`, `domain-training-repository`
`labeling-and-benchmarking`	Defines customer-specific good outcomes, creates eval sets, and supplies labels from former contact-center agents and domain experts.	`conversation-data-ingestion`, `mlops-training-pipeline`
`domain-training-repository`	Aggregates opted-in vertical conversation data for telco, healthcare, finance, airlines, and other domains while preserving single-tenant deployments where needed.	`conversation-data-ingestion`, `mlops-training-pipeline`
`mlops-training-pipeline`	Continuously trains, fine-tunes, deploys, and evaluates models as architecture shifted from LSTMs to transformers and foundation models.	`labeling-and-benchmarking`, `domain-training-repository`, `real-time-suggestion-models`
`real-time-suggestion-models`	Generate response autocomplete, smart compose, and what-to-say-next suggestions during support or sales conversations.	`mlops-training-pipeline`, `agent-desktop-plugin`
`usage-and-roi-telemetry`	Measures suggestion usage, adoption, A/B-test outcomes, and downstream business metrics used for coaching and enterprise sales proof.	`agent-desktop-plugin`

Data flow

Customer conversations stream through the agent desktop and Cresta's Chrome-plugin-style copilot. Historical and live conversation data feed labeling, training, and evaluation pipelines; models generate suggested responses or next-best actions in real time; usage telemetry and A/B tests connect suggestion adoption to business metrics such as conversion, handle time, CSAT, and revenue.

“It listens into, you know, what's happening on the call and then is showing the agent what things to say and to steer them one way or the other to try to improve the customer experience”transcript, stated

“we were just building internal infrastructures we like these ML ops uh sort of pipelines where you can continuously train and deploy models and with continuous evaluation um kind of like devops but for ML models”transcript, stated

Why this architecture

Cresta used a thin copilot layer rather than replacing contact-center platforms because enterprise customers already lived in those systems, while the value came from model quality and measured outcomes. Customer/vertical training and continuous evals were necessary because each enterprise had different products, conversations, and success metrics.

“the user interface surface area is actually pretty small. Just like build with Chrome like a plugin to insert the UI and the suggestions, but a lot of efforts actually improving the quality of the suggestions”transcript, stated

“every customer is slightly different with different conversation with different type of like product they sell and it's like you have to retrain the model in some sense for for a new customer”transcript, stated

Models used: LSTMs · transformers · BERT-era models · GPT-2 fine-tuned on customer data · GPT-3/GPT-4-class foundation models

Scale

Cresta served Fortune 500 enterprise contact-center customers.
“It's called Crusta and we work with a lot of Fortune 500 companies. Um yeah, so happy to talk more.”transcript, stated
An Intuit pilot reportedly drove about $100 million in incremental revenue.
“I remember like we had a pilot that drove like 100 million mean like incremental revenue. Um and we were able to charge a fraction of that.”transcript, stated
About 50% of customers shared conversation data into the domain-specific repository.
“Um so over time we build up a repository like 50% of customers share conversation data hoping that we'll create a domain specific model for them”transcript, stated
Cresta ingested millions of customer conversations as the base layer for training data.
“Well the base layer of the data is the same. We ingest like millions of conversations from customer. Then the question is like what data additional data you need right?”transcript, stated
The company had roughly 300-400 employees and was nearly doubling revenue yearly at the time discussed.
“growing wise like almost doubling every year and we have about three 400 people”transcript, stated

Alternatives considered (roads not taken)

Full chatbot automation of contact-center conversations

Rejected because: The 2017 chatbot wave promised automation but failed in practice, so Cresta focused on augmenting human agents.

“shopping and you know talking to customers and everything will happens through messaging and chat and uh none of that stuff actually worked.”transcript, stated

One generic model applied unchanged to every enterprise customer

Rejected because: Each customer's conversation patterns and products differed enough that the model needed customer or domain adaptation.

“it's not like a language model today where we can apply it to anything it's like a narrow AI narrow domain AI have to train out each customer”transcript, stated

Pure per-customer single-tenant training forever

Rejected because: Cresta built shared vertical repositories when customers opted in, using incentives to improve domain-specific models across customers.

“you'll get some kind of incentive to do that. Um so over time we build up a repository like 50% of customers share conversation data hoping that we'll create a domain specific model for them”transcript, stated

Patterns

`human-in-the-loop-gate` (agent-assist)

Cresta augments human contact-center agents with suggestions rather than replacing them outright.

“we're sort of taking the contrarian view of like AI, we should be augmenting people instead of replacing people”transcript, stated

`fine-tuning-adaptation` (model-operations)

The system trains on customer conversation data and later uses fine-tuned or mid-trained foundation models for domain-specific behavior.

“there? Yeah. So the first thing we do is to ingest what uh the human um conversations like humans already do. So for these large call centers, they already have like either call recordings or chat transcripts”transcript, stated

`eval-driven-development` (evaluation)

Cresta starts customer engagements by defining what good looks like, building a benchmark, and aligning on the eval set before ML work.

“We go to customer we build a benchmark on what is like what does good look like and align with our customer on the eval set u and that's where we start the ML work”transcript, stated

`data-quality-loop` (model-operations)

Model improvements come heavily from data selection, formatting, labeling, and continuous retraining rather than only new model architectures.

“when we working on these models in production it's actually a lot of is actually data engineering it's figuring out what data you train on what mixture what kind of like format you should train it”transcript, stated

`observability-tracing` (operations)

Usage of suggestions acts as an internal proxy metric for whether the copilot can produce downstream business value.

“we call the usage of the suggestion we provide through copilot and it's a proxy metric on how much value you can uh provide”transcript, stated

Tradeoffs

How to price a costly AI feature.

Chose value-based pricing tied to measured lift over ordinary per-seat SaaS pricing.

Rationale: Inference was expensive and the value was tied directly to improved contact-center outcomes.

“because AI was so expensive we had to figure out that maybe we should try charge based on how much value we deliver and then then run into the issues like how do you actually measure the value”transcript, stated

How much UI to build around the AI.

Chose a small surface inserted into existing workflows over replacing the contact-center platform.

Rationale: The transcript says the UI was small and most effort went into suggestion quality.

Gotchas (what bit them)

AI product roadmaps are less deterministic than SaaS feature roadmaps.

Root cause: Improving model accuracy can matter more than shipping a visible feature, and experiments may or may not work.

Resolution: Cresta built MLOps pipelines and continuous evaluation to iterate on model/data experiments quickly.

“when you're building AI product it's actually very different because you're sort of figuring out whether you're building a new feature or maybe you should just like focus all your energy on improving the model”transcript, stated

A pilot that works for one enterprise does not automatically generalize.

Root cause: Early models were narrow-domain and had to be retrained for each customer's products, conversations, and best practices.

Resolution: Cresta started with enterprise customers and built customer/vertical data pipelines before broader scaling.

“it's like a narrow AI narrow domain AI have to train out each customer >> so every customer is slightly different with different conversation”transcript, stated

A good suggestion model still fails if agents ignore it.

Root cause: Veteran agents resisted the tool because they believed they already knew best practices.

Resolution: The team tracked suggestion usage and used customer-success coaching to drive adoption.

“we found that the people who are not using aren't actually veterans where they think they know actually how to do things and it's like a mindset shift”transcript, stated

Evaluation

Cresta uses customer-aligned benchmarks, continuous evaluation, A/B tests against non-AI agents, and usage/adoption proxy metrics.

Metrics: model accuracy · A/B test lift · suggestion usage · conversion/revenue outcomes · agent adoption

“So we run like AB tests of like the agents people using AI and the people not using AI and we show like a AB tests of their uh results”transcript, stated

Outcomes

Cresta demonstrated ROI through A/B tests and customer pilots.
One Intuit pilot is described as driving about $100 million in incremental revenue.
The company grew into Fortune 500 deployments across large contact centers.

The source is founder-interview evidence; some revenue and customer-scale statements are directional rather than audited metrics.

“I remember like we had a pilot that drove like 100 million mean like incremental revenue. Um and we were able to charge a fraction of that.”transcript, stated