Domain-Specific Small Language Models for Call Center Intelligence

deepgram · Telecommunications · 2023 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Call-center speech intelligence

Deepgram wanted to build call-center language AI that was accurate, reliable, and cost-effective without relying on very large generic foundation models that are slow, expensive, and poorly adapted to messy call transcripts.

Constraints:

Call-center transcripts include overlapping speakers, diarization needs, disfluencies, filler words, and business-specific terminology.
Production speech products must combine ASR, understanding, and possibly interaction layers.
Large foundation models can be too slow and expensive for high-volume call-center workloads.
Domain adaptation must preserve low latency while improving transcript understanding and summarization.
The system needs independently optimized components for audio perception, transcript understanding, and text-to-speech interaction.

Architecture

Deepgram describes a multi-model speech-intelligence pipeline. Audio first passes through ASR and diarization to produce speaker-separated transcripts. An understanding layer applies a domain-adapted small language model to summarize, classify topics, or detect sentiment from call-center transcripts. Where an application needs interaction, the generated response can then move through text-to-speech. The proof point uses a 500M-parameter model transfer-learned on call-center transcripts, then used for summarization in an API flow.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  call-audio-input[Call audio input] --> asr-and-diarization[ASR and diarization]
  asr-and-diarization --> domain-adapted-slm[Domain-adapted SLM]
  domain-adapted-slm --> understanding-outputs[Understanding outputs]
  understanding-outputs --> interaction-layer[Interaction layer]

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Call audio input`	Incoming phone-call audio from call-center conversations.	`ASR and diarization`
`ASR and diarization`	Speech recognition plus speaker-turn separation for transcripts.	`Domain-adapted SLM`
`Domain-adapted SLM`	Small language model adapted on call-center transcripts for understanding tasks.	`Understanding outputs`
`Understanding outputs`	Summaries, topic detection, sentiment, or other distilled outputs from transcripts.	`Interaction layer`
`Interaction layer`	Optional response-generation and text-to-speech path for voice-bot or agent-assist interactions.	`Understanding outputs`

Data flow

A call audio file enters Deepgram's API, ASR converts it to text and diarization separates speakers. The transcript is then passed to a call-center-adapted small language model for summarization or other understanding outputs. If the application is interactive, an interaction layer can turn generated responses back into audio with text-to-speech.

“there's going to be a perception layer which is going to be fundamentally ASR something that takes audio and turns it into text”transcript, stated

“transcribes an audio a phone call Audio diarizes the transcript and then sends the transcript to it to our slm to perform summarization”transcript, stated

Why this architecture

Deepgram argues that near-term language AI products will combine existing technologies in a multi-model pipeline, with each layer optimized for its job. The call-center understanding layer uses a small domain-adapted model because the business requirement is the combination of cost effectiveness, reliability, and accuracy rather than general foundation-model breadth.

“the most impactful products that people are going to build are going to combine existing Technologies in a multi-model pipeline”transcript, stated

“businesses are going to derive maximum benefit from language AI products that are cost effective reliable and accurate those three things together you need all three”transcript, stated

Models used: ASR model · diarization model · 500M-parameter small language model · text-to-speech model

Scale

Deepgram had processed over one trillion minutes of audio.
“so far in deep Graham's existence we've processed over one trillion uh minutes of uh of audio”transcript, stated
The demonstrated small language model had 500 million parameters.
“we took a small language model 500 million parameters and that was trained on general internet data”transcript, stated
The demo summarization call took 8.4 seconds.
“you see that that call took 8.4 seconds we're going to print the transcript that we got back from ASR and diarized”transcript, stated
Deepgram was an $85 million Series B company at the time of the talk.
“where a series B company 85 million uh total dollars raised so far so far”transcript, stated

Alternatives considered (roads not taken)

Use large foundation models for call-center understanding.

Rejected because: Deepgram argues call centers require small, fast, domain-specific models to achieve cost, reliability, and accuracy together.

“this is going to require the use of uh small fast domain specific language models as opposed to large foundational language models”transcript, stated

Use general internet-trained small models without adaptation.

Rejected because: Deepgram transfer-learned the 500M model on in-domain call-center transcripts and saw major improvements.

“trained on general internet data and then we transfer learned it on uh an in-domain data set of call center transcripts”transcript, stated

Build one monolithic speech AI model.

Rejected because: The proposed architecture composes ASR, diarization, understanding, and interaction layers so each existing technology can be optimized separately.

“this pipeline is going to have three stages in it there's going to be a perception layer”transcript, stated

Patterns

`fine-tuning-adaptation` (model-operations)

Deepgram transfer-learned a 500M-parameter model on call-center transcripts for domain-specific summarization and understanding.

“we transfer learned it on uh an in-domain data set of call center transcripts we found that the model improved dramatically in all metrics”transcript, stated

`latency-optimization` (operations)

The architecture favors small, fast models and a demonstrated 8.4-second API call over large foundation-model paths for call-center workloads.

“you see that that call took 8.4 seconds we're going to print the transcript that we got back from ASR and diarized”transcript, stated

`model-routing` (model-operations)

Deepgram frames speech intelligence as a pipeline where ASR/diarization, language understanding, and text-to-speech are distinct model stages.

“there's going to be an understanding layer that will take the output of that transcript and apply probably a large language model or perhaps a medium-sized language model or a small language model”transcript, stated

Tradeoffs

Use small domain-specific language models.

Chose A 500M model transfer-learned on call-center transcripts. over Large foundational language models..

Rationale: Small domain-specific models were argued to better satisfy cost, reliability, accuracy, and latency constraints for call centers.

“the purpose of this talk is to argue that this is going to require the use of uh small fast domain specific language models as opposed to large foundational language models”transcript, stated

Compose speech AI from existing technologies.

Chose A multi-model pipeline with perception, understanding, and interaction layers. over A single end-to-end monolithic model..

Rationale: Near-term impactful products can combine existing ASR, language, and speech technologies while optimizing each layer.

“many businesses will start to derive tremendous value from language AI products and in the short term the most impactful products that people are going to build are going to combine existing Technologies”transcript, stated

Gotchas (what bit them)

Real call transcripts are messy enough to hurt generic understanding.

Root cause: The transcript includes overlapping speech, disfluencies, and filler words.

Resolution: Deepgram argued for domain-adapted language models trained on call-center transcripts.

“real transcripts have things like disfluencies where people are basically stuttering and stumbling while they're speaking using filler words”transcript, stated

Businesses need all three of cost, reliability, and accuracy.

Root cause: A model path optimized for only one attribute can fail production requirements.

Resolution: Deepgram proposed small fast domain-specific models inside a multi-model pipeline.

“businesses are going to derive maximum benefit from language AI products that are cost effective reliable and accurate those three things together you need all three”transcript, stated

Call-center domain adaptation needs realistic conversation behavior, not just benchmark text.

Root cause: The adapted model had to continue prompted call-center conversations realistically.

Resolution: Deepgram evaluated next-token loss, perplexity, continuation quality, and summarization output.

“we find that we can continue a call center conversation a prompted call center conversation and it generates very realistic text”transcript, stated

Evaluation

Deepgram evaluated the adapted SLM with next-token prediction loss, perplexity, realistic conversation continuation, and an API demo that transcribed, diarized, summarized, and returned a human-readable result.

Metrics: 500M-parameter model · Next-token prediction loss · Perplexity · 8.4-second demo call · Over one trillion minutes of audio processed historically

“we found that the model improved dramatically in all metrics uh next token prediction loss and perplexity”transcript, stated

“we see that it's highly accurate and we're going to print the summary that we got back and we see a very nice human readable summary”transcript, stated

Outcomes

The transfer-learned SLM improved next-token prediction loss and perplexity on call-center data.
The demo returned a diarized transcript and accurate human-readable summary in 8.4 seconds.
The case established a concrete architecture for production call-center speech intelligence.

Deepgram cited over one trillion processed audio minutes and demonstrated a 500M-parameter model summarization flow taking 8.4 seconds.