Low-Latency Intelligence Extraction from Audio Streams in Contact Centers

fujitsu · Telecommunications · 2026 · source Agentic: no · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Contact-center audio intelligence and after-call work automation

Fujitsu needed to reduce contact-center after-call documentation time by turning messy multi-channel audio into structured CRM-ready intelligence without losing accuracy, security, or operator oversight.

Constraints:

• Contact-center audio is messy, overlapping, emotionally charged, and often multi-channel rather than clean text.
• Operators spent nearly as much time on after-call documentation as on customer conversations.
• Speech-to-text errors, speaker confusion, and hallucinated summaries can corrupt downstream customer records.
• Sensitive information such as credit card numbers, passwords, and PII must be masked before reaching LLM memory or cloud endpoints.

Architecture

Fujitsu built a four-stage low-latency pipeline: voice capture from telephony systems with noise filtering, normalization, channel mapping, and early PII masking; speech-to-text with domain dictionaries, inverse text normalization, and punctuation; an orchestrated generative-AI core for summarization, intent extraction, sentiment/entity recognition, token optimization, and hallucination checks; and a customer-data sync layer that maps structured JSON into CRM fields while keeping operators in a quick validation loop.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  Audio["Telephony audio intake"] --> Capture["Voice capture layer"]
  Capture --> Masking["PII masking layer"]
  Capture --> STT["Speech-to-text engine"]
  Masking --> STT
  STT --> GenAI["Generative AI core"]
  GenAI --> Trust["Trust layer"]
  Trust --> GenAI
  GenAI --> Gateway["API gateway"]
  Gateway --> CRM["CRM system"]
  Gateway --> Validation["Operator validation UI"]
  CRM --> BI["Business intelligence models"]

Components

Data flow

Raw audio enters from telephony systems, is denoised, normalized, split by speaker channel, and masked for sensitive data. The STT engine transcribes with domain language logic and numeric normalization. The LLM core converts time-indexed, confidence-scored transcript text into structured JSON with customer inquiries, operator actions, intents, entities, sentiment, and reasoning. An API gateway maps the JSON into CRM fields, where operators validate before confirmation and BI systems consume the standardized voice-of-customer data.

Why this architecture

Fujitsu used a staged pipeline because audio quality, speaker attribution, numeric formatting, and security all determine whether the LLM can safely create business records. The LLM is not left to infer everything from raw transcripts; upstream audio/STT processing improves input fidelity, prompt orchestration imposes structure, and the API/validation layer keeps humans accountable.

Models used: speech-to-text engine · LLMs · generative AI core · acoustic models · time series analytics

Scale

• Over 50% of contact centers identify hiring, training, and productivity as critical barriers.
  “over 50% of the contact centers, they identify uh, hiring, training, and productivity as their most critical barriers.”transcript, stated
• Baseline calls lasted about 6.5 minutes while after-call work took about 6.3 minutes.
  “the average contact center call it typically last about like 6.5 minutes. However, the average post-processing time where the operator types up the the disposition codes, it it takes like almost 6.3 minutes.”transcript, stated
• Fujitsu targeted about a 50% or better reduction in after-call work.
  “if we can mechanize the summarization and the data extraction, uh, theoretically, we can reduce the post-processing time, um, almost by 50% or even more.”transcript, stated
• The deployed workflow reduced after-call work from 6.3 minutes to 3.1 minutes.
  “under the manual operation, after average after-call work was 6.3 minutes. Uh powered by our AI workflow, that dropped to like 3.1 minutes”transcript, stated
• The business case was framed across 500 seats handling thousands of calls per day.
  “if you calculate that across 500 seats handling thousands of calls a day, you are looking at a massive operational saving”transcript, stated
• STT accuracy needed to exceed 90% for downstream LLM summarization to work effectively.
  “for generative AI or the LLMs to summarize the um data effectively or your response effectively, we found that the speech-to-text, the STT um accuracy must be above 90%.”transcript, stated

Alternatives considered (roads not taken)

Send mixed mono audio downstream

Rejected because: Overlapping speakers make it hard for AI to determine who said what, degrading the final summary.

Ask the LLM for a generic call summary

Rejected because: Basic summarization produced messy narrative paragraphs instead of parseable customer inquiry and operator action fields.

Fully automate CRM updates without human review

Rejected because: The system retained a verification step so operators could visually validate and edit AI-generated records before confirmation.

Let sensitive audio reach cloud LLM endpoints unmasked

Rejected because: PII masking before cloud endpoints was a strict compliance requirement despite adding latency and architectural overhead.

Patterns

structured-output-contract (integration)

The LLM produces bullet-point summaries and clean JSON schemas mapped to CRM templates instead of free-form narrative text.

human-in-the-loop-gate (governance)

Operators review auto-populated summaries, make minor edits, and confirm before CRM updates become authoritative.

safety-policy-enforcement (security)

Early PII masking and hallucination checks prevent sensitive data exposure and ungrounded CRM summaries.

latency-optimization (performance)

The pipeline applies token optimization and low-latency processing so call intelligence can support near-real-time workflow efficiency.

tool-use-orchestration (integration)

The LLM output is orchestrated into REST API calls that update CRM/customer-data fields and feed BI dashboards.

Tradeoffs

Preserve speaker channels before transcription

Chose stereo channel mapping for agent and customer over single mixed mono transcript.

Rationale: Speaker separation is necessary for accurate dialogue stitching and summary attribution.

Use structured few-shot prompting

Chose separate bullet lists and CRM-shaped JSON over plain LLM summarization.

Rationale: Structured prompting made outputs parseable and business-system-ready instead of messy narrative paragraphs.

Keep operator validation in the workflow

Chose auto-populated summaries with quick visual confirmation over unreviewed automatic CRM writes.

Rationale: Human review preserves accountability while still removing most repetitive typing.

Gotchas (what bit them)

Bad audio intake can cause downstream LLM hallucination.

Root cause: Overlapping speakers, noise, and mixed channels make attribution unreliable before the transcript reaches the LLM.

Resolution: Normalize audio, filter noise, and split stereo channels so agent and customer speech remain separable.

STT quality is a bottleneck for the entire generative layer.

Root cause: Heavy accents or poor audio quality leave the LLM with unreliable transcript input.

Resolution: Continuously optimize STT with acoustic modeling, dialect handling, domain dictionaries, and formatting post-processing.

Security layers add latency and architecture overhead.

Root cause: PII masking before cloud endpoints is required, but extra layers slow the pipeline.

Resolution: Keep masking mandatory while optimizing the component design to reduce overhead.

Evaluation

Fujitsu evaluated the deployment through baseline and post-deployment after-call work time, data-quality consistency, STT accuracy requirements, hallucination grounding checks, and operational effects on operator workload.

Metrics: after-call work time from 6.3 to 3.1 minutes · roughly 50% processing-time reduction · STT accuracy above 90% · 500-seat / thousands-of-calls-per-day operational projection · standardized inquiry categorization and call reason tagging

Outcomes

• After-call work dropped from 6.3 minutes to 3.1 minutes.
• Processing time was reduced by about 50%.
• Call categorization moved from operator-dependent notes to standardized logic-based data.
• Structured voice-of-customer data began feeding management dashboards and FAQ candidate discovery.
• The roadmap expands into operator coaching, predictive staffing, and abuse detection for well-being.

The reported production framing is a 500-seat contact-center environment handling thousands of calls per day, with a low-latency pipeline designed for near-real-time workflow improvement rather than offline-only summarization.