AI-Powered Customer Conversation Analytics at Scale

godaddy · Tech · 2025 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Customer-service conversation analytics

GoDaddy needed to turn up to 100,000 daily customer-service transcripts into company-wide operational intelligence within hours, instead of relying on limited manual sampling.

Constraints:

Transcripts are unstructured, messy, colloquial, and can shift topics
Insights must be available within hours rather than weeks
The pipeline must respect security and privacy policies at massive scale

Architecture

GoDaddy's Lighthouse is an event-driven, serverless analytics platform for customer-service transcripts. Load balancer/API Gateway entry points reach Lambda orchestration, S3 stores raw and processed transcript datasets with versioning, GoCaaS centralizes LLM access, OpenSearch performs deterministic lexical filtering, prompt pipelines emit structured JSON with validators and retries, and QuickSight exposes precomputed insights and incident analytics to business users.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  load-balancer-api-gateway["load-balancer-api-gateway"]
  lambda-processing-functions["lambda-processing-functions"]
  s3-versioned-transcript-store["s3-versioned-transcript-store"]
  gocaas-model-wrapper["gocaas-model-wrapper"]
  opensearch-lexical-filter["opensearch-lexical-filter"]
  structured-prompt-pipelines["structured-prompt-pipelines"]
  prompt-validation-and-retry["prompt-validation-and-retry"]
  on-demand-summarizer["on-demand-summarizer"]
  quicksight-dashboard["quicksight-dashboard"]
  regression-evaluation-suite["regression-evaluation-suite"]
  load-balancer-api-gateway --> lambda-processing-functions
  lambda-processing-functions --> s3-versioned-transcript-store
  lambda-processing-functions --> gocaas-model-wrapper
  lambda-processing-functions --> opensearch-lexical-filter
  lambda-processing-functions --> structured-prompt-pipelines
  s3-versioned-transcript-store --> lambda-processing-functions
  gocaas-model-wrapper --> structured-prompt-pipelines
  opensearch-lexical-filter --> lambda-processing-functions
  opensearch-lexical-filter --> on-demand-summarizer
  structured-prompt-pipelines --> prompt-validation-and-retry
  structured-prompt-pipelines --> quicksight-dashboard
  on-demand-summarizer --> gocaas-model-wrapper
  regression-evaluation-suite --> structured-prompt-pipelines

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`load-balancer-api-gateway`	Request ingress for routing, authentication, and throttling before backend Lambda execution.	`lambda-processing-functions`
`lambda-processing-functions`	Elastic orchestration layer for transcript ingestion, LLM invocation, result aggregation, and scheduled pipelines.	`s3-versioned-transcript-store`, `gocaas-model-wrapper`, `opensearch-lexical-filter`, `structured-prompt-pipelines`
`s3-versioned-transcript-store`	Raw and processed transcript storage with versioning for data integrity and historical reprocessing.	`lambda-processing-functions`
`gocaas-model-wrapper`	Internal LLM wrapper centralizing access to different AI providers and selected models.	`structured-prompt-pipelines`
`opensearch-lexical-filter`	Deterministic token-level search with proximity controls for pre-filtering transcripts before LLM analysis.	`lambda-processing-functions`, `on-demand-summarizer`
`structured-prompt-pipelines`	Versioned prompts and scheduled ETL jobs that output validated JSON for topics, intents, sentiment, and anomalies.	`prompt-validation-and-retry`, `quicksight-dashboard`
`prompt-validation-and-retry`	Runtime validators, corrective retry hints, concurrency controls, token budgets, and circuit breakers.	—
`on-demand-summarizer`	Prompt-driven summarization path for incident response and product issue triage over filtered search results.	`gocaas-model-wrapper`
`quicksight-dashboard`	Business-user dashboard surface for precomputed distributions, anomaly detection, and operational insight.	—
`regression-evaluation-suite`	Automated prompt-quality benchmark suite for detecting drift in output quality over curated transcript sets.	`structured-prompt-pipelines`

Data flow

Customer-service transcripts land in S3 as versioned raw data. Lighthouse uses OpenSearch to filter by metadata or text before expensive LLM processing, then Lambda-driven pipelines call GoCaaS and task-specific prompts to produce structured JSON insight outputs. Validators, retry hints, budgets, and circuit breakers protect the pipeline; QuickSight dashboards and on-demand summarization expose aggregate issues and incident investigations.

3 supporting quotes

“Data flows through several key components: S3 manages storage with versioning and compliance support, GoCaaS centralizes model access across different AI providers”article, stated

“OpenSearch enables fast lexical filtering before expensive LLM processing, and QuickSight provides business users direct access to insights through familiar dashboard interfaces.”article, stated

“Each prompt declares an output JSON schema with runtime validators that reject malformed completions and auto-retry with corrective hints”article, stated

Why this architecture

Lighthouse uses serverless batch processing because manual review cannot cover company-wide transcript volume, while OpenSearch pre-filtering and precomputed ETL keep LLM costs and latency under control. Structured JSON contracts, validators, retries, token budgets, circuit breakers, and regression tests turn noisy transcript interpretation into repeatable analytics rather than one-off prompt runs.

“Processing 100,000+ daily transcripts while maintaining low latency required an event-driven, serverless architecture.”article, stated

“Our lexical search engine provides deterministic token-level matching with proximity controls, delivering sub-second response times over large datasets”article, stated

Models used: Claude v2 · large language models via GoCaaS

Scale

GoDaddy processes more than 100,000 daily customer-service transcripts.
“GoDaddy processes 100,000+ daily customer service transcripts using AI to identify pain points and operational issues at company-wide scale”article, stated
The platform's operating target is actionable insight within hours instead of weeks.
“Automated analysis helps catch emerging issues in hours instead of weeks, reducing customer impact and operational costs”article, stated
Lighthouse processes the full 100k+ daily transcript volume in approximately 80 minutes.
“These tools helped our pipeline process the full daily volume of 100k+ records in approximately 80 minutes.”article, stated
OpenSearch lexical search provides sub-second response times over large datasets.
“Our lexical search engine provides deterministic token-level matching with proximity controls, delivering sub-second response times over large datasets”article, stated
Within a week, Lighthouse identified the most common escalation drivers and dissatisfaction factors.
“within a week, Lighthouse identified the most common drivers of escalations and the key factors contributing to customer dissatisfaction.”article, stated

Alternatives considered (roads not taken)

Manual transcript review and small-call sampling

Rejected because: Manual review could not extract broad, company-wide intelligence from high-volume customer conversations quickly enough.

“While evaluating individual interactions can provide meaningful insights into specific cases, the method quickly becomes limiting.”article, stated

Ad hoc prompt runs

Rejected because: The team built a versioned prompt library because informal prompts produced inconsistent analytics across similar transcripts.

“Early attempts with ad hoc prompts produced inconsistent results across similar conversations. To solve this, we built a versioned prompt library”article, stated

One LLM for all analysis tasks

Rejected because: The orchestration layer allows task-specific model choice because models differ by sentiment, issue extraction, token limits, and concurrency limits.

“No single LLM excels at every task - some models perform better at sentiment analysis, while others excel at product issue extraction.”article, stated

Sending every transcript directly to expensive LLM processing

Rejected because: OpenSearch performs deterministic filtering before LLM calls to reduce cost and speed targeted analysis.

“OpenSearch enables fast lexical filtering before expensive LLM processing”article, stated

Patterns

`prompt-versioning` (analytics-prompting)

GoDaddy maintains a versioned prompt library so teams can reproduce, debug, and improve transcript insights.

“The versioning system ensures reproducibility and enables teams to track which prompt generated specific insights”article, stated

`structured-output-contract` (downstream-analytics)

Every prompt emits JSON, with runtime validators and corrective retries to protect downstream dashboards and aggregation.

“Each prompt declares an output JSON schema with runtime validators that reject malformed completions and auto-retry with corrective hints”article, stated

`model-routing` (llm-orchestration)

The platform lets teams choose models by use case because sentiment, issue extraction, token limits, and cost profiles differ.

“No single LLM excels at every task - some models perform better at sentiment analysis, while others excel at product issue extraction.”article, stated

`retrieval-ranking` (cost-aware-filtering)

OpenSearch performs deterministic lexical filtering before expensive LLM processing, making targeted analysis cheaper and faster.

“Our lexical search engine provides deterministic token-level matching with proximity controls, delivering sub-second response times over large datasets and enabling targeted filtering before expensive LLM processing.”article, stated

`batch-offline-processing` (daily-transcript-etl)

Lighthouse processes the full daily transcript corpus in scheduled ETL and precomputes distributions/anomaly signals for dashboards.

“we run pre-generated insight pipelines that perform scheduled ETL operations for category-based aggregation across topics, intents, and sentiment.”article, stated

`eval-driven-development` (prompt-regression)

Automated tests against curated transcript sets detect drift in prompt output quality over time.

“continuous prompt evaluation through regression-style benchmarking detects drift in output quality using automated tests against curated transcript sets.”article, stated

Tradeoffs

How to process a full daily transcript corpus quickly.

Chose serverless batch analytics with pre-filtering and pre-computation over manual sampling and purely on-demand LLM calls.

Rationale: Batch processing gives company-wide coverage and low-latency dashboard access while avoiding unnecessary LLM calls.

“AWS Lambda provided the elasticity needed for variable workloads, while S3 separated raw and processed datasets with versioning for data integrity.”article, implied

Which model to use for high-volume batch processing.

Chose Claude v2 for batch operations over a single default LLM for every analytics task.

Rationale: Claude v2 fit the token limit and cost profile for GoDaddy's high-volume processing needs.

“we chose Claude v2 for our batch processing operations due to its favorable token limits and cost structure for high-volume analysis.”article, stated

Gotchas (what bit them)

Ad hoc prompts produced inconsistent transcript analytics.

Root cause: Similar conversations could receive different interpretations without systematic prompt iteration and SME input.

Resolution: GoDaddy built a versioned prompt library and structured JSON output contracts.

“Early attempts with ad hoc prompts produced inconsistent results across similar conversations.”article, stated

Manual review could not reveal company-wide or emerging issues quickly.

Root cause: Sampling individual calls misses systemic patterns across the full high-volume corpus.

Resolution: Lighthouse analyzes all daily transcripts and gives teams dashboards and summaries within hours.

“Manual review and sampling approaches failed to provide comprehensive intelligence, and even targeted audits could not surface systemic issues”article, stated

Large-scale transcript processing needed protection against upstream overload.

Root cause: LLM APIs have token and concurrency limits, while daily volume and batch processing create bursty workloads.

Resolution: The system uses token budgets, per-prompt concurrency controls, and circuit breakers.

“The system includes per-prompt concurrency controls, token budgets, and circuit breakers to protect upstream systems”article, stated

Evaluation

GoDaddy continuously scores model responses for accuracy, relevance, and consistency, monitors regression benchmarks, and validates structured outputs before analytics consumption.

Metrics: accuracy · relevance · consistency · daily processing latency · malformed completion rate · drift detection

“A comprehensive evaluation framework continuously scores model responses against accuracy, relevance, and consistency metrics.”article, stated

“These tools helped our pipeline process the full daily volume of 100k+ records in approximately 80 minutes.”article, stated

Outcomes

Lighthouse processes more than 100,000 daily transcripts in about 80 minutes.
Teams can identify pain points and operational issues within hours rather than weeks.
A malfunctioning customer-facing link was detected and resolved before becoming a larger crisis.
Within a week, Lighthouse identified top escalation drivers and customer dissatisfaction factors.

GoDaddy processes 100,000-plus English-language customer-service transcripts per day and plans multilingual expansion as models improve.

“GoDaddy processes 100,000+ daily customer service transcripts using AI to identify pain points and operational issues at company-wide scale”article, stated

“Through Lighthouse insights, GoDaddy was able to quickly identify this trending issue and rectify the problem before it escalated into a larger operational crisis.”article, stated