Abstractive Conversation Summarization for Google Chat Spaces

google · Tech · 2022 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.2 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Workplace collaboration

Google wanted to reduce unread-message overload in Chat Spaces with automatically generated conversation summaries.

Constraints:

Architecture

Google Chat conversation summaries combine a Pegasus-based summarization model with production controls for chat-specific quality and latency. ForumSum supplies multi-speaker training data, Pegasus is fine-tuned for conversation summarization, knowledge distillation produces a lower-latency transformer-encoder/RNN-decoder hybrid, and the product layer decides when a Space is worth summarizing, detects low-quality summaries, pre-generates updates on message changes, caches them ephemerally, and displays summary cards when users enter Spaces with unread messages.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  forumsum-dataset[ForumSum dataset] --> pegasus-fine-tuning[Pegasus fine-tuning]
  pegasus-fine-tuning --> knowledge-distillation[Knowledge distillation]
  knowledge-distillation --> hybrid-summarization-model[Hybrid summarization model]
  google-chat-spaces[Google Chat Spaces] --> controlled-triggering[Controlled triggering]
  controlled-triggering --> hybrid-summarization-model
  hybrid-summarization-model --> low-quality-summary-detector[Low-quality summary detector]
  hybrid-summarization-model --> ephemeral-summary-cache[Ephemeral summary cache]
  ephemeral-summary-cache --> summary-card[Summary card]
  summary-card --> google-chat-spaces
  human-evaluation-and-user-feedback[Human evaluation and user feedback] --> low-quality-summary-detector
  human-evaluation-and-user-feedback --> forumsum-dataset
Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component Role Interacts with
Google Chat Spaces Product surface where unread-message summaries appear as cards. Summary card, Controlled triggering
Summary card UI presentation of automatically generated topic summaries for unread conversations. Google Chat Spaces, Ephemeral summary cache
ForumSum dataset Conversation summarization dataset used to adapt the model to multi-speaker chats. Pegasus fine-tuning
Pegasus fine-tuning Fine-tunes Pegasus with conversation words as input and summaries as output. ForumSum dataset, Knowledge distillation
Knowledge distillation Distills Pegasus into a lower-latency hybrid architecture. Pegasus fine-tuning, Hybrid summarization model
Hybrid summarization model Transformer-encoder/RNN-decoder model used for lower latency and memory footprint. Knowledge distillation, Ephemeral summary cache, Low-quality summary detector
Controlled triggering Determines whether a conversation is worth summarizing. Google Chat Spaces, Hybrid summarization model
Low-quality summary detector Detects misattribution, misrepresentation, and weak summaries so they can be withheld. Hybrid summarization model, Summary card
Ephemeral summary cache Stores pre-generated summaries so opening Spaces is smooth. Hybrid summarization model, Summary card
Human evaluation and user feedback Signals used to characterize usefulness, accuracy, and quality issues. Low-quality summary detector, ForumSum dataset

Data flow

Forum conversations are curated into ForumSum and used to fine-tune Pegasus; the Pegasus model is distilled into a lower-latency hybrid summarizer. In production, Google Chat updates a summary whenever messages are sent, edited, or deleted, stores it ephemerally, checks for low-value or low-quality cases, and shows a summary card when a user opens a Space with unread messages.

“First, we fine-tune Pegasus on the ForumSum dataset where the input is the conversation words and the output is the summary words.”article, stated
“To address this issue, we instead generate and update summaries whenever there is a new message sent, edited or deleted.”article, stated

Why this architecture

Google chose an abstractive, chat-adapted, distilled model with triggering and caching because Chat summaries needed human-like wording, lower latency, and guardrails against misattribution or misrepresentation. The product could not simply summarize every conversation on demand with a large model.

“The resulting model has lower latency and memory footprint while maintaining similar quality as the Pegasus model.”article, stated
“Controlled triggering: To make sure summaries bring the most value to our users, we first need to make sure that the chat conversation is worthy of summarization.”article, stated

Models used: Pegasus · hybrid transformer-encoder/RNN-decoder

Scale

Alternatives considered (roads not taken)

Extractive summarization

Rejected because: The system needed abstractive summaries closer to how people summarize conversations rather than concatenated snippets.

“Another approach is to use natural language generation (NLG) techniques to summarize using novel words and phrases not necessarily present in the original text.”article, implied

Serving Pegasus directly

Rejected because: Distillation gave a lower-latency and lower-memory model while maintaining similar quality.

“Second, we use knowledge distillation to distill the Pegasus model into a hybrid architecture”article, stated

Generate summaries only when users open Spaces

Rejected because: Opening-time generation still produced noticeable latency, so summaries were updated on message changes and cached.

“the latency to generate summaries was still noticeable to users when they opened Spaces with unread messages. To address this issue, we instead generate and update summaries whenever there is a new message sent, edited or deleted. Then summaries are cached ephemerally”article, stated

Show every generated summary

Rejected because: The system needed detectors that abstain from low-quality or low-value summaries.

“we still developed methods to detect and abstain from showing such summaries to the user when they are generated.”article, stated

Patterns

fine-tuning-adaptation (model-operations)

Google created ForumSum and adapted summarization models to conversational data.

“To address this we created [ForumSum](https://aclanthology.”article, stated

latency-optimization (serving)

Summaries are generated and refreshed before users open Spaces to reduce visible wait time.

“To address this issue, we instead generate and update summaries whenever there is a new message sent, edited or deleted.”article, stated

Tradeoffs

Deploy a distilled summarization model for chat surfaces.

Chose A hybrid transformer-encoder/RNN-decoder distilled from Pegasus over Serving the larger Pegasus model directly in the product path.

Rationale: Google traded architectural simplicity for lower latency and memory while keeping similar summarization quality.

“The resulting model has lower latency and memory footprint while maintaining similar quality as the Pegasus model.”article, stated

How to serve chat summaries with acceptable latency.

Chose pre-generate and ephemerally cache summaries on message changes over generating summaries only when users opened unread Spaces.

Rationale: Google moved work off the critical open path because even the optimized hybrid model still produced noticeable user-facing latency.

“Then summaries are cached ephemerally to ensure they surface smoothly when users open Spaces with unread messages.”article, stated

How aggressively to show generated summaries.

Chose controlled triggering and low-quality detection over displaying every model-generated summary.

Rationale: Some conversations were too short or active to be worth summarizing, and generated summaries could misattribute or misrepresent content.

“The first one is misattribution, when the model confuses which person or entity said or performed a certain action. The second one is misrepresentation”article, stated

Gotchas (what bit them)

The model sometimes produced low-quality or low-value summaries.

Root cause: Chat conversation style differed from existing summarization datasets.

Resolution: Google added generation restrictions and detection heuristics that abstain from showing weak summaries.

“Detecting low quality summaries: While the two methods above limited low quality and low value summaries, we still developed methods to detect and abstain from showing such summaries to the user when they are generated.”article, stated

Evaluation

Google used human evaluation and user feedback to characterize summary usefulness and accuracy.

Metrics: human evaluation · user feedback · latency

“Based on human evaluation and user feedback, we learned that the summarization model generates useful and accurate summaries most of the time.”article, stated

Outcomes

The production feature is surfaced as summary cards in Spaces.

“When these summaries are available, a card with automatically generated summaries is shown as users enter Spaces with unread messages.”article, stated