Auto-generated Document Summaries Using Abstractive Summarization

google · Tech · 2022 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Tech

Google Docs wanted to suggest concise document summaries while preserving writer control over whether to accept, edit, or ignore the suggestion.

Constraints:

• summaries must be short and useful across document types
• fine-tuning data quality mattered more than sheer quantity
• serving latency mattered for a user-facing feature
• writers needed final control

Architecture

Google Docs auto-generated summaries use a Pegasus-based abstractive summarization pipeline adapted for product serving. The design starts from self-supervised Pegasus pre-training, fine-tunes on a cleaned and consistent summary corpus, distills the large model into a Transformer-encoder/RNN-decoder hybrid for lower latency and memory footprint, serves inference on TPUs, and exposes suggestions under writer control so users can accept, edit, or ignore model output.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  document-text-input["document-text-input"]
  pegasus-summarization-model["pegasus-summarization-model"]
  curated-finetuning-corpus["curated-finetuning-corpus"]
  distilled-serving-model["distilled-serving-model"]
  tpu-inference-service["tpu-inference-service"]
  summary-confidence-gate["summary-confidence-gate"]
  summary-suggestion-ui["summary-suggestion-ui"]
  writer-control-loop["writer-control-loop"]
  document-text-input --> pegasus-summarization-model
  curated-finetuning-corpus --> pegasus-summarization-model
  pegasus-summarization-model --> distilled-serving-model
  distilled-serving-model --> tpu-inference-service
  distilled-serving-model --> summary-confidence-gate
  tpu-inference-service --> summary-suggestion-ui
  summary-confidence-gate --> summary-suggestion-ui
  summary-suggestion-ui --> writer-control-loop

Components

Data flow

Document text enters a summarization model that generates a one-to-two sentence suggestion only when confidence is sufficient. Pegasus pre-training supplies summarization-oriented language capabilities; cleaned fine-tuning data adapts the model to Google Docs-style summaries; knowledge distillation creates a lighter serving model; the UI displays the suggestion to the document writer for final control.

Why this architecture

The production architecture balances research quality, data quality, serving efficiency, and user control. Pegasus gives a summarization-specific pre-training objective, cleaned fine-tuning data prevents the model from learning inconsistent summary styles, distillation and TPUs address decoder latency/memory costs, and confidence gating plus writer control handles unsuitable or ambiguous documents.

Models used: Pegasus · Transformer encoder / RNN decoder distilled model

Scale

• The generated summary target is one to two sentences.
  “generates a 1-2 sentence natural language description of the document content”article, stated
• Pegasus could match Transformer baselines trained on 10,000+ supervised examples with as few as 1,000 fine-tuning examples on some benchmarks.
  “Some summarization benchmarks required as few as 1,000 fine-tuning examples for Pegasus to match the performance of Transformer baselines that saw 10,000+ supervised examples”article, stated
• The serving model was optimized because autoregressive decoding gets slower as summaries get longer.
  “The decoding process becomes noticeably slow when summaries get longer since the decoder attends to all previously generated tokens at each step.”article, implied
• The TPU serving path was chosen to handle more requests per machine.
  “which provide significant speed ups and allow more requests to be handled by a single machine.”article, implied

Alternatives considered (roads not taken)

Use early inconsistent fine-tuning data as-is

Rejected because: The corpus mixed document and summary styles so heavily that the model struggled to learn a coherent relationship.

Prefer more supervised data over cleaner data

Rejected because: Google reduced the data volume but improved consistency, which produced a higher-quality model.

Serve the original large Pegasus-style Transformer decoder directly

Rejected because: Transformer decoding was inefficient for production, so Google distilled into a more efficient hybrid architecture.

Automatically overwrite document summaries

Rejected because: The product leaves writers in control by letting them accept, edit, or ignore suggestions.

Patterns

fine-tuning-adaptation (model-operations)

The model was fine-tuned on manually generated summaries consistent with Google Docs use cases.

data-quality-loop (data)

Google cleaned and filtered the fine-tuning data for consistency, improving quality despite reducing volume.

human-in-the-loop-gate (product)

The generated summary remains a suggestion controlled by the document writer.

Tradeoffs

Prioritize fine-tuning data consistency over dataset size.

Chose cleaned and filtered lower-volume fine-tuning data over more varied but inconsistent supervised data.

Rationale: The source says reducing data volume while improving consistency led to a higher-quality model.

Gotchas (what bit them)

A broad, inconsistent summary corpus confused the model.

Root cause: The training data mixed many document types and summary styles.

Resolution: Google narrowed and cleaned the fine-tuning set for a coherent definition of summaries.

Evaluation

The source refers to model quality improvement from data curation but does not give a detailed evaluation protocol.

Outcomes

• Auto-generated suggestions became available to Google Workspace business customers.

The feature is user-facing inside Google Docs.