Adversarial Grammatical Error Correction at Scale for Writing Assistance

grammarly · Tech · 2021 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.2 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Writing assistance

Grammarly wanted grammatical error correction that improved semantic appropriateness rather than only optimizing n-gram translation metrics.

Constraints:

Semantic consistency
Human-like rewrite quality
Metric mismatch with n-gram objectives

Architecture

Grammarly's GEC architecture reframes correction as an adversarial generator-discriminator training system. A sequence-to-sequence generator translates erroneous sentences into corrected rewrites; a sentence-pair discriminator judges whether each rewrite is both grammatical and appropriate for its source sentence; adversarial min-max training and policy gradients use discriminator rewards to fine-tune the generator. Grammarly evaluates RNN and Transformer generators against standard GEC datasets and compares the sentence-pair discriminator with single-sentence and GLEU-style alternatives.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  erroneous-sentence[Erroneous sentence] --> seq2seq-generator[Seq2seq generator]
  seq2seq-generator --> generated-correction[Generated correction]
  generated-correction --> sentence-pair-discriminator[Sentence-pair discriminator]
  human-correction-distribution[Human correction distribution] --> sentence-pair-discriminator
  sentence-pair-discriminator --> policy-gradient-update[Policy-gradient update]
  policy-gradient-update --> seq2seq-generator
  generated-correction --> gec-evaluation-datasets[GEC evaluation datasets]

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Erroneous sentence`	Input text requiring grammatical correction.	`Seq2seq generator`
`Human correction distribution`	Ground-truth corrected rewrites used as the target distribution.	`Sentence-pair discriminator`
`Seq2seq generator`	RNN or Transformer model that proposes grammatical rewrites.	`Erroneous sentence`, `Generated correction`, `Policy-gradient update`
`Generated correction`	Candidate rewrite produced by the generator.	`Sentence-pair discriminator`
`Sentence-pair discriminator`	Scores whether a generated correction is appropriate for the corresponding input sentence.	`Generated correction`, `Human correction distribution`, `Policy-gradient update`
`Policy-gradient update`	Reward signal that fine-tunes the generator toward task-specific correction quality.	`Seq2seq generator`
`GEC evaluation datasets`	Standard benchmarks used to compare adversarial and non-adversarial models.	`Generated correction`

Data flow

Training examples pair erroneous sentences with human corrections. The generator proposes corrected sentences, the discriminator scores them against the source sentence and human-correction distribution, policy-gradient rewards update the generator, and evaluation compares adversarial RNN/Transformer variants on standard GEC datasets.

3 supporting quotes

“the generator is a sequence-to-sequence (seq2seq) model, which is trained to “translate” a grammatically incorrect sentence to its grammatically correct rewrite.”article, stated

“which rewarded high-quality generated text when conditioned on the source. This improved the generation results.”article, stated

“We found that their adversarially trained analogues (RNN-Adv and Transformer-Adv) using the proposed framework consistently achieved better results”article, stated

Why this architecture

Grammarly chose a sentence-pair adversarial setup because n-gram or edit metrics can reward fluent but semantically inconsistent rewrites, and single-sentence real/fake discrimination does not know what was supposed to be corrected. Conditioning the discriminator on the source sentence made the reward align with grammatical correctness, semantic preservation, and natural phrasing.

3 supporting quotes

“they don’t necessarily lead to high-quality corrections. For example, they might make changes to some other parts of the text, resulting in grammatically correct, albeit semantically inconsistent, text. We explored an approach to GEC that avoids such issues: employing”article, stated

“it was harder for such a classifier to differentiate between a ground-truth correction and a generated sentence”article, stated

“able to better differentiate between low-quality or unsuitably corrected sequences and better understand semantic deviations”article, stated

Models used: RNN attentional encoder-decoder · Transformer · sentence-pair discriminator

Scale

The work was presented as part of Grammarly's core writing-assistant mission and EMNLP 2020 research.
“We describe this in a new paper from the Grammarly research team”article, implied
Adversarial RNN and Transformer variants consistently beat standard counterparts on GEC evaluation datasets.
“adversarially trained analogues (RNN-Adv and Transformer-Adv) using the proposed framework consistently achieved better results”article, stated

Alternatives considered (roads not taken)

Optimize NMT-style n-gram or edit metrics

Rejected because: Those objectives can produce grammatical text that changes the source meaning.

“resulting in grammatically correct, albeit semantically inconsistent, text. We explored an approach to GEC that avoids such issues: employing generative adversarial networks (GANs).”article, stated

Use a single-sentence real-versus-fake discriminator

Rejected because: It could not reliably distinguish corrections that missed intended edits or changed semantics.

“A conventional single-sentence real-versus-fake discriminator would provide the probability of a sentence being grammatically correct as the reward”article, stated

Use GLEU directly as the reward

Rejected because: The sentence-pair discriminator performed better than GLEU or conventional discriminator rewards.

“much better performance compared with the conventional single-sentence real-versus-fake discriminator, or using the GLEU score directly”article, stated

Patterns

`fine-tuning-adaptation` (training)

The correction model is trained with adversarial task-specific rewards.

“Adversarial training between the two models was set up as optimizing a min-max objective.”article, stated

`llm-as-judge` (evaluation)

The discriminator judges whether a correction is appropriate for its input sentence.

“In summary, instead of only distinguishing between real-versus-fake sentences, the discriminator needed to measure the degree of “grammatical correctness” of an output sentence given its corresponding input sentence.”article, stated

Tradeoffs

What reward should train the correction model.

Chose source-conditioned sentence-pair discrimination over n-gram metrics or single-sentence real/fake rewards.

Rationale: The source-conditioned reward better reflects whether a correction preserves the intended meaning while fixing grammar.

“given its corresponding input sentence”article, stated

Gotchas (what bit them)

NMT-style GEC can optimize metrics while making semantically inconsistent rewrites.

Root cause: Traditional translation framing underweights input-specific appropriateness.

Resolution: Grammarly trained a sentence-pair discriminator for task-specific rewards.

“Grammarly, a leading AI-powered writing assistant, tackled the challenge of improving grammatical error correction (GEC) by moving beyond traditional neural machine translation approaches that optimize n-gram metrics but sometimes produce semantically inconsistent corrections.”summary, stated

Evaluation

The experiments compared adversarially trained RNN and Transformer generators with standard baselines.

Metrics: GEC benchmark performance · human correction alignment

“We found that their adversarially trained analogues (RNN-Adv and Transformer-Adv) using the proposed framework consistently achieved better results on standard evaluation datasets for GEC, as shown in the figure below.”article, stated

Outcomes

Adversarially trained models outperformed standard counterparts in the reported experiments.

The discriminator helped identify unsuitable corrections.

“We also found that since the sentence-pair discriminator was able to access the contextual information about what has been corrected, it was able to better differentiate between low-quality or unsuitably corrected sequences and better understand semantic deviations in the generated corrections.”article, stated