Multilingual Text Editing via Instruction Tuning

grammarly · Tech · 2024 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Multilingual writing assistance

Grammarly needed text-editing models that could support multiple edit tasks across multiple languages because general foundation models and earlier specialized editors did not cover that full product surface.

Constraints:

• The system had to support grammatical error correction, simplification, and paraphrasing.
• The training set spanned seven languages and uneven public data availability.
• Instruction language could differ from the language of the text being edited.
• The work was research-stage and not yet integrated into Grammarly production systems.

Architecture

Grammarly built mEdIT by fine-tuning multiple multilingual LLM families on curated instruction-output pairs, then evaluating task, language, and cross-lingual instruction performance with automatic and human evaluation.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  instruction-dataset["Instruction Dataset"]
  human-instruction-review["Human Instruction Review"]
  fine-tuning-jobs["Fine-Tuning Jobs"]
  evaluation-harness["Evaluation Harness"]
  instruction-dataset --> fine-tuning-jobs
  human-instruction-review --> instruction-dataset
  fine-tuning-jobs --> evaluation-harness

Components

Data flow

Public multilingual editing datasets were curated into instruction-output pairs, native speakers reviewed translated instructions, candidate multilingual models were fine-tuned on GPU infrastructure, and the resulting models were evaluated against task-specific metrics and human ratings.

Why this architecture

The team chose instruction tuning over generic prompting because specialized multilingual editing behavior required curated task examples and explicit cross-lingual evaluation.

Models used: mT5 · mT0 · BLOOMZ · PolyLM · Bactrian-X

Scale

• The training corpus contains more than 200,000 instruction-output pairs.
  “over 200,000 carefully curated instruction-output pairs across Arabic, Chinese, English, German, Japanese, Korean, and Spanish”summary, stated
• The models cover seven languages and three editing tasks.
  “support intelligent writing assistance across seven languages and three editing tasks (grammatical error correction, text simplification, and paraphrasing).”summary, stated
• Training used 8 A100 80GB GPUs.
  “All models were fine-tuned on 8xA100 80GB GPU instances, representing significant computational infrastructure.”summary, stated

Alternatives considered (roads not taken)

Use general-purpose foundation models without task fine-tuning

Rejected because: They produced poor quality on multilingual editing tasks.

Keep the prior English-only CoEdIT scope

Rejected because: It did not address multilingual writing support.

Patterns

fine-tuning-adaptation (model-adaptation)

Grammarly fine-tuned multilingual models on curated edit instructions rather than relying on zero-shot foundation model behavior.

eval-driven-development (evaluation)

The project compared architectures, instruction-language settings, and baselines using task-specific metrics plus human evaluation.

human-in-the-loop-gate (data-quality)

Native speakers reviewed translated instructions before they entered the training set.

Tradeoffs

Fine-tune smaller multilingual models

Chose Task-specialized multilingual LLMs over Prompting large general-purpose models.

Rationale: Fine-tuning improved editing quality while keeping deployment feasibility closer to small and medium LLMs.

Cap dataset samples rather than maximize quantity

Chose Randomly sample 10,000 examples where available over Use every available example in larger datasets.

Rationale: Prior experiments showed quality mattered more than raw volume after a threshold.

Gotchas (what bit them)

Training data availability varies sharply by language and task.

Root cause: Public editing datasets are uneven across languages.

Resolution: Sample consistent caps where possible and use all available data for sparse combinations.

Automatic metrics can misread high-quality generated edits.

Root cause: Some metrics depend heavily on overlap with reference text.

Resolution: Document metric limitations and supplement with human evaluation.

Evaluation

The team evaluated with task-specific automatic metrics, zero-shot baselines, architecture comparisons, cross-lingual instruction conditions, and human ratings.

Metrics: M2 F0.5 · ERRANT · GLEU · SARI · BLEU · Self-BLEU · mUSE · Human ratings

Outcomes

• mEdIT achieved strong performance across tasks and languages.
• The models generalized to unseen languages.
• Causal language models performed best in the reported experiments.

The work covers seven languages, three edit tasks, and over 200,000 curated examples, but it was not yet deployed in Grammarly production.