Specialized Text Editing LLM Development through Instruction Tuning

grammarly · Tech · 2023 · source Agentic: no · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Text-editing model development

Grammarly wanted a smaller, higher-quality model specialized for text editing because general-purpose LLMs were not trained on dense editing tasks and suffered from usability, quality, performance, and reproducibility gaps.

Constraints:

• Text editing quality is subjective and lacks accepted quantitative measures for some desired qualities.
• General-purpose LLMs train on sparse task distributions rather than dense editing tasks.
• The approach needed to cover fluency, coherence, clarity, style, adjacent tasks, and composite edits.
• The resulting models had to be smaller and more accessible than frontier-scale editing models.

Architecture

Grammarly built CoEdIT by translating text-editing categories into natural-language instructions, adding paraphrased instruction templates, fine-tuning FLAN-T5 L/XL/XXL models on the CoEdIT dataset, and evaluating the resulting specialist models against supervised editing baselines, instruction-tuned LLMs, decoder-only LLMs, human preferences, adjacent tasks, and composite editing tasks.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  iterate_r_editing_dataset["IteraTeR+ editing dataset"] --> instruction_template_builder["Instruction-template builder"]
  instruction_template_builder --> coedit_fine_tuning_dataset["CoEdIT fine-tuning dataset"]
  coedit_fine_tuning_dataset --> flan_t5_base_models["FLAN-T5 base models"]
  flan_t5_base_models --> coedit_model_family["CoEdIT model family"]
  coedit_model_family --> benchmark_evaluation_suite["Benchmark evaluation suite"]
  coedit_model_family --> human_evaluation_panel["Human evaluation panel"]

Components

Data flow

Editing categories from IteraTeR+ are converted into natural-language instructions and enriched with sub-intentions plus paraphrased templates. Those examples fine-tune FLAN-T5 variants into CoEdIT models. The models are then evaluated on benchmark test sets, human preference comparisons, adjacent editing tasks, and composite-task prompts.

Why this architecture

Grammarly chose task-specific instruction tuning because text editing is a dense task family: specializing on closely related edits should improve quality, interpretability, and adjacent-task generalization while using models far smaller than general-purpose editing baselines.

Models used: CoEdIT-L · CoEdIT-XL · CoEdIT-XXL · FLAN-T5 · GPT3-Edit · ChatGPT · LLaMA

Scale

• The fine-tuned FLAN-T5 variants ranged from 770 million to 11 billion parameters.
  “FLANT5² LLM (L: 770 million parameters, XL: 3 billion parameters, XXL: 11 billion parameters)”article, stated
• CoEdIT was reported as up to 60 times smaller than GPT-3-Edit and ChatGPT while outperforming them on writing tasks.
  “CoEdIT is an open-source LLM that is not only up to 60 times smaller than popular LLMs like GPT-3-Edit (175 billion parameters) and ChatGPT”article, stated
• Human evaluators preferred CoEdIT-XL output 64% of the time versus 10% for GPT3-Edit.
  “Evaluators preferred CoEdIT’s output 64 percent of the time, compared to just 10 percent for GPT3-Edit.”article, stated

Alternatives considered (roads not taken)

General-purpose LLM training

Rejected because: Grammarly argues sparse general task distributions limit quality for the well-scoped text-editing domain.

Existing supervised editing models

Rejected because: CoEdIT was evaluated against supervised text-editing models and the smallest CoEdIT model still outperformed them.

GPT3-Edit for text editing

Rejected because: Human preference and composite-task evaluations favored CoEdIT variants over GPT3-Edit in the reported comparisons.

Patterns

fine-tuning-adaptation (model adaptation)

Grammarly fine-tuned FLAN-T5 variants on a task-specific editing instruction dataset to produce specialist CoEdIT models.

eval-driven-development (evaluation)

CoEdIT was evaluated with benchmark suites, model comparison groups, human preference tests, adjacent tasks, and composite-task prompts.

data-quality-loop (training data)

The dataset work added natural-language instructions, sub-intentions, and paraphrases to make editing examples consistent and robust to instruction phrasing.

Tradeoffs

Train a dense text-editing specialist.

Chose Task-specific instruction tuning on editing examples over General-purpose task training.

Rationale: The article argues that related editing tasks create a dense distribution where specialist models can outperform generalists, including on adjacent tasks.

Use smaller FLAN-T5 variants.

Chose 770M, 3B, and 11B CoEdIT models over Relying on 175B-parameter GPT3-Edit-style models.

Rationale: CoEdIT sought accessible, smaller models that still outperformed much larger baselines on editing tasks.

Include paraphrased instruction templates.

Chose Multiple phrasings for the same editing instruction over Single canonical instruction wording.

Rationale: The model needed to understand different phrasings like write versus rewrite rather than overfitting to one template.

Gotchas (what bit them)

Broad instruction corpora are a poor fit for specialized text-editing behavior.

Root cause: Sparse general tasks are not closely related to one another, so they do not teach the dense editing skill distribution Grammarly wanted.

Resolution: Construct a dense editing dataset from IteraTeR+ categories, sub-intentions, and paraphrased editing instructions.

Subjective editing quality required more than automated benchmarks.

Root cause: Writing and edit quality depend on human judgment, and accepted quantitative measures are missing for some target qualities.

Resolution: Use both quantitative benchmarks and human evaluations comparing fluency, accuracy, and preservation of meaning.

Composite editing tasks still showed room for improvement.

Root cause: Composite-task comparisons were closer than familiar-task evaluations.

Resolution: Grammarly created CoEdIT-Composite and identified future enhancements for complex multi-part editing.

Evaluation

CoEdIT used quantitative benchmark test sets, comparison groups covering baselines and LLM families, human preference evaluations, adjacent-task tests, and composite-task comparisons.

Metrics: CoEdIT achieved state-of-the-art performance on multiple benchmark test sets. · CoEdIT-L used 12 to 60 times fewer parameters while outperforming compared models. · Human evaluators preferred CoEdIT-XL 64% of the time versus 10% for GPT3-Edit. · CoEdIT-Composite was preferred 38% versus 34% for GPT3-Edit and 34% versus 21% for CoEdIT-XL.

Outcomes

• CoEdIT outperformed multiple text-editing baselines with smaller models.
• Human evaluators strongly preferred CoEdIT-XL to GPT3-Edit in the reported comparison.
• CoEdIT generalized to adjacent tasks and composite editing tasks.

The public model family spans FLAN-T5 L, XL, and XXL sizes from 770M to 11B parameters and is reported as up to 60 times smaller than GPT-3-Edit.