On-Device Unified Spelling and Grammar Correction Model

grammarly · Tech · 2025 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: On-device writing assistance

Grammarly needed offline spelling and grammar correction that preserved writing assistance quality without internet connectivity and without relying on multiple specialized cloud models.

Constraints:

• The model had to run on-device.
• Correction quality needed to preserve user voice.
• Real-time feedback required high token throughput.
• The team needed to consolidate multiple specialized models into one efficient local model.

Architecture

Grammarly selected Llama as a base, trained on synthetic correction data, and optimized a compact 1B-parameter model with Grouped Query Attention, MLX integration, and 4-bit quantization for Apple silicon.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  synthetic-training-data["Synthetic Training Data"]
  n-1b-on-device-llm["1B On-Device LLM"]
  optimization-stack["Optimization Stack"]
  writing-assistance-ui["Writing Assistance UI"]
  synthetic-training-data --> n-1b-on-device-llm
  n-1b-on-device-llm --> optimization-stack
  n-1b-on-device-llm --> writing-assistance-ui
  optimization-stack --> n-1b-on-device-llm

Components

Data flow

Synthetic data covering writing styles and error types trains the correction model, optimization techniques compress and accelerate inference, and the on-device model provides offline spelling and grammar feedback in real time.

Why this architecture

A compact optimized LLM allowed Grammarly to consolidate spelling and grammar correction locally while preserving real-time performance on Apple hardware.

Models used: Llama · 1B-parameter on-device LLM · Grouped Query Attention · MLX · 4-bit quantization

Scale

• The model has 1B parameters.
  “compact 1B-parameter on-device LLM”summary, stated
• The model achieves 210 tokens per second on M2 Mac hardware.
  “achieves 210 tokens/second on M2 Mac hardware”summary, stated
• The deployment uses 4-bit quantization.
  “4-bit quantization.”summary, stated

Alternatives considered (roads not taken)

Cloud-only spelling and grammar correction

Rejected because: The feature needed to work without internet connectivity.

Maintain multiple specialized models

Rejected because: A single optimized model could consolidate multiple correction capabilities.

Patterns

fine-tuning-adaptation (model-adaptation)

The model was trained on synthetic correction data covering writing styles and error types.

cost-aware-routing (edge-efficiency)

GQA, MLX, and 4-bit quantization make on-device inference fast enough for real-time correction.

model-routing (model-consolidation)

The work consolidates multiple specialized correction models into one unified on-device model.

Tradeoffs

Use a compact 1B model

Chose On-device Llama-based LLM over Larger cloud-hosted correction models.

Rationale: The compact model preserves offline capability and real-time local performance.

Quantize and optimize for Apple silicon

Chose MLX, GQA, and 4-bit quantization over Unoptimized local inference.

Rationale: Optimization was necessary to reach 210 tokens/second on M2 hardware.

Gotchas (what bit them)

Offline writing assistance requires local quality without cloud fallbacks.

Root cause: Users may lack internet connectivity while still expecting correction feedback.

Resolution: Deploy a compact on-device LLM.

On-device LLMs must be heavily optimized for real-time UX.

Root cause: Correction feedback needs low-latency local inference on consumer hardware.

Resolution: Use GQA, MLX on Apple silicon, and 4-bit quantization.

Evaluation

Grammarly reports throughput on M2 Mac hardware and qualitative correction-quality preservation.

Metrics: Tokens per second · Correction quality · Offline capability

Outcomes

• The model provides offline spelling and grammar correction.
• It achieves 210 tokens/second on M2 Mac hardware.
• It consolidates multiple specialized models into one on-device solution.

The model is 1B parameters, uses 4-bit quantization, and runs at 210 tokens/second on M2 Mac hardware.