AI-Powered Lesson Generation System for Language Learning

duolingo · Education · 2023 · source Agentic: no · promotional bias: med · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Language-learning content creation

Duolingo needed to create and maintain lesson exercises faster for a very large learner base without giving up pedagogical control, grammar constraints, CEFR targeting, vocabulary appropriateness, or expert review.

Constraints:

• content must align with lesson theme, grammar focus, vocabulary, exercise type, and CEFR level
• LLM output can satisfy formal rules while sounding stilted or unnatural
• teaching experts must retain final authority before content reaches learners
• course teams serve more than 21 million daily users with fewer than 1,000 employees
• new content historically shipped only a few times per year for most courses
• the source does not identify the specific LLM provider or report quantitative quality metrics

Architecture

Duolingo added an LLM-assisted workflow to its lesson-authoring process. Learning Designers plan the lesson's theme, grammar, vocabulary, and exercise type. Engineering infrastructure fills some prompt-template blanks automatically, such as language, CEFR level, and theme, while the Learning Designer fills pedagogical parameters such as exercise type and grammar focus. The prompt combines fixed rules, including answer-option count and character limits, with variable lesson parameters. The LLM generates multiple candidate exercises in seconds. Teaching experts then choose, edit, and approve the best options before they enter the app.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  curriculum-plan["curriculum-plan"] --> prompt-template["prompt-template"]
  automatic-parameter-fill["automatic-parameter-fill"] --> prompt-template
  designer-parameter-fill["designer-parameter-fill"] --> prompt-template
  prompt-template --> llm-generator["llm-generator"]
  llm-generator --> candidate-exercises["candidate-exercises"]
  candidate-exercises --> learning-designer-review["learning-designer-review"]
  learning-designer-review --> teaching-expert-approval["teaching-expert-approval"]
  teaching-expert-approval --> duolingo-app-content["duolingo-app-content"]
  duolingo-app-content --> birdbrain-personalization["birdbrain-personalization"]

Components

Data flow

A Learning Designer starts with curriculum intent: theme, grammar, vocabulary, and exercise type. The internal tool fills template variables such as language, CEFR level, and theme, while the designer supplies remaining blanks like grammar focus and exercise type. The prompt template includes fixed constraints such as two answer options and fewer than 75 characters. The LLM produces ten candidate exercises in seconds. The Learning Designer chooses three favorites, edits for naturalness, learning value, and vocabulary, and Spanish teaching experts have the final say before publication.

Why this architecture

Duolingo kept curriculum design and final approval with teaching experts because educational quality depends on pedagogy, naturalness, vocabulary fit, and learner progression. The Mad Lib prompt template makes lesson generation scalable by separating fixed exercise rules from variable course parameters. Automatic parameter fill reduces repetitive setup, while Learning Designer inputs steer the model toward the target grammar and exercise type. Generating multiple candidates in seconds gives experts options, but the review gate catches unnatural or low-value outputs before publication.

The benefits of using AI for lesson creation

At the time of writing this, Duolingo has fewer than 1,000 employees and over 21 million daily users."

— article, stated

Models used: large language model · Birdbrain

Scale

• Duolingo had over 21 million daily users at the time of writing.
  “Duolingo has fewer than 1,000 employees and over 21 million daily users.”article, stated
• Duolingo had fewer than 1,000 employees at the time of writing.
  “Duolingo has fewer than 1,000 employees and over 21 million daily users.”article, stated
• Most courses released new content only a few times per year before faster AI-assisted production.
  “most courses releasing new content only a few times a year.”article, stated
• The example prompt constrains the exercise to two answer options.
  “The exercise must have two answer options.”article, stated
• The example prompt constrains the exercise to fewer than 75 characters.
  “The exercise must be fewer than 75 characters.”article, stated
• The example prompt targets A2 CEFR Spanish.
  “The exercise must be written in A2 CEFR level SPANISH.”article, stated
• The example generation produced ten exercises in seconds.
  “In a matter of seconds, the AI model outputs ten exercises that fit the difficulty level, the grammar, and the theme.”article, stated
• The Learning Designer selected three favorite generated sentences for possible editing.
  “The Learning Designer then chooses their three favorite sentences”article, stated

Alternatives considered (roads not taken)

Fully manual exercise creation

Rejected because: Every exercise had previously been written, reviewed, edited, and translated by human experts, which limited content-update frequency.

1. Delve deeper into the CEFR scale to teach more advanced concepts"

— article, stated

Let the LLM generate lesson content without expert steering

Rejected because: The model needs detailed instructions and guardrails to produce pedagogically useful exercises in the right direction.

Publish generated exercises automatically

Rejected because: Some generated sentences follow rules but sound unnatural, so teaching experts choose, edit, and approve before publication.

Use a one-off prompt for every lesson

Rejected because: The Mad Lib template separates fixed rules from lesson-specific variables so the workflow can scale across languages, levels, and grammar targets.

Patterns

structured-output-contract (contracts)

The prompt constrains generated exercises by answer-option count, character limit, CEFR level, target language, and grammar focus.

guardrails-io-filtering (safety)

Duolingo frames prompt instructions as guardrails that steer the LLM toward acceptable lesson outputs.

human-in-the-loop-gate (control)

Learning Designers and Spanish teaching experts choose, edit, and approve generated exercises before they enter the app.

prompt-versioning (prompting)

The article implies active prompt maintenance by describing constant adjustment of instructions, though it does not describe a formal versioning system.

model-routing (inference)

Duolingo already uses Birdbrain for personalized exercise difficulty and a separate LLM workflow for generating exercise candidates.

Tradeoffs

Use AI as a draft generator

Chose LLM-generated candidate exercises with expert review over fully manual exercise writing for every candidate.

Rationale: Generating options in seconds gives learning experts more raw material while preserving editorial control.

Keep curriculum planning human-driven

Chose Learning Designer planning of theme, grammar, vocabulary, and exercise type over letting the LLM decide the lesson objective.

Rationale: The article positions educational strategy as expert work and uses the model only after humans define the lesson target.

Use prompt templates

Chose fixed and variable prompt fields over free-form prompt writing for each exercise.

Rationale: Template blanks allow engineers and designers to combine stable exercise constraints with lesson-specific variables.

Require final teaching-expert review

Chose human editing and final say over automatic publication of rule-following outputs.

Rationale: Generated exercises may meet formal constraints but still need naturalness, learning value, and vocabulary judgment.

Gotchas (what bit them)

Formally valid generated content can still be pedagogically weak.

Root cause: The LLM can satisfy grammar and format constraints while producing unnatural or low-value sentences.

Resolution: Learning Designers edit for naturalness, learning value, and vocabulary, and teaching experts keep final approval.

Prompt templates need ongoing expert adjustment.

Root cause: The model can generate text from examples, but it needs detailed steering to align with Duolingo's lesson goals.

Resolution: Duolingo keeps teaching experts in the prompt-development loop and continuously adjusts instructions.

The article does not quantify production quality improvements.

Root cause: The company blog describes convenience, speed, and productivity qualitatively but does not provide measured content velocity or learner outcome metrics.

Resolution: Treat the claimed benefits as directional and preserve the known process details rather than overstating impact.

The model provider and monitoring setup are not disclosed.

Root cause: The article is an introductory workflow description, not an infrastructure or LLMOps post.

Resolution: Record the LLM workflow while leaving provider, evaluation, and monitoring details unknown.

Evaluation

The article describes expert review rather than a numeric evaluation harness: Learning Designers inspect generated candidates, edit for naturalness, learning value, and vocabulary, and Spanish teaching experts have final approval before content enters the app.

Metrics: rule compliance · naturalness · learning value · vocabulary appropriateness · expert approval · course content production speed

Outcomes

• Duolingo added LLM-assisted exercise generation to its internal content creation workflow.
• The example workflow generated ten Spanish exercises in seconds from a structured prompt.
• Learning Designers select and edit candidates before publication, with Spanish teaching experts retaining final authority.
• The intended benefits are faster course production, deeper CEFR coverage, more ancillary features such as Stories and DuoRadio, and better support for smaller courses.
• The article does not provide measured content-velocity, quality, or learner-outcome improvements.

The article reports fewer than 1,000 employees, over 21 million daily users, most courses releasing new content only a few times per year, an example prompt with two answer options, a 75-character limit, A2 CEFR Spanish, ten generated exercises in seconds, and three selected favorites.