Structured LLM Conversations for Language Learning Video Calls

duolingo · Education · 2025 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: AI-powered language-learning video conversations

Duolingo needed Video Call with Lily to produce natural spoken language practice that stays at the learner's CEFR level, preserves Lily's character voice, personalizes across sessions, and adapts mid-call without letting a general LLM roam freely.

Constraints:

• The feature must stay pedagogically appropriate for a learner's CEFR level.
• Lily must sound like a sarcastic teenage character rather than a generic chatbot.
• The conversation should follow a predictable structure while allowing learner-led turns.
• Memory must personalize future calls without making the prompt overloaded.
• Calls must end rather than continue indefinitely.

Architecture

Duolingo structures Video Call with Lily around system instructions, a conversation blueprint, separate conversation-prep prompting for the first question, a main conversation prompt, a memory pipeline that extracts learner facts from transcripts, and mid-call evaluations that decide whether Lily should follow or abandon the planned topic.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  learner["Learner"] --> lily_assistant["Lily assistant"]
  system_prompt["System prompt"] --> conversation_prep_prompt["Conversation prep prompt"]
  conversation_prep_prompt --> main_conversation_prompt["Main conversation prompt"]
  system_prompt --> main_conversation_prompt
  system_prompt --> lily_assistant
  main_conversation_prompt --> lily_assistant
  lily_assistant --> mid_call_evaluation["Mid-call evaluation"]
  learner_memory_facts["Learner memory facts"] --> system_prompt
  transcript_fact_extractor["Transcript fact extractor"] --> learner_memory_facts

Components

Data flow

Before a call, the system prepares a first question at the learner's CEFR level and injects prior learner facts into Lily's instructions. The main conversation starts with a system-controlled greeting and opener, then Lily reacts to learner turns. During the call, mid-call checks inspect what the learner said and adjust the conversation direction; after the call, the transcript is summarized into memory facts for next time.

Why this architecture

Duolingo splits conversation prep, main conversation, memory, and mid-call evaluation because one broad prompt can overload the LLM. Structured prompts let Lily maintain character, CEFR level, conversation shape, personalization, and learner-led adaptability without drifting.

Models used: LLM · Video Call with Lily

Scale

• Lily conversations are aligned to CEFR learner levels such as A2.
  “You’re talking to a learner who’s at A2 CEFR level.”article, stated
• Engineers built a cycle of greetings for each CEFR level.
  “Our engineers have built a cycle of greetings that Lily will go through for each CEFR level.”article, stated
• The system runs an intervention after a certain number of back-and-forths to end the call.
  “After a certain number of back-and-forths, engineers have created a program where the System jumps in”article, stated

Alternatives considered (roads not taken)

Let the LLM roam freely

Rejected because: Duolingo says targeted instructions and predictable structure are needed to ensure learning value and character consistency.

Put first-question instructions into the main conversation prompt

Rejected because: Combining those instructions overloaded the LLM and produced undesirable results.

Force Lily to stick to the planned topic

Rejected because: Learners sometimes want to lead the conversation, so the system now checks when to ignore the planned topic.

Patterns

prompt-chaining (conversation architecture)

Duolingo separates conversation prep, main conversation, memory extraction, and mid-call evaluation into distinct LLM prompt stages.

agent-memory (personalization)

The system extracts learner facts from prior transcripts and includes them in future calls for personalized follow-ups.

structured-output-contract (prompt control)

The system uses a three-character System/Assistant/User structure and a conversation blueprint to constrain free-form LLM behavior.

Tradeoffs

Separate conversation prep from live conversation.

Chose Two-stage prompting over One large prompt for question generation and conversation management.

Rationale: Separating the first question avoids overload and preserves vocabulary, difficulty, and conversation quality.

Use memory facts for personalization.

Chose Transcript-derived List of Facts over Stateless conversations.

Rationale: Memory allows Lily to reference prior learner facts and make calls feel personalized.

Gotchas (what bit them)

A single overloaded prompt degrades conversation quality.

Root cause: First-question generation and full conversation management ask the LLM to satisfy too many constraints at once.

Resolution: Prepare the first question separately and feed it into the main conversation prompt.

Rigid topic plans can ignore learner intent.

Root cause: Learners may introduce a more meaningful topic than the planned focus.

Resolution: Add mid-call evaluations that let Lily follow the learner when appropriate.

Evaluation

Duolingo reports qualitative product-learning checks: prompt overload symptoms, mid-call evaluation behavior, CEFR-level alignment, and conversation continuation/end controls.

Metrics: CEFR-level alignment. · Mid-call learner-lead detection. · Conversation length control after a certain number of turns.

Outcomes

• Video Call with Lily can maintain a consistent character, level, and conversation structure.
• The feature remembers learner facts across sessions.
• Mid-call evaluations let learners lead when appropriate.

The article describes product architecture and design decisions, not usage volume, latency, or learning-outcome metrics.