AI-Powered Teacher Assistant for Core Curriculum Alignment in K-5 Education

espark · Education · 2024 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: K-5 education technology

eSpark needed to help teachers align its supplemental K-5 reading and math activities to district-mandated core curricula, while making the workflow fast enough for overwhelmed teachers and credible enough for administrators who wanted core-curriculum fidelity.

Constraints:

• teachers work under district mandates and often have limited training on new core curricula
• hundreds of core curricula and district-created scopes/sequences make manual alignment unsustainable
• eSpark has tens of thousands of activities curated over 15 years
• teachers did not respond well to blank-chatbot interfaces during planning
• student-facing and teacher-facing AI outputs needed strong evaluation and observability before production

Architecture

eSpark built a teacher assistant that matches a teacher's current core-curriculum lesson to relevant eSpark assignments. The system moved away from a chatbot-first interface toward a lesson dropdown: the teacher selects the lesson they are on, a core-curriculum retrieval tool pulls that lesson into the prompt, an embeddings/RAG tool retrieves semantically relevant eSpark activities, and the LLM recommends a small set of assignments plus AI-generated follow-up questions. BrainTrust provides prompt/version tooling, tool-call traces, datasets, code-based and human scoring, and a go/no-go eval loop before new book curricula or model changes reach production.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  teacher-lesson-selector["teacher-lesson-selector"] --> core-curriculum-retriever["core-curriculum-retriever"]
  teacher-lesson-selector["teacher-lesson-selector"] --> assignment-recommendation-agent["assignment-recommendation-agent"]
  core-curriculum-retriever["core-curriculum-retriever"] --> assignment-recommendation-agent["assignment-recommendation-agent"]
  metadata-enrichment-pipeline["metadata-enrichment-pipeline"] --> espark-activity-embedding-index["espark-activity-embedding-index"]
  espark-activity-embedding-index["espark-activity-embedding-index"] --> assignment-recommendation-agent["assignment-recommendation-agent"]
  assignment-recommendation-agent["assignment-recommendation-agent"] --> follow-up-question-generator["follow-up-question-generator"]
  assignment-recommendation-agent["assignment-recommendation-agent"] --> braintrust-eval-observability["braintrust-eval-observability"]
  product-learning-design-review["product-learning-design-review"] --> braintrust-eval-observability["braintrust-eval-observability"]
  product-learning-design-review["product-learning-design-review"] --> metadata-enrichment-pipeline["metadata-enrichment-pipeline"]
  assignment-recommendation-agent["assignment-recommendation-agent"] --> support-routing-layer["support-routing-layer"]

Components

Data flow

A teacher selects their core-curriculum lesson rather than typing an open prompt. The assistant retrieves that exact lesson text, which fits in the prompt window, and calls an embeddings database over enriched eSpark activity metadata. The LLM combines the current lesson with retrieved activities to recommend the best two or three assignments and generate three follow-up question suggestions. Interactions, prompts, tool calls, Pinecone returns, traces, and scores are logged in BrainTrust so product, learning-design, and engineering staff can evaluate relevance, recall, breadth, JSON shape, and human teacher impression before production rollout.

Why this architecture

eSpark settled on tool-calling RAG because core curricula and eSpark's activity catalog were too large and dense to stuff into prompts. The selected lesson could be retrieved directly, while eSpark assignments needed semantic search over richer embeddings. The UI moved away from open chat because teachers did not know what to ask and needed planning speed. BrainTrust became necessary because product and learning-design teams needed prompt/version control, traces, non-engineer scoring, and repeatable evals across many curricula before production changes.

Models used: OpenAI APIs · OpenAI Assistants API · LLMs · Pinecone · BrainTrust · Python scoring · LLM-based scores

Scale

• eSpark serves kindergarten through fifth grade reading and math.
  “students in kindergarten through fifth grade will take a short diagnostic test”transcript, stated
• The teacher assistant was eSpark's fourth major LLM/AI feature after the ChatGPT launch era.
  “We're currently shipping now our fourth sort of LLM based or AI based major feature”transcript, stated
• The team had tens of thousands of activities curated or created over 15 years.
  “we had this tens of thousands of eSpark activities that we curated or created our own over the last 15 years.”transcript, stated
• The first teacher-assistant idea emerged around April 2024.
  “The beginning of the teachers thinking of thinking about teachers is I think it was April 24th.”transcript, stated
• Teachers may face 30 students while planning the next day on Sunday night.
  “This happens on Sunday night. They have the Sunday scaries, and they have tomorrow there's 30 kids.”transcript, stated
• A single core-curriculum lesson can have 24 learning objectives and up to roughly 50 possible matching assignments.
  “a certain core curriculum that have 24 different learning objectives each learning objective could have three or four assignments”transcript, stated

  “for just a single lesson you could have 50 assignments that could theoretically be a good match.”transcript, stated
• They limit recommendations to a maximum of five to avoid overwhelming teachers.
  “there's a maximum of five. If our data set has if our expected column has like 30, it's limited what you can score there.”transcript, stated
• The team expects thousands of teachers to use the assistant daily and provide feedback.
  “finally have it out there and have thousands of teachers on it every day and giving us feedback.”transcript, stated

Alternatives considered (roads not taken)

Manual alignment by curriculum provider

Rejected because: Hundreds of curricula, grade levels, and district-created sequences made one-provider-at-a-time mapping unsustainable for a small team.

Open-text chatbot as the main teacher interface

Rejected because: Teachers did not know what to type and did not have time for a conversational planning workflow.

Stuffing all curriculum and activity data directly into prompts

Rejected because: The data was too dense and the activity catalog too vast, making token cost and context quality problematic.

Same semantic-search approach for curricula and eSpark activities

Rejected because: The selected core lesson could be retrieved directly, while eSpark activity matching required richer semantic embeddings.

Patterns

rag-retrieval (retrieval)

The teacher assistant augments prompts with the selected core lesson and retrieved eSpark activities rather than relying on prompt-only generation.

tool-use-orchestration (agent architecture)

The LLM decides when to call an assignment-retrieval tool and receives tool outputs describing relevant activities.

retrieval-ranking (retrieval)

The team enriched metadata and tuned embeddings so retrieval could rank the best few eSpark activities instead of dumping many weak matches.

eval-driven-development (evaluation)

BrainTrust datasets, traces, code scores, human scores, and go/no-go loops gate curriculum publication and model changes.

observability-tracing (operations)

BrainTrust traces make the non-deterministic prompt/tool/RAG behavior inspectable when a recommendation fails.

Tradeoffs

Use a lesson dropdown rather than open chat for the primary workflow.

Chose Guided lesson selection over Conversational blank text box.

Rationale: Teachers know what lesson they are on and need fewer clicks, not a prompt-writing task during lesson planning.

Use different retrieval approaches for each data source.

Chose Direct lesson retrieval plus semantic activity search over One RAG pattern for both core curricula and eSpark content.

Rationale: The selected lesson fits in context and does not require semantic search, while activity matching needs enriched embeddings.

Limit recommendation count.

Chose Best two or three assignments, max five in eval framing over Returning all potentially relevant assignments.

Rationale: Teachers need actionable recommendations, and too many options are overwhelming even if many matches are technically valid.

Gotchas (what bit them)

The chatbot-first interface did not fit teacher planning behavior.

Root cause: The team projected their own ChatGPT usage onto teachers, but teachers were time-constrained and anxious about typing the right thing.

Resolution: Replace the blank text box with lesson selection, recommended assignments, and suggested follow-up questions.

Sparse activity metadata made embeddings retrieval poor.

Root cause: The original assignment table had titles, short descriptions, URLs, grade levels, and standards, but not enough semantic context for retrieval.

Resolution: Use LLM-assisted asynchronous enrichment to add learning objectives, grade level, keywords, topic, subtopic, and domain before embedding.

Expected-answer datasets were expensive and imperfect.

Root cause: A single curriculum lesson can have many learning objectives and many acceptable assignments, so 100% complete golden sets are unrealistic.

Resolution: Combine code scores with human teacher-impression and breadth scoring to calibrate realistic thresholds.

Teachers used open text for customer-support questions.

Root cause: A general ask-anything box invited support requests outside the planned assignment-recommendation use case.

Resolution: Add basic support knowledge and route users toward Intercom when the assistant receives support-style questions.

Evaluation

Evaluation combines BrainTrust traces, prompt/version management, code-based precision/recall/JSON scores, human teacher-impression and breadth scores, learning-designer review, and go/no-go loops before curriculum publication or model switches.

Metrics: precision of recommended assignments against expected matches · recall or coverage across learning objectives · JSON output correctness · human teacher impression score · breadth of recommendations · trace debugging of RAG and tool calls · thumbs-up/thumbs-down teacher feedback

Outcomes

• The product moved from an open chatbot proof of concept to a guided lesson-selection assistant with assignment recommendations and follow-up suggestions.
• BrainTrust made debugging, evals, prompt changes, and production go/no-go decisions less stressful and accessible to non-engineers.
• The team planned next to incorporate student data for more context-aware recommendations while monitoring teacher feedback.

The source reports tens of thousands of activities over 15 years, hundreds of curricula, thousands of teachers expected daily, and recommendation/eval limits shaped by lessons with up to 50 plausible matching assignments.