Building a Systematic SNAP Benefits LLM Evaluation Framework

propel · Government · 2025 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Public benefits and safety-net services

Propel needed a domain-specific LLM evaluation approach for SNAP questions because general model evals under-cover high-stakes public-benefits nuance and bad answers can materially harm low-income users.

Constraints:

• SNAP has federal law, federal regulations, state policy variation, and practical navigation nuance.
• General-purpose AI evals are unlikely to cover niche safety-net benefits questions deeply.
• Incorrect or unhelpful answers can affect eligibility, benefit access, or user safety.
• Evaluation criteria must encode domain-expert judgment, not only string matching or vibes.

Architecture

Propel's SNAP eval approach starts with a domain expert using many models, noting nuanced good/meh/bad responses, and turning those observations into automated question/answer tests. The eval compares foundation models, prompts, and context/document configurations, can use model-judges for harder semantic checks, and is intended to inform routing, guardrails, and safer product development.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  snap-domain-expert[SNAP Domain Expert] --> hydra-slackbot[Hydra Slackbot]
  hydra-slackbot --> foundation-models[Foundation Models]
  snap-domain-expert --> eval-test-cases[Eval Test Cases]
  policy-context[Policy Context] --> foundation-models
  policy-context --> eval-test-cases
  foundation-models --> eval-test-cases
  eval-test-cases --> llm-judge[LLM Judge]
  eval-test-cases --> routing-and-guardrails[Routing and Guardrails]
  eval-test-cases --> public-eval[Public Eval]

Components

Data flow

A SNAP domain expert explores frontier models directly or through Hydra, records nuanced failure modes such as asset-limit answers, converts representative prompts into eval test cases, compares model/prompt/context configurations, and uses results to decide routing, guardrail topics, and product-design choices for SNAP users.

Why this architecture

Propel treats evals as test-driven product infrastructure: domain experts first discover what good answers require, then automated tests compare models, prompts, and document-context strategies so high-stakes product choices can be empirical rather than vibe-based.

Models used: OpenAI · Anthropic · Google · Llama · Deepseek · ChatGPT · Claude · Gemini · o1-mini

Scale

• SNAP is implemented across 50+ state-level policy regimes.
  “With 50+ state-level implementations of federal SNAP policy, answers often depend on jurisdiction, creating a combinatorial challenge for evaluation.”summary, stated
• Propel compared broad model families including GPT-3, GPT-3.5, GPT-4, and o1-mini.
  “* How ChatGPT vs. Claude vs. Gemini (vs. Google AI-generated search answers) perform on SNAP questions/prompts * The pace of improvement on SNAP questions of a given class of model (GPT-3 vs. 3.5 vs. 4 vs. o1-mini…)”article, stated
• The asset-limit example includes a $10,000-bank-balance prompt and federal limits of $2,750 or $4,250.
  “Strictly speaking, the asset limit is $2,750, or $4,250 for a household with an elderly or disabled member. You can read that in federal policy.”article, stated
• Only 13 states still impose asset limits in the cited SNAP asset-limit scenario.
  “As a result of this, in reality only 13 states still impose asset limits. All other states have asset limits only in very limited circumstances. A good answer to this question must include this critical nuance — that for most people, assets won’t affect your eligibility.”article, stated
• A hypothetical 100-question SNAP policy eval reaching 99 correct answers is used as the confidence threshold example.
  “If our SNAP policy eval has 100 question/answer pairs, and we iterate to a system (a choice of a model, a prompt, and context/documents) that gets 99 of those questions right”article, stated

Alternatives considered (roads not taken)

Relying on broad public model evaluations

Rejected because: SNAP is a niche safety-net domain unlikely to be covered deeply by general AI evaluations.

Manual vibe-check assessment of model output

Rejected because: The eval should let Propel test many models, prompts, and contexts without inconsistent subjective assessment.

Simple keyword checks for semantic requirements

Rejected because: Checking for words like eliminated or BBCE may mislabel answers; semantic model judging may be needed.

Including all federal and state policy documents by default

Rejected because: The article explicitly notes that all federal policy might confuse the model more than a state manual alone.

Patterns

eval-driven-development (evaluation)

Propel starts with domain-expert model exploration and converts nuanced failures into reproducible eval tests before building products on top of models.

llm-as-judge (evaluation)

For semantic checks that cannot be reduced to a word match, Propel proposes using a model to evaluate another model's answer.

model-routing (inference)

The eval can identify where cheaper/faster models are good enough for a class of SNAP questions and route traffic accordingly.

safety-policy-enforcement (safety)

The eval codifies risky SNAP edge cases and identifies topics/forms of harmful output that require guardrails.

confidence-thresholding (safety)

The article frames production confidence around knowing and filtering the small set of failure cases that remain after high eval performance.

Tradeoffs

Starting point for eval construction

Chose Domain expert exploration before code automation over Immediately writing automated tests.

Rationale: The domain expert can discover nuanced good, bad, and ambiguous answers faster by using models directly first.

Model selection and routing

Chose Use eval results to route cheaper/faster models when they perform well over Use the most expensive or most capable model for every SNAP question.

Rationale: SNAP-specific evals can identify narrow task classes where latency and cost can be improved without sacrificing quality.

Context strategy

Chose Evaluate state manuals alone versus state plus federal sources over Assume all potentially relevant documents should always be retrieved.

Rationale: The team explicitly expects that too much federal policy context may confuse answers in some cases.

Public-good sharing

Chose Publish representative portions of the SNAP eval over Keep all eval cases private to Propel.

Rationale: Public tests could help all models improve on SNAP knowledge and help adjacent safety-net domains build evals.

Gotchas (what bit them)

Technically correct SNAP answers can still be practically harmful or misleading.

Root cause: SNAP rules include state-level policy options and practical navigation nuance; federal asset limits alone omit the fact that most applicants face no asset limit.

Resolution: Use domain-expert eval criteria that require both strict accuracy and practical helpfulness.

Keyword-based evals can misclassify answers that satisfy or miss a semantic requirement.

Root cause: The word eliminated or BBCE is only a proxy for the actual requirement that most states do not apply asset limits.

Resolution: Use a semantic check, potentially LLM-as-judge, for claims that require substantive meaning rather than exact keywords.

More source context can make policy answers worse rather than better.

Root cause: Federal policy mixed with state policy may confuse the model in some SNAP policy-answering setups.

Resolution: Evaluate context strategies separately instead of assuming more documents always improves answers.

Domain experts and AI researchers rarely overlap in niche public-benefits domains.

Root cause: SNAP experts are typically in government, legal aid, or benefits organizations rather than AI research labs.

Resolution: Publish representative domain evals and share methodology so models improve on under-covered high-impact domains.

Evaluation

The work itself is an evaluation design: domain-expert exploration, model comparison, SNAP-specific question/answer pairs, semantic criteria for practical helpfulness, potential LLM-as-judge grading, and experiments over models, prompts, and context documents.

Metrics: Strict factual accuracy · Practical helpfulness · State-specific nuance · Safe fallback behavior · Model comparison by topic · Cost and latency tradeoffs · Failure-case filtering

Outcomes

• The article defines a methodology rather than reporting production deployment outcomes.
• Propel had built Hydra for internal multi-model exploration.
• Propel planned to publish a version of the SNAP eval as a public good.
• The asset-limit sniff test established a reusable example of practical-helpfulness evaluation.

This is part 1 of a series; it reports methodology and examples, not production performance metrics.