AI-Powered SNAP Benefits Notice Interpretation System

propel · Government · 2025 · source Agentic: no · promotional bias: med · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Public benefits notice interpretation

Propel wanted to help SNAP recipients understand confusing official notices, distinguish informational letters from urgent benefit-risk notices, and identify concrete next steps without exposing vulnerable users to unsafe or overconfident AI advice.

Constraints:

• SNAP notices carry legal requirements and can affect denials, reductions, overpayments, and appeal rights
• users face cognitive load from dense boilerplate, legal language, and hidden action items
• the tool was still alpha-stage and not ready for direct deployment to recipients
• outputs needed to stay grounded in the notice to reduce hallucination risk
• notices may contain PII and sensitive benefit information sent to external model APIs

Architecture

Propel prototyped a notice helper that takes a real SNAP notice, optionally a user question, and asks an LLM to produce two user-facing outputs: importance level and required action. The prototype used Claude 3.5 Sonnet behind a Streamlit interface while the team compared models, prompts, designs, and external context on real notices. Safety work centered on expert feedback before deployment, keeping answers grounded in the notice, possible response-checking chains, PII redaction, and edge-case handling for critical boilerplate or incorrect notices.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  notice-input["notice-input"] --> streamlit-prototype["streamlit-prototype"]
  optional-user-question["optional-user-question"] --> streamlit-prototype["streamlit-prototype"]
  pii-redaction-layer["pii-redaction-layer"] --> streamlit-prototype["streamlit-prototype"]
  streamlit-prototype["streamlit-prototype"] --> snap-specialized-prompt["snap-specialized-prompt"]
  streamlit-prototype["streamlit-prototype"] --> program-expert-review["program-expert-review"]
  snap-specialized-prompt["snap-specialized-prompt"] --> claude-3-5-sonnet["claude-3-5-sonnet"]
  snap-specialized-prompt["snap-specialized-prompt"] --> response-grounding-check["response-grounding-check"]
  response-grounding-check["response-grounding-check"] --> notice-input["notice-input"]
  program-expert-review["program-expert-review"] --> snap-specialized-prompt["snap-specialized-prompt"]

Components

Data flow

A SNAP recipient or program expert supplies a notice, and sometimes a specific question about it. The prototype passes that notice and question to a SNAP-specialized prompt that asks the LLM to classify importance and explain what action the recipient should take in simple language. The response is inspected against real notice examples during alpha testing. Before user rollout, program experts and safety checks are meant to harden the output, with possible chained model verification that information in the answer appears in the original notice.

"* The model used here was Anthropic’s Claude 3.5 Sonnet (newest)

• The prompt used here is the following:
  • “You are a legal aid attorney specializing in SNAP benefits."

Why this architecture

Propel chose a narrow notice-interpretation workflow because the hardest user problem is language friction in documents the user already has, making LLM summarization safer than open-ended benefits advice. The two-output prompt narrows the product to importance and action. Expert review and possible chained grounding checks reflect the risk level of SNAP benefits and the team's decision not to deploy directly to users until the tool is hardened.

Models used: Anthropic Claude 3.5 Sonnet · Streamlit · Microsoft Presidio · ChatGPT · Gemini

Scale

• About 50% of surveyed Propel users say they call the SNAP agency when they have a question.
  “about 50% of people say they call when they have a question”article, stated
• The prototype prompt asks for two core output sections: importance and action.
  “Say high importance, medium, or low. And say why. And 2. What the user needs should do, if anything.”article, stated
• The example notice required finding an action in the third paragraph and the phone number in the top right.
  “The 3rd paragraph has an action (call the phone number) but the number itself is not included, but rather at the top right”article, stated
• The team is inviting alpha access for expert users who help SNAP clients with notices.
  “We can also provide access to our alpha tool so you can put in notices you may have and test for scenarios you have.”article, stated

Alternatives considered (roads not taken)

Open-ended chatbot for benefits navigation

Rejected because: Propel viewed notice processing as safer because the answer can be constrained to information already included in the notice.

Deploying the prototype directly to SNAP recipients in its current state

Rejected because: The team is explicitly keeping the system in expert-feedback hardening before user testing.

Only summarizing notices without user-specific questions

Rejected because: The team found specific user questions important, including cases where an off-ramp to other help may be needed.

Patterns

structured-output-contract (prompting)

The prompt constrains the model to a small, repeatable output contract: importance level plus specific user action in simple language.

eval-driven-development (evaluation)

Propel treats real notices, model/prompt variants, and expert feedback as the evaluation loop before exposing the tool to SNAP recipients.

guardrails-io-filtering (safety)

The planned safety layer narrows generated content to the original notice, considers response checking, and explores PII redaction before using external model APIs.

Tradeoffs

Constrain the first AI experience to notice interpretation.

Chose Document-grounded importance and action extraction over Open-ended benefits advice.

Rationale: The team judged notice interpretation safer because most content can be anchored in a document the user already received.

Hold the alpha back from direct recipient deployment.

Chose Expert hardening and real-example testing over Immediate public rollout.

Rationale: SNAP users could face severe consequences from AI failures, so the team prioritized risk discovery first.

Balance concise answers against complete notice coverage.

Chose Highlight the most important information over Repeating all legally required notice content.

Rationale: The UX value comes from reducing overload, but too much filtering can omit information that is critical in edge cases.

Gotchas (what bit them)

Filtering can hide rare but important boilerplate.

Root cause: The product goal is to reduce notice overload, but some low-frequency legal information can matter critically to a particular recipient.

Resolution: Consider reminding users to read the full notice and offering proactive follow-ups for less common edge cases.

External LLM APIs create PII handling risk for benefit notices.

Root cause: SNAP notices can include personally identifying and sensitive information, and the prototype may send notice content to external providers.

Resolution: Evaluate redaction or no-persistence strategies, including local deidentification models such as Presidio.

Grounding answers to a notice does not fix an incorrect notice.

Root cause: If the original government notice is wrong, faithfully explaining it may still produce adverse outcomes.

Resolution: Add checks or supplemental policy information for highly consequential situations where a notice may be erroneous.

Evaluation

Alpha-stage evaluation using real notices and real social-media questions, comparisons across models/prompts/designs/external context, SNAP expert feedback, and planned recipient testing after hardening.

Metrics: importance classification quality · action clarity · grounding of response content in the original notice · expert feedback before user deployment · ability to avoid overwhelming users

Outcomes

• The prototype produced clearer importance/action guidance for example notices than raw notices alone.
• Propel identified a safety plan focused on expert review, response grounding, PII handling, and incorrect-notice risks.
• The tool remained alpha-stage rather than deployed directly to SNAP recipients.

The article reports alpha testing and expert feedback, not production usage. It cites a user survey where about half of respondents call the SNAP agency when they have questions.