Building and Scaling OG Assist: An AI Agent Platform for Government ERP Software

opengov · Government · 2026 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Government ERP AI agents

OpenGov needed to deploy a unified AI agent, OG Assist, across government ERP product suites so users could ask questions, invoke domain tools, act on product data, and receive contextual UI help while preserving safety, observability, evaluation, long-context behavior, and human control over sensitive operations.

Constraints:

• government ERP workflows span budgeting, procurement, asset management, permitting, utility billing, and other product domains
• all product teams needed a way to contribute domain-specific tools and skills behind one agent interface
• mutating operations and code/file execution require human approval and sandboxing
• long conversations overload token windows unless earlier context is summarized
• agent behavior crosses product APIs and platform capabilities, so tracing and evals are required for production diagnosis
• framework control became important as the agents team and use cases scaled

Architecture

OG Assist is a first-party agent button embedded across OpenGov product suites. A custom Effect-native agent loop, built after moving from LangGraph, coordinates language-model streaming, dependency-injected model swapping, A2A-style agent route/schema contracts, domain toolkits, product-suite tools, human approval interrupts, ephemeral sandboxes, rolling-summary memory, generative UI primitives, CI evals, user feedback, and Effect tracing. Product teams build tools and skills for their domains, while the agent loop can inspect page context, make tool calls, query suite data, highlight UI next steps, generate files in sandboxes, and render forms at runtime. OpenGov also reuses the same tools/skills pattern internally with Claude, Cursor, and Claude agents to accelerate engineering work.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  opengov-product-suites[OpenGov product suites] --> og-assist-chat-interface[OG Assist chat interface]
  opengov-product-suites --> product-suite-tools[Product suite tools]
  og-assist-chat-interface --> effect-native-agent-loop[Effect native agent loop]
  og-assist-chat-interface --> human-approval-ui[Human approval UI]
  og-assist-chat-interface --> generative-ui-primitives[Generative UI primitives]
  effect-native-agent-loop --> language-model-abstraction[Language model abstraction]
  effect-native-agent-loop --> a2a-route-schema[A2A route schema]
  effect-native-agent-loop --> toolkits-and-skills[Toolkits and skills]
  toolkits-and-skills --> product-suite-tools
  effect-native-agent-loop --> human-approval-ui
  effect-native-agent-loop --> sandbox-runner[Sandbox runner]
  effect-native-agent-loop --> rolling-summary-memory[Rolling summary memory]
  effect-native-agent-loop --> tracing-and-observability[Tracing and observability]
  effect-native-agent-loop --> evaluation-and-feedback-loop[Evaluation and feedback loop]
  toolkits-and-skills --> internal-development-agents[Internal development agents]

Components

Data flow

A user opens OG Assist from the product navigation bar. The agent receives product and page context, follows an A2A-influenced route/schema contract, and runs through the Effect-native agent loop. The loop streams text from a dependency-injected language model, registers toolkits built by product teams, and calls suite-specific tools to look up data or act on the page. If a tool requires approval, the loop deterministically interrupts and presents accept/reject UI. If code or file creation is needed, the agent spins up an ephemeral sandbox and tears it down after use. Long conversations are condensed into rolling summaries plus recent messages. Feedback thumbs and CI evals over real completions feed improvements to tools, skills, prompts, and the harness, while Effect spans and distributed traces expose bottlenecks and failures.

Why this architecture

OpenGov chose a custom Effect-native agent loop because LangGraph was adequate early on but became limiting as the team and use cases scaled. Effect gave them one foundation for schema, error handling, logging, traces, structured concurrency, and agent-loop control. A2A-style contracts aligned frontend and backend expectations, while tools and skills let product teams extend one unified assistant across domains. Human approvals, sandboxing, rolling summaries, evals, and tracing address production risks specific to government ERP: mutating actions, code/file execution, long-context recall, regression detection, and cross-service debugging.

Models used: Effect AI language model abstraction · Claude · Cursor · Claude agents

Scale

• OpenGov was founded about 14 years before the presentation.
  “we were founded about 14 years ago”transcript, stated
• OG Assist is embedded across all OpenGov products and product suites.
  “OG Assist is this little button on the top of all of our products in the in the navigation bar.”transcript, stated

  “all of our product suites and product teams have built tools and skills in order to power this button.”transcript, stated
• OG Assist is available to OpenGov government customers according to the ZenML summary.
  “OG Assist is deployed across all OpenGov product suites and available to all their government customers.”summary, stated
• The rolling-summary strategy keeps roughly the last five or ten messages plus summarized older context.
  “maybe you only want the like n minus five most recent messages or n minus 10 most recent messages”transcript, stated
• The transcript describes recall over context from about a hundred messages earlier.
  “you may want to refer to uh context earlier like a hundred messages above”transcript, stated
• The sandbox example generated a PDF for AI Engineer Conference 2026.
  “create a PDF uh for the folks of the AI Engineer Conference 2026.”transcript, stated

Alternatives considered (roads not taken)

Continue using LangGraph

Rejected because: LangGraph was fine early, but the agents team needed more control as the team scaled and use cases evolved.

Let each suite build its own isolated assistant

Rejected because: OpenGov wanted one first-party OG Assist experience powered by tools and skills from all product teams.

Always include all conversation messages

Rejected because: Long conversations overloaded context windows and legacy model token limits.

Let the agent execute sensitive actions directly

Rejected because: Mutating operations and risky tool calls need explicit user control in government ERP workflows.

Run agent code and file creation in production context

Rejected because: Code execution and file generation needed isolation from production systems.

Patterns

tool-use-orchestration (agent-runtime)

OG Assist registers toolkits and product-specific skills with the model so the agent can query suite data, highlight UI elements, create files, and act inside OpenGov products.

human-in-the-loop-gate (safety)

The agent loop deterministically interrupts for approval-required tool calls and presents accept/reject UI to the user.

privacy-preserving-processing (safety)

Code execution and file creation happen inside ephemeral isolated sandboxes so agent actions do not risk production systems.

context-compaction (memory)

OG Assist summarizes older conversation turns and keeps only recent messages to manage token limits in long conversations.

agent-memory (memory)

The agent can recall earlier conversation context from the rolling summary when a user refers back to something discussed much earlier.

observability-tracing (operations)

Effect spans feed distributed traces that show request paths, timings, bottlenecks, failures, and cross-service behavior.

eval-driven-development (quality)

OpenGov combines user thumbs feedback with CI evals over real completions to improve prompts, tools, skills, and the harness.

structured-output-contract (integration)

The A2A protocol and agent cards provide a rigorous schema/contract for frontend and backend agent routes.

model-routing (model-ops)

Dependency injection lets the Effect loop hot-swap language models without changing core loop logic.

Tradeoffs

Agent runtime

Chose custom Effect-native agent loop over continuing with LangGraph.

Rationale: The team traded framework convenience for full loop control, Effect-native tracing, structured concurrency, logging, and model hot-swap flexibility.

Architecture contract

Chose A2A-inspired protocol and agent cards over ad hoc frontend/backend contracts.

Rationale: A rigorous shared spec improved development alignment and extensibility.

Context strategy

Chose rolling summaries plus recent-message window over stuffing all latest/recent messages into context.

Rationale: Summaries preserve earlier conversation memory while limiting token growth.

Action safety

Chose human approval UI for approval-required tool calls over fully autonomous mutating operations.

Rationale: Explicit user approval builds trust and keeps humans in control of sensitive government ERP changes.

Agent extension model

Chose tools and skills owned by product teams over one monolithic assistant capability set.

Rationale: Distributed tool development lets each product suite add domain capabilities behind one shared OG Assist interface.

Gotchas (what bit them)

A generic agent framework became limiting as production use cases grew.

Root cause: LangGraph was acceptable early, but the team needed full control of the agent loop for complex features and fine-grained tracing/logging/concurrency.

Resolution: Migrate to a custom Effect-native agent loop.

Sensitive tool calls require deterministic interruption rather than model discretion.

Root cause: Mutating operations in government ERP data can have real business impact.

Resolution: Interrupt the loop for approval-required tool calls and require explicit accept/reject input.

Agent code execution can threaten production systems.

Root cause: File creation and code execution may have side effects if run in ordinary application context.

Resolution: Use on-demand ephemeral sandboxes for code, files, and generated artifacts.

Long conversations overloaded context limits.

Root cause: Including all recent messages became ineffective as conversations grew longer, especially with legacy models.

Resolution: Use rolling summarization plus a small window of recent messages and recall over the summary.

Agent failures can be difficult to debug across product-team APIs.

Root cause: Agentic systems integrate with multiple teams, APIs, and platform capabilities.

Resolution: Use Effect's automatic spans and traces to profile bottlenecks, failures, and cross-service data.

Evaluation

OpenGov evaluates OG Assist with in-product thumbs-up/thumbs-down feedback, calls and emails, automated CI evals against real completions, prompt checks for expected tool hits and intended actions, and observability traces that show request flow, timings, bottlenecks, failures, and cross-service behavior.

Metrics: thumbs-up/thumbs-down feedback · support calls and emails · CI eval pass/fail · expected tool hit · intended action completion · trace spans · component execution time · bottlenecks · cross-service failure context

Outcomes

• OG Assist is deployed as a shared agent button across OpenGov product suites
• product teams built domain-specific tools and skills behind the shared assistant
• the team migrated from LangGraph to a custom Effect-native loop for control and observability
• human approval interrupts protect sensitive tool calls
• ephemeral sandboxes isolate code execution and file generation
• rolling summarization handles long-context conversation memory
• generative UI primitives let the agent render forms at runtime
• OpenGov applies similar agent/tool patterns internally with Claude, Cursor, and Claude agents

The case describes a real production agent workload across all OpenGov product suites and government customers, with cross-team tool ownership and production controls for evals, feedback, tracing, sandboxing, and approval gates.