Production AI Agents for Accounting Automation: Engineering Process Daemons at Scale

digits · Finance · 2025 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Accounting automation agents in production

Digits needed to run LLM-based agents for accounting workflows in production without buying into agent hype, excessive framework dependencies, weak observability, duplicated security logic, or unbounded autonomy in financial workflows.

Constraints:

Architecture

Digits reframes agents as process daemons: background processes with objectives, an LLM, tools, and a response, surrounded by production infrastructure. They run agents for vendor enrichment, client onboarding, and complex user questions; implement their own lightweight agent loop; generate curated tool schemas from existing Go handlers; reuse existing APIs and permissions; trace decisions with OpenTelemetry; expose memory as a tool; use a separate LLM for guardrail assessment; and feed user feedback into reward functions and future fine-tuning.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  process-daemon-objective["Process daemon objective"] --> core-agent-loop["Core agent loop"]
  core-agent-loop["Core agent loop"] --> process-daemon-objective["Process daemon objective"]
  core-agent-loop["Core agent loop"] --> llm-reasoning-model["LLM reasoning model"]
  llm-reasoning-model["LLM reasoning model"] --> core-agent-loop["Core agent loop"]
  core-agent-loop["Core agent loop"] --> curated-api-tools["Curated API tools"]
  curated-api-tools["Curated API tools"] --> core-agent-loop["Core agent loop"]
  go-reflection-schema-generator["Go reflection schema generator"] --> curated-api-tools["Curated API tools"]
  curated-api-tools["Curated API tools"] --> go-reflection-schema-generator["Go reflection schema generator"]
  core-agent-loop["Core agent loop"] --> response-surface["Response surface"]
  response-surface["Response surface"] --> core-agent-loop["Core agent loop"]
  core-agent-loop["Core agent loop"] --> opentelemetry-tracing["OpenTelemetry tracing"]
  opentelemetry-tracing["OpenTelemetry tracing"] --> core-agent-loop["Core agent loop"]
  core-agent-loop["Core agent loop"] --> memory-tool["Memory tool"]
  memory-tool["Memory tool"] --> core-agent-loop["Core agent loop"]
  core-agent-loop["Core agent loop"] --> guardrail-evaluator["Guardrail evaluator"]
  guardrail-evaluator["Guardrail evaluator"] --> core-agent-loop["Core agent loop"]
  response-surface["Response surface"] --> feedback-loop["Feedback loop"]
  feedback-loop["Feedback loop"] --> response-surface["Response surface"]
Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component Role Interacts with
Process daemon objective Task definition for vendor enrichment, onboarding, or complex accounting questions. Core agent loop
Core agent loop Small custom loop combining objective, LLM, tools, and response. Process daemon objective, LLM reasoning model, Curated API tools, Response surface, OpenTelemetry tracing, Memory tool, Guardrail evaluator
LLM reasoning model Language model that plans, reasons, and decides tool use for the process daemon. Core agent loop
Go reflection schema generator Introspects function handlers and generates JSON schemas for tool inputs and outputs. Curated API tools
Curated API tools Existing Digits APIs exposed as selected agent tools while inheriting access rights. Core agent loop, Go reflection schema generator
OpenTelemetry tracing Decision traceability integrated into Digits' broader monitoring stack. Core agent loop
Memory tool Provider-agnostic memory capability combining semantic search with relational or graph context. Core agent loop
Guardrail evaluator Separate LLM or guardrail framework that evaluates outputs from the primary model. Core agent loop
Response surface Output delivered back to users or systems for the accounting workflow. Core agent loop, Feedback loop
Feedback loop User feedback, reward functions, and fine-tuning exploration for continuous improvement. Response surface

Data flow

A process daemon receives an objective from an accounting workflow, asks an LLM to reason about it, invokes curated tools generated from existing APIs, and returns a response to a user or system. Tool schemas are generated from Go function handlers while preserving existing access controls. Execution emits OpenTelemetry traces and prompt-comparison data. Memory is invoked as a tool when context across conversations is needed. A separate guardrail model evaluates responses, and user feedback is captured for reward functions and future agent-specific model improvement.

3 supporting quotes
“At its core, it's about 100 lines of code that combines four elements:”article, stated
“Our solution leverages Go's reflection capabilities to dynamically introspect function handlers and generate JSON schemas for inputs and outputs.”article, stated
“We use OpenTelemetry to integrate with our existing traces, making agent observability part of our broader system monitoring.”article, stated

Why this architecture

Digits chose custom lightweight infrastructure because the valuable part of production agents was not a framework abstraction but reliable control around existing APIs, permissions, traces, guardrails, and feedback. The process-daemon framing sets expectations that these are supervised background systems, not magical autonomous assistants.

3 supporting quotes
“Implement the core agent loop ourselves. It's more work upfront, but it gives us control, reduces dependencies, and ensures production readiness.”article, stated
“Production environments already have APIs. Rather than manually defining agent tools (too time-consuming) or automatically exposing all RPCs (too noisy), we wanted a curated approach.”article, stated
“The future of AI in production isn't about magical agents. It's about well-engineered process daemons that reliably execute specific tasks”article, stated

Models used: LLMs with native tool calling · reasoning models · separate LLM guardrail evaluators · agent-specific fine-tuned models

Scale

Alternatives considered (roads not taken)

Use dependency-heavy agent frameworks such as LangChain or CrewAI as-is

Rejected because: Digits found these frameworks too complex and dependency-heavy for production environments.

“We evaluated numerous agent frameworks—LangChain, CrewAI, and others—but found them often too complex with too many dependencies for production environments.”article, stated

Manually define every agent tool

Rejected because: Manual tool definition was too time-consuming and hard to maintain.

“Rather than manually defining agent tools (too time-consuming) or automatically exposing all RPCs (too noisy), we wanted a curated approach.”article, stated

Automatically expose all RPCs as tools

Rejected because: Exposing all RPCs creates too much noise for the LLM and risks uncontrolled tool surfaces.

“automatically exposing all RPCs (too noisy)”article, stated

Use provider-specific memory solutions

Rejected because: Digits implements memory as a tool to avoid vendor lock-in and keep flexibility as memory requirements evolve.

“We implement memory as a tool rather than relying on provider-specific memory solutions.”article, stated
“This avoids vendor lock-in and gives us flexibility as the landscape evolves.”article, stated

Trust the primary LLM to police itself

Rejected because: Digits uses a different LLM to evaluate responses and says never to trust a single model to police itself.

“Use a different LLM to evaluate responses.”article, stated
“The key principle: never trust a single model to police itself.”article, stated

Patterns

tool-use-orchestration (agent-architecture)

Digits exposes curated existing APIs as agent tools, generating JSON schemas from Go handlers while preserving access controls.

“Our solution leverages Go's reflection capabilities to dynamically introspect function handlers and generate JSON schemas for inputs and outputs.”article, stated
“Crucially, the existing APIs already handle access rights, so we're not creating new security concerns.”article, stated

observability-tracing (operations)

Digits integrates agent traces into its existing OpenTelemetry monitoring stack for lightweight decision traceability.

“### Observability is non-negotiable”article, stated
“We use OpenTelemetry to integrate with our existing traces, making agent observability part of our broader system monitoring.”article, stated

agent-memory (context-management)

Digits treats memory as a tool that combines semantic search with relational or graph databases for cross-conversation context.

“True agent memory combines semantic search with relational or graph databases, establishing context across conversations”article, stated
“We implement memory as a tool rather than relying on provider-specific memory solutions. This avoids vendor lock-in and gives us flexibility as the landscape evolves.”article, stated

llm-as-judge (quality)

The system uses a different LLM to evaluate responses from the primary model for guardrail checks.

“Use a different LLM to evaluate responses. Simple guardrails can be implemented via LLM assessment, while more complex scenarios benefit from specialized guardrail frameworks. The key principle: never trust a single model to police itself.”article, stated
“Use a different LLM to evaluate responses. Simple guardrails can be implemented via LLM assessment, while more complex scenarios benefit from specialized guardrail frameworks. The key principle: never trust a single model to police itself.”article, stated

safety-policy-enforcement (safety)

Digits layers guardrails, prompt-injection vigilance, permissions, and team notifications around agents used in financial workflows.

“### Guardrails: don't trust your LLM”article, stated
“Stay vigilant about prompt injection attacks. And perhaps most importantly, set realistic expectations—both internally and with users.”article, stated

eval-driven-development (quality)

Digits compares prompts, captures user feedback, designs reward functions, and explores reinforcement learning for agent-specific improvement.

“We use OpenTelemetry to integrate with our existing traces, making agent observability part of our broader system monitoring. Prompt comparison capabilities help us understand when changes improve or degrade performance.”article, stated
“We capture user feedback about agent responses, design reward functions, and explore reinforcement learning to fine-tune agent-specific models.”article, stated

structured-output-contract (tooling)

Tool inputs and outputs are generated as JSON schemas from Go handlers, creating explicit contracts for agent tool invocation.

“Our solution leverages Go's reflection capabilities to dynamically introspect function handlers and generate JSON schemas for inputs and outputs. This gives us the best of both worlds: automated tool generation with curation.”article, stated

Tradeoffs

Build the core agent loop in-house.

Chose Custom lightweight core loop over Dependency-heavy frameworks.

Rationale: The custom loop required more upfront work but provided control, fewer dependencies, and production readiness.

“Our approach: Implement the core agent loop ourselves. It's more work upfront, but it gives us control, reduces dependencies, and ensures production readiness.”article, stated

Reuse existing APIs as agent tools.

Chose Curated tool generation from production APIs over Parallel agent-specific interfaces or exposing all RPCs.

Rationale: Existing APIs already encode business logic and access rights; curation avoids noisy tool surfaces.

“Production environments already have APIs. Rather than manually defining agent tools (too time-consuming) or automatically exposing all RPCs (too noisy), we wanted a curated approach.”article, stated
“Crucially, the existing APIs already handle access rights, so we're not creating new security concerns.”article, stated

Make memory provider-agnostic.

Chose Memory as an agent tool over Provider-specific memory solutions.

Rationale: This avoids vendor lock-in and keeps memory implementations swappable as the landscape changes.

“We implement memory as a tool rather than relying on provider-specific memory solutions. This avoids vendor lock-in and gives us flexibility as the landscape evolves.”article, stated

Add a separate model for guardrails.

Chose Different LLM evaluating the primary model's responses over Single-model self-policing.

Rationale: Independent assessment reduces reliance on the same model that generated the response.

“Use a different LLM to evaluate responses. Simple guardrails can be implemented via LLM assessment, while more complex scenarios benefit from specialized guardrail frameworks. The key principle: never trust a single model to police itself.”article, stated

Let applications drive infrastructure.

Chose Infrastructure built from real deployed use cases over Abstract platform work ahead of application needs.

Rationale: Digits treats production application requirements as the guide for which infrastructure to build.

“Let applications drive infrastructure - Don't build infrastructure in a vacuum. Let real application needs guide your architectural decisions.”article, stated
“Let applications drive infrastructure - Don't build infrastructure in a vacuum. Let real application needs guide your architectural decisions.”article, stated

Gotchas (what bit them)

The term agent created unrealistic autonomy expectations.

Root cause: The word suggested autonomous assistants handling everything independently, while production systems need bounded background processes.

Resolution: Digits reframed agents as process daemons with oversight, observability, and specific tasks.

“Let's address the elephant in the room: the term "agent" sets the wrong expectations. It conjures images of autonomous assistants bringing you martinis and handling everything independently. The reality is far more nuanced—and far less glamorous.”article, stated
“That's why, we think they should be called Process Daemons — a term that better captures what they actually are: background processes that execute tasks with oversight and observability.”article, stated

Framework convenience threatened production control.

Root cause: Agent frameworks carried complexity and dependency chains that could become reliability or security risks.

Resolution: Digits wrote the core loop themselves and scrutinized frameworks for production readiness.

“All major LLM providers now offer models with tool-calling capabilities, and open source alternatives are increasingly viable. We evaluated numerous agent frameworks—LangChain, CrewAI, and others—but found them often too complex with too many dependencies for production environments.”article, stated
“Our approach: Implement the core agent loop ourselves. It's more work upfront, but it gives us control, reduces dependencies, and ensures production readiness.”article, stated

Automatically exposing every RPC would overwhelm the model and increase risk.

Root cause: The full internal API surface is noisy and not all endpoints are appropriate agent tools.

Resolution: Use Go reflection for automated schema generation, but keep curation and existing API access controls.

“Production environments already have APIs. Rather than manually defining agent tools (too time-consuming) or automatically exposing all RPCs (too noisy), we wanted a curated approach.”article, stated
“This gives us the best of both worlds: automated tool generation with curation. Crucially, the existing APIs already handle access rights, so we're not creating new security concerns.”article, stated

Agent storage is often mistaken for memory.

Root cause: Simple persistence does not provide semantic/relational/graph context across conversations.

Resolution: Digits models memory as a tool combining semantic search with relational or graph databases.

“Here's a critical distinction: storage isn't memory. True agent memory combines semantic search with relational or graph databases, establishing context across conversations in a fundamentally different way than simple data persistence.”article, stated
“Here's a critical distinction: storage isn't memory. True agent memory combines semantic search with relational or graph databases, establishing context across conversations in a fundamentally different way than simple data persistence.”article, stated

Debugging nondeterministic agents is nearly impossible without traces.

Root cause: LLM responses cannot be replayed deterministically, so failures need forensic traceability.

Resolution: Use OpenTelemetry and prompt comparison as part of the broader monitoring stack.

“Understanding what happens under the hood isn't optional in production. We needed lightweight decision traceability to debug issues and understand agent behavior.”article, stated
“non-deterministic nature of LLM responses means you cannot simply replay requests and expect identical outcomes.”summary, stated

Evaluation

Digits evaluates production agent systems through observability traces, prompt comparison, user feedback, reward functions, guardrail assessments, and real application outcomes across vendor enrichment, onboarding, and complex question-answering use cases.

Metrics: Over two years of production agent operation · Three primary deployed use cases · Prompt comparisons showing improved or degraded performance · User feedback about agent responses · Guardrail evaluations by a separate LLM

3 supporting quotes
“We use OpenTelemetry to integrate with our existing traces, making agent observability part of our broader system monitoring. Prompt comparison capabilities help us understand when changes improve or degrade performance.”article, stated
“The work doesn't stop at deployment. We capture user feedback about agent responses, design reward functions, and explore reinforcement learning to fine-tune agent-specific models. Each iteration makes our agents more effective at their specific tasks.”article, stated
“Use a different LLM to evaluate responses. Simple guardrails can be implemented via LLM assessment, while more complex scenarios benefit from specialized guardrail frameworks. The key principle: never trust a single model to police itself.”article, stated

Outcomes

Digits reports over two years of production operation, three primary accounting use cases, an approximately 100-line core loop, and an engineering team with over a decade of collective ML deployment experience.

4 supporting quotes
“At Digits, we've been running AI agents in production for over 2 years at this point, and we want to share what we've learned—starting with why we should probably stop calling them "agents" altogether.”article, stated
“At Digits, we've deployed agents for three primary use cases:”article, stated
“Focus on the fundamentals: clear objectives, appropriate tooling, comprehensive observability, and robust guardrails.”article, stated
“well-engineered process daemons that reliably execute specific tasks with appropriate oversight and permissions.”article, stated