Building Production AI Agents for Enterprise HR, IT, and Finance Platform

rippling · HR · 2025 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Enterprise HR, IT, finance, and payroll AI platform

Rippling needed to embed AI agents into a sensitive enterprise system of record covering HR, payroll, IT, identity, devices, finance, and travel while enabling distributed product teams to ship agent use cases without rebuilding the AI stack or violating customer permissions.

Constraints:

• Agents operate over sensitive employee, payroll, device, identity, benefits, and finance data.
• Enterprise permissions are complex and must carry through agent behavior.
• Demo data gives misleading signals; production agents need realistic production snapshots and internal dogfooding.
• Traditional software teams must adapt to probabilistic ML experimentation where some agent experiments fail.
• Rigid domain routers fail when humans phrase equivalent questions differently.
• Potentially hundreds of tools across HR, payroll, IT, finance, time, and benefits cannot all be exposed in a flat initial context.

Architecture

Rippling built a shared AI foundation for product teams: a Databricks-backed data layer, an agent layer developed with LangChain/LangGraph/deep agents, evaluation systems, LangSmith tracing, enterprise model access, responsible-AI review, and paved-path primitives for prototype-to-production. Product teams explore during hack weeks and co-build production candidates with the AI team. Agents increasingly use a deep-agent paradigm: LLMs receive rich context, planning steps, and domain tool sets, while deterministic workflows for critical actions are encapsulated as tools. Domain subgraphs progressively disclose scoped tools so payroll, IT, or finance agents do not overload the context window. Product-facing agents inherit the invoking user's permissions, and external AI pilots must meet zero-retention/no-training standards.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  rippling-product-suite["Rippling product suite"] --> permission-aware-product-agent["Permission-aware product agent"]
  rippling-product-suite --> dogfooding-users["Dogfooding users"]
  shared-ai-foundation["Shared AI foundation"] --> databricks-data-layer["Databricks data layer"]
  shared-ai-foundation --> langchain-agent-layer["LangChain agent layer"]
  shared-ai-foundation --> evaluation-system["Evaluation system"]
  shared-ai-foundation --> enterprise-model-access["Enterprise model access"]
  shared-ai-foundation --> responsible-ai-review["Responsible AI review"]
  databricks-data-layer --> permission-aware-product-agent
  langchain-agent-layer --> domain-subgraphs["Domain subgraphs"]
  langchain-agent-layer --> permission-aware-product-agent
  langchain-agent-layer --> langsmith-tracing["LangSmith tracing"]
  domain-subgraphs --> deterministic-action-tools["Deterministic action tools"]
  deterministic-action-tools --> permission-aware-product-agent
  permission-aware-product-agent --> langsmith-tracing
  langsmith-tracing --> evaluation-system
  evaluation-system --> dogfooding-users["Dogfooding users"]
  ai-enablement-program["AI enablement program"] --> shared-ai-foundation

Components

Data flow

A product team builds or prototypes an AI feature using shared foundation primitives. The agent receives customer/user context, production-like data snapshots, and a scoped tool or subgraph set. It reasons over ambiguous HR/IT/payroll/finance questions, invokes deterministic tools or workflows for actions, and returns answers inside Rippling's product permission boundary. LangSmith traces and user feedback show where routing, graph execution, formatting, or model behavior fails. Production candidates are dogfooded internally, reviewed through eval systems and responsible-AI practices, and generalized into reusable primitives for the next team.

Why this architecture

Rippling's architecture reflects both organizational and product constraints: many independent teams need to innovate, but sensitive enterprise workflows require shared primitives, real production validation, permission inheritance, tracing, and responsible-AI controls. The technical agent design moved away from rigid deterministic routing because user language crosses domain boundaries, while retaining deterministic tools for transactional actions where sequence and accuracy matter.

Models used: OpenAI models · Anthropic models · Google models · LangChain/LangGraph/deep agents · Cursor · BugBot · Codex/Claude Code-style coding tools

Scale

• Rippling's suite spans HR, payroll, benefits, IT identity/device management, corporate card, travel, and expense.
  “HR payroll benefits on one side with IT uh IT solutions with identity and device management and then on the finance side with corporate card travel expense.”transcript, stated
• The platform serves companies with about 5,000 to 6,000 employees on the suite.
  “larger companies like Android which are running almost 5,000 6,000 employee on entire platform”transcript, stated
• Rippling runs AI hack weeks every six months.
  “we typically 6 months we do hack week.”transcript, stated
• The last hack week had 150 AI projects, with about 50 reaching completion.
  “The last hack week we had had 150 projects with AI out of which I think 50 of them reached completion.”transcript, stated
• Rippling has about 150 product-team founders running teams independently.
  “we have almost 150 founders at rippling today running different product teams.”transcript, stated
• Anker had been at Rippling for about seven months during the discussion.
  “since the time I've been now repling at 7 months”transcript, stated
• The deterministic-to-deep-agent iteration happened over roughly three months, with the deep-agent exploration starting about a month before the talk.
  “last 3 months we've gone through multiple iterations”transcript, stated

  “last month or so uh we really started exploring with deep agent paradigm”transcript, stated
• The tool-context discussion cites 10 to 15 tools as a typical threshold before teams should consider progressive disclosure.
  “we've typically seen like order 10 to 15 is like a good to kind of like, you know, if you start to add more than that, you should maybe start to look at it.”transcript, stated
• Progressive-disclosure examples mention theoretical access to about 300 tools while loading only needed tools.
  “have access in theory to like 300 tools but you just look at the ones that that you need at given point in time.”transcript, stated

Alternatives considered (roads not taken)

Let product teams build AI infrastructure independently.

Rejected because: Rippling's founder-first structure needed shared primitives and a paved path so distributed teams could move fast without rebuilding data, agent, eval, and model infrastructure.

Validate agents primarily on demo instances.

Rejected because: The team found production data snapshots were necessary to see real data variability and validate agent behavior.

Use deterministic domain routers for ambiguous enterprise questions.

Rejected because: Human phrasing crosses domain boundaries, making rigid routers unreliable for queries that mean the same thing in different words.

Model every agent as a deterministic workflow.

Rejected because: Real-world situations produce unknown cases, while LLMs can reason, plan, and handle edge cases when given enough context and tools.

Allow the agent to access every enterprise tool at once.

Rejected because: Too many tools overload context, so Rippling scopes tools into domain subgraphs and tests those subgraphs independently.

Patterns

tool-use-orchestration (agent_tools)

Rippling scopes domain tools into subgraphs and treats deterministic workflows as tools that an LLM can invoke when action accuracy and sequencing matter.

eval-driven-development (evaluation)

Evaluation systems, controlled tests, human feedback, and production traces are part of Rippling's shared AI foundation and prototype-to-production loop.

observability-tracing (operations)

Teams inspect LangSmith traces from controlled tests and production behavior to understand LLM calls, routing, graph execution, and output failures.

safety-policy-enforcement (safety)

Product agents run inside the existing product security boundary and inherit the invoking user's permissions.

privacy-preserving-processing (safety)

External AI pilots and partners must satisfy Rippling's responsible-AI checklist, including zero data retention and no training on customer data.

human-in-the-loop-gate (quality)

Rippling keeps humans accountable for agent outputs, from AE validation of sales briefings to engineer ownership of AI-generated code.

llm-as-judge (evaluation)

The discussion frames eval as the mechanism for deciding how many tools a subagent can handle and for comparing agent behavior, though detailed judge metrics are not disclosed.

Tradeoffs

Build a paved-path AI foundation for distributed teams.

Chose Central primitives plus AI-team co-building for production priorities over Purely decentralized AI infrastructure.

Rationale: This lets 150 independent product-team founders innovate while the AI team captures missing primitives from each production deployment.

Dogfood agents internally before broader rollout.

Chose Internal alpha testing and fast Slack/user feedback over Relying only on external user research or synthetic evaluation.

Rationale: Rippling's own employees and super-admins can provide immediate real-world feedback on product fit and failure modes.

Lean into deep-agent reasoning for ambiguous questions.

Chose LLM planning with tools and trajectories over Fully deterministic routers and workflows.

Rationale: Ambiguous enterprise questions and unknown real-world cases need reasoning and judgment, but action execution still uses deterministic tools where needed.

Treat agents as extensions of the invoking user.

Chose Permission inheritance inside the product over Privileged agent access divorced from user permissions.

Rationale: This preserves customer trust and existing enterprise authorization boundaries for sensitive HR/payroll/finance data.

Frame internal AI adoption as productivity rather than efficiency.

Chose Automating parts of work so teams can do more over Positioning AI as reducing headcount or replacing work.

Rationale: This reduces fear, encourages adoption, and lets teams look for parts of their day job to automate safely.

Gotchas (what bit them)

Software engineers can misread failed AI experiments as fixable bugs.

Root cause: Traditional software development expects deterministic fixes, while ML/LLM experimentation sometimes fails and should be abandoned.

Resolution: Teach teams to run experiments, measure outcomes, and move on when a use case does not work.

Demo data hides real production complexity.

Root cause: Agents behave differently on real enterprise data, edge cases, and user behavior than on simple demo instances.

Resolution: Curate production snapshots and production instances for AI innovation and validation.

Transcript summarization is easy; role-relevant summaries require iteration.

Root cause: A generic transcript summary does not necessarily surface the signals an interviewer needs for a specific role or level.

Resolution: Iterate on context and prompting so summaries highlight role-specific evaluation signals.

Rigid domain routing breaks on ordinary language variation.

Root cause: Users can ask the same business question with different words that cross deterministic domain boundaries.

Resolution: Give LLMs rich context, tools, and planning steps instead of forcing every query through fixed routers.

Action-taking agents still need deterministic sequences.

Root cause: Read-only answers tolerate more flexibility, but actions require transactionality and ordered data capture.

Resolution: Encapsulate deterministic action sequences as tools that the LLM can invoke when it chooses an action.

Tool overload degrades context management.

Root cause: Enterprise agents may span devices, payroll, time, benefits, email, and many more tools, and flat tool exposure overloads the context.

Resolution: Use domain subgraphs and progressive disclosure so agents load or see only the relevant tool set.

AI-generated code can create review overload.

Root cause: Coding tools make it easy to generate large PRs faster than humans can review them.

Resolution: Keep engineers accountable, add AI code-review feedback loops, manage coding context, and consider guardrails such as PR-size limits.

Evaluation

Rippling evaluates agents through curated production snapshots, internal dogfooding, thumbs-up/down and stakeholder feedback, LangSmith trace review, controlled internal tests generating many traces, human feedback on desired outputs, eval systems in the shared foundation, and responsible-AI gates for external tools.

Metrics: Production snapshot behavior · Internal dogfooding feedback · Thumbs up/down on output quality · LangSmith trace success/failure patterns · Routing and graph behavior · Human feedback on desired outputs · Date/time formatting correctness by user locale/timezone · Responsible AI checklist pass/fail · Code-review feedback and PR quality

Outcomes

• Rippling established a shared AI foundation with data, agent, eval, model-access, and tracing primitives.
• The company moved from summarization and standalone AI products toward agents acting as system analysts for admins and end users.
• AI hack weeks produced 150 AI projects with about 50 completed in the cited cycle.
• The agent architecture shifted from deterministic domain routers toward deep-agent reasoning with domain subgraphs and deterministic tools for actions.
• Product agents preserve user permission boundaries, while external AI pilots must meet responsible-AI requirements.

The source reports an HR/IT/finance suite used by companies with 5,000-6,000 employees, biannual hack weeks, 150 AI hack-week projects with 50 completed, almost 150 internal product founders, seven months of AI-team buildout, three months of architecture iteration, a tool-count heuristic of about 10-15 before progressive disclosure, and theoretical access to around 300 tools.