Improving Error Handling for AI Agents in Production

fewsats · Tech · 2025 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: API error handling for tool-using AI agents

fewsats found that AI agents using the Sherlock Domains Python SDK could get trapped in repeated failed API calls because standard HTTP exception handling exposed only generic status codes, not the response bodies that explained what to fix. Agents needed complete, agent-readable error details so they could self-correct tool calls across models and platforms.

Constraints:

• agents were making API calls through an SDK originally designed for human developers
• response.raise_for_status() raised exceptions while discarding response-body details
• HTTP APIs expose errors through diverse status-code and body conventions
• GraphQL APIs can return 200 OK while placing errors in the response body
• smaller or local models were more sensitive to missing information
• SDK interfaces needed to serve both human developers and AI agents

Architecture

fewsats changed the Sherlock Domains SDK error interface so AI agents can see complete error details, including HTTP response payloads. Instead of surfacing only HTTP Error 422: Unprocessable Entity, the SDK preserves validation details such as missing first_name and last_name fields. With that information visible, the Replit Agent immediately split a single name field into the required fields and proceeded. The article generalizes this into an SDK design principle: keep traditional structured exceptions for humans, but include verbose payload details that agents can interpret, and monitor intermediate tool-call inputs and outputs rather than only the final answer.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  AI_agent["AI agent"] --> Sherlock_Domains_SDK["Sherlock Domains SDK"]
  Claudette_Claude_wrapper["Claudette Claude wrapper"] --> Sherlock_Domains_SDK
  Local_Llama_3_agent["Local Llama 3 agent"] --> Sherlock_Domains_SDK
  Replit_Agent["Replit Agent"] --> Sherlock_Domains_SDK
  llms_txt_documentation["llms.txt documentation"] --> Replit_Agent
  Sherlock_Domains_SDK --> HTTP_API["HTTP API"]
  HTTP_API --> HTTP_response_body["HTTP response body"]
  HTTP_response_body --> HTTP_error_handler["HTTP error handler"]
  Sherlock_Domains_SDK --> HTTP_error_handler
  HTTP_error_handler --> Agent_visible_error_message["Agent-visible error message"]
  Agent_visible_error_message --> AI_agent
  HTTP_error_handler --> Human_developer_exception["Human developer exception"]
  Human_developer_exception --> Human_developer["Human developer"]
  Human_developer --> Sherlock_Domains_SDK
  Tool_call_monitoring["Tool-call monitoring"] --> AI_agent
  Tool_call_monitoring --> Sherlock_Domains_SDK
  Tool_call_monitoring --> HTTP_error_handler

Components

Data flow

An AI agent invokes a Sherlock Domains SDK method, such as setting contact information. The SDK sends an HTTP request to the domain API. If the API returns an error, the old path raised a status-code-only exception and hid the response body, causing the agent to retry blindly. The improved path raises or returns an error that includes both the original status and payload details. The agent reads the validation error, revises its tool arguments, and retries with corrected fields.

Why this architecture

The architecture keeps SDK ergonomics for humans while making tool failures legible to agents. Complete error payloads let agents repair arguments without human intervention, whereas prompts and documentation cannot compensate when the tool interface hides the actual validation error. The same principle must cover REST, minimalist errors, custom formats, 200-with-error bodies, and GraphQL errors.

Models used: Claude via Claudette · Llama 3 running locally · Replit Agent · LLM agents

Scale

• The source is a 9-minute read published on February 28, 2025.

  "9 min read

Feb 28, 2025"

— article, stated

• The Replit Agent error involved HTTP 422 Unprocessable Entity.
  “It would show only: HTTP Error 422: Unprocessable Entity”article, stated
• The article documents five classes of API error format.

  "The Many Faces of API Errors

When working with HTTP APIs, error responses come in various formats."

— article, stated

• The team tested the SDK against Claude, local Llama 3, and Replit Agent integrations.

  "Observations Across Different Models and Platforms

The case study provides interesting observations about how different AI agents handled the SDK integration"

— summary, stated

Alternatives considered (roads not taken)

Fix the agent with prompts or more documentation

Rejected because: Prompt and llms.txt tweaks did not solve the root cause: the agent could not see the error details.

Use response.raise_for_status() alone

Rejected because: It raises on 4xx/5xx but loses the response body the agent needs to fix its call.

Return plain text for all SDK responses

Rejected because: Humans still need structured data, exceptions, type hints, and debugging interfaces during this transitional period.

Assume one API error convention

Rejected because: REST, custom APIs, success-with-error bodies, and GraphQL communicate failures differently.

Patterns

tool-use-orchestration (agent-architecture)

Agents use the Sherlock Domains SDK as a tool for searching, purchasing, and configuring domains and DNS records.

fallback-degradation (resilience)

Complete error details let agents recover from failed tool calls by correcting arguments instead of looping or giving up.

observability-tracing (operations)

The article extends LLM observability to intermediate API calls and error responses, not just final inputs and outputs.

structured-output-contract (quality)

Error payloads must preserve structured validation details such as missing fields so agents can reason about corrective action.

eval-driven-development (quality)

The SDK was exercised across Claude, local Llama 3, and Replit Agent, revealing failures that stronger models hid.

Tradeoffs

Enhanced exceptions

Chose preserve structured exceptions plus detailed payloads over status-code-only raise_for_status() errors.

Rationale: Human developers keep conventional error handling while agents see enough detail to self-correct.

Verbose agent-readable errors

Chose full response-body visibility over concise human-oriented status messages.

Rationale: Verbose payloads are tedious for humans but let LLMs infer the required corrective action.

Hybrid SDK design

Chose interfaces that work for humans and agents over plain text only for AI consumption.

Rationale: Humans still build, test, debug, and collaborate with agents, so structured interfaces remain necessary today.

Cross-model integration testing

Chose test SDK behavior with Claude, local Llama 3, and Replit Agent over assuming a Claude-successful SDK is broadly reliable.

Rationale: Smaller models and different agent platforms exposed information-availability issues that stronger integrations masked.

Gotchas (what bit them)

Generic HTTP status errors make agents blind.

Root cause: Status-code-only exceptions hide response-body validation details.

Resolution: Surface complete error details, including the response body, to the agent.

Agents can enter doom loops when the real validation error is hidden.

Root cause: The Replit Agent kept trying a nonexistent name field because it could not see that first_name and last_name were missing.

Resolution: Expose the validation payload so the agent can split the field correctly.

Prompt and documentation tweaking can distract from tool-interface defects.

Root cause: The team kept modifying llms.txt under hackathon pressure even though the SDK error layer was the actual blocker.

Resolution: Debug the tool/SDK information surface before assuming the prompt is deficient.

API error formats are not uniform.

Root cause: Some APIs use status-code errors, some use custom JSON formats, some return 200 OK with errors in the body, and GraphQL uses an errors array.

Resolution: Adapt the agent-visible error layer to each API's failure conventions.

SDKs now have two audiences with different ergonomics.

Root cause: Humans prefer structured data and concise abstractions, while agents may need verbose plain-text context and full payloads.

Resolution: Use enhanced exceptions with detailed payloads as a pragmatic middle ground.

Evaluation

The article evaluates the fix qualitatively across agent integrations rather than with a formal benchmark. It reports that Claude through Claudette worked well initially, local Llama 3 struggled with chained calls, Replit Agent exposed a repeated first_name/last_name validation failure, and after surfacing complete error details the agent immediately corrected its arguments.

Metrics: agent could search domains and handle purchase flow with Claude wrapper · Llama 3 struggled with chaining multiple API calls · Replit Agent repeatedly used a wrong name field · HTTP 422 response body identified missing first_name and last_name fields · agent immediately adapted after complete error details were surfaced

Outcomes

• The Replit Agent stopped blindly retrying and corrected the missing first_name/last_name issue after seeing the full response payload.
• The team identified error visibility as more important than prompt/documentation tweaks for this failure mode.
• The case generalizes into a production SDK guideline: surface complete error payloads to tool-using agents.
• The article frames enhanced exceptions as a practical transition design for both human and AI consumers.
• The source predicts future AI-only interfaces may favor plain text and natural-language format negotiation.

This is a small but high-signal agent-integration case. It should not be overread as a broad benchmark, but the gotcha is extremely reusable: if agents cannot see full tool errors, they cannot self-correct.