Building Production AI Agents and Agentic Platforms at Scale

vercel · Tech · 2025 · source Agentic: yes · promotional bias: high · source-backed · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Production AI agent platforms

AWS and Vercel frame the production problem as a platform problem: AI POCs fail when teams bolt model calls onto traditional software lifecycles without model-switching, evaluation, observability, governance, orchestration, and durable agent execution.

Constraints:

• AI platforms must keep up with rapid model churn, provider-specific APIs, and regional model availability differences.
• Teams need model-specific quality, cost, latency, and safety evaluation rather than generic benchmarks alone.
• Production agents need end-to-end tracing across LLM calls, tools, RAG systems, costs, latency, tokens, and errors.
• Governance must apply consistently across heterogeneous models and providers.
• Agent workflows can run longer than serverless request timeouts once they use browsers, code interpreters, or other tools.
• The same platform has to support customer-facing apps, code-review agents, internal business agents, and analytics bots.

Architecture

The architecture combines Amazon Bedrock platform primitives with Vercel's agent application stack. Bedrock supplies the model catalog, Converse API for a unified model interface, Strands Agent SDK for model-agnostic agents, Guardrails for input/output governance, AgentCore Evaluations for live and custom model/agent scoring, and CloudWatch/AgentCore Observability for traces and metrics. Vercel builds applications on top with the Vercel AI SDK for provider-agnostic tool-calling agents, AI Gateway as a model-call control plane, and Workflow Development Kit to compile TypeScript functions into durable workflows. Customer-facing products such as v0, Vercel Agent, and internal agents reuse the same Bedrock-backed platform.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  bedrock-model-catalog["Bedrock model catalog"] --> converse-api["Converse API"]
  bedrock-model-catalog --> strands-agent-sdk["Strands Agent SDK"]
  bedrock-model-catalog --> vercel-ai-sdk["Vercel AI SDK"]
  converse-api --> bedrock-guardrails["Bedrock Guardrails"]
  bedrock-guardrails --> cloudwatch-and-agentcore-observability["CloudWatch and AgentCore Observability"]
  strands-agent-sdk --> agentcore-tools["AgentCore tools"]
  vercel-ai-sdk --> ai-gateway["AI Gateway"]
  vercel-ai-sdk --> agentcore-tools
  vercel-ai-sdk --> workflow-development-kit["Workflow Development Kit"]
  ai-gateway --> bedrock-model-catalog
  ai-gateway --> cloudwatch-and-agentcore-observability
  workflow-development-kit --> durable-agent["Durable Agent"]
  workflow-development-kit --> agentcore-tools
  durable-agent --> agentcore-tools
  durable-agent --> vercel-application-agents["Vercel application agents"]
  vercel-application-agents --> internal-business-agents["Internal business agents"]
  agentcore-evaluations["AgentCore Evaluations"] --> cloudwatch-and-agentcore-observability

Components

Data flow

A Vercel or customer agent is implemented with the Vercel AI SDK and routed through AI Gateway to Bedrock models. Bedrock-facing calls can use unified Converse-style interfaces, model-agnostic agent SDKs, and Guardrails at the input and output boundaries. When the agent needs tools, it can call AgentCore browser or code-interpreter tools through AI-SDK-compatible interfaces. For long-running work, WDK wraps the agent in a workflow with durable queue-backed steps. Evaluation logs and OpenTelemetry-compatible traces flow into AgentCore Evaluations, AgentCore Observability, CloudWatch dashboards, or third-party observability systems.

Why this architecture

The stack is designed around the three production gaps the speakers say teams neglect: model switching, evaluation, and observability. Bedrock supplies unified model access, guardrails, evaluations, and observability so model and safety changes are platform capabilities rather than one-off application rewrites. Vercel then adds an SDK, gateway, and WDK durability layer because its production agents need provider abstraction, model-call visibility, and long-running tool execution without giving up serverless-style ergonomics.

Models used: Amazon Bedrock foundation models · Anthropic Claude Sonnet 4.5 · OpenAI-compatible models via provider switching · LLM-as-a-judge models · Amazon Nova · Llama · DeepSeek

Scale

• The cited analyst statistic says companies cancel 46% of AI POC projects before production.
  “companies and organizations on average are canceling 46% of their AI POC projects before they reach production.”transcript, stated
• AWS says it has met with hundreds of customers discussing POC-to-production problems.
  “We've met with hundreds of customers around the world that have talked to us about this problem going from POC to production.”transcript, stated
• Bedrock's AgentCore Evaluations offers 13 built-in evaluators.
  “You can sample and score live interactions with 13 built in out of the box evaluators for common quality dimensions”transcript, stated
• Vercel describes global-scale deployment and globally distributed production as part of the platform context.
  “we provide what we call self-driving infrastructure to deploy and run those apps at global scale.”transcript, stated

  “merge to Maine and then that'll just get deployed out there to a globally distributed production system.”transcript, stated
• Vercel says it has run production AI applications since transformer models emerged in 2019.
  “applications in production since transformer models really hit the scene in 2019.”transcript, stated
• The durable-agent demo specifically targets tool use that can run for multiple minutes.
  “It might be going out there and using the browser for multiple minutes at a time.”transcript, stated

Alternatives considered (roads not taken)

Retrofit model switching after the platform is already built.

Rejected because: The speakers argue model switching must be architected from day one because provider APIs differ and models churn quickly.

Use generic benchmark datasets as the primary evaluation basis.

Rejected because: The talk says open-source datasets are useful but often fail to match a team's business case and use case.

Rely entirely on human or programmatic evaluation.

Rejected because: Bedrock promotes LLM-as-a-judge as a way to automate parts of evaluation while still retaining human checks for golden datasets.

Manually instrument every agent for observability.

Rejected because: AgentCore Observability is positioned as a way to avoid manual instrumentation while still sending OTEL-compatible telemetry to CloudWatch or third-party tools.

Run complex tool-using agents as ordinary serverless requests.

Rejected because: Vercel says browser/code-interpreter agents can exceed serverless timeouts, so WDK turns tool use into durable queue-backed steps.

Patterns

model-routing (model_operations)

The platform abstracts model access through Bedrock APIs, Strands, Vercel AI SDK, and AI Gateway so teams can change model providers or versions without rewriting application logic.

guardrails-io-filtering (safety)

Bedrock Guardrails apply safety checks to both model inputs and outputs, with failures logged and application fallback left to the calling system.

eval-driven-development (evaluation)

The system treats evaluation as an iterative production loop using business-specific datasets, automated scoring, human checks for golden data, live sampling, custom evaluators, and LLM-as-a-judge.

observability-tracing (operations)

CloudWatch, AgentCore Observability, and Vercel AI Gateway provide traces, metrics, logs, costs, latency, tokens, and model-call attribution across agents and providers.

durable-execution (reliability)

Vercel's WDK turns TypeScript orchestration and tool calls into pausable, resumable, queue-backed durable workflows for agents that exceed normal serverless timeouts.

tool-use-orchestration (agent_tools)

The Vercel demo starts with text generation, adds AgentCore browser tools, then adds code-interpreter tools before wrapping tool use in durable execution.

human-in-the-loop-gate (evaluation)

The evaluation process retains human checks, at least for sampled or golden datasets, before automated evaluators become the standard for future runs.

Tradeoffs

Use unified model interfaces and gateways.

Chose Provider/model abstraction over Direct provider-specific calls embedded in application logic.

Rationale: Abstraction makes model switching faster and safer, at the cost of introducing platform layers such as Bedrock APIs, Vercel AI SDK, and AI Gateway.

Combine automated evaluation with human checks.

Chose Managed and LLM-based scoring plus golden-dataset human review over Fully manual evaluation or fully automated evaluation alone.

Rationale: Managed evaluators reduce setup and operational burden, but human checks remain necessary to calibrate quality for future evaluations.

Use Guardrails as a cross-model policy layer.

Chose One safe-AI policy applied to inputs and outputs over Separate safety handling for each model/provider.

Rationale: A unified policy gives governance continuity during model switching, but the application still needs to decide fallback behavior when checks fail.

Adopt durable workflow execution for complex agents.

Chose WDK durable workflows and queue-backed steps over Simple short-lived serverless agent requests.

Rationale: Durability lets agents use browsers and code interpreters for multi-minute work while preserving serverless-style scaling and developer ergonomics.

Keep observability portable through OpenTelemetry-compatible emissions.

Chose CloudWatch/AgentCore dashboards plus third-party export options over A single mandatory observability backend.

Rationale: The talk promotes CloudWatch as the default view while preserving compatibility for teams that use other observability systems.

Gotchas (what bit them)

Static AI platforms fall behind the model ecosystem.

Root cause: Model versions, capabilities, regional availability, costs, and provider interfaces change faster than traditional development cycles.

Resolution: Design the platform around day-one model switching through unified APIs, model-agnostic agent SDKs, and gateway routing.

Model switching is risky without evaluation.

Root cause: Different models have different APIs, behaviors, and quality/cost/latency tradeoffs, so replacement cannot be treated as a pure endpoint swap.

Resolution: Tie model switching to evaluation frameworks that use business-specific data, automated scoring, human review, and production feedback.

Production observability is not automatic.

Root cause: CloudWatch model invocation logging is disabled by default, and AI workflows span LLMs, agents, tools, and knowledge bases.

Resolution: Explicitly enable logging, collect traces/metrics, and route OTEL-compatible telemetry into CloudWatch, AgentCore Observability, or third-party systems.

Tool-rich agents can exceed ordinary serverless execution limits.

Root cause: Browser use and code execution may take minutes, while serverless functions have timeouts.

Resolution: Wrap long-running tool use in WDK durable workflows and queue-backed steps with retry, observability, and error handling.

Initial evaluation setup is not the same as meaningful evaluation.

Root cause: Teams must still choose metrics, datasets, custom evaluators, and human review standards suited to their own use case.

Resolution: Use managed evaluators to reduce operational work while retaining custom evaluators and golden-dataset checks for domain quality.

Evaluation

The session's evaluation design is itself a platform capability: evaluate candidate models and agents against business-specific data, quality/cost/latency goals, safety behavior, built-in and custom evaluators, LLM-as-a-judge, human checks for golden datasets, and continuous scoring of live interactions. It does not disclose measured production accuracy for Vercel's applications.

Metrics: Quality, cost, and latency tradeoffs · Bias and trusted behavior monitoring · 13 built-in AgentCore evaluators · Correctness, helpfulness, and goal success rate · Live interaction sampling · Latency, token counts, throttles, error counts, timing, tool usage, and knowledge lookups · Model-call latency and cost by agent in AI Gateway

Outcomes

• AWS positions Bedrock and AgentCore as the platform layer for secure, governed, observable, evaluated agent deployments.
• Vercel reports using Bedrock as the foundation for its AI and agentic workloads.
• Vercel built and reused the AI SDK, AI Gateway, and Workflow Development Kit across v0, Vercel Agent, and internal agents.
• The demonstrated durable-agent path moves from a single text-generation call to browser tools, code-interpreter tools, and WDK-backed durable execution.
• The talk provides platform architecture evidence but no independent production KPI results for the specific Vercel agents.

The source reports a 46% AI POC cancellation statistic, hundreds of AWS customer conversations, 13 AgentCore evaluators, Vercel AI production experience since 2019, globally distributed deployment, and multi-minute agent tool use. The case does not provide adoption counts, cost savings, accuracy deltas, or reliability metrics for Vercel's deployed agents.