Building AI-Native Platforms: Agentic Systems, Infrastructure Evolution, and Production LLM Deployment

delphi_/seam_ai/_apisec · Tech · 2025 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: AI-native production platform evolution

Delphi, Seam AI, and APIsec were evolving from early LLM prompting into production agentic systems while managing serverless infrastructure, cost, latency, deterministic control, tool loops, observability, and ROI.

Constraints:

Different products span consumer AI profiles, sales/marketing agents, and API security testing.
The systems evolved from small-context single-shot prompting toward agentic workflows.
Some security workflows still require deterministic state-machine control.
High token volume and model choice create cost and latency pressure.

Architecture

The panel describes a converging architecture: serverless and scalable infrastructure, agents modeled as tools in a loop, state machines where deterministic control matters, less-opinionated agent architectures that let models choose tools, and observability through Pydantic AI/Logfire-style distributed traces of agent decisions and tool calls.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  user-or-customer-task[User or Customer Task] --> agent-loop[Agent Loop]
  agent-loop --> tool-layer[Tool Layer]
  agent-loop --> state-machine-layer[State Machine Layer]
  agent-loop --> knowledge-graph[Knowledge Graph]
  agent-loop --> trace-instrumentation[Trace Instrumentation]
  state-machine-layer --> serverless-infrastructure[Serverless Infrastructure]
  serverless-infrastructure --> cost-and-latency-controls[Cost and Latency Controls]
  cost-and-latency-controls --> agent-loop

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`User or Customer Task`	Consumer profile interaction, sales/marketing automation, or API security work request.	`Agent Loop`
`Agent Loop`	LLM-driven loop that decides whether to call tools, APIs, MCP servers, or other LLM calls.	`User or Customer Task`, `Tool Layer`, `State Machine Layer`, `Knowledge Graph`, `Trace Instrumentation`
`Tool Layer`	Functions, LLM calls, API calls, and MCP tools available to the agent.	`Agent Loop`
`State Machine Layer`	Deterministic workflow control for security or high-confidence decision paths.	`Agent Loop`, `Serverless Infrastructure`
`Knowledge Graph`	Representation used as one of the agent's tools, such as Delphi's person's representation graph.	`Agent Loop`
`Serverless Infrastructure`	Scalable runtime substrate for AI-native systems and workflows.	`State Machine Layer`, `Cost and Latency Controls`
`Cost and Latency Controls`	Model choice, optimization timing, fine-tuning decisions, and latency/cost tradeoff management.	`Serverless Infrastructure`, `Agent Loop`
`Trace Instrumentation`	Distributed traces and agent logs for tool calls, decisions, and debugging.	`Agent Loop`

Data flow

AI-native products receive user or system tasks, route them through agent loops or state machines, let the model decide which tools, API calls, MCP servers, or knowledge graphs to use, log decisions/tool calls through tracing, and optimize cost/latency after validating that the product behavior works.

“the simplest way to think about agents is literally just tools in a loop where you have a system prompt and that it has things that it can do”transcript, stated

“creating full distributed traces that then include all the agentic logs for all the decisions it made and the tools that it called.”transcript, stated

Why this architecture

The panelists moved toward less-opinionated agents and serverless infrastructure where the model can reason with tools, while retaining state machines when deterministic behavior matters.

“completely redoing the mind architecture from scratch to be the least opinionated possible to have an agent that has a tool”transcript, stated

“the deterministic nature of those state machines is very important to us because as you're advising an organization to activate an engineering resource to go fix something, you have to have extremely high confidence in that. And”transcript, stated

Models used: GPT-3 · OpenAI models · LLMs · fine-tuned models · Pydantic AI

Scale

The panel describes moving from GPT-3-era 4,000-token contexts to agentic workflows and fully agentic systems.
“windows were really small. Back in 2023, we had like 4,000 tokens with GBT3. So you were doing a lot of like question and answer work. Um and it was you know really great use cases for that search and summarization and in our world data”transcript, stated
“really got to expand beyond this just like back and forth question answer to you know agentic workflows and now like these fully agentic systems.”transcript, stated
One company had been publicly recognized for one trillion OpenAI tokens.
“I was on the one trillion token and that was one of the most embarrassing things to be publicly recognized for because it's like your open bill is huge and uh but I think also I I want to take a step back before and just clarify what Del is a little bit more because I t”transcript, stated
The infrastructure discussion includes serverless and infinitely scalable platforms.
“being serverless and infinitely scalable. And so there's a couple things plan transitioned sometime last year I think to being completely serverless.”transcript, stated
“And then you have things like planet scale that are basically infinitely scalable. And that combined with things like temporal for being able to program in reactive programming styles almost automatically built in a lot of what you”transcript, stated

Alternatives considered (roads not taken)

Keep systems as single-shot prompting and question answering.

Rejected because: The companies' use cases matured into workflows and fully agentic systems.

“was kind of these three phases. Phase one is kind of what we call single shot prompting where, you know, the context windows were really small. Back in 2023, we had like 4,000 tokens with GBT3. So you were doing a lot of like question and answer work. Um and it was you know”transcript, stated

“like really got to expand beyond this just like back and forth question answer to you know agentic workflows and now like these fully agentic systems. So for us we've like kind of followed that journey. I think like when we initially started like it was a lot of ingesting”transcript, stated

Optimize cost before validating product behavior.

Rejected because: For consumer category creation, Delphi prioritized proving user behavior before optimizing spend.

“cost is the last consideration. And so we're trying to first actually prove that people will want to use the website”transcript, stated

Fine-tune as a default production optimization.

Rejected because: The panelist recommends fine-tuning only for narrow use cases where fixed cost or latency is the specific goal.

“we don't do any fine tuning right now and I would recommend that everyone here doesn't do it unless you have a very specific use case”transcript, stated

“to save money I would say that's usually save money or latency and you want to have a fixed cost where you have something that you want to throw on a uh a fireworks GPU or something over a long period of time. But a lot of times any task that you're trying to do will”transcript, stated

Patterns

`tool-use-orchestration` (agent-runtime)

Agents are described as LLM loops deciding among functions, LLM calls, API calls, and MCP tools.

“things that it can do and they might be functions, it might be LM calls, might be a API call MCP that hits a server”transcript, stated

`durable-execution` (workflow)

State machines and serverless workflow primitives keep some agentic systems deterministic and scalable.

“we use a lot of state machines in order to do reasoning because a lot of the work that we do is not just”transcript, stated

“we're using a lot of serverless as you said state machines like step functions type stuff”transcript, stated

`observability-tracing` (operations)

Pydantic AI/Logfire instrumentation records agentic logs, tool calls, and decisions in distributed traces.

“Pyantic called Logfire that automatically instruments is very good and we started to use them a ton”transcript, stated

“full distributed traces that then include all the agentic logs for all the decisions it made and the tools that it called.”transcript, stated

`cost-aware-routing` (model-operations)

The panel treats model choice, fine-tuning, chips, latency, and token spend as product-specific optimization decisions.

“the cost is directly proportional to latency.”transcript, stated

“using the most intelligent models with the fact that those are usually inherent inherently more latent sensitive”transcript, stated

Tradeoffs

Architecture control

Chose Less-opinionated agents with tools over Rigid pre-scripted model reasoning paths.

Rationale: Delphi wanted a digital mind architecture that let the model use tools rather than over-constraining reasoning.

“least opinionated possible to have an agent that has a tool that has tools one of which being a knowledge graph”transcript, stated

Determinism boundary

Chose State machines for high-confidence/security paths over Fully autonomous model reasoning for every step.

Rationale: Security analysis needs deterministic flow control when advising organizations about risky behavior.

“the deterministic nature of those state machines is very important to us because as you're advising an organization”transcript, stated

Optimization sequence

Chose Prove new product behavior before cost optimization over Constrain early exploration by token cost.

Rationale: For a consumer category, product pull was the gating risk.

“cost is the last consideration. And so we're trying to first actually prove that people will want to use the website”transcript, stated

Gotchas (what bit them)

Agentic systems can burn enormous token volume before cost is optimized.

Root cause: Consumer products may need to prove a new user behavior before optimizing spend.

Resolution: Treat cost optimization as a later stage once the product behavior works, while avoiding obvious waste.

“we're trying to first actually prove that people will want to use the website and want to create the delifies.”transcript, stated

“Obviously don't make unforced errors and spend a ton of money unnecessarily but prove that it actually works and then go and optimize the cost”transcript, stated

Security workflows cannot abandon deterministic control just because models improve.

Root cause: API security requires judging what users should be allowed to do, not only what an API can technically do.

Resolution: Keep state machines and bring in humans or organizational data for complex should/can decisions.

“not just can somebody do something should they be able to do that as well.”transcript, stated

“So we have to bring in either humans or other uh data in the organization.”transcript, stated

Fine-tuning can waste time if the base-model frontier moves faster than the dataset effort.

Root cause: By the time a team builds a good fine-tuning dataset, the task may become better, cheaper, and faster on newer base models.

Resolution: Reserve fine-tuning for narrow fixed-cost or latency use cases.

“any task that you're trying to do will become better, cheaper, faster by the time that you figure out how to how to how to have a good data set”transcript, stated

Evaluation

The panel frames evaluation around product ROI, outcome metrics, cost/latency behavior, security correctness, and trace-level observability rather than only model-level accuracy.

Metrics: ROI · Usage-based cost · Latency · Agent tool-call traces · Security decision correctness

“And then for latency uh or versus cost I would say there are two companies that sort of break this paradigm”transcript, stated