Durable Agent Execution through Snapshot and Restore Infrastructure

trigger.dev · Tech · 2026 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Durable infrastructure for production agents

Trigger.dev needed backend infrastructure for agents that run meaningful work for hours or days, survive failures and code changes, and preserve both conversation context and execution-machine state more flexibly than stateless or replay-only systems.

Constraints:

Traditional shared-nothing web architecture keeps compute stateless and pushes state into databases.
Replay-based durable execution requires wrapping side effects and keeping code deterministic outside steps.
Long-running agents are sessions, not finite transactions, and may accumulate large context and machine state.
Agents need files, memory, subprocesses, repositories, dev servers, datasets, and other compute-layer state.
Snapshots must be fast and cheap enough for production scale.

Architecture

Trigger.dev separates agent durability into two state layers. The context layer is an append-only durable log of system messages, user messages, tool calls, tool results, and assistant responses. The execution layer is a Firecracker microVM whose whole machine state can be snapshotted and restored. Seekable compression and snapshot layering reduce snapshot size and restore only needed memory pages. FC Run packages the Firecracker VM start, snapshot, fork, and restore workflow for a future stateful compute layer.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  agent-loop[Agent loop] --> context-log[Context log]
  agent-loop --> execution-vm[Execution VM]
  execution-vm --> firecracker-snapshotter[Firecracker snapshotter]
  firecracker-snapshotter --> seekable-compression[Seekable compression]
  fc-run[FC Run] --> firecracker-snapshotter

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Agent loop`	Long-running LLM/tool session doing production work.	`Context log`, `Execution VM`
`Context log`	Append-only durable record of messages, tool calls, tool results, and assistant responses.	`Agent loop`
`Execution VM`	Stateful compute environment containing files, memory, subprocesses, datasets, and servers.	`Firecracker snapshotter`
`Firecracker snapshotter`	VM-level snapshot and restore system for the entire machine.	`Seekable compression`, `FC Run`
`Seekable compression`	Compressed layered snapshot storage that restores memory pages lazily.	`Firecracker snapshotter`
`FC Run`	Tooling for running containers in Firecracker VMs and snapshotting, forking, and restoring them.	`Firecracker snapshotter`

Data flow

An agent runs inside a VM with its context recorded in a durable append-only log. When the agent needs to pause, recover, or preserve compute-layer state, the Firecracker VM is snapshotted with compressed layered memory. On restore, the context log recreates conversation continuity while the VM snapshot brings back files, memory, subprocesses, and runtime state so execution resumes where it left off.

“the context is first and the most important, and it's just an append-only log of sort of everything that happened”transcript, stated

“everything that's is on a machine on a VM, we can snapshot it, and then we can restore it, and it pick up right where it left off”transcript, stated

Why this architecture

Trigger.dev argues that replay gives useful durable transactions but is too rigid for agents as open-ended sessions with compute-layer state. Splitting context durability from execution snapshots lets code versions, failures, and turns survive through a simple append-only log while preserving the machine state that agents increasingly rely on.

“replay gave us these like sort of durable transactions, but you know, an agent isn't like a transaction, it's like a session”transcript, stated

“you also have this sort of execution layer. And as agents are like more complicated, doing more things, they kind of want a machine”transcript, stated

Models used: LLM agent loop · Firecracker microVMs · FC Run

Scale

Agents were described as already doing meaningful work for a few hours, with days expected soon.
“right now we're on about like a few hours, but like not too long from now we'll be on like multiple days of length as these agents build to actually do meaningful work.”transcript, stated
Naive snapshots of a 512 MB VM would create 512 MB on disk.
“say you have a default machine size of 512 megabytes, you know, uh, if you do a snapshot, it's 512 megabytes on disk.”transcript, stated
Compression can reduce snapshots to about 14 MB.
“we can get the the, um, snapshot down to like 14 megabytes compressed.”transcript, stated
Snapshot time is slightly under a second and restore time is a couple hundred milliseconds.
“snapshots are like slightly under a second, and restores are a couple hundred milliseconds.”transcript, stated
FC Run benchmarked about 15,000 VM starts per minute.
“we're doing like 15,000 VM starts per minute.”transcript, stated

Alternatives considered (roads not taken)

Traditional shared-nothing stateless compute.

Rejected because: Agents need durable state across turns, code versions, errors, and long-running work, not only request-plus-database responses.

“shared nothing sort of means that the compute layer is stateless, right? There's nothing There's no meaningful state like in the compute.”transcript, stated

Replay-only durable execution.

Rejected because: Replay requires rigid step wrapping and deterministic code outside steps, which is awkward for agent sessions.

“you sort of have to wrap everything in in these steps, and then everything outside of steps is has to be deterministic”transcript, stated

Naive full-memory VM snapshots.

Rejected because: A 512 MB VM snapshot would be 512 MB on disk, creating storage and network costs.

“if you do a snapshot, it's 512 megabytes on disk. So, that's that's not great.”transcript, stated

CRIU/container snapshots.

Rejected because: The talk says container-compatible snapshotting got slow with registries and did not capture all file-system state.

“if you're working with the file system, it has to be open at the time of snapshot, or you won't get snapshot.”transcript, stated

Patterns

`durable-execution` (orchestration)

Trigger.dev combines append-only context logs with VM-level execution snapshots to make long-running agent sessions resumable across turns, failures, and code versions.

“we've sort of made it durable now by doing two different things, context log and execution snapshot.”transcript, stated

`agent-memory` (context)

The context log records all messages, tool calls, results, and assistant responses as durable agent memory.

“this is all all your your system messages, user messages, tool calls, tool results, assistant responses”transcript, stated

`fallback-degradation` (reliability)

The platform is designed so agents can recover from errors and pick up where they left off after crashes.

“maybe the machine crashes, and you can still that saved somewhere, so you can pick up where you left off”transcript, stated

Tradeoffs

Durability model

Chose Append-only context log plus execution snapshot. over Replay-only workflows or stateless compute..

Rationale: Agents are long-running sessions with both conversational and machine state.

“sessions keep going for as long as possible. So, if we sort of take a step back and think about what an agent needs to be durable for”transcript, stated

Snapshot boundary

Chose Firecracker microVM snapshots. over Container-level snapshotting approaches..

Rationale: VM snapshots can capture the entire machine and restore no matter what was happening.

“Firecracker micro VMs. And this allows us to sort of snapshot like the entire machine”transcript, stated

Snapshot storage strategy

Chose Seekable compression and layered snapshots. over Storing full uncompressed memory snapshots..

Rationale: Compression cuts storage and transfer costs while restoring memory lazily.

“there's a lot of memory there that's not actually being used. Uh, so we actually solved this with, uh, compressing it.”transcript, stated

Gotchas (what bit them)

Replay durable execution is too rigid for agent sessions.

Root cause: Replay requires deterministic code outside wrapped steps and the replay journal becomes tricky across versions.

Resolution: Trigger.dev separates durable context logs from VM-level execution snapshots.

“you kind of get this like uh rigid structure. You have to write your code in a certain way, or things break.”transcript, stated

Agents need more than conversational context.

Root cause: Useful agents create files, install packages, hold datasets in memory, and run processes inside compute.

Resolution: The execution layer is snapshotted as a whole Firecracker VM.

“Maybe you've cloned a GitHub repo, you know, you've installed some packages, you've got some data sets in memory, you're running a dev server”transcript, stated

VM snapshots can become too expensive if stored naively.

Root cause: A default 512 MB machine produces a 512 MB snapshot on disk.

Resolution: Trigger.dev uses seekable compression and layered snapshots to restore needed memory pages lazily.

“we actually use a a seekable compression. Um, so when we restore, we actually don't restore all the memory pages at once.”transcript, stated

Evaluation

Trigger.dev evaluates the infrastructure through snapshot and restore latency, compressed snapshot size, and VM start throughput benchmarks.

Metrics: Snapshots slightly under a second · Restores a couple hundred milliseconds · 512 MB naive snapshot size · About 14 MB compressed snapshot size · 15,000 VM starts per minute

“snapshots are like slightly under a second, and restores are a couple hundred milliseconds.”transcript, stated

“this is, uh, but we're doing like 15,000 VM starts per minute.”transcript, stated

Outcomes

Produced a two-part durability model for agents: context log and execution snapshot.
Reduced snapshot storage from a naive 512 MB VM image to about 14 MB compressed.
Achieved sub-second snapshots and couple-hundred-millisecond restores.
Benchmarked about 15,000 VM starts per minute for the FC Run layer.

The talk reports a 512 MB naive snapshot reduced to about 14 MB compressed, snapshots slightly under a second, restores in a couple hundred milliseconds, and 15,000 VM starts per minute.

“So, we got durability across versions, durability across turns, across failures, and I think this will lead to a future of, you know, stateful compute.”transcript, stated

“we can get the the, um, snapshot down to like 14 megabytes compressed.”transcript, stated