Niki Birkner

Founding PM at Together AI, working on inference and growth. Co-founder of Babel, a community for people building AI in San Francisco.

← AI lessons

A checkpoint saves state; durable execution also restarts the run.

Systems · 3 min read

The Mon/Wed/Fri briefing is AI news.

Some AI agents run for minutes. Some run for days, especially when they wait on a person to approve something. Over that time, servers crash, new code gets deployed, and machines get recycled. What keeps the agent from losing everything, or worse, starting over and doing something different?

The idea

Durable execution means the system keeps a journal of completed steps, so after a crash it resumes from the last finished step instead of replaying the whole job.

That’s different from a checkpoint, which saves the agent’s state. As Temporal’s writeup puts it: a checkpoint persists data. Durable execution also restarts the run when the process dies. Saved state alone doesn’t bring anything back to life.

Three ways teams do it

Temporal puts the overall plan in a “Workflow” and the actual model calls and tool calls in “Activities.” Every finished Activity is recorded in an Event History. On recovery, Temporal replays the history and skips work that already finished. A call that died halfway retries from the start of that step, so tools that change things (charging a card, sending an email) need idempotency keys, a unique ID that makes doing it twice safe. Waiting for a human costs nothing: the Workflow parks with zero worker CPU until the answer arrives. Temporal’s LangGraph plugin (public preview, July 2026) runs graph steps as Activities so the run itself keeps going after a crash.

Restate wraps ordinary code. You journal model responses and wrap tool calls, and after a crash it restarts the handler, replays completed entries, and continues from the first unfinished step. Each session gets its own state with one writer at a time, so two messages to the same agent queue up instead of colliding.

LangGraph checkpointers save the graph’s state after each step, keyed by a thread ID. In-memory savers vanish on restart; a Postgres saver persists for production. To close the “who restarts it?” gap, LangSmith Deployment’s Agent Server adds a task queue and worker leases, so another worker can pick up a crashed job.

Why it matters

Money. A research agent that spent twenty minutes on tools has already paid for those tokens. Restarting from scratch pays again, and might take a different path.

Temporal’s vacation-booking example shows the danger. The first run booked flights to Whistler, then crashed while booking hotels. A naive restart booked Japan instead. Replaying from the history kept the Whistler plan.

Reliability. If an agent forgets it was waiting for approval, its trace just ends. There’s no error to catch, which is why tests rarely find this bug.

Deploys. Anthropic’s multi-agent research writeup describes rainbow deployments: keep several versions running so in-flight sessions finish on the version that started them, while new sessions go to the latest. The catch is sessions lasting days that never drain unless you set a timeout. And “version” means the whole bundle: code, model release, prompts and tool definitions together.

An analogy: the recipe with checkmarks

You’re cooking a complicated meal, checking off each step on the recipe card. The power goes out. When it comes back, you don’t restart from “preheat the oven.” You look at the checkmarks and pick up at step 7. That’s a checkpoint.

Durable execution is having a sous-chef whose whole job is noticing the power came back, reading the checkmarks, and getting cooking again, even if you’ve gone to bed. And if step 7 was “order the fish,” they check whether it was already ordered before calling again.

Try this

Think of a multi-step errand you do online, like booking a trip or ordering for a party. Write the steps down. Mark the ones that change something in the world (pay, book, send) with a star. Those are the steps where “accidentally doing it twice” is a real problem. For each starred step, ask: if my browser crashed right after this, how would I know whether it went through? That’s exactly the question durable execution answers for agents.

Use it at dinner

“Saving an AI agent’s progress isn’t enough. Something also has to notice it crashed, restart it, and make sure it doesn’t book the same flight twice.”

Sources

Subscribe

↑↓ to move · enter to open · esc to close · t toggles theme