An agent's unit of work is not a stateless HTTP request. It is a session: long-running, stateful across both memory and the local filesystem, and idle 50% to 95% of its lifecycle while waiting on LLM token generation or a human approval click. On a laptop, an agent harness (such as Claude Code, Codex, or Gemini CLI) runs inside a single local process where compute scheduling and state persistence are invisible. When teams move those same harnesses into production on Kubernetes, standard web and ML serving patterns break down. Running one always-on Pod or VM per active session burns massive compute on idle wait times, hits Kubernetes control-plane pod churn limits, and drops active sessions during rolling deploys.
This talk shares what broke (and what finally worked) across internal agent deployments at Google, and how those failures led us to split where an agent runs from how its session is served using two open-source Apache 2.0 primitives: Agent Substrate (github.com/agent-substrate/substrate) and AX / Agent Executor (github.com/google/ax). Think of it as the Erlang BEAM actor model rebuilt for isolated agent sandboxes on Kubernetes.
We will walk through the mechanics of session-tenant compute in production:
- Sub-200ms Sandbox Hibernation: How Agent Substrate runs untrusted, model-generated code inside hardened gVisor sandboxes (or MicroVMs), snapshots memory and filesystem state in ~460ms when an actor blocks on an LLM call or human input (freeing 92% of RAM for 10x compute density), and restores onto a warm pool in under 200ms without touching the Kubernetes control plane.
- Disaggregating the Brain from the Hands: How AX separates the harness controller from sandbox execution, enforcing single-writer session ordering, resumable event streams with disconnect backfill, and OpenTelemetry trajectory lineage.
- Why Event Replay Is Not Enough (ax fork): Why tool-using agents leave filesystem side effects that cannot be reconstructed from a conversation log alone, and how container-level memory and disk snapshots let you branch live agent sessions in milliseconds.
