Google AX agent runtime: the four things it keeps for an agent task, and the one it does not track

What is the Google AX agent runtime, and what does it keep for you?#

Google AX is an open-source, Apache 2.0 runtime that declares an agent task in four Kubernetes-style manifests and runs it sandboxed on Agent Substrate. The project's page at agentexecutor.io opens with one reason: "Agents are a new kind of workload." Because they accumulate state, they need strict isolation. Also, they call out to model APIs and tool servers. And they can burn money in a loop when no person is watching.

In practice, AX answers that with four small primitives. Task is the isolated execution: untrusted agent code in a sandbox with CPU and memory limits. The page calls a task "Cheap to create, suspend, and throw away." Workspace is the setup: the Git repos, MCP servers and skills your agent needs, prepared before the task starts. Gateway is the network boundary: an explicit allowlist of hosts and ports. Model is one place for the provider, the model name, the parameters and the secret that holds the key.

The naming needs one paragraph. Google Cloud announced the project as Agent Executor on 21 May 2026. The repository is github.com/google/ax, and the repository describes itself as "Google's open agentic orchestrator". It runs on Agent Substrate, a separate open-source project that maps many suspended actors onto a smaller pool of ready workers. Finally, the README sets the frame in one line: "If you have used Kubernetes, ax will feel similar."

What does a runtime like AX change for a team running coding agents?#

Idle agents stop costing a machine each: Agent Substrate's own demo multiplexes about 250 stateful actors across 8 pods, and AX inherits that. For example, a coding agent opens a pull request and waits a day for review. While it waits, it holds no CPU or RAM. When the review lands, the task resumes with its files intact.

The figures below are Google's statements about Substrate. They come from Agent Substrate's README, fetched on 21 September 2026, and they describe a demo cluster rather than a benchmark you can rerun.

Therefore the value a platform lead reports upward is operational. First, a fleet of waiting agents no longer reserves a container each. Second, a waiting agent comes back to the same workspace rather than a fresh clone. Third, the egress allowlist and the model secret live in two manifests instead of in every agent's environment. Say, for example, your team runs 300 concurrent coding tasks. At the demo's ratio that is about ten worker pods, and the arithmetic is illustrative because your tasks are not the demo's.

What do Task, Workspace, Gateway and Model each declare?#

A Task names an image, a command, CPU and memory limits, one Gateway and one or more Workspaces; the other three kinds are what it binds to. Every manifest carries apiVersion: ax.io/v1alpha1. The manifests page prints a complete example of each kind. Also, all four can sit in one multi-document YAML file.

The concepts page defines each one. A Task is "the smallest unit of isolated execution". Then a Workspace populates the filesystem and tool landscape: Git repositories cloned into subdirectories, MCP servers and registries, and skill registries. Next, a Gateway declares the listeners a task exposes and an egress allowlist of hosts and ports. In particular, a Model "is not a model" in the docs' own words. It is a named configuration: provider, model identifier, generation parameters and a reference to the Kubernetes secret holding the API key.

A task then moves through phases: Running, Suspended, Failed, Terminating. Conditions carry the detail, and the one to wait on is Ready.

What happens inside a task from boot to suspend, and what survives a resume?#

Every task container starts ax-task-runner as PID 1, which prepares each workspace once, serves readiness on port 80, then starts and supervises your command. The sandbox page lists the boot steps. First, the runner loads the Task and Workspace specs and starts a metadata daemon. Then it clones the repos and sets up skills. When a workspace carries a goal, it hands that goal to a setup agent with ten minutes by default. Only then does spec.command start, in the first workspace's directory.

Above the container, the control plane is small. Because "Storing millions of short-lived tasks as Kubernetes CRDs pushes etcd past its comfort zone", as DESIGN.md puts it, AX keeps task state in Redis. Redis Streams is the queue between ax-server and a pool of ax-controller workers. In turn, the controllers drive Agent Substrate over gRPC.

Suspend snapshots the /workspace volume and stops the process tree. Resume restores the files into a fresh container with a new process tree. So your agent's files survive, and its in-memory state does not. The runner page is explicit: "The /workspace volume is what survives suspend and resume." On stop or suspend, the runner sends SIGTERM to the command's process group. It waits ten seconds, then kills what is left. As a result, an agent that keeps its plan in memory loses it on resume. An agent that writes its plan to the workspace does not.

What does AX do when your agent's command fails?#

Nothing by itself: the runner logs the exit code and keeps serving, and the docs state the control plane does not currently read the command's exit status back. The runner page says it plainly. "The control plane does not currently read the command's exit status back from the container." That is the whole sentence. Meanwhile, the runner stays up as PID 1 after the command exits, so ax ssh and the metadata server keep working.

However, the same page gives you the hook. A custom runner that embeds the runner Go package gets an OnCommandExit callback with the exit code. The docs suggest uploading artifacts or notifying a webhook from it. That is where a harness learns the task finished, and it is code you write.

Retries, step-level recovery and the decision that a task is finished belong to the harness or a workflow engine above AX; AX guarantees the sandbox and the files, not the outcome. Google's announcement of Agent Executor on 21 May 2026 listed five native capabilities. They were durable execution, secure isolation, session consistency, connection recovery and trajectory branching. In contrast, the September docs describe suspend, resume and the workspace snapshot, and they do not describe step retries. So read the announcement as intent, and the docs as the contract you can build on today.

How does AX differ from LangGraph, Temporal and CrewAI?#

They sit at different layers: AX is a sandboxed runtime on a cluster, LangGraph and CrewAI are in-process frameworks that persist graph or flow state, and Temporal is a durable execution engine. Each product's own documentation places it. So the table below is layers, not a scorecard.

LangGraph's persistence page describes checkpointers that persist a thread's graph state, and stores that persist data across threads. Temporal's overview describes Durable Execution. When a Workflow crashes, it resumes where it stopped, with local variables and progress intact, replayed from the Event History. CrewAI's Flows page describes shared state and a @persist decorator that can resume or fork a run from a saved state. Yet none of the three gives your agent a sandbox with an egress allowlist. That is the layer AX occupies.

The honest comparison is by layer, not by feature count. A team picks a framework for your agent's logic, and optionally an engine for durable steps. Then it picks AX for where and how your agent's process runs. For instance, LangChain's own comparison page says of LangGraph and Temporal that "Many teams run both". The same stacking applies here, with AX underneath.

What does an enterprise need in place before the first task runs?#

The quick start needs six things: a Kubernetes cluster, a reachable Agent Substrate control API, a container registry, ko, Redis, and a Kubernetes secret holding the model key. The README's deploy step is make deploy AX_IMAGE_REPO=<your-registry>. It deploys Redis, then builds and deploys the control plane images with ko into the ax-system namespace.

AX is a control plane you deploy, not a service you sign up for, so the prerequisites are the platform team's before any agent code is written. In short, read the pages in this order:

1. Agent Substrate README: what Substrate is, and its status line, before you commit a cluster to it.
2. google/ax README: install the CLI with go install, the deploy prerequisites, and the everyday commands.
3. Manifests: the four kinds, and the kubectl create secret line for the model key.
4. Networking: how a request reaches a task through the atenet-router and the ate-target-actor header.
5. Runners: how to extend the default image, or embed the runner package to get the exit hook.

What is the smallest working example, and how mature is AX today?#

The smallest example is one YAML file with a Workspace and a Task, applied with ax apply, watched with ax watch, and paused with ax suspend. Both manifests below are the ones the README prints, and the commands are its quick start.

# task.yaml: a Workspace and a Task, as printed in the google/ax README
apiVersion: ax.io/v1alpha1
kind: Workspace
metadata:
  name: golang
spec:
  git:
    - repo: https://github.com/golang/go.git
      branch: "my-fix"
---
apiVersion: ax.io/v1alpha1
kind: Task
metadata:
  name: test
spec:
  workspaces:
    - name: golang
  goal: "Ensure that Go tool chain is available and is built from source"
  debug: true

ax apply -f task.yaml          # creates workspace.ax.io/golang and task.ax.io/test
ax watch task test             # streams phase and condition changes
ax ssh test -- ls /workspace   # needs spec.debug: true
ax suspend task test           # checkpoint actor state and pause
ax resume task test            # pick up where it left off

That is five steps in YAML and the ax CLI. Write the Workspace and the Task from the manifests page. Then apply, watch, suspend and resume with the README's commands. Still, the README warns of major breaking changes before a stable release, and the releases page lists six tags between 20 May and 20 September 2026. Its warning box says so. "We will likely to introduce major breaking changes prior to a stable release." Likewise, Agent Substrate's README says of Substrate: "It is not ready for production use, and the APIs are almost guaranteed to change."

Also, Google Cloud's post of 16 September 2026 says Agent Substrate is available on GKE for non-production workloads, with production support by allowlist. In short, AX is usable today for a pilot on a cluster you control, with an API that will move under you.

When is AX the wrong fit?#

AX is the wrong fit when you have no Kubernetes platform team, when your agents are single-turn calls, or when you need the runtime to retry steps for you. A single-turn assistant needs no sandbox lifecycle. A team without a cluster needs a managed sandbox. And a team that wants step retries needs a workflow engine, with or without AX underneath.

For instance, a chat assistant that answers one question per turn never suspends and never needs a workspace. So a plain model API call is the right size. A team without a cluster, or without anyone to run Redis and a registry, is better served by a managed sandbox until that changes. Finally, some teams have a different problem: step three failed at 3am and no retry ran. They need Temporal or a similar engine, because AX keeps the sandbox alive and does not know step three failed.

Where to go next#

The durable-execution layer AX leaves to you is the subject of the Cloudflare Workflows post, and the agentic SDLC post places these tasks in a delivery process. Read Cloudflare Workflows for AI for how retries stack across layers. Then read The Agentic SDLC for where agent tasks sit in delivery, and Laravel queues at scale for the same failure shapes in a familiar stack. For the platform work, the DevOps and CI/CD page covers the cluster, registry and secrets side. Also, enterprise software covers the harness and workflow layer above the runtime. The AX docs are short enough to read in an hour, and they are enough on their own.