AXP CLI

axp is the command-line interface for the AXP agent experimentation platform.

Installing, pinning a version, and uninstalling are done with the install script rather than with axp subcommands; see Installation. You can update either way — with axp update (below) or by re-running the installer.

axp <COMMAND>

Global Options

axp --help
axp --version

axp --version also checks for a newer stable release. It prints axp is up to date. when you're current, or how to update with axp update when a newer release is available. (Other commands print the same upgrade notice when one is available, but never the "up to date" line.)

The generated help command prints the top-level help or the help for a given subcommand.

axp help [COMMAND]

Command index

Author experiments

Run experiments

Inspect results

Manage platform access

Maintain the CLI

Author experiments

experiment create

Authoring commands live under axp experiment (create, schema, validate, resolve).

Write a templated <name>.yaml in the current directory.

axp experiment create <NAME>

intro

Write a curated four-variant CLI-install experiment for first-time local use. The command is non-interactive and writes YAML for a local experiment. The smoke command and optional output assertion are hidden test oracles, not agent-visible guidance. The generated variants share the same task description — your --target-description — and vary only the install pointer: no pointer, CLI name, install docs URL, then exact install command.

axp intro <NAME> \
  --cli <CLI_NAME> \
  --target-description <DESCRIPTION> \
  --install-docs <URL> \
  --install-command <COMMAND> \
  --smoke-command <COMMAND> \
  [--smoke-output-contains <TEXT>]

Example:

axp intro my-first-experiment \
  --cli mycli \
  --target-description 'a CLI for managing OLAP databases' \
  --install-docs https://example.com/install \
  --install-command 'npm install mycli' \
  --smoke-command 'mycli --version' \
  --smoke-output-contains 'mycli version'

experiment schema

Print an experiment schema document to stdout.

axp experiment schema [OPTIONS]

experiment validate

Schema-validate an experiment YAML. On success, prints a confirmation and the next axp run command to stdout; on schema failure, prints a single-line error and exits non-zero. Valid experiments may also print non-fatal lint warnings to stderr without changing the exit code.

axp experiment validate <EXPERIMENT>

Successful validation output:

Validated <EXPERIMENT>
Next: run it with `axp run <EXPERIMENT>`

Advisory warnings use warning[<rule>]: <location>: <message> on stderr. For example, warning[polls-for-file-in-setup] flags variant setup scripts that appear to wait for files that another isolated variant might create.

AI review

After the schema checks, axp experiment validate runs a best-effort AI review of the experiment's design quality — grading it against AXP authoring best practices such as a single clear evaluation question, task-focused prompts, axes/extensions that isolate a meaningful difference, tests that verify a real success artifact, and sensible secrets and limits. The review is advisory: it prints to stderr and never changes the exit code, so the result on stdout stays machine-readable.

• Signed in — with a saved login (axp auth login) or AXP_API_KEY set — the review runs a fast model through 514-managed model access and prints a prioritized, severity-tagged list of suggestions under an AI review header. It makes one short model call that counts toward your organization's managed-access usage.
• Signed out — no network call. axp experiment validate prints a one-line note that signing in unlocks AI hints.
• Before sending YAML for review, AXP elides large unbroken inline payloads such as base64 or hex fixture data, while keeping the surrounding YAML, prompts, and setup commands visible to the reviewer.
• If the review can't run (for example, the platform is unreachable, or the experiment is still too large after eliding embedded blobs), it degrades to a one-line AI review skipped note and validation still succeeds.

Pass --no-ai (or set AXP_NO_AI_LINT=1) to skip the review entirely — useful in CI, scripts, or when offline. Set AXP_AI_LINT_MODEL to review with a different model. Color and emoji appear only on an interactive terminal and honor NO_COLOR. Signing in is covered in Secrets and auth.

experiment resolve

Resolve an experiment's variant coordinates and fingerprints using the canonical Rust resolver. This is mainly useful for local development, debugging platform resolution, and checking the exact variant ids a YAML file produces.

axp experiment resolve [OPTIONS] <EXPERIMENT>

Run experiments

run

Submit an experiment to the AXP platform and poll it to completion. The platform dispatches one managed sandbox per (variant × repeat) job; results land in your organization's data and the run details page. For local Docker runs, use local run.

axp run [OPTIONS] <EXPERIMENT>

Requires axp auth login (or AXP_API_KEY). The platform supplies model access for agent: claude, agent: codex, and agent: cursor variants, so you do not pass a model key. Experiments that reference org secrets via environment_variables (axp://secrets/<slug>) are supported — dispatch resolves them server-side from your org's secret store; a referenced slug that doesn't exist fails fast at submit. See Runs for the full platform-vs-local comparison.

axp run prints a run id. Pass it to runs status to check progress later, to cancel to stop queued or in-flight jobs, or to axp download and axp runs list to find or fetch results once the run completes.

cancel

Cancel some or all jobs in a platform run. Queued jobs are canceled immediately; in-flight jobs are signalled and finalize shortly after.

axp cancel [OPTIONS] <RUN_REQUEST_ID>

runs status

Show one platform run's status: its lifecycle roll-up, every (variant × repeat) run's current status and phase, and a status summary. Pass the run id printed by axp run. This reads the same status axp run polls, so you can check on a run you submitted with --detach or detached from with Ctrl-C.

axp runs status [OPTIONS] <RUN_REQUEST_ID>

The command exits non-zero when the run is terminal with any failed or canceled run, so it can gate a script. A still-running run exits zero.

runs list

List recent runs, newest first. By default, axp runs list combines local runs from ./.axp/runs/ with remote platform runs. If you are signed out or offline, it skips the remote list with a note and still shows local runs. axp list is a kept alias of this command.

axp runs list [OPTIONS]

The table columns are ID, EXPERIMENT, STATUS, TYPE, and CREATED. The TYPE column is local or remote. Use a remote ID with runs status or axp download, or a local ID with axp local query.

local

Local-only commands.

axp local <COMMAND>

local run

Run an experiment locally in Docker across its resolved variants, writing results under ./.axp/runs/<run-id>/. This is the pre-platform axp run: local sandboxes, local artifacts, optional axp upload afterwards. On first run it pulls 514labs/axp-base from Docker Hub.

axp local run [OPTIONS] [EXPERIMENT]

Requires Docker, plus a model credential for each selected agent (ANTHROPIC_API_KEY / OPENAI_API_KEY / CURSOR_API_KEY) — or --managed-model-access to have the platform mint one. Like axp run, it resolves environment_variables (axp://secrets/<slug> references come from your org's secret store); additionally, local sources (./.env / --env-file / host / --env) override the YAML for a matching name, and --file stages local files. See Environment and Auth for precedence.

sandbox clean

List or remove orphaned sandbox containers left behind by a crashed or SIGKILL'd axp local run. Containers are matched by the ai.514.axp.* labels AXP attaches at create time. Without --yes this is a dry run that only lists the orphans it would remove.

axp sandbox clean [OPTIONS]

Inspect results

local query

Run SQL against a local raw run under .axp/runs/<run-id>/ or downloaded platform Parquet under .axp/downloads/<run-id>/. For raw runs, the CLI derives query Parquet under .axp/derived/<run-id>/query/ on demand. It exposes runs, agent_events, tool_calls, messages, harness_spans, tests, artifacts, and experiment_runs.

axp local query <RUN_ID> <SQL>

Example:

axp local query my-run-01HX... "SELECT * FROM runs LIMIT 10" --table

query

Run read-only ClickHouse SQL against uploaded Source/Facts tables through the AXP platform. Unlike axp local query, this command is scoped to the active organization and can query across all uploaded runs in that org. It requires axp auth login or axp auth connect.

Available tables are artifacts, runs, tests, tool_calls, messages, harness_spans, and agent_events. SHOW TABLES and DESCRIBE [TABLE] <table> are supported for discovery.

axp query <SQL>

Example:

axp query "SELECT run_request_id, count() AS variants FROM runs GROUP BY run_request_id" --table

upload

Upload a run to platform-app. The CLI uploads artifacts.parquet; the platform derives the analytical tables from those raw artifacts. On success, the CLI prints that derivation is starting and points you to the Experiments area, where the platform shows experiment, run, and variant details for uploaded data.

axp upload [OPTIONS] [RUN_ID]

download

Download a run's Parquet files from the AXP platform.

axp download [OPTIONS] [RUN_ID]

prompt

Print a templated prompt for a run to stdout, designed for piping into a local agent CLI. <PROMPT_NAME> selects the template (matching prompts/<name>.md); only explore ships today.

axp prompt <PROMPT_NAME> [RUN_ID]

Example:

axp prompt explore 01HX... | claude

send-debug

Bundle a run, its source experiment, and host metadata into a tar.gz and upload it to the AXP platform for debugging and reproduction. Bundles include agent and test logs verbatim, so the CLI warns and prompts before uploading unless you pass --yes.

axp send-debug [OPTIONS] [RUN_ID]

Manage platform access

auth

Authenticate with the AXP platform by saving an API key for the CLI.

axp auth <COMMAND>

auth login

Log in by pasting an API key, or pass --token.

axp auth login [OPTIONS]

auth connect

Log in by redeeming a one-time code returned by the AXP MCP connector.

axp auth connect --code <CODE>

auth logout

Forget the local credentials file. Does not revoke the Clerk key.

axp auth logout

auth whoami

Show the identity resolved by the platform API.

axp auth whoami [OPTIONS]

auth orgs

List organizations available to the current API key. Without options, this defaults to list output.

axp auth orgs [OPTIONS]

secrets

Manage your organization's secret store. Stored values are encrypted, addressed by a kebab-case slug, and referenced from experiments as axp://secrets/<slug> in environment_variables. Reads never return a value — there is no command that prints a secret. Requires axp auth login (or AXP_API_KEY); all subcommands operate on the active org (or --org <ORG_ID>).

axp secrets set <slug> [--description <text>]   # create or overwrite; prompts for the value (never echoed) or reads stdin
axp secrets list [--limit <n>] [--page-token <t>] [--json]   # slugs + audit metadata only
axp secrets rm <slug>                            # idempotent

See Environment and Auth for the store model and who can resolve secrets.

Maintain the CLI

release-notes

Fetch release-note metadata from the stable download index and print the resolved notes as Markdown.

axp release-notes [OPTIONS] [VERSION]

When an exact patch version has no dedicated notes, AXP falls back to the newest release-note entry from the same minor line that is not newer than the requested version. The rendered output includes the release title, date, source URL, and a fallback notice when the resolved notes differ from the requested version.

update

Download and install the latest stable AXP release. This runs the same steps as the installer — it fetches the latest stable build, verifies it, and switches axp over to it — so it's the quickest way to get the newest version. The update takes effect the next time you run axp.

axp update

To see the exact install command without running anything — for example to review it or run it yourself — use --print:

axp update --print

axp update always installs the latest stable release. To install a specific version or a branch build, run the installer directly with an axp@<version> spec (see Installation).