Search documentation

Search the Fumadocs-backed documentation index.

Overview

What AXP does and where to start.

AI agents are becoming a new kind of user for your product. AXP helps you find out whether they can actually use it.

AXP lets you run sandboxed experiments that evaluate any product surface an agent can interact with, such as CLIs, SDKs, APIs, web apps, docs, etc.

You can use AXP to get user feedback/test results on new features, improvements, and bug fixes before you release them to the public. You can answer questions like:

  • How do agents use your product?
  • Do changes to your product make agents better or worse at completing tasks?
  • Which models or coding agents perform best with your product? Which perform worse?

You can also run AXP experiments to evaluate competing products that you are considering adopting, buying, or investing in. You can answer questions like:

  • Which product performs better with agents for your use case?
  • Which product works best with the coding agents you already use?
  • Which product fails in ways your team can debug and recover from?
  • Which product is cheaper or faster for agents to use?

How AXP Works

AXP has three main pieces:

  1. Create an Experiment — define the task, the variants you want to compare (e.g. agents, prompts, product versions, or competing products), and the tests that measure how the agent performed the task.
  2. Run the Experiment — execute the experiment in a sandbox, either on the AXP platform with axp run or locally with axp local run, and capture data about what happened: logs, file changes, test results, token usage, and cost.
  3. Generate Results — compare the variant outcomes so you can see what worked, what failed, and what to change next.
Experiments
The spec that defines the task, variants, setup, and tests you want an agent to run.
Runs
A completed execution of an experiment, including variant outputs, costs, logs, and artifacts.
$0.57
Avg cost
5m00s
Avg wall clock
321
Tool failures
100%
Test pass rate
Group by: Variant
X-axis: Wall clock
Y-axis: Cost
Cost vs wall clock, grouped by variant
0m00s
13m20s
Results
The summaries you generate from runs to compare variants and decide what to change next.

Installation

The fastest way to get started is to use the official AXP CLI installer:

bash <(curl -fsSL https://dl.514.ai/install.sh) axp
axp --help

See Installation for more details.

Supported Coding Agents

AXP currently supports Anthropic Claude Code, OpenAI Codex, and Cursor.

Need support for another coding agent? Ask us to add it.

Next

Start with Installation. Then use Getting Started for your first run.