The ax evaluators commands are currently in ALPHA. The API may change without notice. A one-time warning is emitted on first use.
ax evaluators commands let you create and manage LLM-as-judge evaluators and their versions on the Arize platform.
ax evaluators list
List evaluators, optionally filtered by space.
ax evaluators list [--space <id>] [--name <filter>] [--limit <n>] [--cursor <cursor>]
| Option | Description |
|---|
--space | Filter evaluators by space name or ID |
--name | Case-insensitive substring filter on evaluator name |
--limit | Maximum number of results to return (default: 15) |
--cursor | Pagination cursor for the next page |
ax evaluators list --space sp_abc123
ax evaluators list --space sp_abc123 --output evaluators.json
ax evaluators create
Create a new evaluator with an initial version. Required options will be prompted interactively if not passed as flags.
ax evaluators create \
--name <name> \
--space <id> \
--commit-message <message> \
--template-name <name> \
--template <template-string> \
--ai-integration-id <id> \
--model-name <model>
| Option | Description |
|---|
--name | Evaluator name (must be unique within the space) |
--space | Space name or ID to create the evaluator in |
--commit-message | Commit message for the initial version |
--template-name | Eval column name (alphanumeric, spaces, hyphens, underscores) |
--template | Prompt template string with {{variable}} placeholders |
--ai-integration-id | AI integration global ID |
--model-name | Model name (e.g. gpt-4o) |
--description | Optional evaluator description |
--include-explanations | Include reasoning explanation alongside the score (flag) |
--use-function-calling | Prefer structured function-call output when supported (flag) |
--invocation-params | JSON object of model invocation parameters (e.g. '{"temperature": 0}') |
--provider-params | JSON object of provider-specific parameters |
ax evaluators create \
--name "Relevance" \
--space sp_abc123 \
--commit-message "Initial version" \
--template-name "Relevance" \
--template "Is the response relevant to the query?\nQuery: {{input.value}}\nResponse: {{output.value}}" \
--ai-integration-id ai_xyz789 \
--model-name gpt-4o \
--include-explanations \
--invocation-params '{"temperature": 0}'
ax evaluators get
Get an evaluator by name or ID, with its resolved version.
ax evaluators get <name-or-id> [--space <id>] [--version-id <id>]
| Option | Description |
|---|
--space | Space name or ID (required when using evaluator name instead of ID) |
--version-id | Specific version ID to retrieve (default: latest version) |
ax evaluators get ev_abc123
ax evaluators get "Relevance" --space sp_abc123
ax evaluators get ev_abc123 --version-id evv_xyz789
ax evaluators update
Update an evaluator’s name or description. At least one of --name or --description is required.
ax evaluators update <name-or-id> [--space <id>] [--name <name>] [--description <desc>]
| Option | Description |
|---|
--space | Space name or ID (required when using evaluator name instead of ID) |
--name | New evaluator name |
--description | New evaluator description |
ax evaluators update ev_abc123 --name "Relevance v2" --description "Updated scoring rubric"
ax evaluators delete
Delete an evaluator and all its versions. This operation is irreversible.
ax evaluators delete <name-or-id> [--space <id>] [--force]
| Option | Description |
|---|
--space | Space name or ID (required when using evaluator name instead of ID) |
--force | Skip the confirmation prompt |
ax evaluators delete ev_abc123
ax evaluators delete ev_abc123 --force
ax evaluators delete "Relevance" --space sp_abc123 --force
ax evaluators list-versions
List all versions of an evaluator.
ax evaluators list-versions <name-or-id> [--space <id>] [--limit <n>] [--cursor <cursor>]
| Option | Description |
|---|
--space | Space name or ID (required when using evaluator name instead of ID) |
--limit | Maximum number of versions to return (default: 15) |
--cursor | Pagination cursor for the next page |
ax evaluators list-versions ev_abc123
ax evaluators create-version
Create a new version of an existing evaluator. Versions are immutable once created; the new version becomes the latest immediately. Required options will be prompted interactively if not passed as flags.
ax evaluators create-version <name-or-id> \
--commit-message <message> \
--template-name <name> \
--template <template-string> \
--ai-integration-id <id> \
--model-name <model>
| Option | Description |
|---|
--space | Space name or ID (required when using evaluator name instead of ID) |
--commit-message | Commit message describing the changes in this version |
--template-name | Eval column name |
--template | Updated prompt template string with {{variable}} placeholders |
--ai-integration-id | AI integration global ID |
--model-name | Model name (e.g. gpt-4o) |
--include-explanations | Include reasoning explanation alongside the score (flag) |
--use-function-calling | Prefer structured function-call output when supported (flag) |
--invocation-params | JSON object of model invocation parameters |
--provider-params | JSON object of provider-specific parameters |
ax evaluators create-version ev_abc123 \
--commit-message "Improved prompt for edge cases" \
--template-name "Relevance" \
--template "Rate the relevance of the response on a scale of 0 to 1.\nQuery: {{input.value}}\nResponse: {{output.value}}" \
--ai-integration-id ai_xyz789 \
--model-name gpt-4o
ax evaluators get-version
Get a specific evaluator version by ID.
ax evaluators get-version <version-id>
ax evaluators get-version evv_xyz789
