> ## Documentation Index
> Fetch the complete documentation index at: https://arize-ax.mintlify.site/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Get evaluator version
> Get a specific evaluator version by its unique identifier.
This endpoint is in beta, read more [here](https://arize.com/docs/ax/rest-reference#api-version-stages).
## OpenAPI
````yaml https://api.arize.com/v2/spec.yaml get /v2/evaluator-versions/{version_id}
openapi: 3.0.3
info:
title: Arize REST API
version: 2.0.0
description: |
API specification for the backend data server. The API is hosted globally
at https://api.arize.com/v2 or in your own environment.
license:
name: Apache-2.0
url: https://www.apache.org/licenses/LICENSE-2.0
servers:
- description: Global
url: https://api.arize.com
- description: Regional
url: https://api.{region}.arize.com
variables:
region:
default: eu-west-1a
enum:
- eu-west-1a
- ca-central-1a
- description: Custom Host
url: https://{host}
variables:
host:
default: api.arize.com
security:
- bearerAuth: []
tags:
- name: AI Integrations
description: |
AI integrations configure access to external LLM providers (e.g. OpenAI,
Azure OpenAI, AWS Bedrock, Vertex AI). Integrations can be scoped to the
entire account, a specific organization, or a specific space.
- name: Annotation Configs
description: >
Annotation configs allow you to define consistent annotation schemas that
can be reused across your workspace, ensuring evaluations are structured
and
comparable over time.
- name: Annotation Queues
description: >
Annotation queues help you organize and manage human evaluation workflows.
Use queues to assign spans or examples to annotators for review and
labeling.
- name: API Keys
description: >
API keys are used to authenticate requests to the Arize API. List your
keys
to view metadata; the raw secret is never returned after creation.
- name: Audit Logs
description: >
Audit logs record authenticated user actions within an account, providing
a
chronological trail for security and compliance review. Access requires
account admin privileges and audit logging to be enabled.
- name: Datasets
description: |
Datasets are structured, version-controlled example collections you use to
run, evaluate, and track LLM experiments.
- name: Evaluators
description: >
Evaluators are reusable evaluation configurations used to assess the
quality
of LLM outputs. They can be template-based (using LLM judges) or
code-based.
- name: Experiments
description: >
Experiments let you systematically test prompt/model changes using
datasets,
tasks, and evaluators.
- name: Integrations
description: >
Integrations configure access to external LLM providers (e.g. OpenAI,
Azure OpenAI, AWS Bedrock, Vertex AI), notifications services (e.g.
PagerDuty, Slack), and
your own agents. Integrations can be scoped to the entire account, a
specific
organization, or a specific space.
- name: Monitors
description: >
Monitors continuously track a metric over your model or LLM application
data
and alert you when it crosses a threshold. Each monitor watches a single
metric - data quality, model performance, drift, a custom metric, or a
tracing metric - and moves between statuses as the metric passes in and
out of its
healthy range.
- name: Organizations
description: >
Organizations are top-level containers within an Arize AX account for
grouping spaces.
- name: Projects
description: |
Projects represent LLM applications being monitored in Arize where you can
observe traces and spans.
- name: Prompts
description: >
Prompts are reusable, versioned templates for LLM interactions. Use
prompts
to standardize and manage how you interact with LLMs across your
application.
- name: Resource Restrictions
description: |
Endpoints for restricting and unrestricting resources (projects, models).
- name: Role Bindings
description: |
Role bindings assign a role to a user on a resource. REST currently
supports space- and project-scoped bindings.
- name: Roles
description: >
Roles define sets of permissions that can be assigned to users within an
account. Create custom roles to tailor access control to your team's
needs.
- name: Spaces
description: >
Spaces are containers within an organization for grouping related
projects,
datasets, and experiments, enabling collaboration or isolated
experimentation
with role-based access control.
- name: Spans
description: |
Spans represent individual operations within a trace. A span captures the
timing, status, and attributes of a single operation in your application.
- name: Tasks
description: |
Tasks are configurable units of work that tie one or more evaluators to a
data source (project or dataset). Use tasks to automate evaluation of LLM
outputs, with support for continuous evaluation and backfill runs.
- name: Traces
description: |
A trace is the collection of spans sharing a trace ID, representing a
single end-to-end request through an LLM application. Use the Traces
endpoint to retrieve traces with all of their spans in one call.
- name: Users
description: >
Users represent members of an account. The Users endpoints allow creating,
listing, updating (display name), and removing users from the account
programmatically.
paths:
/v2/evaluator-versions/{version_id}:
get:
tags:
- Evaluators
summary: Get evaluator version
description: >
Get a specific evaluator version by its unique identifier.
This endpoint is in beta, read more
[here](https://arize.com/docs/ax/rest-reference#api-version-stages).
operationId: get_evaluator_version
parameters:
- $ref: '#/components/parameters/EvaluatorVersionIdPathParam'
responses:
'200':
$ref: '#/components/responses/EvaluatorVersion'
'400':
$ref: '#/components/responses/BadRequest'
'401':
$ref: '#/components/responses/Unauthorized'
'404':
$ref: '#/components/responses/NotFound'
'429':
$ref: '#/components/responses/RateLimitExceeded'
components:
parameters:
EvaluatorVersionIdPathParam:
name: version_id
in: path
required: true
description: The unique evaluator version identifier (base64)
schema:
$ref: '#/components/schemas/Id'
example: RXZhbHVhdG9yVmVyc2lvbjoxMjM0NQ==
responses:
EvaluatorVersion:
description: Returns an evaluator version
content:
application/json:
schema:
$ref: '#/components/schemas/EvaluatorVersion'
examples:
template_version:
summary: Template evaluator version
value:
id: RXZhbHVhdG9yVmVyc2lvbjoxMDpYeVp3
evaluator_id: RXZhbHVhdG9yOjEyOmFCY0Q=
commit_hash: a3b1c9e4f7d2a0518e6c3bb9217f87d1c4e810f2
commit_message: Initial version
type: TEMPLATE
template_config:
name: hallucination
template: You are an evaluation assistant...
include_explanations: true
use_function_calling_if_available: true
classification_choices:
hallucinated: 0
factual: 1
direction: MAXIMIZE
data_granularity: SPAN
llm_config:
ai_integration_id: TGxtSW50ZWdyYXRpb246MTI6YUJjRA==
model_name: gpt-4o
invocation_parameters:
temperature: 0
provider_parameters: {}
created_at: '2026-02-16T22:05:48.143Z'
created_by_user_id: VXNlcjoxOm5OYkM=
code_version:
summary: Managed code evaluator version
value:
id: RXZhbHVhdG9yVmVyc2lvbjoxMjpEZUZn
evaluator_id: RXZhbHVhdG9yOjEzOmNEZUY=
commit_hash: d6e4f2g7b0c3d1841h9f6ee2540i10g4f7h143i5
commit_message: Tune keywords
type: CODE
code_config:
type: MANAGED
name: keywords_eval
managed_evaluator: CONTAINS_ANY_KEYWORD
variables:
- output
static_params:
- name: keywords
type: STRING_ARRAY
default_value:
- alpha
- beta
data_granularity: SPAN
query_filter: null
created_at: '2026-02-16T22:05:48.143Z'
created_by_user_id: VXNlcjoxOm5OYkM=
BadRequest:
description: Invalid request
content:
application/problem+json:
schema:
$ref: '#/components/schemas/Problem'
example:
status: 400
title: Invalid request parameters
detail: The 'name' field is required and must be a non-empty string.
instance: /resource
type: https://arize.com/docs/ax/rest-reference/errors#invalid-request
Unauthorized:
description: Authentication is required
content:
application/problem+json:
schema:
$ref: '#/components/schemas/Problem'
example:
status: 401
title: Authentication required
detail: You must be authenticated to access this resource.
instance: /resource
type: >-
https://arize.com/docs/ax/rest-reference/errors#authentication-required
NotFound:
description: Not found
content:
application/problem+json:
schema:
$ref: '#/components/schemas/Problem'
example:
status: 404
title: Resource not found
detail: The requested resource with ID '12345' was not found.
instance: /resource/12345
type: https://arize.com/docs/ax/rest-reference/errors#resource-not-found
RateLimitExceeded:
description: Rate limit exceeded
headers:
Retry-After:
description: |
When throttled (429), how long to wait before retrying. Value is
either a delta-seconds integer.
schema:
type: integer
minimum: 0
example: 42
content:
application/problem+json:
schema:
$ref: '#/components/schemas/Problem'
example:
status: 429
title: Rate limit exceeded
detail: >-
You have exceeded the allowed number of requests. Please try again
later.
instance: /resource
type: >-
https://arize.com/docs/ax/rest-reference/errors#rate-limit-exceeded
schemas:
Id:
type: string
description: A universally unique identifier (base64-encoded opaque string).
example: RW50aXR5OjEyMzQ1
EvaluatorVersion:
oneOf:
- $ref: '#/components/schemas/EvaluatorVersionTemplate'
- $ref: '#/components/schemas/EvaluatorVersionCode'
- $ref: '#/components/schemas/EvaluatorVersionHarness'
- $ref: '#/components/schemas/EvaluatorVersionRemote'
discriminator:
propertyName: type
mapping:
TEMPLATE:
$ref: '#/components/schemas/EvaluatorVersionTemplate'
CODE:
$ref: '#/components/schemas/EvaluatorVersionCode'
HARNESS:
$ref: '#/components/schemas/EvaluatorVersionHarness'
REMOTE:
$ref: '#/components/schemas/EvaluatorVersionRemote'
description: |
A versioned snapshot of an evaluator's configuration. The `type` field
discriminates the branch and matches the parent evaluator's `type`.
Problem:
type: object
description: RFC 9457 Problem Details
properties:
title:
type: string
description: A short, human-readable summary of the problem type
status:
type: integer
description: >-
The HTTP status code generated by the origin server for this
occurrence of the problem
type:
type: string
format: uri-reference
description: A URI reference that identifies the problem type
detail:
type: string
description: >-
A human-readable explanation specific to this occurrence of the
problem
instance:
type: string
format: uri-reference
description: >-
A URI reference that identifies the specific occurrence of the
problem
required:
- title
- status
additionalProperties: false
EvaluatorVersionTemplate:
allOf:
- $ref: '#/components/schemas/EvaluatorVersionCommon'
- type: object
required:
- template_config
properties:
type:
type: string
enum:
- TEMPLATE
description: >-
Discriminator identifying this as a template evaluator version.
Always `TEMPLATE` for this variant.
template_config:
$ref: '#/components/schemas/TemplateConfig'
description: The LLM template configuration for this version
additionalProperties: false
description: Evaluator version carrying a template (LLM) configuration.
EvaluatorVersionCode:
allOf:
- $ref: '#/components/schemas/EvaluatorVersionCommon'
- type: object
required:
- code_config
properties:
type:
type: string
enum:
- CODE
description: >-
Discriminator identifying this as a code evaluator version.
Always `CODE` for this variant.
code_config:
$ref: '#/components/schemas/CodeConfig'
description: The code evaluator configuration for this version
additionalProperties: false
description: Evaluator version carrying a code configuration.
EvaluatorVersionHarness:
allOf:
- $ref: '#/components/schemas/EvaluatorVersionCommon'
- type: object
properties:
type:
type: string
enum:
- HARNESS
description: Discriminator identifying this as a harness evaluator version.
additionalProperties: false
description: |
Evaluator version backed by a harness evaluation config. Only common
version metadata (id, commit info, timestamps) is returned — the harness
configuration is not yet accessible and will be a future addition.
EvaluatorVersionRemote:
allOf:
- $ref: '#/components/schemas/EvaluatorVersionCommon'
- type: object
properties:
type:
type: string
enum:
- REMOTE
description: Discriminator identifying this as a remote evaluator version.
additionalProperties: false
description: |
Evaluator version backed by a remote evaluation config. Only common
version metadata (id, commit info, timestamps) is returned — the remote
configuration is not yet accessible and will be a future addition.
EvaluatorVersionCommon:
type: object
required:
- id
- evaluator_id
- commit_hash
- commit_message
- created_at
- created_by_user_id
- type
properties:
id:
type: string
description: The unique identifier for this version
evaluator_id:
type: string
description: The parent evaluator ID
commit_hash:
type: string
description: A unique hash identifying this version
commit_message:
type: string
nullable: true
description: A message describing the changes in this version
created_at:
type: string
format: date-time
description: When this version was created
created_by_user_id:
type: string
nullable: true
description: The unique identifier for the user who created this version
type:
$ref: '#/components/schemas/EvaluatorType'
TemplateConfig:
type: object
required:
- name
- template
- include_explanations
- use_function_calling_if_available
- llm_config
properties:
name:
type: string
description: Eval column name. Must match ^[a-zA-Z0-9_\s\-&()]+$
template:
type: string
description: The prompt template with variable placeholders
include_explanations:
type: boolean
description: Whether to include explanations in the evaluation output
use_function_calling_if_available:
type: boolean
description: Whether to use function calling if the model supports it
use_structured_output:
type: boolean
default: true
description: Whether to use structured output if the model supports it
classification_choices:
type: object
nullable: true
additionalProperties:
type: number
description: >-
Map of choice label to numeric score (e.g. {"relevant": 1,
"irrelevant": 0}). When omitted, the evaluator produces freeform
(non-classification) output.
direction:
allOf:
- $ref: '#/components/schemas/OptimizationDirection'
default: MAXIMIZE
description: >-
Direction for optimization applied to this template's evaluation
scores. Defaults to `MAXIMIZE` when omitted.
data_granularity:
allOf:
- $ref: '#/components/schemas/DataGranularity'
nullable: true
description: Data granularity level. Defaults to null when omitted.
llm_config:
$ref: '#/components/schemas/EvaluatorLlmConfig'
description: The LLM configuration for executing the template
additionalProperties: false
CodeConfig:
description: >
Discriminated union representing either a managed (built-in) or custom
(user-supplied Python) code evaluator configuration, resolved by the
nested `type` field (`MANAGED` -> `ManagedCodeConfig`, `CUSTOM` ->
`CustomCodeConfig`).
This inner `type` is independent of the parent evaluator version's
`type` (which is always `CODE` here).
oneOf:
- $ref: '#/components/schemas/ManagedCodeConfig'
- $ref: '#/components/schemas/CustomCodeConfig'
discriminator:
propertyName: type
mapping:
MANAGED:
$ref: '#/components/schemas/ManagedCodeConfig'
CUSTOM:
$ref: '#/components/schemas/CustomCodeConfig'
EvaluatorType:
type: string
enum:
- TEMPLATE
- CODE
- HARNESS
- REMOTE
description: >
The evaluator type:
- `TEMPLATE` — LLM-based evaluator.
- `CODE` — managed built-in evaluators or custom Python code (both are
subtypes of `CODE`, discriminated by the nested `CodeConfig.type` =
`MANAGED` | `CUSTOM`).
- `HARNESS` — test harness evaluator.
- `REMOTE` — remote evaluator.
Applies to both the parent `Evaluator.type` field and every version's
`type`
discriminator — a version's `type` must always match its parent
evaluator's `type`.
OptimizationDirection:
type: string
enum:
- MAXIMIZE
- MINIMIZE
- NONE
default: NONE
description: |
The direction for optimization. Defaults to `NONE` when omitted.
- MAXIMIZE: higher scores are better
- MINIMIZE: lower scores are better
- NONE: higher or lower scores are neither better nor worse
DataGranularity:
type: string
enum:
- SPAN
- TRACE
- SESSION
description: |
Data granularity level for evaluation.
- SPAN - Evaluate at the individual span level.
- TRACE - Evaluate at the full trace level.
- SESSION - Evaluate at the session level.
EvaluatorLlmConfig:
type: object
required:
- ai_integration_id
- model_name
- invocation_parameters
- provider_parameters
properties:
ai_integration_id:
type: string
description: AI integration identifier (base64)
model_name:
type: string
description: Model name (e.g. gpt-4o)
invocation_parameters:
$ref: '#/components/schemas/InvocationParams'
provider_parameters:
$ref: '#/components/schemas/ProviderParams'
additionalProperties: false
ManagedCodeConfig:
type: object
required:
- type
- name
- managed_evaluator
- variables
properties:
data_granularity:
allOf:
- $ref: '#/components/schemas/DataGranularity'
nullable: true
description: >
Data granularity level for evaluation. When omitted or null, no
granularity
filter is applied (span-level evaluation is used by default on the
server).
query_filter:
type: string
nullable: true
description: >
Optional filter query over the chosen data granularity. When omitted
or null,
no filter is applied.
type:
type: string
enum:
- MANAGED
description: >-
Discriminator identifying this as a managed (built-in) code
evaluator
name:
type: string
description: Eval column name. Must match ^[a-zA-Z0-9_\s\-&()]+$
pattern: ^[a-zA-Z0-9_\s\-&()]+$
managed_evaluator:
$ref: '#/components/schemas/ManagedCodeEvaluator'
description: Which managed code evaluator implementation to use
variables:
type: array
items:
type: string
description: >
Dataset columns or span attributes passed into the evaluator (order
and count
must match the managed evaluator's requirements).
static_params:
type: array
items:
$ref: '#/components/schemas/StaticParam'
description: >
Static parameters for the managed evaluator (see registry `args`).
When omitted,
the registry's required arguments must be satisfied by defaults on
the evaluator
class; otherwise validation fails with 400. If the registry has no
args, omitting
this field is equivalent to an empty list.
additionalProperties: false
CustomCodeConfig:
type: object
required:
- type
- name
- code
- variables
properties:
data_granularity:
allOf:
- $ref: '#/components/schemas/DataGranularity'
nullable: true
description: >
Data granularity level for evaluation. When omitted or null, no
granularity
filter is applied (span-level evaluation is used by default on the
server).
query_filter:
type: string
nullable: true
description: >
Optional filter query over the chosen data granularity. When omitted
or null,
no filter is applied.
type:
type: string
enum:
- CUSTOM
description: >-
Discriminator identifying this as a custom (user-supplied Python)
code evaluator
name:
type: string
description: Eval column name. Must match ^[a-zA-Z0-9_\s\-&()]+$
pattern: ^[a-zA-Z0-9_\s\-&()]+$
code:
type: string
description: Python source defining the evaluator class
imports:
type: string
nullable: true
description: Optional package import block prepended when running the evaluator
variables:
type: array
items:
type: string
description: Dataset columns or span attributes mapped to evaluate() arguments
static_params:
type: array
items:
$ref: '#/components/schemas/StaticParam'
description: >
Optional typed defaults accessible on the evaluator instance. Omit
or pass an
empty array when the custom class does not read any static
parameters.
additionalProperties: false
InvocationParams:
type: object
description: Parameters for the LLM invocation
properties:
temperature:
type: number
description: Sampling temperature (higher = more random)
max_tokens:
type: integer
description: Maximum number of tokens to generate
max_completion_tokens:
type: integer
description: Maximum number of completion tokens to generate
top_p:
type: number
description: Nucleus sampling parameter
frequency_penalty:
type: number
description: Frequency penalty (-2.0 to 2.0)
presence_penalty:
type: number
description: Presence penalty (-2.0 to 2.0)
stop:
type: array
items:
type: string
description: Stop sequences
response_format:
allOf:
- $ref: '#/components/schemas/ResponseFormat'
description: >-
Response format configuration. Optional. When omitted, no structured
output constraint is applied (the provider's default plain-text
behavior is used).
tool_config:
allOf:
- $ref: '#/components/schemas/ToolConfig'
description: >-
Tool configuration for the LLM invocation. Optional. When omitted,
no tools are made available to the model.
top_k:
type: integer
description: >-
Top-K sampling parameter. A top-K of 1 means the next selected token
is the most probable (greedy decoding).
thinking_level:
type: string
description: >-
Controls how much reasoning the model performs before responding.
Supported by Gemini 3.x models. Accepted values: 'low', 'high'.
thinking_budget:
type: integer
description: >-
Maximum tokens the model may use for internal reasoning. Supported
by Gemini 2.5 models. Range: 0-24576 (Flash/Flash-Lite) or 128-32768
(Pro). Set 0 to disable thinking on Flash models.
reasoning_effort:
type: string
description: >-
Controls how much reasoning the model performs before responding.
Supported by OpenAI o-series and GPT-5 models. o-series: 'low' |
'medium' | 'high'. GPT-5: 'none' | 'low' | 'medium' | 'high' |
'xhigh'.
verbosity:
type: string
description: >-
Controls the verbosity of model output. Supported by OpenAI GPT-5
series. Accepted values: 'low' | 'medium' | 'high'.
additionalProperties: true
ProviderParams:
type: object
description: Provider-specific parameters
properties:
azure_params:
allOf:
- $ref: '#/components/schemas/AzureParams'
description: Azure OpenAI specific parameters
anthropic_headers:
allOf:
- $ref: '#/components/schemas/AnthropicHeaders'
description: Anthropic-specific headers
anthropic_version:
type: string
description: Anthropic API version
bedrock_options:
allOf:
- $ref: '#/components/schemas/BedrockOptions'
description: AWS Bedrock options
region:
type: string
description: Region for the model deployment
additionalProperties: true
ManagedCodeEvaluator:
type: string
enum:
- MATCHES_REGEX
- JSON_PARSEABLE
- CONTAINS_ANY_KEYWORD
- CONTAINS_ALL_KEYWORDS
- EXACT_MATCH
description: Built-in managed code evaluator name
StaticParam:
type: object
required:
- name
- type
- default_value
properties:
name:
type: string
description: Parameter name (matches the managed evaluator's argument name)
type:
$ref: '#/components/schemas/StaticParamType'
default_value:
oneOf:
- type: string
- type: array
items:
type: string
description: >
Default value. Must be a string when `type` is STRING or REGEX, and
a string
array when `type` is STRING_ARRAY. Mismatches are rejected with 400
by the server.
additionalProperties: false
ResponseFormat:
type: object
description: Response format configuration
properties:
type:
allOf:
- $ref: '#/components/schemas/ResponseFormatType'
default: TEXT
description: The response format type. Defaults to `TEXT` if not specified.
json_schema:
allOf:
- $ref: '#/components/schemas/JsonSchemaConfig'
description: JSON schema configuration (when type is JSON_SCHEMA)
ToolConfig:
type: object
description: Tool configuration for the LLM invocation
properties:
tools:
type: array
description: List of tool definitions available to the model
items:
$ref: '#/components/schemas/ToolDefinition'
tool_choice:
description: Tool choice configuration
AzureParams:
type: object
description: Azure OpenAI specific parameters
properties:
azure_deployment_name:
type: string
description: The Azure deployment name
azure_openai_endpoint:
type: string
description: The Azure OpenAI endpoint URL
azure_openai_version:
type: string
description: The Azure OpenAI API version
AnthropicHeaders:
type: object
description: Anthropic-specific headers
properties:
anthropic_beta:
type: array
description: Anthropic beta feature flags
items:
type: string
nullable: true
nullable: true
BedrockOptions:
type: object
description: AWS Bedrock options
properties:
use_converse_endpoint:
type: boolean
default: false
description: >-
Whether to use the AWS Bedrock Converse endpoint. Defaults to
`false`.
StaticParamType:
type: string
enum:
- STRING
- STRING_ARRAY
- REGEX
description: |
Argument type for static evaluator parameters.
- STRING - A single string value.
- STRING_ARRAY - An array of string values.
- REGEX - A regular expression string.
ResponseFormatType:
type: string
enum:
- TEXT
- JSON_OBJECT
- JSON_SCHEMA
description: The response format type
JsonSchemaConfig:
type: object
description: JSON schema configuration (when type is JSON_SCHEMA)
properties:
name:
type: string
description: The name of the JSON schema
description:
type: string
description: A description of the JSON schema
schema:
type: object
description: The JSON schema object
strict:
type: boolean
default: false
description: Whether to enforce strict schema validation. Defaults to `false`.
ToolDefinition:
type: object
description: A tool definition available to the model
securitySchemes:
bearerAuth:
type: http
scheme: bearer
bearerFormat:
description: >
Most Arize AI endpoints require authentication. For those endpoints that
require authentication, include your API key in the request header using
the format
``` Authorization: Bearer
```
````