curl --request POST \
  --url https://api.arize.com/v2/evaluators \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "space_id": "U3BhY2U6NDkzOkJaSkc=",
  "name": "Hallucination Eval",
  "description": "Detects hallucinated content in LLM responses",
  "type": "TEMPLATE",
  "version": {
    "commit_message": "Initial version",
    "template_config": {
      "name": "hallucination",
      "template": "You are an evaluation assistant. Given the following input and output, determine if the output contains hallucinated content.\n\nInput: {input}\nOutput: {output}\nReference: {reference}",
      "include_explanations": true,
      "use_function_calling": true,
      "classification_choices": {
        "hallucinated": 0,
        "factual": 1
      },
      "direction": "MAXIMIZE",
      "data_granularity": "SPAN",
      "llm_config": {
        "ai_integration_id": "TGxtSW50ZWdyYXRpb246MTI6YUJjRA==",
        "model_name": "gpt-4o",
        "invocation_parameters": {
          "temperature": 0
        },
        "provider_parameters": {}
      }
    }
  }
}
'

{
  "id": "RXZhbHVhdG9yOjEyOmFCY0Q=",
  "name": "Hallucination Eval",
  "description": "Detects hallucinated content in LLM responses",
  "type": "TEMPLATE",
  "space_id": "U3BhY2U6NDkzOkJaSkc=",
  "created_at": "2026-02-16T22:05:47.900Z",
  "updated_at": "2026-02-16T22:05:48.150Z",
  "created_by_user_id": "VXNlcjoxOm5OYkM=",
  "version": {
    "id": "RXZhbHVhdG9yVmVyc2lvbjoxMDpYeVp3",
    "evaluator_id": "RXZhbHVhdG9yOjEyOmFCY0Q=",
    "commit_hash": "a3b1c9e4f7d2a0518e6c3bb9217f87d1c4e810f2",
    "commit_message": "Initial version",
    "type": "TEMPLATE",
    "template_config": {
      "name": "hallucination",
      "template": "You are an evaluation assistant...",
      "include_explanations": true,
      "use_function_calling": true,
      "classification_choices": {
        "hallucinated": 0,
        "factual": 1
      },
      "direction": "MAXIMIZE",
      "data_granularity": "SPAN",
      "llm_config": {
        "ai_integration_id": "TGxtSW50ZWdyYXRpb246MTI6YUJjRA==",
        "model_name": "gpt-4o",
        "invocation_parameters": {
          "temperature": 0
        },
        "provider_parameters": {}
      }
    },
    "created_at": "2026-02-16T22:05:48.143Z",
    "created_by_user_id": "VXNlcjoxOm5OYkM="
  }
}

{
  "status": 400,
  "title": "Invalid request parameters",
  "detail": "The 'name' field is required and must be a non-empty string.",
  "instance": "/resource",
  "type": "https://arize.com/docs/ax/rest-reference/errors#invalid-request"
}

{
  "status": 401,
  "title": "Authentication required",
  "detail": "You must be authenticated to access this resource.",
  "instance": "/resource",
  "type": "https://arize.com/docs/ax/rest-reference/errors#authentication-required"
}

{
  "status": 403,
  "title": "Access forbidden",
  "detail": "You do not have permission to access this resource.",
  "instance": "/resource/12345",
  "type": "https://arize.com/docs/ax/rest-reference/errors#access-forbidden"
}

{
  "status": 404,
  "title": "Resource not found",
  "detail": "The requested resource with ID '12345' was not found.",
  "instance": "/resource/12345",
  "type": "https://arize.com/docs/ax/rest-reference/errors#resource-not-found"
}

{
  "status": 409,
  "title": "Resource conflict",
  "detail": "A resource with the given identifier already exists.",
  "instance": "/resource",
  "type": "https://arize.com/docs/ax/rest-reference/errors#resource-conflict"
}

{
  "status": 422,
  "title": "Unprocessable Entity",
  "detail": "One or more fields failed validation.",
  "instance": "/resource/12345",
  "type": "https://arize.com/docs/ax/rest-reference/errors#unprocessable-entity"
}

{
  "status": 429,
  "title": "Rate limit exceeded",
  "detail": "You have exceeded the allowed number of requests. Please try again later.",
  "instance": "/resource",
  "type": "https://arize.com/docs/ax/rest-reference/errors#rate-limit-exceeded"
}

Evaluators

Create evaluator

Creates a new evaluator with an initial version.

Payload Requirements

The evaluator name must be unique within the given space.
type (top-level) selects the evaluator kind: TEMPLATE, CODE, or REMOTE. With TEMPLATE, provide version.template_config. With CODE, provide version.code_config — where code_config.type is MANAGED or CUSTOM (a separate discriminator within code_config, independent of the top-level type: CODE). With REMOTE, provide version.remote_config.integration_id referencing an accessible EVALUATOR integration. Remote evaluator creation requires the remote evaluators feature to be enabled; otherwise the request returns 403.
For template evaluators: version.template_config.name is the eval column name; must match ^[a-zA-Z0-9_\s\-&()]+$.
For template evaluators: version.template_config.template is the prompt template; use {variable} for placeholders (f-string format, e.g. {input}, {output}).
For template evaluators: version.template_config.classification_choices is required and maps choice labels to numeric scores (e.g. {"relevant": 1, "irrelevant": 0}).
For code evaluators: see CodeConfig — managed evaluators (code_config.type: MANAGED) use managed_evaluator and variables; custom evaluators (code_config.type: CUSTOM) use code, optional imports, and variables.
System-managed fields (id, created_at, updated_at, created_by_user_id) are rejected on input.

Valid example (template evaluator)

{
  "name": "Hallucination Detector",
  "space_id": "U3BhY2U6MTpWNEth",
  "type": "TEMPLATE",
  "version": {
    "commit_message": "Initial version",
    "template_config": {
      "name": "hallucination",
      "template": "Given the input: {input}\nand the output: {output}\nIs the output a hallucination?",
      "include_explanations": true,
      "use_function_calling": true,
      "classification_choices": {"hallucinated": 0, "factual": 1},
      "llm_config": {
        "ai_integration_id": "TGxtSW50ZWdyYXRpb246MTI6YUJjRA==",
        "model_name": "gpt-4o",
        "invocation_parameters": {"temperature": 0},
        "provider_parameters": {}
      }
    }
  }
}

Invalid example (type/config mismatch — TEMPLATE type with code_config)

{
  "name": "Bad Evaluator",
  "space_id": "U3BhY2U6MTpWNEth",
  "type": "TEMPLATE",
  "version": {
    "commit_message": "Wrong config",
    "code_config": {
      "type": "CUSTOM",
      "name": "my_eval",
      "code": "class Evaluator: ...",
      "variables": ["input"]
    }
  }
}

This endpoint is in beta, read more here.

POST

evaluators

curl --request POST \
  --url https://api.arize.com/v2/evaluators \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "space_id": "U3BhY2U6NDkzOkJaSkc=",
  "name": "Hallucination Eval",
  "description": "Detects hallucinated content in LLM responses",
  "type": "TEMPLATE",
  "version": {
    "commit_message": "Initial version",
    "template_config": {
      "name": "hallucination",
      "template": "You are an evaluation assistant. Given the following input and output, determine if the output contains hallucinated content.\n\nInput: {input}\nOutput: {output}\nReference: {reference}",
      "include_explanations": true,
      "use_function_calling": true,
      "classification_choices": {
        "hallucinated": 0,
        "factual": 1
      },
      "direction": "MAXIMIZE",
      "data_granularity": "SPAN",
      "llm_config": {
        "ai_integration_id": "TGxtSW50ZWdyYXRpb246MTI6YUJjRA==",
        "model_name": "gpt-4o",
        "invocation_parameters": {
          "temperature": 0
        },
        "provider_parameters": {}
      }
    }
  }
}
'

{
  "id": "RXZhbHVhdG9yOjEyOmFCY0Q=",
  "name": "Hallucination Eval",
  "description": "Detects hallucinated content in LLM responses",
  "type": "TEMPLATE",
  "space_id": "U3BhY2U6NDkzOkJaSkc=",
  "created_at": "2026-02-16T22:05:47.900Z",
  "updated_at": "2026-02-16T22:05:48.150Z",
  "created_by_user_id": "VXNlcjoxOm5OYkM=",
  "version": {
    "id": "RXZhbHVhdG9yVmVyc2lvbjoxMDpYeVp3",
    "evaluator_id": "RXZhbHVhdG9yOjEyOmFCY0Q=",
    "commit_hash": "a3b1c9e4f7d2a0518e6c3bb9217f87d1c4e810f2",
    "commit_message": "Initial version",
    "type": "TEMPLATE",
    "template_config": {
      "name": "hallucination",
      "template": "You are an evaluation assistant...",
      "include_explanations": true,
      "use_function_calling": true,
      "classification_choices": {
        "hallucinated": 0,
        "factual": 1
      },
      "direction": "MAXIMIZE",
      "data_granularity": "SPAN",
      "llm_config": {
        "ai_integration_id": "TGxtSW50ZWdyYXRpb246MTI6YUJjRA==",
        "model_name": "gpt-4o",
        "invocation_parameters": {
          "temperature": 0
        },
        "provider_parameters": {}
      }
    },
    "created_at": "2026-02-16T22:05:48.143Z",
    "created_by_user_id": "VXNlcjoxOm5OYkM="
  }
}

{
  "status": 400,
  "title": "Invalid request parameters",
  "detail": "The 'name' field is required and must be a non-empty string.",
  "instance": "/resource",
  "type": "https://arize.com/docs/ax/rest-reference/errors#invalid-request"
}

{
  "status": 401,
  "title": "Authentication required",
  "detail": "You must be authenticated to access this resource.",
  "instance": "/resource",
  "type": "https://arize.com/docs/ax/rest-reference/errors#authentication-required"
}

{
  "status": 403,
  "title": "Access forbidden",
  "detail": "You do not have permission to access this resource.",
  "instance": "/resource/12345",
  "type": "https://arize.com/docs/ax/rest-reference/errors#access-forbidden"
}

{
  "status": 404,
  "title": "Resource not found",
  "detail": "The requested resource with ID '12345' was not found.",
  "instance": "/resource/12345",
  "type": "https://arize.com/docs/ax/rest-reference/errors#resource-not-found"
}

{
  "status": 409,
  "title": "Resource conflict",
  "detail": "A resource with the given identifier already exists.",
  "instance": "/resource",
  "type": "https://arize.com/docs/ax/rest-reference/errors#resource-conflict"
}

{
  "status": 422,
  "title": "Unprocessable Entity",
  "detail": "One or more fields failed validation.",
  "instance": "/resource/12345",
  "type": "https://arize.com/docs/ax/rest-reference/errors#unprocessable-entity"
}

{
  "status": 429,
  "title": "Rate limit exceeded",
  "detail": "You have exceeded the allowed number of requests. Please try again later.",
  "instance": "/resource",
  "type": "https://arize.com/docs/ax/rest-reference/errors#rate-limit-exceeded"
}

Authorizations

Authorization

string

header

required

Most Arize AI endpoints require authentication. For those endpoints that require authentication, include your API key in the request header using the format

Body

application/json

Body containing evaluator creation parameters with an initial version.

type: TEMPLATE, type: CODE, and type: REMOTE are accepted on creation.

Body containing evaluator creation parameters with an initial version.

space_id

string

required

Space identifier (base64)

name

string

required

Evaluator name (must be unique within the space)

type

enum<string>

required

The evaluator type:

TEMPLATE — LLM-based evaluator.
CODE — managed built-in evaluators or custom Python code (both are subtypes of CODE, discriminated by the nested CodeConfig.type = MANAGED | CUSTOM).
HARNESS — test harness evaluator.
REMOTE — remote evaluator.

Applies to both the parent Evaluator.type field and every version's type discriminator — a version's type must always match its parent evaluator's type.

Available options:

TEMPLATE,

CODE,

HARNESS,

REMOTE

version

object

required

Payload for an evaluator version: exactly one of template_config, code_config, or remote_config. Used both when creating an evaluator (initial version) and when appending a version.

Option 1
Option 2
Option 3

Show child attributes

description

string

Evaluator description

Response

Returns an evaluator with a resolved version

An evaluator defines reusable evaluation logic that can be attached to evaluation tasks. The type field determines the kind of evaluation: TEMPLATE (LLM-based template evaluation), CODE (custom code evaluation), or REMOTE (externally hosted evaluation).

string

required

The unique identifier for the evaluator

name

string

required

The name of the evaluator

type

enum<string>

required

The evaluator type:

TEMPLATE — LLM-based evaluator.
CODE — managed built-in evaluators or custom Python code (both are subtypes of CODE, discriminated by the nested CodeConfig.type = MANAGED | CUSTOM).
HARNESS — test harness evaluator.
REMOTE — remote evaluator.

Applies to both the parent Evaluator.type field and every version's type discriminator — a version's type must always match its parent evaluator's type.

Available options:

TEMPLATE,

CODE,

HARNESS,

REMOTE

space_id

string

required

The unique identifier for the space the evaluator belongs to

created_at

string<date-time>

required

When the evaluator was created

updated_at

string<date-time>

required

When the evaluator was last updated

created_by_user_id

string | null

required

The unique identifier for the user who created the evaluator

version

object

required

The resolved version of the evaluator

Option 1
Option 2
Option 3
Option 4

Show child attributes

description

string | null

The description of the evaluator

Skills

MCP server

API reference

API clients

Create evaluator

Authorizations

Body

Response