Create an annotation queue

curl --request POST \
  --url https://api.arize.com/v2/annotation-queues \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "name": "Quality Review Queue",
  "space_id": "spc_xyz789",
  "annotation_config_ids": [
    "ac_abc123"
  ],
  "annotator_emails": [
    "annotator1@example.com"
  ],
  "assignment_method": "all"
}
'

{
  "id": "aq_abc123",
  "name": "Quality Review Queue",
  "space_id": "spc_xyz789",
  "instructions": "Review each response for accuracy and helpfulness",
  "annotation_configs": [
    {
      "id": "ac_001",
      "name": "accuracy",
      "type": "continuous",
      "space_id": "spc_xyz789",
      "minimum_score": 0,
      "maximum_score": 10,
      "optimization_direction": "maximize",
      "created_at": "2024-01-10T08:00:00Z"
    },
    {
      "id": "ac_002",
      "name": "sentiment",
      "type": "categorical",
      "space_id": "spc_xyz789",
      "values": [
        {
          "label": "positive",
          "score": 1
        },
        {
          "label": "neutral",
          "score": 0
        },
        {
          "label": "negative",
          "score": -1
        }
      ],
      "optimization_direction": "maximize",
      "created_at": "2024-01-10T08:00:00Z"
    }
  ],
  "annotators": [
    {
      "id": "usr_001",
      "email": "reviewer@example.com"
    },
    {
      "id": "usr_002",
      "email": "annotator@example.com"
    }
  ],
  "created_at": "2024-01-15T10:30:00Z",
  "updated_at": "2024-01-15T10:30:00Z"
}

Annotation Queues

Create an annotation queue

Create a new annotation queue.

Payload Requirements

The annotation queue name must be unique within the given space (among active queues).
At least one annotation_config_id is required, and all configs must belong to the specified space.
Do not include system-managed fields on input: id, created_at, updated_at.
If assignment_method is not provided, it defaults to "all".

Valid example

{
  "name": "Quality Review Queue",
  "space_id": "spc_xyz789",
  "annotation_config_ids": ["ac_abc123"],
  "annotator_emails": ["reviewer@example.com"],
  "assignment_method": "all"
}

Valid example with records

{
  "name": "Quality Review Queue",
  "space_id": "spc_xyz789",
  "annotation_config_ids": ["ac_abc123"],
  "annotator_emails": ["reviewer@example.com"],
  "records": [
    {"record_type": "span", "project_id": "prj_abc", "start_time": "2024-01-15T00:00:00Z", "end_time": "2024-01-16T00:00:00Z", "span_ids": ["span_001"]},
    {"record_type": "example", "dataset_id": "ds_xyz", "example_ids": ["ex_001", "ex_002"]}
  ]
}

Invalid example (missing required annotation_config_ids)

{
  "name": "My Queue",
  "space_id": "spc_xyz789"
}

This endpoint is in beta, read more here.

POST

annotation-queues

Create an annotation queue

curl --request POST \
  --url https://api.arize.com/v2/annotation-queues \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "name": "Quality Review Queue",
  "space_id": "spc_xyz789",
  "annotation_config_ids": [
    "ac_abc123"
  ],
  "annotator_emails": [
    "annotator1@example.com"
  ],
  "assignment_method": "all"
}
'

{
  "id": "aq_abc123",
  "name": "Quality Review Queue",
  "space_id": "spc_xyz789",
  "instructions": "Review each response for accuracy and helpfulness",
  "annotation_configs": [
    {
      "id": "ac_001",
      "name": "accuracy",
      "type": "continuous",
      "space_id": "spc_xyz789",
      "minimum_score": 0,
      "maximum_score": 10,
      "optimization_direction": "maximize",
      "created_at": "2024-01-10T08:00:00Z"
    },
    {
      "id": "ac_002",
      "name": "sentiment",
      "type": "categorical",
      "space_id": "spc_xyz789",
      "values": [
        {
          "label": "positive",
          "score": 1
        },
        {
          "label": "neutral",
          "score": 0
        },
        {
          "label": "negative",
          "score": -1
        }
      ],
      "optimization_direction": "maximize",
      "created_at": "2024-01-10T08:00:00Z"
    }
  ],
  "annotators": [
    {
      "id": "usr_001",
      "email": "reviewer@example.com"
    },
    {
      "id": "usr_002",
      "email": "annotator@example.com"
    }
  ],
  "created_at": "2024-01-15T10:30:00Z",
  "updated_at": "2024-01-15T10:30:00Z"
}

Authorizations

Authorization

string

header

required

Most Arize AI endpoints require authentication. For those endpoints that require authentication, include your API key in the request header using the format

Body

application/json

Body containing annotation queue creation parameters

name

string

required

The name of the annotation queue. Must be unique within the space for active queues.

Maximum string length: 255

Example:

"Quality Review Queue"

space_id

string

required

The space ID that the annotation queue belongs to

Example:

"spc_xyz789"

annotation_config_ids

string[]

required

IDs of annotation configs to associate with this queue. All configs must belong to the same space.

Minimum array length: 1

Example:

["ac_abc123", "ac_def456"]

annotator_emails

string<email>[]

required

Email addresses of annotators to assign to the queue. Emails are resolved to user IDs server-side.

Minimum array length: 1

An email address

Example:

[
  "annotator1@example.com",
  "annotator2@example.com"
]

instructions

string

Instructions for annotators working on this queue

Maximum string length: 5000

Example:

"Review each response for accuracy and helpfulness"

assignment_method

enum<string>

default:all

How records are assigned to annotators. Defaults to all when omitted.

Available options:

all,

random

record_sources

object[]

Record sources to add to the annotation queue on creation. At most 2 record sources (projects or datasets) may be provided in a single create request. Additional records from other sources can be added after creation.

Maximum array length: 2

Option 1
Option 2

Show child attributes

Response

Returns the created annotation queue

string

required

The unique identifier for the annotation queue

Example:

"aq_abc123"

name

string

required

The name of the annotation queue

Example:

"Quality Review Queue"

space_id

string

required

The space id the annotation queue belongs to

Example:

"spc_xyz789"

annotators

object[]

required

Users assigned as annotators to this queue

Show child attributes

created_at

string<date-time>

required

The timestamp for when the annotation queue was created

Example:

"2024-01-15T10:30:00Z"

updated_at

string<date-time>

required

The timestamp for when the annotation queue was last updated

Example:

"2024-01-20T14:45:00Z"

instructions

string | null

The instructions for the annotation queue

Example:

"Review each response for accuracy and helpfulness"

annotation_configs

object[]

The annotation configs associated with this queue

Option 1
Option 2
Option 3

Show child attributes

List annotation queues Get an annotation queue

⌘I