Create an annotation queue

curl --request POST \
  --url https://api.arize.com/v2/annotation-queues \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "name": "Quality Review Queue",
  "space_id": "spc_xyz789",
  "annotation_config_ids": [
    "ac_abc123"
  ],
  "assignment_method": "all"
}
'

{
  "id": "aq_abc123",
  "name": "Quality Review Queue",
  "space_id": "spc_xyz789",
  "instructions": "Review each response for accuracy and helpfulness",
  "created_at": "2024-01-15T10:30:00Z",
  "updated_at": "2024-01-15T10:30:00Z"
}

Annotation Queues

Create an annotation queue

Create a new annotation queue.

Payload Requirements

The annotation queue name must be unique within the given space (among active queues).
At least one annotation_config_id is required, and all configs must belong to the specified space.
Do not include system-managed fields on input: id, created_at, updated_at.
If assignment_method is not provided, it defaults to "all".
If annotator_emails are provided, all emails must resolve to existing users with access to the space.

Valid example

{
  "name": "Quality Review Queue",
  "space_id": "spc_xyz789",
  "annotation_config_ids": ["ac_abc123"],
  "assignment_method": "all"
}

Valid example with records

{
  "name": "Quality Review Queue",
  "space_id": "spc_xyz789",
  "annotation_config_ids": ["ac_abc123"],
  "annotator_emails": ["reviewer@example.com"],
  "records": [
    {"record_type": "span", "project_id": "prj_abc", "start_time": "2024-01-15T00:00:00Z", "end_time": "2024-01-16T00:00:00Z", "span_ids": ["span_001"]},
    {"record_type": "example", "dataset_id": "ds_xyz", "example_ids": ["ex_001", "ex_002"]}
  ]
}

Invalid example (missing required annotation_config_ids)

{
  "name": "My Queue",
  "space_id": "spc_xyz789"
}

This endpoint is in alpha, read more here.

POST

annotation-queues

Create an annotation queue

curl --request POST \
  --url https://api.arize.com/v2/annotation-queues \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "name": "Quality Review Queue",
  "space_id": "spc_xyz789",
  "annotation_config_ids": [
    "ac_abc123"
  ],
  "assignment_method": "all"
}
'

{
  "id": "aq_abc123",
  "name": "Quality Review Queue",
  "space_id": "spc_xyz789",
  "instructions": "Review each response for accuracy and helpfulness",
  "created_at": "2024-01-15T10:30:00Z",
  "updated_at": "2024-01-15T10:30:00Z"
}

Authorizations

Authorization

string

header

required

Most Arize AI endpoints require authentication. For those endpoints that require authentication, include your API key in the request header using the format

Body

application/json

Body containing annotation queue creation parameters

name

string

required

The name of the annotation queue. Must be unique within the space for active queues.

Maximum string length: 255

Example:

"Quality Review Queue"

space_id

string

required

The space ID that the annotation queue belongs to

Example:

"spc_xyz789"

annotation_config_ids

string[]

required

IDs of annotation configs to associate with this queue. All configs must belong to the same space.

Minimum array length: 1

Example:

["ac_abc123", "ac_def456"]

annotator_emails

string<email>[]

required

Email addresses of annotators to assign to the queue. Emails are resolved to user IDs server-side.

Minimum array length: 1

Example:

[
  "annotator1@example.com",
  "annotator2@example.com"
]

instructions

string

Instructions for annotators working on this queue

Maximum string length: 5000

Example:

"Review each response for accuracy and helpfulness"

assignment_method

enum<string>

default:all

How records are assigned to annotators. Defaults to "all".

all: Every annotator is assigned to every record.
random: Each record is randomly assigned to one annotator.

Available options:

all,

random

records

object[]

Records to add to the annotation queue on creation. At most 2 record sources (projects or datasets) may be provided in a single create request. Additional records from other sources can be added after creation.

Maximum array length: 2

Option 1
Option 2

Show child attributes

Response

Returns the created annotation queue

string

required

The unique identifier for the annotation queue

Example:

"aq_abc123"

name

string

required

The name of the annotation queue

Example:

"Quality Review Queue"

space_id

string

required

The space id the annotation queue belongs to

Example:

"spc_xyz789"

created_at

string<date-time>

required

The timestamp for when the annotation queue was created

Example:

"2024-01-15T10:30:00Z"

updated_at

string<date-time>

required

The timestamp for when the annotation queue was last updated

Example:

"2024-01-20T14:45:00Z"

instructions

string | null

The instructions for the annotation queue

Example:

"Review each response for accuracy and helpfulness"

annotation_configs

object[]

The annotation configs associated with this queue

Option 1
Option 2
Option 3

Show child attributes

Example:

[
  {
    "id": "ac_001",
    "name": "accuracy",
    "type": "continuous",
    "space_id": "spc_xyz789",
    "minimum_score": 0,
    "maximum_score": 10,
    "optimization_direction": "maximize",
    "created_at": "2024-01-10T08:00:00Z"
  },
  {
    "id": "ac_002",
    "name": "sentiment",
    "type": "categorical",
    "space_id": "spc_xyz789",
    "values": [
      { "label": "positive", "score": 1 },
      { "label": "neutral", "score": 0 },
      { "label": "negative", "score": -1 }
    ],
    "optimization_direction": "maximize",
    "created_at": "2024-01-10T08:00:00Z"
  }
]

List annotation queues Get an annotation queue

⌘I

REST API

Reference

Create an annotation queue

Authorizations

Body

Response