Annotation Configs

The annotation_configs client methods are currently in BETA. The API may change without notice. A one-time warning is emitted on first use.

Create custom labels that can be added to represent this human feedback. Annotation configs allow teams and subject matter experts to label data and curate high-quality datasets.

Key Capabilities

Create categorical, continuous, and freeform annotation configs within spaces
List annotation configs with pagination support
Retrieve annotation configs by name or ID
Update annotation config fields by type
Delete annotation configs when no longer needed

List Annotation Configs

List all annotation configs you have access to, with optional filtering by space or name.

resp = client.annotation_configs.list(
    space="your-space-name-or-id",  # optional
    name="accuracy",                # optional substring filter
    limit=50,
)

print(resp)

For details on pagination, field introspection, and data conversion (to dict/JSON/DataFrame), see Response Objects.

Create an Annotation Config

Three types are supported, each created with a dedicated method: create_categorical, create_continuous, and create_freeform. Annotation config names must be unique within the target space.

Categorical

A categorical annotation config with discrete labeled values and optional scores.

from arize.annotation_configs.types import (
    CategoricalAnnotationValue,
    OptimizationDirection,
)

resp = client.annotation_configs.create_categorical(
    name="accuracy",
    space="your-space-name-or-id",
    values=[
        CategoricalAnnotationValue(label="accurate", score=1),
        CategoricalAnnotationValue(label="inaccurate", score=0),
    ],
    optimization_direction=OptimizationDirection.MAXIMIZE,  # optional
)

print(resp)

Continuous

A continuous annotation config with a numeric score range.

from arize.annotation_configs.types import OptimizationDirection

resp = client.annotation_configs.create_continuous(
    name="relevance",
    space="your-space-name-or-id",
    minimum_score=0.0,
    maximum_score=1.0,
    optimization_direction=OptimizationDirection.MAXIMIZE,  # optional
)

print(resp)

Freeform

A freeform annotation config for free-text feedback with no structured scoring.

resp = client.annotation_configs.create_freeform(
    name="reviewer-notes",
    space="your-space-name-or-id",
)

print(resp)

Get an Annotation Config

Retrieve a specific annotation config by name or ID. When using a name, provide space to disambiguate.

resp = client.annotation_configs.get(
    annotation_config="annotation-config-name-or-id",
    space="your-space-name-or-id",  # required when using a name
)

print(resp)

Update an Annotation Config

Update an annotation config by type. Only the fields you pass are changed; omitted fields are left unchanged. The stored config must already be of the matching type.

Update Categorical

from arize.annotation_configs.types import (
    CategoricalAnnotationValue,
    OptimizationDirection,
)

resp = client.annotation_configs.update_categorical(
    annotation_config="annotation-config-name-or-id",
    space="your-space-name-or-id",  # required when using a name
    name="accuracy-v2",                                  # optional
    values=[                                             # optional; replaces the full label set
        CategoricalAnnotationValue(label="accurate", score=1),
        CategoricalAnnotationValue(label="partial", score=0.5),
        CategoricalAnnotationValue(label="inaccurate", score=0),
    ],
    optimization_direction=OptimizationDirection.MAXIMIZE,  # optional
)

print(resp)

Update Continuous

from arize.annotation_configs.types import OptimizationDirection

resp = client.annotation_configs.update_continuous(
    annotation_config="annotation-config-name-or-id",
    space="your-space-name-or-id",  # required when using a name
    name="relevance-v2",              # optional
    minimum_score=0.0,                # optional
    maximum_score=5.0,                # optional
    optimization_direction=OptimizationDirection.MAXIMIZE,  # optional
)

print(resp)

Update Freeform

resp = client.annotation_configs.update_freeform(
    annotation_config="annotation-config-name-or-id",
    space="your-space-name-or-id",  # required when using a name
    name="reviewer-notes-v2",         # optional
)

print(resp)

Delete an Annotation Config

Delete an annotation config by name or ID. This operation is irreversible. There is no response from this call.

client.annotation_configs.delete(
    annotation_config="annotation-config-name-or-id",
    space="your-space-name-or-id",  # required when using a name
)

print("Annotation config deleted successfully")

API reference

API clients

Key Capabilities

List Annotation Configs

Create an Annotation Config

Categorical

Continuous

Freeform

Get an Annotation Config

Update an Annotation Config

Update Categorical

Update Continuous

Update Freeform

Delete an Annotation Config

​Key Capabilities

​List Annotation Configs

​Create an Annotation Config

​Categorical

​Continuous

​Freeform

​Get an Annotation Config

​Update an Annotation Config

​Update Categorical

​Update Continuous

​Update Freeform

​Delete an Annotation Config

Key Capabilities

List Annotation Configs

Create an Annotation Config

Categorical

Continuous

Freeform

Get an Annotation Config

Update an Annotation Config

Update Categorical

Update Continuous

Update Freeform

Delete an Annotation Config