> ## Documentation Index
> Fetch the complete documentation index at: /docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Webhooks

> Send prompt, evaluator, and evaluation task events from Arize AX to an HTTPS endpoint.

Webhooks notify your systems when a prompt or evaluator changes or when an evaluation task run fails. Arize AX sends the event to your endpoint as it happens, so you can react right away, for example by redeploying when a prompt version is labeled or alerting when a task starts failing, and your systems do not need to poll for changes.

## Events

A **webhook** defines the destination URL and authentication. A **subscription** connects one webhook to one event on one source.

| Subscription event | Payload `event` | Source |
| - | - | - |
| `PROMPT_VERSION_CREATED` | `prompt.version.created` | Prompt |
| `PROMPT_VERSION_LABELED` | `prompt.version.labeled` | Prompt |
| `PROMPT_VERSION_UNLABELED` | `prompt.version.unlabeled` | Prompt |
| `EVALUATOR_VERSION_CREATED` | `evaluator.version.created` | Evaluator |
| `ONLINE_TASK_RUN_FAILED` | `online_task.run.failed` | Evaluation task or space |

`ONLINE_TASK_RUN_FAILED` fires when a task run fails, including when a run writes some evaluations and then stops on an error. It is sent once when a task starts failing and again each time the cause changes.

## Create a webhook

1. Go to **Settings → Org Settings → Webhooks** and click **Add Webhook**.
2. Enter a **Name** and the **URL** that receives events. The name must be unique within the organization, and a description is optional.
3. Enter an **Authorization Token** if your endpoint expects a bearer token or secret, and add custom headers if it needs them.
4. Click **Add Webhook**.

![Add Webhook panel with Name, Description, URL, and Authorization Token fields and an Add Custom Headers button](https://storage.googleapis.com/arize-assets/doc-images/webhooks-add-dialog.webp)

The Add Webhook panel, with the name, URL, and optional authorization token.

The REST API can also create a webhook:

```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
curl -X POST https://api.arize.com/v2/webhooks \
  -H "Authorization: Bearer $ARIZE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "organization_id": "YOUR_ORG_ID",
    "name": "prompt-changes",
    "url": "https://example.com/hooks/arize",
    "auth_type": "HMAC_SHA256"
  }'
```

Webhooks created in the UI use `BEARER`. `HMAC_SHA256` webhooks are created with the REST API, the [Python SDK](/docs/api-clients/python/version-8/client-resources/webhooks.md), the [TypeScript SDK](/docs/api-clients/typescript/version-1/client-resources/webhooks.md), the [Go SDK](/docs/api-clients/go/version-2/client-resources/webhooks.md), or the [`ax webhooks` CLI](/docs/api-clients/cli/webhooks).

Arize AX does not send events to private, loopback, or link-local addresses, and it does not follow redirects, so the URL must be the exact public address that receives events.

## Authenticate deliveries

The auth type is fixed at creation. To switch, create a new webhook.

### Bearer token

`BEARER` sends the saved token verbatim in the `Authorization` header, so the token must include the scheme your endpoint expects, such as `Bearer my-secret-token`. The token is write-only and never returned after saving. Updating the webhook replaces or clears it.

### HMAC signature

`HMAC_SHA256` signs every delivery with a signing secret Arize AX generates. The secret is returned once, in the create response, and cannot be rotated or retrieved again; afterwards the webhook exposes only a `signing_secret_hint` such as `whsec_…abcd`. If the secret is lost, create a new webhook.

Each signed delivery includes these headers.

| Header | Value |
| - | - |
| `X-Arize-Webhook-Id` | Event ID. Matches `event_uuid` in the payload and stays the same across retries. |
| `X-Arize-Webhook-Event` | Dotted event name |
| `X-Arize-Webhook-Timestamp` | Delivery time in Unix seconds |
| `X-Arize-Webhook-Signature` | `v1=<hex HMAC-SHA256>` |

The signature is HMAC-SHA256 over `<timestamp>.<raw request body>`. The key is the full signing secret, including its `whsec_` prefix.

1. Read the raw request body as bytes, before JSON parsing.
2. Compute HMAC-SHA256 over `<X-Arize-Webhook-Timestamp>.<raw body>` with the full signing secret as the key.
3. Hex-encode the result, prefix `v1=`, and compare to `X-Arize-Webhook-Signature` in constant time.
4. Reject timestamps older than your tolerance, for example five minutes. Each retry carries a fresh timestamp and signature.

```python theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
import hashlib, hmac

def is_valid(secret: str, timestamp: str, body: bytes, signature: str) -> bool:
    signed = timestamp.encode() + b"." + body
    digest = hmac.new(secret.encode(), signed, hashlib.sha256).hexdigest()
    return hmac.compare_digest("v1=" + digest, signature)
```

### Custom headers

Both auth types support up to 20 write-only custom headers, for example a routing key. Connection headers such as `Host` and `Content-Length` are rejected. `Content-Type: application/json`, the bearer token, and the signature headers override custom headers with the same names.

## Subscribe to events

Subscriptions are created with the REST API:

```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
curl -X POST https://api.arize.com/v2/webhook-subscriptions \
  -H "Authorization: Bearer $ARIZE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "webhook_id": "YOUR_WEBHOOK_ID",
    "source_type": "PROMPT",
    "source_id": "YOUR_PROMPT_ID",
    "event": "PROMPT_VERSION_LABELED"
  }'
```

The webhook and the source must belong to the same organization, and each event works only with its matching source type. Prompt events use `PROMPT` sources, evaluator events use `EVALUATOR` sources, and task run events use `ONLINE_TASK` and `SPACE` sources.

An `ONLINE_TASK` source must be an eval task; other task types return `422`. A `SPACE` subscription covers every eval task in the space, including ones created later. A webhook subscribed to both a task and its space receives each alert once.

## Task failure alerts

`online_task.run.failed` covers tasks that run LLM-as-a-judge, code, agent-as-a-judge, and remote evaluators. Playground experiments and prompt optimization tasks never send it.

### When it fires

| Run outcome | Run status in Arize AX | Payload `status` |
| - | - | - |
| No part of the run succeeded. | `failed` | `failed` |
| The run scored some evals, then stopped on an error such as repeated LLM rate limits. | `cancelled` | `partial_completed` |
| The run scored its evals but could not save them, because the save failed or the run hit its time limit first. | `failed` | `failed` |
| The task's config is invalid, for example it has no LLM integration or its integration is no longer available to the space. | `failed` | `failed` |

Only the webhook payload uses `partial_completed`; the UI, REST API, and SDKs show that run as `CANCELLED`.

An invalid config adds one `failed` run with the config error to the task's run logs each time the task starts failing on config. This applies to scheduled runs only; Playground and date-range runs show the error directly and record no run.

### One alert per cause

Arize AX sends one alert when a task starts failing, and another each time the cause changes.

* Runs that keep failing with the same `error_type` send nothing more, even when they alternate between `failed` and `partial_completed`.
* A run with a new `error_type` sends a new alert.
* A completed run ends the streak. The next failure alerts again.
* A run that finds no data neither alerts nor ends the streak.

Each webhook tracks its own streak. Subscribing to a task that is already failing alerts on the task's next failed run.

### When it does not fire

* A user stops the run. It ends `cancelled` with the counts it had scored.
* The run had nothing to evaluate, or found no data.
* Some parts could not be scored, but the scored parts saved.
* A single eval fails, such as one LLM call returning an unparsable label. The run still succeeds.
* The run is a sub-run. Only the top-level run alerts, and it completes silently when its sub-runs split between partly finished and completed with none failed.
* The run never reaches a final status.

## Payloads

Each delivery is an HTTP `POST` with `Content-Type: application/json`. Event deliveries share these top-level fields. All IDs are opaque strings.

| Field | Description |
| - | - |
| `event` | The event, for example `prompt.version.created`. |
| `event_uuid` | A unique ID for the event. Retries reuse it, so use it to drop duplicates. |
| `space_id` | The ID of the space that holds the prompt, evaluator, or task. |
| `space_name` | The name of that space. |
| `account_organization_id` | The ID of the organization. |
| `account_organization_name` | The name of the organization. |
| `timestamp` | When the event happened, in RFC 3339 format (UTC). |
| `data` | Fields specific to the event. |

`data` fields by event. `changed_by` is the ID of the user who made the change.

> [accordion] prompt.version.created
>
> ```json theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
> {
>   "prompt_id": "UHJvbXB0OjEyMzQ1",
>   "prompt_version_id": "UHJvbXB0VmVyc2lvbjo2Nzg=",
>   "prompt_name": "support-agent",
>   "changed_by": "VXNlcjo0Mg=="
> }
> ```

> [accordion] prompt.version.labeled and prompt.version.unlabeled
>
> `label` is the label that was added or removed.
>
> ```json theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
> {
>   "prompt_id": "UHJvbXB0OjEyMzQ1",
>   "prompt_version_id": "UHJvbXB0VmVyc2lvbjo2Nzg=",
>   "prompt_name": "support-agent",
>   "label": "production",
>   "changed_by": "VXNlcjo0Mg=="
> }
> ```

> [accordion] evaluator.version.created
>
> ```json theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
> {
>   "evaluator_id": "RXZhbHVhdG9yOjQ1Ng==",
>   "evaluator_version_id": "RXZhbHVhdG9yVmVyc2lvbjo4OQ==",
>   "evaluator_name": "hallucination-check",
>   "changed_by": "VXNlcjo0Mg=="
> }
> ```

> [accordion] online\_task.run.failed
>
> ```json theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
> {
>   "task_id": "T25saW5lVGFzazo4ODE=",
>   "task_name": "hallucination-check",
>   "run_id": "VGVtcGxhdGVFdmFsdWF0aW9uVGFza1J1bjo1NTA5",
>   "project_id": "TW9kZWw6MjEw",
>   "status": "failed",
>   "error_type": "authentication",
>   "failure_reason": "LLM Provider Error [AUTHENTICATION]: Incorrect API key provided",
>   "url": "https://app.arize.com/organizations/.../tasks?runId=...&taskId=...&taskPane=logs"
> }
> ```
>
> The payload carries `project_id` or `dataset_id`, whichever the task runs on. `status` is `failed` or `partial_completed`; treat any other value as a failure. `failure_reason` is the run log text, up to 1,000 characters. `error_type` is missing when the failure could not be classified, and a run with several evaluators reports the first failing evaluator's type in name order.
>
> | `error_type` | Meaning |
> | - | - |
> | `authentication`, `access_denied` | The LLM provider rejected the credentials, or they lack access to the model. |
> | `rate_limit`, `quota_exceeded` | The provider rate-limited the requests or the account ran out of quota. |
> | `context_length_exceeded`, `bad_request` | The prompt was too long or the request was invalid. |
> | `model_not_found` | The model does not exist or was retired. |
> | `timeout`, `service_unavailable`, `provider_error` | The provider timed out, was down, or returned another error. |
> | `content_policy`, `llm_refusal` | The content filter blocked the request or the model refused. |
> | `integration_config`, `invalid_config` | The LLM integration or the task config is set up wrong. |
> | `file_size_limit_exceeded` | A file in the request exceeded the provider size limit. |
> | `write_failed`, `run_timeout` | Evals were scored but not saved, or the run hit its time limit first. Lower the sampling rate so each run scores less data. |

New fields may be added; ignore fields you do not use. A `test.test_event` delivery carries only `event`, `event_uuid`, and `timestamp`.

## Retries

Arize AX counts any `2xx` response as delivered. It attempts a delivery up to 6 times in total, with retries about 1, 4, 16, 64, and 256 minutes after the previous attempt. Connection failures, timeouts, and HTTP `408`, `429`, and `5xx` responses are retried. Other `4xx` responses and `3xx` redirects are not.

Retries resend the same payload and `event_uuid`, so deduplicate on `event_uuid`. Return `2xx` quickly and do slow work afterwards.

## Delivery history

Every delivery attempt is recorded. Fetch them with `GET /v2/webhooks/{webhook_id}/delivery-attempts` or from the webhook's **Event Delivery** tab.

```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
curl "https://api.arize.com/v2/webhooks/YOUR_WEBHOOK_ID/delivery-attempts?limit=50" \
  -H "Authorization: Bearer $ARIZE_API_KEY"
```

| Field | Description |
| - | - |
| `event_id` | The event ID. Matches `event_uuid` and `X-Arize-Webhook-Id`. |
| `attempt_number` | The attempt number for this event. |
| `payload` | The payload that was sent. |
| `status_code` | The HTTP status your endpoint returned. `null` when no response came back. |
| `error_message` | The error, if any. |
| `created_at` | When the attempt was made. |

## Test a webhook

Send a test event from the webhook's page or with `POST /v2/webhooks/{webhook_id}/test`. The test sends one `test.test_event` request and reports your endpoint's HTTP status, or `502` when no response came back. Tests do not retry, do not appear in delivery history, and do not work for `HMAC_SHA256` webhooks; subscribe to an event and trigger it instead.

## Limits

* Webhook names must be unique within an organization.
* The delivery timeout is between 1 and 60 seconds. The default is 30 seconds.
* A webhook can have at most 20 custom headers.
* At most 200 webhooks can subscribe to the same event on one source.
* A webhook can subscribe to a given event on a given source only once.

## Permissions

| Action | Who can do it |
| - | - |
| View webhooks | Any organization role, including Read-only Member |
| Create a webhook, send a test, or view delivery history | Organization Admin or Member |
| Update or delete a webhook | Organization Admins, and the Member who created the webhook |
| Add or remove a subscription on a prompt, evaluator, or task | Anyone who can edit that prompt, evaluator, or task, such as a space Admin or Member |
| Add or remove a subscription on a space | Space Admins, Organization Admins, and Account Admins |

Account Admins can do everything an Organization Admin can. See [Role-based access control](/docs/ax/security-and-settings/sso-and-rbac).
