Skip to main content
Your dashboards show you the current state, but you can’t watch them all day. Models change, user patterns shift, and what worked last week can silently degrade. You need to know the moment something goes wrong. Monitors watch your metrics continuously and alert you when something crosses a threshold. Set them up once, and they run in the background.

​
Create a monitor

  1. Go to Monitors in the sidebar and click + New Monitor
  2. Select Tracing Project Monitor
Monitors page showing monitor list with statuses and New Monitor dropdown
  1. Select a project to monitor
  2. Choose a metric source:
    • Span attribute — any attribute from your traces (latency, status, token counts, eval labels, custom metadata)
    • Custom metric — any AQL metric you’ve defined
  3. Configure the evaluation window and threshold (see below)
  4. Set up notifications
  5. Click Create
Monitor creation form showing metric source, chart with threshold, and data configuration

​
What can you monitor?

What to trackHow to set it up
Hallucination rate exceeds 5%Metric source: custom metric
Latency spikeSpan attribute: latency_ms, static threshold
Token usage increaseSpan attribute: token count, automatic threshold
Eval quality dropsSpan attribute: eval label (e.g., eval.groundedness.label)

​
Monitor settings

​
Evaluation window

The time range of data the monitor evaluates each time it runs. For example, a 24-hour window means each check looks at the last 24 hours of data. Default is 24 hours for Tracing Project Monitors.

​
Frequency

How often the monitor runs. Default is every hour. The monitor evaluates its window of data at this cadence.

​
Delay window

Getting false alerts because your data arrives in batches? Set a delay window to give your system time to stabilize before the monitor evaluates.

​
API-triggered monitors

Instead of running on a schedule, you can trigger a monitor manually via the GraphQL API. This is useful for evaluating after batch ingestion or a pipeline step: 
mutation {
    triggerMonitor(input: {monitorId: "MONITOR_ID"}) {
        success
        monitor { name id }
    }
}

​
Thresholds

Two ways to set when alerts fire:
  • Automatic — Arize sets a threshold based on your historical data. Adjust sensitivity (high = more alerts, low = fewer alerts).
  • Static — you set the exact number (e.g., “alert if accuracy drops below 85%”)
You can set both upper and lower bounds for more precise alerting.

​
Viewing a monitor

Once running, click into any monitor to see its alert history, threshold line, traffic, and evaluation window:
Monitor detail view showing Hallucination rate alert history with threshold and evaluation window
Your monitor is always in one of these states:
StatusMeaning
Healthy (green)Everything looks good
No Data (yellow)No recent data to evaluate
Triggered (red)Threshold crossed, investigate

​
Notifications and integrations

A monitor is only useful if it reaches the right people. When a monitor triggers, get notified through:
  • Email — recipient doesn’t need an Arize account
  • Slack — send to a team channel
  • PagerDuty — for on-call escalation
  • OpsGenie — for incident management
  • Webhooks — send to any HTTP endpoint
Configure integrations in Alert Integrations (left sidebar in Organization Settings) or from the Config tab within a project.
Alert Integrations page showing available notification services
  1. Go to Alert Integrations and click Slack
  2. Click Connect to Slack, select a channel, and click Allow
  3. Use Test Integration to verify the connection
  4. Assign to monitors via the Config tab or individual monitor settings
Each Slack integration is tied to a specific channel. Add multiple for different teams.
You can also set up integrations programmatically using the GraphQL API.
You can set alert destinations at two levels:
  • Project level — all monitors for a project send to the same destination (set in Config tab)
  • Monitor level — individual monitors send to different destinations (set in monitor settings)

​
Additional features

  • Mute monitors — temporarily silence alerts during maintenance or known issues
  • Downtime windows — schedule recurring periods where alerts are suppressed
  • Duplicate monitors — copy an existing monitor’s configuration to create a similar one
  • Audit log — track every change made to a monitor (who changed it, when, and what was modified)

​
Best practices

  • Start with a simple latency or token count monitor, then add more as you learn what matters
  • Don’t monitor everything. Focus on metrics tied to business outcomes or SLAs
  • Start with automatic thresholds and adjust sensitivity over time
  • Set a delay window if your data arrives in batches
  • Don’t change too many monitor settings at once. Adjust gradually

​
You’ve completed the Observe workflow

You now have full observability for your LLM application. You can explore traces and sessions, understand agent behavior, define the metrics that matter, visualize them on dashboards, and get alerted the moment something goes wrong. When a monitor fires, the loop restarts: go back to your traces, investigate the issue, understand the pattern, and take action. That’s the Observe workflow.