Use the open-source phoenix-evals library to grade Arize AX traces and as evaluators in Arize AX experiments.
arize-phoenix-evals is Arize’s open-source, backend-agnostic evaluation library — the same library the LLM-as-a-judge provider guides wire a judge model into. It ships composable primitives (LLM, create_classifier, evaluate_dataframe) and prebuilt metric evaluators (hallucination, correctness, relevance, and more), and runs entirely in-process: it needs no separate server, and its results feed Arize AX through the arize SDK.This guide shows both ways to wire phoenix-evals into Arize AX: Flow 1 grades existing Arize AX traces with a classifier and writes the scores back via client.spans.update_evaluations(...); Flow 2 uploads a small dataset, runs an Arize AX experiment with the same classifier wrapped as an evaluator, and surfaces the scores in Datasets+Experiments. For the concepts behind the offline pattern, see Offline evaluation; for running these evals on a schedule inside the platform, see Run evals on traces.Both flows share the same setup. Run the code blocks below in order inside a single Python session — each block builds on imports and variables from earlier ones.
Prerequisites
Python 3.11+
An ARIZE_SPACE_ID and ARIZE_API_KEY from your Arize AX space settings
An OPENAI_API_KEY from OpenAI Platform (used as both the model under trace and the phoenix-evals judge LLM)
Launch Arize AX
If you don’t already have an Arize AX account, sign up at arize.com and grab your ARIZE_SPACE_ID and ARIZE_API_KEY from Settings → Space Settings.
The shared setup: a create_classifier correctness evaluator backed by a GPT-4.1 judge, the canonical 2-row dataset both flows score, and an Arize SDK client. create_classifier relies on the judge’s tool-calling / structured-output support, so use a non-reasoning model such as GPT-4.1. The {input}, {output}, and {reference} placeholders in the template are filled from dataframe columns of the same name.
# combined.pyimport osimport timefrom datetime import datetime, timedelta, timezoneimport pandas as pdfrom arize import ArizeClientfrom phoenix.evals import LLM, create_classifier, evaluate_dataframefrom phoenix.evals.utils import to_annotation_dataframeSPACE_ID = os.environ["ARIZE_SPACE_ID"]API_KEY = os.environ["ARIZE_API_KEY"]TIMESTAMP = int(time.time())# The judge model. GPT-4.1 is a non-reasoning model that supports the# tool-calling / structured output create_classifier relies on.judge = LLM(provider="openai", model="gpt-4.1")TEMPLATE = ( "You are grading whether a response is factually correct.\n" "[Question]: {input}\n" "[Response]: {output}\n" "[Reference]: {reference}\n\n" "Is the response factually correct given the reference? " "Answer 'correct' or 'incorrect'.")correctness = create_classifier( name="correctness", prompt_template=TEMPLATE, llm=judge, choices={"correct": 1.0, "incorrect": 0.0}, direction="maximize",)# Canonical 2-row dataset — row 0 is factual (answer matches the reference),# row 1 is hallucinated. Both flows grade these same rows.ROWS = [ { "input": "What is the capital of France?", "output": "Paris is the capital of France.", "reference": "Paris is the capital and most populous city of France.", }, { "input": "What is the capital of France?", "output": "Berlin is the capital of France.", "reference": "Paris is the capital and most populous city of France.", },]arize = ArizeClient(api_key=API_KEY)
create_classifier builds a custom categorical evaluator. For common metrics, phoenix.evals.metrics ships prebuilt evaluators — HallucinationEvaluator, CorrectnessEvaluator, DocumentRelevanceEvaluator, RefusalEvaluator, and others — that you can pass to evaluate_dataframe in place of the classifier above.
Flow 1 — Evaluate existing traces
Source the spans
Instrument OpenAI with OpenInference, make two calls (each forced to echo a known answer so the trace contains predictable text), then pull the resulting spans back from Arize AX.
from openinference.instrumentation.openai import OpenAIInstrumentorfrom opentelemetry import tracefrom opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporterfrom opentelemetry.sdk.resources import Resourcefrom opentelemetry.sdk.trace import TracerProviderfrom opentelemetry.sdk.trace.export import BatchSpanProcessorfrom openai import OpenAIPROJECT_NAME = f"phoenix-evals-example-{TIMESTAMP}"resource = Resource.create( { "service.name": PROJECT_NAME, "openinference.project.name": PROJECT_NAME, "model_id": PROJECT_NAME, })provider = TracerProvider(resource=resource)provider.add_span_processor( BatchSpanProcessor( OTLPSpanExporter( endpoint="https://otlp.arize.com:443", headers={ "authorization": API_KEY, "arize-space-id": SPACE_ID, "arize-interface": "python", }, ) ))trace.set_tracer_provider(provider)OpenAIInstrumentor().instrument(tracer_provider=provider)sync_oai = OpenAI()for row in ROWS: sync_oai.chat.completions.create( model="gpt-5.4-mini", messages=[ { "role": "system", "content": ( "You are a fact-recall assistant. The user states the " "exact answer to use; reply with that verbatim." ), }, { "role": "user", "content": ( f"Question: {row['input']}\n" f"Answer (reply verbatim): {row['output']}" ), }, ], )provider.force_flush(timeout_millis=10_000)print(f"Project: {PROJECT_NAME}")# Spans take ~5–15s to be queryable after flush. Poll defensively: Arize's# OTLP ingest and Flight export use different catalogs and the new project# can briefly appear "unauthorized" to the export endpoint while still# accepting span writes via OTLP, so swallow transient errors and retry.start = datetime.now(timezone.utc) - timedelta(minutes=5)end = datetime.now(timezone.utc) + timedelta(minutes=1)spans_df = Nonelast_err: Exception | None = Nonefor _ in range(12): time.sleep(5) try: spans_df = arize.spans.export_to_df( space_id=SPACE_ID, project_name=PROJECT_NAME, start_time=start, end_time=end, ) except Exception as e: last_err = e continue if spans_df is not None and len(spans_df) >= len(ROWS): breakelse: raise RuntimeError( f"Spans never appeared after 60s (last error: {last_err})" )spans_df = spans_df.sort_values("start_time").reset_index(drop=True)
Run the evaluators
Map the span attributes to the template variables, then run the classifier over the whole dataframe in one pass. evaluate_dataframe returns the input dataframe plus a correctness_score column (a dict holding the label, score, and explanation) and a correctness_execution_details column. The reference value isn’t on the span, so it’s sourced from ROWS by position.
spans_df["input"] = spans_df["attributes.input.value"]spans_df["output"] = spans_df["attributes.output.value"]spans_df["reference"] = [r["reference"] for r in ROWS]results = evaluate_dataframe(dataframe=spans_df, evaluators=[correctness])
For larger batches, async_evaluate_dataframe(dataframe=..., evaluators=[...]) is the async equivalent and runs the judge calls concurrently.
Log evaluations to Arize AX
to_annotation_dataframe flattens the nested correctness_score dict into score, label, and explanation columns while preserving context.span_id (the join key export_to_df already provides). Rename those to the reserved eval.<name>.{score,label,explanation} columns Arize AX expects, then upload. For trace or session evals use the trace_eval.<name> / session_eval.<name> prefixes instead.
annotations = to_annotation_dataframe(dataframe=results)upload_df = annotations.rename( columns={ "label": "eval.correctness.label", "score": "eval.correctness.score", "explanation": "eval.correctness.explanation", })[[ "context.span_id", "eval.correctness.label", "eval.correctness.score", "eval.correctness.explanation",]]arize.spans.update_evaluations( space_id=SPACE_ID, project_name=PROJECT_NAME, dataframe=upload_df,)# Print the scores so they appear in stdout for verification.print("Flow 1 results:")print(upload_df[["eval.correctness.label", "eval.correctness.score"]].to_string())
Open the project named phoenix-evals-example-<timestamp> (the value printed above) in your Arize AX space. Each ChatCompletion span now carries a correctness annotation column showing the score and label written by update_evaluations(...).
Flow 2 — Run an experiment
Create a dataset
The dataset is the same two rows. The space= / examples= kwarg names match the v8 SDK exactly (note: not space_id= and not dataframe=).
The task function receives the dataset row and returns whatever the experiment should grade. The parameter name must be one of input, output, metadata, or dataset_row — a single-arg task with an unrecognized name is bound to dataset_row by default. A real workflow would call an LLM here; this passthrough keeps the example deterministic.
Experiment evaluators run inside an asyncio loop, so use async def and the classifier’s async_evaluate(...) — it returns a list of Score objects, one per evaluator. Convert the first Score to an EvaluationResult with score and label and explanation populated: leaving any of those reserved fields as None triggers unsupported cast from null to <type>: reserved column cannot be coerced to canonical type at upload time.
from arize.experiments.evaluators.types import EvaluationResultasync def correctness_eval(output, dataset_row) -> EvaluationResult: scores = await correctness.async_evaluate( { "input": dataset_row["input"], "output": output if isinstance(output, str) else str(output), "reference": dataset_row["reference"], } ) s = scores[0] return EvaluationResult( score=s.score, label=s.label, explanation=s.explanation or "no explanation", )
EvaluationResult’s constructor order is (score, label, explanation, metadata) — not(label, score, ...) as the alphabetical reading might suggest. Always use keyword arguments; positional calls silently swap label and score.
Flow 2 results: output score label0 Paris is the capital of France. 1.0 correct1 Berlin is the capital of France. 0.0 incorrect
Verify in Arize AX
Open the Datasets + Experiments tab in Arize AX. The dataset phoenix-evals-example-ds-<timestamp> and the experiment phoenix-evals-example-<timestamp> (names printed above) appear with one run per dataset row, each carrying the correctness score and label columns.
Troubleshooting
create_classifier fails or returns empty labels. The judge model must support tool calling / structured output. Reasoning models (GPT-5-family or o-series ids) can also reject standard sampling params — use a non-reasoning judge such as gpt-4.1.
column "eval.<name>.label": unsupported cast from null to string: reserved column cannot be coerced to canonical type. In Flow 2 your evaluator returned a bare number or string instead of a fully-populated EvaluationResult(score=..., label=..., explanation=...). Arize AX’s Flight server rejects null values in reserved eval columns — populate all three fields.
Cannot call sync ... from an async context. Your Flow 2 evaluator is calling the classifier’s sync evaluate(...) instead of async_evaluate(...). Experiment evaluators run inside asyncio; use the async API. Flow 1 uses evaluate_dataframe(...) because it runs outside any loop.
Spans never appear after 60s. Span flush + ingest typically takes 5–15s. If the loop times out, check that ARIZE_SPACE_ID + ARIZE_API_KEY are right and that you’re connecting to the correct region’s OTLP endpoint (otlp.arize.com for US, otlp.eu.arize.com for EU). Evals can only be applied to spans up to 14 days old; for older spans contact support@arize.com.
task failed for example id .... Your task function’s parameter name isn’t one of the recognized names (input, output, metadata, dataset_row). Rename it to dataset_row if you want the whole row, or pick the field you actually need.
Experiment runs duplicate or the dataset already exists. Both names embed TIMESTAMP = int(time.time()) so a single re-run produces unique names. If you re-execute the same combined.py quickly, regenerate TIMESTAMP first.
Resources
Phoenix Evals reference
arize-phoenix-evals on GitHub
Offline evaluation concepts
Logging evaluations to Arize AX
Was this page helpful?
{"docsConfig":{"theme":"mint","$schema":"https://mintlify.com/docs.json","name":"Arize AX Docs","colors":{"primary":"#7A0047","light":"#FF8AC8","dark":"#FF74BF"},"logo":{"light":"https://mintcdn.com/arize-ax/V-uvingrGLOCK4VD/logo/light.svg?fit=max\u0026auto=format\u0026n=V-uvingrGLOCK4VD\u0026q=85\u0026s=cc58ee9388d5b5253d39b65cc662908f","dark":"https://mintcdn.com/arize-ax/V-uvingrGLOCK4VD/logo/dark.svg?fit=max\u0026auto=format\u0026n=V-uvingrGLOCK4VD\u0026q=85\u0026s=b16a0266f5913934131301a1b60af51e","href":"https://arize.com"},"favicon":"https://mintcdn.com/arize-ax/1gcABHkqP5TIGbru/favicon.ico?fit=max\u0026auto=format\u0026n=1gcABHkqP5TIGbru\u0026q=85\u0026s=aef538e87b54a3c819583bc40acfc2f8","appearance":{"default":"dark"},"background":{"color":{"light":"#FFFFFF","dark":"#0A0A12"}},"navbar":{"links":[],"primary":{"type":"button","label":"Get started","href":"https://app.arize.com/auth/join"}},"footer":{"socials":{"x":"https://x.com/arizeai","bluesky":"https://bsky.app/profile/arize.bsky.social","threads":"https://www.threads.com/@arize.ai","instagram":"https://www.instagram.com/arize_ai","linkedin":"https://www.linkedin.com/company/arizeai","youtube":"https://youtube.com/@arizeai","slack":"https://arize-ai.slack.com/ssb/redirect","website":"https://fosstodon.org/@arizeai"},"links":[{"header":"Support","items":[{"label":"Chat Us On Slack","href":"https://arize-ai.slack.com/ssb/redirect"},{"label":"support@arize.com","href":"mailto:support@arize.com"},{"label":"System Status","href":"https://status.arize.com"}]},{"header":"Get Started","items":[{"label":"Signup For Free","href":"https://arize.com/join"},{"label":"Book A Demo","href":"https://arize.com/request-a-demo"}]},{"header":"Resources","items":[{"label":"Blog","href":"https://arize.com/blog"}]}]},"seo":{"metatags":{"canonical":"https://arize.com/docs"},"indexing":"all"},"fonts":{"heading":{"family":"Geist","weight":600},"body":{"family":"Geist"}},"styling":{"codeblocks":{"theme":{"light":"github-light-default","dark":"github-dark-default"}},"latex":true},"integrations":{"gtm":{"tagId":"GTM-W96XM35"}},"contextual":{"options":["copy","view","chatgpt","claude"],"display":"header"},"interaction":{"drilldown":false},"navigation":{"pages":[]}},"deployment":{"deploymentMetadataValue":{"subdomain":"arize-ax","hasAudioNarration":false,"actualSubdomain":"arize-ax","customDomains":[],"gitSource":{"type":"github","owner":"arize-ai","repo":"docs","deployBranch":"main","contentDirectory":"","isPrivate":true,"mountPath":"","label":""},"feedback":{"thumbs":true,"contextualFeedback":{"allowEmail":true,"emailDisclaimer":"We will only use your email to follow up with you about this feedback."}},"entitlements":{"AI_CHAT":{"status":"ENABLED"},"REMOVE_BRANDING":{"status":"ENABLED"},"CONTEXTUAL_FEEDBACK":{"status":"ENABLED"},"CODE_SNIPPET_FEEDBACK":{"status":"ENABLED"},"WEB_MCP":{"status":"ENABLED"},"NARRATION":{"status":"ENABLED"}},"buildId":"6afa81bdfa6d377b6d1d4433","clientVersion":"0.0.4143","searchFilterCounts":[{"language":"default","tag":"APIs, SDKs, Skills \u0026 MCP","count":369},{"language":"default","tag":"Get Started","count":256},{"language":"default","tag":"API reference","count":214},{"language":"default","tag":"REST","count":181},{"language":"default","tag":"Endpoints","count":178},{"language":"default","tag":"API clients","count":143},{"language":"default","tag":"Machine Learning","count":139},{"language":"default","tag":"ML observability","count":139},{"language":"default","tag":"Integrations","count":129},{"language":"default","tag":"Python SDK","count":67}]},"assistantConfig":{"enableAskAiButton":true,"enableStarterQuestions":true,"enableAskAiSearchBar":true,"deflection":{"enabled":true,"email":"support@arize.com"}},"basePath":"/docs","hasChatPermissions":true,"hasWebMcpPermissions":true,"toggles":[{"name":"kill-isr","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-rbac-roles","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-mintlify-slack","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-integrations-settings","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"rbac-expansion","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-pr-grouping","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-prefix","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"analytics-v7","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"user-flows-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"no-publish-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-onboarding","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving-live","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"stripTemperatureMiddleware","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"support-platform-pylon-zendesk","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"instant-rollback","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications-server","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-fable","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fixed-ai-action-pricing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automation-resolve-merge-conflict","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"scoped-publishing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"mintql","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"page-level-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-version-history-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"billing-page-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"full-page-search","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workspace-projects","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"assistant-v5","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fork-pr-previews","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"revalidation-jwt","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-sso-scim","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"hobby-abuse-scan","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"dashboard-inbox","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-signup","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-dual-write","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-enabled","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"approve-review-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-metadata","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"sso-setup-deadline-banner","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-serve","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"live-agents","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"llms-txt-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false}],"starterQuestions":[{"_id":"6944513fe59e82150b5700b3","deploymentId":"6894ef048eafbd2a7c20796e","questionText":"Need more help? Ask our team in #arize-ax-platform-support on community Slack.","status":"active","generatedBy":"manual","createdAt":"2025-12-18T19:08:47.802Z","updatedAt":"2025-12-18T19:15:01.436Z","__v":0}],"googleFontsInlined":true},"customCss":[{"filePath":"style.css","content":"/* cyrillic-ext */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 300;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrodmgPn.woff2) format('woff2');\n unicode-range: U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;\n}\n/* cyrillic */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 300;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrMdmgPn.woff2) format('woff2');\n unicode-range: U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;\n}\n/* symbols2 */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 300;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFg08vz7ehw.woff2) format('woff2');\n unicode-range: U+2000-2001, U+2004-2008, U+200A, U+23B8-23BD, U+2500-259F;\n}\n/* vietnamese */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 300;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrgdmgPn.woff2) format('woff2');\n unicode-range: U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB;\n}\n/* latin-ext */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 300;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrkdmgPn.woff2) format('woff2');\n unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C4, U+2113, U+2C60-2C7F, U+A720-A7FF;\n}\n/* latin */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 300;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrcdmg.woff2) format('woff2');\n unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;\n}\n/* cyrillic-ext */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 400;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrodmgPn.woff2) format('woff2');\n unicode-range: U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;\n}\n/* cyrillic */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 400;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrMdmgPn.woff2) format('woff2');\n unicode-range: U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;\n}\n/* symbols2 */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 400;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFg08vz7ehw.woff2) format('woff2');\n unicode-range: U+2000-2001, U+2004-2008, U+200A, U+23B8-23BD, U+2500-259F;\n}\n/* vietnamese */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 400;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrgdmgPn.woff2) format('woff2');\n unicode-range: U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB;\n}\n/* latin-ext */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 400;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrkdmgPn.woff2) format('woff2');\n unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C4, U+2113, U+2C60-2C7F, U+A720-A7FF;\n}\n/* latin */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 400;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrcdmg.woff2) format('woff2');\n unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;\n}\n/* cyrillic-ext */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 500;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrodmgPn.woff2) format('woff2');\n unicode-range: U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;\n}\n/* cyrillic */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 500;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrMdmgPn.woff2) format('woff2');\n unicode-range: U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;\n}\n/* symbols2 */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 500;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFg08vz7ehw.woff2) format('woff2');\n unicode-range: U+2000-2001, U+2004-2008, U+200A, U+23B8-23BD, U+2500-259F;\n}\n/* vietnamese */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 500;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrgdmgPn.woff2) format('woff2');\n unicode-range: U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB;\n}\n/* latin-ext */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 500;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrkdmgPn.woff2) format('woff2');\n unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C4, U+2113, U+2C60-2C7F, U+A720-A7FF;\n}\n/* latin */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 500;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrcdmg.woff2) format('woff2');\n unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;\n}\n/* cyrillic-ext */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 600;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrodmgPn.woff2) format('woff2');\n unicode-range: U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;\n}\n/* cyrillic */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 600;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrMdmgPn.woff2) format('woff2');\n unicode-range: U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;\n}\n/* symbols2 */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 600;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFg08vz7ehw.woff2) format('woff2');\n unicode-range: U+2000-2001, U+2004-2008, U+200A, U+23B8-23BD, U+2500-259F;\n}\n/* vietnamese */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 600;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrgdmgPn.woff2) format('woff2');\n unicode-range: U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB;\n}\n/* latin-ext */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 600;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrkdmgPn.woff2) format('woff2');\n unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C4, U+2113, U+2C60-2C7F, U+A720-A7FF;\n}\n/* latin */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 600;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrcdmg.woff2) format('woff2');\n unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;\n}\nhtml:root{--font-family-mono-custom: 'Geist Mono'}html:root{--gray-50: 248 248 246;--gray-100: 242 242 237;--gray-200: 230 230 223;--gray-300: 205 205 200;--gray-400: 179 179 191;--gray-500: 138 138 150;--gray-600: 91 91 105;--gray-700: 62 62 78;--gray-800: 46 46 61;--gray-900: 21 21 31;--gray-950: 10 10 18}html:root{--ax-radius-sm: 2px;--ax-radius-md: 6px;--ax-radius-lg: 8px;--ax-accent-bg: #FF74BF;--ax-panel: #CDBEF7;--ax-panel-2: #E3DBFC;--ax-panel-code: #FFFFFF;--ax-panel-border: rgba(19, 22, 64, 0.22);--ax-panel-icon: #4827C1;--ax-panel-t1: #121221;--ax-panel-t2: #2A2A38;--ax-panel-accent: #6E1046;--ax-panel-code-kw: #3F4110;--ax-panel-code-flag: #5A3200;--ax-panel-code-url: #004A70;--ax-panel-selection: #FFE9F4}html:not(.dark){--ax-bg: #FFFFFF;--ax-surface: #F8F8F4;--ax-code-bg: #F5F5ED;--ax-chip: #EDEDE2;--ax-line: rgba(19, 22, 64, 0.20);--ax-t1: #121221;--ax-t2: #2E2E3D;--ax-t3: #3E3E4E;--ax-accent: #7A0047;--ax-accent-line: rgba(255, 98, 182, 0.55);--ax-accent-soft: #FFE9F4;--ax-on-accent: #121221;--ax-note: #E8F3EC;--ax-note-line: rgba(30, 70, 50, 0.24);--ax-note-t: #1B4230;--ax-warn: #FBEEE0;--ax-warn-line: rgba(90, 50, 0, 0.24);--ax-warn-t: #54300A;--ax-blue: #E7F3FD;--ax-blue-line: rgba(0, 82, 125, 0.24);--ax-blue-t: #00527D;--ax-danger: #FFEBE9;--ax-danger-line: rgba(207, 34, 46, 0.28);--ax-danger-t: #A40E26}html.dark{--ax-bg: #0A0A12;--ax-surface: #15151F;--ax-code-bg: #05050B;--ax-chip: #221B3A;--ax-line: rgba(247, 247, 252, 0.16);--ax-t1: #F5F5FA;--ax-t2: #DCDCE6;--ax-t3: #B3B3C4;--ax-accent: #FF8AC8;--ax-accent-line: rgba(255, 138, 200, 0.45);--ax-accent-soft: #2A1425;--ax-on-accent: #0A0A12;--ax-note: #10241A;--ax-note-line: rgba(159, 224, 188, 0.30);--ax-note-t: #9FE0BC;--ax-warn: #2A1B0A;--ax-warn-line: rgba(255, 201, 138, 0.30);--ax-warn-t: #FFC98A;--ax-blue: #0A1A24;--ax-blue-line: rgba(57, 195, 255, 0.30);--ax-blue-t: #97D8FF;--ax-danger: #2A0F12;--ax-danger-line: rgba(255, 138, 148, 0.30);--ax-danger-t: #FFB3B8}#content{line-height:1.75}#page-title,#content h1{font-size:clamp(2rem, 1.4rem + 2.1vw, 2.875rem);line-height:1.12;letter-spacing:-0.02em;font-weight:600}#content h2,#content h3,#content h4{letter-spacing:-0.01em;font-weight:600}#content code,#content kbd,#content samp{font-feature-settings:'tnum' 1}#content a:not(.card){color:var(--ax-accent);text-decoration-thickness:1px;text-underline-offset:3px}:focus-visible{outline:2px solid var(--ax-accent-line);outline-offset:2px}html.dark :focus-visible{outline-color:var(--ax-accent-line)}::selection{background:var(--ax-accent-soft)}#navbar{background:var(--ax-bg);border-bottom:1px solid var(--ax-line)}#navbar div:has(>.nav-tabs){position:relative}#navbar div:has(>.nav-tabs)::before{content:'';position:absolute;top:0;left:50%;width:100vw;margin-left:-50vw;border-top:1px solid var(--ax-line);pointer-events:none}.nav-tabs-item[data-active]{box-shadow:inset 0 -2px 0 var(--ax-accent)}.ax-product-selector{position:relative;display:inline-flex;align-items:stretch;flex:0 0 auto}.ax-product-selector--desktop{margin-left:0.75rem;align-self:center}.page-context-dropdown__label{font-family:inherit;font-weight:400}.page-context-dropdown__chevron>span{display:inline-block;transition:transform 150ms ease}.page-context-dropdown__chevron svg{display:block;width:1rem;height:1rem}.page-context-dropdown__chevron[aria-expanded=\"true\"]>span{transform:rotate(180deg)}.page-context-dropdown__menu{position:absolute;z-index:60;top:calc(100% + 0.55rem);right:0;min-width:12.5rem;padding:0.4rem;background:var(--ax-bg);border:1px solid var(--ax-line);border-radius:var(--ax-radius-md);box-shadow:0 0.75rem 1.5rem rgb(10 10 18 / 0.18)}.ax-product-selector--desktop .page-context-dropdown__menu{right:auto;left:0}.page-context-dropdown__menu[hidden]{display:none}.page-context-dropdown__item{display:flex;align-items:center;justify-content:space-between;min-height:2.5rem;padding:0.55rem 0.65rem;color:var(--ax-t1);border-radius:var(--ax-radius-sm);font-size:0.9375rem;font-weight:400;text-decoration:none;transition:background-color 150ms ease, color 150ms ease}.page-context-dropdown__item:hover,.page-context-dropdown__item:focus-visible{color:var(--ax-accent);background:var(--ax-accent-soft)}.page-context-dropdown__item[data-selected=\"true\"]{color:var(--ax-accent);font-weight:500}.page-context-dropdown__item[data-selected=\"true\"]::after{content:'✓';margin-left:1.5rem;color:var(--ax-accent);font-size:0.875rem}@media (prefers-reduced-motion: reduce){.page-context-dropdown__chevron>span,.page-context-dropdown__item{transition:none}}@media (max-width: 767px){#navbar .ax-product-selector--desktop{display:none}#mobile-nav-content .ax-product-selector--mobile{margin:0.5rem 0 1rem}#mobile-nav-content .page-context-dropdown__label{flex:1 1 auto}#mobile-nav-content .page-context-dropdown__chevron{flex:0 0 auto}#mobile-nav-content .page-context-dropdown__menu{position:static;margin-top:0.4rem}}#topbar-cta-button{background:var(--ax-accent-bg);color:var(--ax-on-accent);border-radius:var(--ax-radius-md);border:none}#topbar-cta-button span,#topbar-cta-button svg{color:var(--ax-on-accent)!important}#search-bar-entry{background:var(--ax-surface);border:1px solid var(--ax-line);border-radius:var(--ax-radius-md)}#assistant-entry{background:var(--ax-surface);border:1px solid var(--ax-accent-line);border-radius:var(--ax-radius-md)}#assistant-entry svg{color:var(--ax-accent)}#page-context-menu,.page-context-dropdown{font-size:0.9375rem;align-items:stretch}#page-context-menu-button,.page-context-dropdown__label{display:inline-flex;align-items:center;padding:10px 14px;border-radius:var(--ax-radius-md) 0 0 var(--ax-radius-md);background:var(--ax-surface);border-color:var(--ax-line);color:var(--ax-t1);border:1px solid var(--ax-line);text-decoration:none}#page-context-menu-button>div{font-size:0.9375rem;gap:9px}#page-context-menu>button[aria-haspopup],.page-context-dropdown__chevron{height:auto;align-self:stretch;aspect-ratio:auto;padding:10px 11px;border-radius:0 var(--ax-radius-md) var(--ax-radius-md) 0;background:var(--ax-surface);border-color:var(--ax-line);color:var(--ax-t3);border:1px solid var(--ax-line);border-left:0;cursor:pointer}#sidebar-content li[data-active]>a{background:var(--ax-accent-soft);color:var(--ax-accent);border-radius:var(--ax-radius-md);font-weight:500}.sidebar-group-header{font-weight:600;color:var(--ax-t1)}.toc-item[data-active],.toc-item[data-active] a{color:var(--ax-accent);font-weight:400}.card.card{background:var(--ax-surface);border-color:var(--ax-line);border-radius:var(--ax-radius-lg);box-shadow:none}.frame.frame{background:var(--ax-surface);border:1px solid var(--ax-line);border-radius:var(--ax-radius-lg)}.accordion.accordion,.accordion-group.accordion-group{border-radius:var(--ax-radius-md);border-color:var(--ax-line)}.accordion.accordion{background:var(--ax-surface)}.code-block.code-block,.code-group.code-group{background:var(--ax-code-bg);border-color:var(--ax-line);border-radius:var(--ax-radius-md)}.code-block-background{background:var(--ax-code-bg)!important}html.dark .code-block-background{background:var(--ax-code-bg)!important}#content :not(pre)>code{background:var(--ax-chip);border-radius:var(--ax-radius-sm)}.code-block:has([data-floating-buttons]) [data-fade-overlay]{background-image:none}[data-floating-buttons] button{background:var(--ax-chip)}.callout .code-block.code-block,.callout .code-group.code-group,.callout .code-block-background,html.dark .callout .code-block-background{background:var(--ax-bg)!important}.callout{border-radius:var(--ax-radius-lg)}.callout[data-callout-type=\"callout\"],.callout[data-callout-type=\"info\"]{background:var(--ax-surface);border-color:var(--ax-line)}.callout[data-callout-type=\"callout\"] [data-component-part=\"callout-content\"],.callout[data-callout-type=\"info\"] [data-component-part=\"callout-content\"]{color:var(--ax-t2)!important}.callout[data-callout-type=\"tip\"],.callout[data-callout-type=\"check\"]{background:var(--ax-note);border-color:var(--ax-note-line)}.callout[data-callout-type=\"tip\"] [data-component-part=\"callout-content\"],.callout[data-callout-type=\"check\"] [data-component-part=\"callout-content\"]{color:var(--ax-note-t)!important}.callout[data-callout-type=\"warning\"]{background:var(--ax-warn);border-color:var(--ax-warn-line)}.callout[data-callout-type=\"warning\"] [data-component-part=\"callout-content\"]{color:var(--ax-warn-t)!important}.callout[data-callout-type=\"note\"]{background:var(--ax-blue);border-color:var(--ax-blue-line)}.callout[data-callout-type=\"note\"] [data-component-part=\"callout-content\"]{color:var(--ax-blue-t)!important}.callout[data-callout-type=\"danger\"]{background:var(--ax-danger);border-color:var(--ax-danger-line)}.callout[data-callout-type=\"danger\"] [data-component-part=\"callout-content\"]{color:var(--ax-danger-t)!important}[data-component-part=\"step-number\"]>div{background:var(--ax-accent-soft);border:1px solid var(--ax-accent-line);color:var(--ax-accent);font-weight:500}.update.update-container{flex-direction:column;gap:0.5rem;border-top:1px solid var(--ax-line)}.update.update-container:first-of-type{border-top:none}.update.update-container>div:first-child{width:100%;position:static}.update.update-container>div:last-child{width:100%;max-width:none}#content th{color:var(--ax-t1);font-weight:600;border-bottom:1px solid var(--ax-line)}#content td{border-bottom:1px solid var(--ax-line)}footer{border-top:1px solid var(--ax-line)}.ax-hero .callout{background:var(--ax-panel);border-color:var(--ax-panel-border);border-radius:var(--ax-radius-lg)}.ax-hero .callout [data-component-part=\"callout-content\"],.ax-hero .callout p,.ax-hero .callout li{color:var(--ax-panel-t2)!important}.ax-hero .callout h2,.ax-hero .callout h3,.ax-hero .callout strong{color:var(--ax-panel-t1)!important}.ax-hero .callout [data-component-part=\"callout-icon\"] svg{background:var(--ax-panel-icon)!important}.ax-hero .code-block,.ax-hero .code-group,.ax-hero .code-block-background{background:var(--ax-panel-code)!important;border-color:var(--ax-panel-border)}.ax-hero .code-block,.ax-hero .code-group{padding:0;overflow:hidden}html.dark .ax-hero .code-block,html.dark .ax-hero .code-group,html.dark .ax-hero .code-block-background,html.dark .ax-hero .shiki,html:not(.dark) .ax-hero .code-block,html:not(.dark) .ax-hero .code-group,html:not(.dark) .ax-hero .code-block-background{background:var(--ax-panel-code)!important}html.dark .ax-hero .shiki,html.dark .ax-hero .shiki span{color:var(--ax-panel-t1)!important}html.dark .ax-hero .shiki span[style*=\"#FFA657\"]{color:var(--ax-panel-code-kw)!important}html.dark .ax-hero .shiki span[style*=\"#79C0FF\"]{color:var(--ax-panel-code-flag)!important}html.dark .ax-hero .shiki span[style*=\"#A5D6FF\"]{color:var(--ax-panel-code-url)!important}.ax-hero .code-group>div:first-child,html.dark .ax-hero .code-group>div:first-child{background:var(--ax-panel-2)!important;border-bottom:1px solid var(--ax-panel-border)}.ax-hero .code-group [role=\"tab\"]{color:var(--ax-panel-t2)!important}.ax-hero .code-group [role=\"tab\"][aria-selected=\"true\"]{color:var(--ax-panel-accent)!important}.ax-hero .code-group [role=\"tab\"][aria-selected=\"true\"]>div:last-child{background:var(--ax-panel-accent)!important}.ax-hero .code-block svg,.ax-hero .code-group svg{color:var(--ax-panel-t2)}html.dark .ax-hero [data-fade-overlay]{background-image:linear-gradient(\n to right,\n rgb(255 255 255 / 0) 0,\n rgb(255 255 255 / 0.2) 10px,\n rgb(255 255 255 / 0.5) 25px,\n rgb(255 255 255 / 0.8) 35px,\n rgb(255 255 255) 45px\n )}.ax-hero .callout a{color:var(--ax-panel-accent)!important}.ax-hero::selection,.ax-hero ::selection{background:var(--ax-panel-selection)}.card-group .card img{max-height:197px!important}.card-group[class$=\"-2\"] .card img{height:160px!important}.card-group[class$=\"-3\"] .card img{height:150px!important}table{display:table!important}.step-title{font-size:1.5rem!important}"}],"page":{"pagePath":"ax/integrations/evaluation-integrations/phoenix-evals","slug":"/ax/integrations/evaluation-integrations/phoenix-evals","pageMetadata":{"title":"Phoenix Evals","description":"Use the open-source phoenix-evals library to grade Arize AX traces and as evaluators in Arize AX experiments.","keywords":["phoenix evals","arize-phoenix-evals","evals","evaluation","llm-as-judge","create_classifier","evaluate_dataframe","experiments","python"],"href":"/ax/integrations/evaluation-integrations/phoenix-evals"},"description":{"compiledSource":"\"use strict\";\nconst {jsx: _jsx} = arguments[0];\nconst {useMDXComponents: _provideComponents} = arguments[0];\nfunction _createMdxContent(props) {\n const _components = {\n p: \"p\",\n ..._provideComponents(),\n ...props.components\n };\n return _jsx(_components.p, {\n children: \"Use the open-source phoenix-evals library to grade Arize AX traces and as evaluators in Arize AX experiments.\"\n });\n}\nfunction MDXContent(props = {}) {\n const {wrapper: MDXLayout} = {\n ..._provideComponents(),\n ...props.components\n };\n return MDXLayout ? _jsx(MDXLayout, {\n ...props,\n children: _jsx(_createMdxContent, {\n ...props\n })\n }) : _createMdxContent(props);\n}\nreturn {\n default: MDXContent\n};\n","frontmatter":{},"scope":{}},"pageType":"default","mdx":{"mdxSource":{"compiledSource":"\"use strict\";const{Fragment:_Fragment,jsx:_jsx,jsxs:_jsxs}=arguments[0];const{useMDXComponents:_provideComponents}=arguments[0];function _createMdxContent(props){const _components={a:`a`,code:`code`,em:`em`,li:`li`,p:`p`,pre:`pre`,span:`span`,strong:`strong`,ul:`ul`,..._provideComponents(),...props.components},{Card,CardGroup,CodeBlock,Heading,Tip,Warning}=_components;if(!Card)_missingMdxReference(`Card`,true);if(!CardGroup)_missingMdxReference(`CardGroup`,true);if(!CodeBlock)_missingMdxReference(`CodeBlock`,true);if(!Heading)_missingMdxReference(`Heading`,true);if(!Tip)_missingMdxReference(`Tip`,true);if(!Warning)_missingMdxReference(`Warning`,true);return _jsxs(_Fragment,{children:[_jsxs(_components.p,{children:[_jsx(_components.a,{href:`https://pypi.org/project/arize-phoenix-evals/`,children:_jsx(_components.code,{children:`arize-phoenix-evals`})}),` is Arize’s open-source, backend-agnostic evaluation library — the same library the `,_jsx(_components.a,{href:`/ax/integrations`,children:`LLM-as-a-judge provider guides`}),` wire a judge model into. It ships composable primitives (`,_jsx(_components.code,{children:`LLM`}),`, `,_jsx(_components.code,{children:`create_classifier`}),`, `,_jsx(_components.code,{children:`evaluate_dataframe`}),`) and prebuilt metric evaluators (hallucination, correctness, relevance, and more), and runs entirely in-process: it needs no separate server, and its results feed Arize AX through the `,_jsx(_components.code,{children:`arize`}),` SDK.`]}),`\n`,_jsxs(_components.p,{children:[`This guide shows both ways to wire phoenix-evals into Arize AX: Flow 1 grades existing Arize AX traces with a classifier and writes the scores back via `,_jsx(_components.code,{children:`client.spans.update_evaluations(...)`}),`; Flow 2 uploads a small dataset, runs an Arize AX experiment with the same classifier wrapped as an evaluator, and surfaces the scores in Datasets+Experiments. For the concepts behind the offline pattern, see `,_jsx(_components.a,{href:`/ax/concepts/evaluators/offline-evaluation-with-phoenix`,children:`Offline evaluation`}),`; for running these evals on a schedule inside the platform, see `,_jsx(_components.a,{href:`/ax/evaluate/run-evals-on-traces`,children:`Run evals on traces`}),`.`]}),`\n`,_jsx(_components.p,{children:`Both flows share the same setup. Run the code blocks below in order inside a single Python session — each block builds on imports and variables from earlier ones.`}),`\n`,_jsx(Heading,{level:`2`,id:`prerequisites`,children:`Prerequisites`}),`\n`,_jsxs(_components.ul,{children:[`\n`,_jsx(_components.li,{children:`Python 3.11+`}),`\n`,_jsxs(_components.li,{children:[`An `,_jsx(_components.code,{children:`ARIZE_SPACE_ID`}),` and `,_jsx(_components.code,{children:`ARIZE_API_KEY`}),` from your Arize AX space settings`]}),`\n`,_jsxs(_components.li,{children:[`An `,_jsx(_components.code,{children:`OPENAI_API_KEY`}),` from `,_jsx(_components.a,{href:`https://platform.openai.com/api-keys`,children:`OpenAI Platform`}),` (used as both the model under trace and the phoenix-evals judge LLM)`]}),`\n`]}),`\n`,_jsx(Heading,{level:`2`,id:`launch-arize-ax`,children:`Launch Arize AX`}),`\n`,_jsxs(_components.p,{children:[`If you don’t already have an Arize AX account, sign up at `,_jsx(_components.a,{href:`https://arize.com/`,children:`arize.com`}),` and grab your `,_jsx(_components.code,{children:`ARIZE_SPACE_ID`}),` and `,_jsx(_components.code,{children:`ARIZE_API_KEY`}),` from Settings → Space Settings.`]}),`\n`,_jsx(Heading,{level:`2`,id:`install`,children:`Install`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`1`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`1`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`pip`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` install`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` 'arize>=8.0.0'`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` arize-phoenix-evals`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` openai`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` openinference-instrumentation-openai`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` opentelemetry-sdk`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` opentelemetry-exporter-otlp-proto-grpc`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` pandas`})]}),`\n`]})})}),`\n`,_jsx(Heading,{level:`2`,id:`configure-credentials`,children:`Configure credentials`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`3`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`3`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`export`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` ARIZE_SPACE_ID`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"\u003cyour-space-id>\"`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`export`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` ARIZE_API_KEY`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"\u003cyour-api-key>\"`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`export`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` OPENAI_API_KEY`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"\u003cyour-openai-api-key>\"`})]}),`\n`]})})}),`\n`,_jsx(Heading,{level:`2`,id:`define-evaluators`,children:`Define evaluators`}),`\n`,_jsxs(_components.p,{children:[`The shared setup: a `,_jsx(_components.code,{children:`create_classifier`}),` correctness evaluator backed by a GPT-4.1 judge, the canonical 2-row dataset both flows score, and an Arize SDK client. `,_jsx(_components.code,{children:`create_classifier`}),` relies on the judge’s tool-calling / structured-output support, so use a non-reasoning model such as GPT-4.1. The `,_jsx(_components.code,{children:`{input}`}),`, `,_jsx(_components.code,{children:`{output}`}),`, and `,_jsx(_components.code,{children:`{reference}`}),` placeholders in the template are filled from dataframe columns of the same name.`]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`51`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`51`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# combined.py`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` os`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` time`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`from`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` datetime `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` datetime, timedelta, timezone`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` pandas `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`as`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` pd`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`from`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` arize `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` ArizeClient`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`from`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` phoenix.evals `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:` LLM`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`, create_classifier, evaluate_dataframe`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`from`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` phoenix.evals.utils `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` to_annotation_dataframe`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`SPACE_ID`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` =`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` os.environ[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"ARIZE_SPACE_ID\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`API_KEY`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` =`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` os.environ[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"ARIZE_API_KEY\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`TIMESTAMP`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` =`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:` int`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(time.time())`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# The judge model. GPT-4.1 is a non-reasoning model that supports the`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# tool-calling / structured output create_classifier relies on.`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`judge `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` LLM(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`provider`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"openai\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`, `}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`model`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"gpt-4.1\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`TEMPLATE`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` =`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` (`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"You are grading whether a response is factually correct.`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`\\\\n`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"[Question]: `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`{input}\\\\n`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"[Response]: `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`{output}\\\\n`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"[Reference]: `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`{reference}\\\\n\\\\n`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"Is the response factually correct given the reference? \"`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"Answer 'correct' or 'incorrect'.\"`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`correctness `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` create_classifier(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` name`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"correctness\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` prompt_template`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`TEMPLATE`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` llm`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`judge,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` choices`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`{`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"correct\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`1.0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"incorrect\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`0.0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`},`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` direction`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"maximize\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# Canonical 2-row dataset — row 0 is factual (answer matches the reference),`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# row 1 is hallucinated. Both flows grade these same rows.`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`ROWS`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` =`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` [`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` {`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"input\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"What is the capital of France?\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"output\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"Paris is the capital of France.\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"reference\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"Paris is the capital and most populous city of France.\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` },`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` {`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"input\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"What is the capital of France?\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"output\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"Berlin is the capital of France.\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"reference\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"Paris is the capital and most populous city of France.\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` },`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`]`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`arize `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` ArizeClient(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`api_key`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`API_KEY`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})]}),`\n`]})})}),`\n`,_jsx(Tip,{children:_jsxs(_components.p,{children:[_jsx(_components.code,{children:`create_classifier`}),` builds a custom categorical evaluator. For common metrics, `,_jsx(_components.code,{children:`phoenix.evals.metrics`}),` ships prebuilt evaluators — `,_jsx(_components.code,{children:`HallucinationEvaluator`}),`, `,_jsx(_components.code,{children:`CorrectnessEvaluator`}),`, `,_jsx(_components.code,{children:`DocumentRelevanceEvaluator`}),`, `,_jsx(_components.code,{children:`RefusalEvaluator`}),`, and others — that you can pass to `,_jsx(_components.code,{children:`evaluate_dataframe`}),` in place of the classifier above.`]})}),`\n`,_jsx(Heading,{level:`2`,id:`flow-1-—-evaluate-existing-traces`,children:`Flow 1 — Evaluate existing traces`}),`\n`,_jsx(Heading,{level:`3`,id:`source-the-spans`,children:`Source the spans`}),`\n`,_jsx(_components.p,{children:`Instrument OpenAI with OpenInference, make two calls (each forced to echo a known answer so the trace contains predictable text), then pull the resulting spans back from Arize AX.`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`86`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`86`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`from`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` openinference.instrumentation.openai `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` OpenAIInstrumentor`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`from`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` opentelemetry `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` trace`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`from`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` opentelemetry.exporter.otlp.proto.grpc.trace_exporter `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` OTLPSpanExporter`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`from`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` opentelemetry.sdk.resources `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` Resource`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`from`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` opentelemetry.sdk.trace `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` TracerProvider`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`from`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` opentelemetry.sdk.trace.export `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` BatchSpanProcessor`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`from`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` openai `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` OpenAI`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`PROJECT_NAME`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` =`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"phoenix-evals-example-`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`{`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`TIMESTAMP`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`resource `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` Resource.create(`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` {`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"service.name\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`PROJECT_NAME`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"openinference.project.name\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`PROJECT_NAME`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"model_id\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`PROJECT_NAME`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` }`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`provider `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` TracerProvider(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`resource`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`resource)`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`provider.add_span_processor(`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` BatchSpanProcessor(`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` OTLPSpanExporter(`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` endpoint`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"https://otlp.arize.com:443\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` headers`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`{`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"authorization\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`API_KEY`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"arize-space-id\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`SPACE_ID`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"arize-interface\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"python\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` },`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` )`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` )`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`trace.set_tracer_provider(provider)`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`OpenAIInstrumentor().instrument(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`tracer_provider`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`provider)`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`sync_oai `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` OpenAI()`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`for`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` row `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`in`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:` ROWS`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`:`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` sync_oai.chat.completions.create(`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` model`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"gpt-5.4-mini\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` messages`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`[`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` {`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"role\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"system\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"content\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: (`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"You are a fact-recall assistant. The user states the \"`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"exact answer to use; reply with that verbatim.\"`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` ),`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` },`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` {`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"role\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"user\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"content\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: (`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"Question: `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`row[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`'input'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`]`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`}\\\\n`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"Answer (reply verbatim): `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`row[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`'output'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`]`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` ),`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` },`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` ],`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` )`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`provider.force_flush(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`timeout_millis`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`10_000`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`print`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"Project: `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`{`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`PROJECT_NAME`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# Spans take ~5–15s to be queryable after flush. Poll defensively: Arize's`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# OTLP ingest and Flight export use different catalogs and the new project`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# can briefly appear \"unauthorized\" to the export endpoint while still`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# accepting span writes via OTLP, so swallow transient errors and retry.`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`start `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` datetime.now(timezone.utc) `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`-`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` timedelta(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`minutes`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`5`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`end `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` datetime.now(timezone.utc) `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`+`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` timedelta(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`minutes`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`1`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`spans_df `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:` None`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`last_err: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`Exception`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` |`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:` None`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` =`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:` None`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`for`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` _ `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`in`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:` range`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`12`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`):`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` time.sleep(`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`5`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` try`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` spans_df `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` arize.spans.export_to_df(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` space_id`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`SPACE_ID`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` project_name`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`PROJECT_NAME`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` start_time`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`start,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` end_time`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`end,`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` )`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` except`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:` Exception`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` as`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` e:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` last_err `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` e`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` continue`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` if`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` spans_df `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`is`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` not`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:` None`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` and`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:` len`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(spans_df) `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`>=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:` len`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`ROWS`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`):`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` break`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`else`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` raise`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:` RuntimeError`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"Spans never appeared after 60s (last error: `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`last_err`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`)\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` )`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`spans_df `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` spans_df.sort_values(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"start_time\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`).reset_index(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`drop`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`True`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})]}),`\n`]})})}),`\n`,_jsx(Heading,{level:`3`,id:`run-the-evaluators`,children:`Run the evaluators`}),`\n`,_jsxs(_components.p,{children:[`Map the span attributes to the template variables, then run the classifier over the whole dataframe in one pass. `,_jsx(_components.code,{children:`evaluate_dataframe`}),` returns the input dataframe plus a `,_jsx(_components.code,{children:`correctness_score`}),` column (a dict holding the label, score, and explanation) and a `,_jsx(_components.code,{children:`correctness_execution_details`}),` column. The `,_jsx(_components.code,{children:`reference`}),` value isn’t on the span, so it’s sourced from `,_jsx(_components.code,{children:`ROWS`}),` by position.`]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`5`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`5`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`spans_df[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"input\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`] `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` spans_df[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"attributes.input.value\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`spans_df[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"output\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`] `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` spans_df[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"attributes.output.value\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`spans_df[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"reference\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`] `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` [r[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"reference\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`] `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`for`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` r `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`in`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:` ROWS`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`]`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`results `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` evaluate_dataframe(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`dataframe`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`spans_df, `}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`evaluators`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`[correctness])`})]}),`\n`]})})}),`\n`,_jsx(Tip,{children:_jsxs(_components.p,{children:[`For larger batches, `,_jsx(_components.code,{children:`async_evaluate_dataframe(dataframe=..., evaluators=[...])`}),` is the async equivalent and runs the judge calls concurrently.`]})}),`\n`,_jsx(Heading,{level:`3`,id:`log-evaluations-to-arize-ax`,children:`Log evaluations to Arize AX`}),`\n`,_jsxs(_components.p,{children:[_jsx(_components.code,{children:`to_annotation_dataframe`}),` flattens the nested `,_jsx(_components.code,{children:`correctness_score`}),` dict into `,_jsx(_components.code,{children:`score`}),`, `,_jsx(_components.code,{children:`label`}),`, and `,_jsx(_components.code,{children:`explanation`}),` columns while preserving `,_jsx(_components.code,{children:`context.span_id`}),` (the join key `,_jsx(_components.code,{children:`export_to_df`}),` already provides). Rename those to the reserved `,_jsx(_components.code,{children:`eval.\u003cname>.{score,label,explanation}`}),` columns Arize AX expects, then upload. For trace or session evals use the `,_jsx(_components.code,{children:`trace_eval.\u003cname>`}),` / `,_jsx(_components.code,{children:`session_eval.\u003cname>`}),` prefixes instead.`]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`24`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`24`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`annotations `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` to_annotation_dataframe(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`dataframe`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`results)`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`upload_df `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` annotations.rename(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` columns`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`{`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"label\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"eval.correctness.label\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"score\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"eval.correctness.score\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"explanation\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"eval.correctness.explanation\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` }`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)[[`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"context.span_id\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"eval.correctness.label\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"eval.correctness.score\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"eval.correctness.explanation\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`]]`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`arize.spans.update_evaluations(`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` space_id`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`SPACE_ID`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` project_name`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`PROJECT_NAME`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` dataframe`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`upload_df,`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# Print the scores so they appear in stdout for verification.`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`print`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"Flow 1 results:\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`print`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(upload_df[[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"eval.correctness.label\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"eval.correctness.score\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`]].to_string())`})]}),`\n`]})})}),`\n`,_jsx(Heading,{level:`3`,id:`expected-output`,children:`Expected output`}),`\n`,_jsx(CodeBlock,{filename:``,wrap:`true`,numberOfLines:`5`,language:`text`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`text`,children:_jsxs(_components.code,{language:`text`,numberOfLines:`5`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`Flow 1 results:`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` eval.correctness.label eval.correctness.score`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`0 correct 1.0`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`1 incorrect 0.0`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})})]})})}),`\n`,_jsx(Heading,{level:`3`,id:`verify-in-arize-ax`,children:`Verify in Arize AX`}),`\n`,_jsxs(_components.p,{children:[`Open the project named `,_jsx(_components.code,{children:`phoenix-evals-example-\u003ctimestamp>`}),` (the value printed above) in your Arize AX space. Each `,_jsx(_components.code,{children:`ChatCompletion`}),` span now carries a `,_jsx(_components.code,{children:`correctness`}),` annotation column showing the score and label written by `,_jsx(_components.code,{children:`update_evaluations(...)`}),`.`]}),`\n`,_jsx(Heading,{level:`2`,id:`flow-2-—-run-an-experiment`,children:`Flow 2 — Run an experiment`}),`\n`,_jsx(Heading,{level:`3`,id:`create-a-dataset`,children:`Create a dataset`}),`\n`,_jsxs(_components.p,{children:[`The dataset is the same two rows. The `,_jsx(_components.code,{children:`space=`}),` / `,_jsx(_components.code,{children:`examples=`}),` kwarg names match the v8 SDK exactly (note: not `,_jsx(_components.code,{children:`space_id=`}),` and not `,_jsx(_components.code,{children:`dataframe=`}),`).`]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`7`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`7`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`DATASET_NAME`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` =`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"phoenix-evals-example-ds-`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`{`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`TIMESTAMP`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`arize.datasets.create(`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` name`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`DATASET_NAME`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` space`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`SPACE_ID`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` examples`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`pd.DataFrame(`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`ROWS`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`),`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`print`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"Dataset: `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`{`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`DATASET_NAME`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})]}),`\n`]})})}),`\n`,_jsx(Heading,{level:`3`,id:`define-the-task`,children:`Define the task`}),`\n`,_jsxs(_components.p,{children:[`The task function receives the dataset row and returns whatever the experiment should grade. The parameter name `,_jsx(_components.strong,{children:`must`}),` be one of `,_jsx(_components.code,{children:`input`}),`, `,_jsx(_components.code,{children:`output`}),`, `,_jsx(_components.code,{children:`metadata`}),`, or `,_jsx(_components.code,{children:`dataset_row`}),` — a single-arg task with an unrecognized name is bound to `,_jsx(_components.code,{children:`dataset_row`}),` by default. A real workflow would call an LLM here; this passthrough keeps the example deterministic.`]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`2`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`2`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`def`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#D2A8FF`},children:` task`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(dataset_row):`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` return`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` dataset_row[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"output\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`]`})]}),`\n`]})})}),`\n`,_jsx(Heading,{level:`3`,id:`wrap-the-evaluators`,children:`Wrap the evaluators`}),`\n`,_jsxs(_components.p,{children:[`Experiment evaluators run inside an `,_jsx(_components.code,{children:`asyncio`}),` loop, so use `,_jsx(_components.code,{children:`async def`}),` and the classifier’s `,_jsx(_components.code,{children:`async_evaluate(...)`}),` — it returns a list of `,_jsx(_components.code,{children:`Score`}),` objects, one per evaluator. Convert the first `,_jsx(_components.code,{children:`Score`}),` to an `,_jsx(_components.code,{children:`EvaluationResult`}),` with score `,_jsx(_components.strong,{children:`and`}),` label `,_jsx(_components.strong,{children:`and`}),` explanation populated: leaving any of those reserved fields as `,_jsx(_components.code,{children:`None`}),` triggers `,_jsx(_components.code,{children:`unsupported cast from null to \u003ctype>: reserved column cannot be coerced to canonical type`}),` at upload time.`]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`17`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`17`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`from`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` arize.experiments.evaluators.types `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` EvaluationResult`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`async`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` def`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#D2A8FF`},children:` correctness_eval`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(output, dataset_row) -> EvaluationResult:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` scores `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` await`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` correctness.async_evaluate(`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` {`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"input\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: dataset_row[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"input\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"output\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: output `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`if`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:` isinstance`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(output, `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`str`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`) `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`else`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:` str`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(output),`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"reference\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: dataset_row[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"reference\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`],`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` }`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` )`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` s `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` scores[`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` return`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` EvaluationResult(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` score`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`s.score,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` label`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`s.label,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` explanation`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`s.explanation `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`or`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"no explanation\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` )`})}),`\n`]})})}),`\n`,_jsx(Warning,{children:_jsxs(_components.p,{children:[_jsx(_components.code,{children:`EvaluationResult`}),`’s constructor order is `,_jsx(_components.code,{children:`(score, label, explanation, metadata)`}),` — `,_jsx(_components.em,{children:`not`}),` `,_jsx(_components.code,{children:`(label, score, ...)`}),` as the alphabetical reading might suggest. Always use keyword arguments; positional calls silently swap label and score.`]})}),`\n`,_jsx(Heading,{level:`3`,id:`run-the-experiment`,children:`Run the experiment`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`20`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`20`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`EXPERIMENT_NAME`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` =`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"phoenix-evals-example-`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`{`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`TIMESTAMP`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`experiment, runs_df `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` arize.experiments.run(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` space`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`SPACE_ID`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` name`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`EXPERIMENT_NAME`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` dataset`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`DATASET_NAME`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` task`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`task,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` evaluators`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`{`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"correctness\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: correctness_eval},`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`print`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"Experiment: `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`{`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`EXPERIMENT_NAME`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`print`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"Flow 2 results:\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`print`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` runs_df[[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"output\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"eval.correctness.score\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"eval.correctness.label\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`]]`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` .rename(`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` columns`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`{`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"eval.correctness.score\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"score\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"eval.correctness.label\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"label\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` }`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` )`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` .to_string()`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`]})})}),`\n`,_jsx(Heading,{level:`3`,id:`expected-output-2`,children:`Expected output`}),`\n`,_jsx(CodeBlock,{filename:``,wrap:`true`,numberOfLines:`5`,language:`text`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`text`,children:_jsxs(_components.code,{language:`text`,numberOfLines:`5`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`Flow 2 results:`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:` output score label`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`0 Paris is the capital of France. 1.0 correct`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`1 Berlin is the capital of France. 0.0 incorrect`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})})]})})}),`\n`,_jsx(Heading,{level:`3`,id:`verify-in-arize-ax-2`,children:`Verify in Arize AX`}),`\n`,_jsxs(_components.p,{children:[`Open the `,_jsx(_components.strong,{children:`Datasets + Experiments`}),` tab in Arize AX. The dataset `,_jsx(_components.code,{children:`phoenix-evals-example-ds-\u003ctimestamp>`}),` and the experiment `,_jsx(_components.code,{children:`phoenix-evals-example-\u003ctimestamp>`}),` (names printed above) appear with one run per dataset row, each carrying the `,_jsx(_components.code,{children:`correctness`}),` score and label columns.`]}),`\n`,_jsx(Heading,{level:`2`,id:`troubleshooting`,children:`Troubleshooting`}),`\n`,_jsxs(_components.ul,{children:[`\n`,_jsxs(_components.li,{children:[_jsxs(_components.strong,{children:[_jsx(_components.code,{children:`create_classifier`}),` fails or returns empty labels.`]}),` The judge model must support tool calling / structured output. Reasoning models (GPT-5-family or o-series ids) can also reject standard sampling params — use a non-reasoning judge such as `,_jsx(_components.code,{children:`gpt-4.1`}),`.`]}),`\n`,_jsxs(_components.li,{children:[_jsxs(_components.strong,{children:[_jsx(_components.code,{children:`column \"eval.\u003cname>.label\": unsupported cast from null to string: reserved column cannot be coerced to canonical type`}),`.`]}),` In Flow 2 your evaluator returned a bare number or string instead of a fully-populated `,_jsx(_components.code,{children:`EvaluationResult(score=..., label=..., explanation=...)`}),`. Arize AX’s Flight server rejects null values in reserved eval columns — populate all three fields.`]}),`\n`,_jsxs(_components.li,{children:[_jsxs(_components.strong,{children:[_jsx(_components.code,{children:`Cannot call sync ... from an async context`}),`.`]}),` Your Flow 2 evaluator is calling the classifier’s sync `,_jsx(_components.code,{children:`evaluate(...)`}),` instead of `,_jsx(_components.code,{children:`async_evaluate(...)`}),`. Experiment evaluators run inside `,_jsx(_components.code,{children:`asyncio`}),`; use the async API. Flow 1 uses `,_jsx(_components.code,{children:`evaluate_dataframe(...)`}),` because it runs outside any loop.`]}),`\n`,_jsxs(_components.li,{children:[_jsx(_components.strong,{children:`Spans never appear after 60s.`}),` Span flush + ingest typically takes 5–15s. If the loop times out, check that `,_jsx(_components.code,{children:`ARIZE_SPACE_ID`}),` + `,_jsx(_components.code,{children:`ARIZE_API_KEY`}),` are right and that you’re connecting to the correct region’s OTLP endpoint (`,_jsx(_components.code,{children:`otlp.arize.com`}),` for US, `,_jsx(_components.code,{children:`otlp.eu.arize.com`}),` for EU). Evals can only be applied to spans up to 14 days old; for older spans contact `,_jsx(_components.a,{href:`mailto:support@arize.com`,children:`support@arize.com`}),`.`]}),`\n`,_jsxs(_components.li,{children:[_jsxs(_components.strong,{children:[_jsx(_components.code,{children:`task failed for example id ...`}),`.`]}),` Your task function’s parameter name isn’t one of the recognized names (`,_jsx(_components.code,{children:`input`}),`, `,_jsx(_components.code,{children:`output`}),`, `,_jsx(_components.code,{children:`metadata`}),`, `,_jsx(_components.code,{children:`dataset_row`}),`). Rename it to `,_jsx(_components.code,{children:`dataset_row`}),` if you want the whole row, or pick the field you actually need.`]}),`\n`,_jsxs(_components.li,{children:[_jsx(_components.strong,{children:`Experiment runs duplicate or the dataset already exists.`}),` Both names embed `,_jsx(_components.code,{children:`TIMESTAMP = int(time.time())`}),` so a single re-run produces unique names. If you re-execute the same `,_jsx(_components.code,{children:`combined.py`}),` quickly, regenerate `,_jsx(_components.code,{children:`TIMESTAMP`}),` first.`]}),`\n`]}),`\n`,_jsx(Heading,{level:`2`,id:`resources`,children:`Resources`}),`\n`,_jsxs(CardGroup,{children:[_jsx(Card,{icon:`book-open`,href:`https://arize-phoenix.readthedocs.io/projects/evals/`,title:`Phoenix Evals reference`,horizontal:true}),_jsx(Card,{icon:`github`,href:`https://github.com/Arize-ai/phoenix/tree/main/packages/phoenix-evals`,title:`arize-phoenix-evals on GitHub`,horizontal:true}),_jsx(Card,{icon:`book-open`,href:`/ax/concepts/evaluators/offline-evaluation-with-phoenix`,title:`Offline evaluation concepts`,horizontal:true}),_jsx(Card,{icon:`book-open`,href:`/api-clients/python/version-8/client-resources/spans#update-evaluations`,title:`Logging evaluations to Arize AX`,horizontal:true})]})]})}function MDXContent(props={}){const{wrapper:MDXLayout}={..._provideComponents(),...props.components};return MDXLayout?_jsx(MDXLayout,{...props,children:_jsx(_createMdxContent,{...props})}):_createMdxContent(props)}return{default:MDXContent};function _missingMdxReference(id,component){throw new Error(`Expected `+(component?`component`:`object`)+\" `\"+id+\"` to be defined: you likely forgot to import, pass, or provide it.\")}","frontmatter":{},"scope":{"codeStyling":{"theme":{"light":"github-light-default","dark":"github-dark-default"}},"config":{},"pageMetadata":{"title":"Phoenix Evals","description":"Use the open-source phoenix-evals library to grade Arize AX traces and as evaluators in Arize AX experiments.","keywords":["phoenix evals","arize-phoenix-evals","evals","evaluation","llm-as-judge","create_classifier","evaluate_dataframe","experiments","python"],"href":"/ax/integrations/evaluation-integrations/phoenix-evals"}}},"mdxSourceWithNoJs":null},"mdxExtracts":{"tableOfContents":[{"title":"Prerequisites","slug":"prerequisites","depth":2,"children":[]},{"title":"Launch Arize AX","slug":"launch-arize-ax","depth":2,"children":[]},{"title":"Install","slug":"install","depth":2,"children":[]},{"title":"Configure credentials","slug":"configure-credentials","depth":2,"children":[]},{"title":"Define evaluators","slug":"define-evaluators","depth":2,"children":[]},{"title":"Flow 1 — Evaluate existing traces","slug":"flow-1-—-evaluate-existing-traces","depth":2,"children":[{"title":"Source the spans","slug":"source-the-spans","depth":3,"children":[]},{"title":"Run the evaluators","slug":"run-the-evaluators","depth":3,"children":[]},{"title":"Log evaluations to Arize AX","slug":"log-evaluations-to-arize-ax","depth":3,"children":[]},{"title":"Expected output","slug":"expected-output","depth":3,"children":[]},{"title":"Verify in Arize AX","slug":"verify-in-arize-ax","depth":3,"children":[]}]},{"title":"Flow 2 — Run an experiment","slug":"flow-2-—-run-an-experiment","depth":2,"children":[{"title":"Create a dataset","slug":"create-a-dataset","depth":3,"children":[]},{"title":"Define the task","slug":"define-the-task","depth":3,"children":[]},{"title":"Wrap the evaluators","slug":"wrap-the-evaluators","depth":3,"children":[]},{"title":"Run the experiment","slug":"run-the-experiment","depth":3,"children":[]},{"title":"Expected output","slug":"expected-output-2","depth":3,"children":[]},{"title":"Verify in Arize AX","slug":"verify-in-arize-ax-2","depth":3,"children":[]}]},{"title":"Troubleshooting","slug":"troubleshooting","depth":2,"children":[]},{"title":"Resources","slug":"resources","depth":2,"children":[]}],"codeExamples":{}},"jsLinks":["/docs/_mintlify/custom/arize-ax/6a30b7a67da02161/analytics.js","/docs/_mintlify/custom/arize-ax/67a4d27b62fababa/product-selector.js"],"subdomain":"arize-ax","siteName":"Arize AX Docs","title":"Phoenix Evals - Arize AX Docs","navMaps":{"firstHrefInVersion":{},"firstHrefInLanguage":{}},"lastModified":"2026-08-18T18:08:20.512Z","originalFileLocation":"ax/integrations/evaluation-integrations/phoenix-evals.mdx"},"headHtml":"\u003cscript data-mint-head>(function(){var d=document.documentElement,p=null;if(!p){try{p=localStorage.getItem(\"isDarkMode\")}catch(e){}if(p==='true')p='dark';else if(p==='false')p='light';if(p!=='light'\u0026\u0026p!=='dark'\u0026\u0026p!=='system')p=\"dark\"}var r=p==='system'?(matchMedia(\"(prefers-color-scheme: dark)\").matches?'dark':'light'):p;d.classList.remove(r==='dark'?'light':'dark');d.classList.add(r);d.setAttribute(\"data-theme-preference\",p)})();\u003c/script>\u003cmeta data-mint-head name=\"description\" content=\"Use the open-source phoenix-evals library to grade Arize AX traces and as evaluators in Arize AX experiments.\">\u003cmeta data-mint-head name=\"application-name\" content=\"Arize AX Docs\">\u003cmeta data-mint-head name=\"generator\" content=\"Mintlify\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/docs/mintlify-assets/_mintlify/favicons/arize-ax/vxzD83QsbAzzm43O/_generated/favicon/android-chrome-192x192.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"apple-touch-icon\" type=\"image/png\" sizes=\"180x180\" href=\"/docs/mintlify-assets/_mintlify/favicons/arize-ax/vxzD83QsbAzzm43O/_generated/favicon/apple-touch-icon.png\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/docs/mintlify-assets/_mintlify/favicons/arize-ax/vxzD83QsbAzzm43O/_generated/favicon/favicon-16x16.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/docs/mintlify-assets/_mintlify/favicons/arize-ax/vxzD83QsbAzzm43O/_generated/favicon/favicon-32x32.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/docs/mintlify-assets/_mintlify/favicons/arize-ax/vxzD83QsbAzzm43O/_generated/favicon/favicon.ico\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/docs/mintlify-assets/_mintlify/favicons/arize-ax/vxzD83QsbAzzm43O/_generated/favicon-dark/android-chrome-192x192.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/docs/mintlify-assets/_mintlify/favicons/arize-ax/vxzD83QsbAzzm43O/_generated/favicon-dark/favicon-16x16.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/docs/mintlify-assets/_mintlify/favicons/arize-ax/vxzD83QsbAzzm43O/_generated/favicon-dark/favicon-32x32.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/docs/mintlify-assets/_mintlify/favicons/arize-ax/vxzD83QsbAzzm43O/_generated/favicon-dark/favicon.ico\" media=\"(prefers-color-scheme: dark)\">\u003cmeta data-mint-head property=\"og:title\" content=\"Phoenix Evals - Arize AX Docs\">\u003cmeta data-mint-head property=\"og:description\" content=\"Use the open-source phoenix-evals library to grade Arize AX traces and as evaluators in Arize AX experiments.\">\u003cmeta data-mint-head property=\"og:site_name\" content=\"Arize AX Docs\">\u003cmeta data-mint-head property=\"og:url\" content=\"https://arize.com/docs/ax/integrations/evaluation-integrations/phoenix-evals\">\u003cmeta data-mint-head property=\"og:type\" content=\"website\">\u003cmeta data-mint-head property=\"og:image\" content=\"https://arize-ax.mintlify.site/docs/_mintlify/api/og/v2?division=Evaluation\u0026amp;title=Phoenix+Evals\u0026amp;description=Use+the+open-source+phoenix-evals+library+to+grade+Arize+AX+traces+and+as+evaluators+in+Arize+AX+experiments.\u0026amp;theme=6b2e1c1a14346acb1262754f\u0026amp;signature=d29af3be17e5962c11c8fa1b03caed4add21d87bd512b2067e57b2a918800b38\">\u003cmeta data-mint-head property=\"og:image:width\" content=\"1200\">\u003cmeta data-mint-head property=\"og:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"twitter:card\" content=\"summary_large_image\">\u003cmeta data-mint-head name=\"twitter:title\" content=\"Phoenix Evals - Arize AX Docs\">\u003cmeta data-mint-head name=\"twitter:description\" content=\"Use the open-source phoenix-evals library to grade Arize AX traces and as evaluators in Arize AX experiments.\">\u003cmeta data-mint-head name=\"twitter:image\" content=\"https://arize-ax.mintlify.site/docs/_mintlify/api/og/v2?division=Evaluation\u0026amp;title=Phoenix+Evals\u0026amp;description=Use+the+open-source+phoenix-evals+library+to+grade+Arize+AX+traces+and+as+evaluators+in+Arize+AX+experiments.\u0026amp;theme=6b2e1c1a14346acb1262754f\u0026amp;signature=d29af3be17e5962c11c8fa1b03caed4add21d87bd512b2067e57b2a918800b38\">\u003cmeta data-mint-head name=\"twitter:image:width\" content=\"1200\">\u003cmeta data-mint-head name=\"twitter:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"msapplication-config\" content=\"/docs/mintlify-assets/_mintlify/favicons/arize-ax/vxzD83QsbAzzm43O/_generated/favicon/browserconfig.xml\">\u003cmeta data-mint-head name=\"apple-mobile-web-app-title\" content=\"Arize AX Docs\">\u003cmeta data-mint-head name=\"msapplication-TileColor\" content=\"#7A0047\">\u003clink data-mint-head rel=\"canonical\" href=\"https://arize.com/docs/ax/integrations/evaluation-integrations/phoenix-evals\">\u003clink data-mint-head rel=\"alternate\" type=\"application/xml\" href=\"/docs/sitemap.xml\">\u003clink data-mint-head rel=\"alternate\" type=\"text/markdown\" href=\"/docs/ax/integrations/evaluation-integrations/phoenix-evals.md\">\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https://arize.com/#organization\",\"name\":\"Arize AX Docs\",\"url\":\"https://arize.com\",\"logo\":{\"@type\":\"ImageObject\",\"url\":\"https://mintcdn.com/arize-ax/V-uvingrGLOCK4VD/logo/light.svg?fit=max\u0026auto=format\u0026n=V-uvingrGLOCK4VD\u0026q=85\u0026s=cc58ee9388d5b5253d39b65cc662908f\"}},{\"@type\":\"WebSite\",\"@id\":\"https://arize.com/docs#website\",\"name\":\"Arize AX Docs\",\"url\":\"https://arize.com/docs\",\"publisher\":{\"@id\":\"https://arize.com/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https://arize.com/docs/ax/integrations/evaluation-integrations/phoenix-evals#webpage\",\"url\":\"https://arize.com/docs/ax/integrations/evaluation-integrations/phoenix-evals\",\"name\":\"Phoenix Evals\",\"description\":\"Use the open-source phoenix-evals library to grade Arize AX traces and as evaluators in Arize AX experiments.\",\"dateModified\":\"2026-08-18T18:08:20.512Z\",\"isPartOf\":{\"@id\":\"https://arize.com/docs#website\"},\"breadcrumb\":{\"@id\":\"https://arize.com/docs/ax/integrations/evaluation-integrations/phoenix-evals#breadcrumb\"}},{\"@type\":\"BreadcrumbList\",\"@id\":\"https://arize.com/docs/ax/integrations/evaluation-integrations/phoenix-evals#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Evaluation\",\"item\":\"https://arize.com/docs/ax/integrations/llm-providers/amazon-bedrock/amazon-bedrock-evals\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Phoenix Evals\",\"item\":\"https://arize.com/docs/ax/integrations/evaluation-integrations/phoenix-evals\"}]},{\"@type\":[\"Article\",\"TechArticle\"],\"@id\":\"https://arize.com/docs/ax/integrations/evaluation-integrations/phoenix-evals#article\",\"headline\":\"Phoenix Evals\",\"name\":\"Phoenix Evals\",\"description\":\"Use the open-source phoenix-evals library to grade Arize AX traces and as evaluators in Arize AX experiments.\",\"url\":\"https://arize.com/docs/ax/integrations/evaluation-integrations/phoenix-evals\",\"mainEntityOfPage\":{\"@id\":\"https://arize.com/docs/ax/integrations/evaluation-integrations/phoenix-evals#webpage\"},\"dateModified\":\"2026-08-18T18:08:20.512Z\",\"keywords\":\"phoenix evals, arize-phoenix-evals, evals, evaluation, llm-as-judge, create_classifier, evaluate_dataframe, experiments, python\",\"publisher\":{\"@id\":\"https://arize.com/#organization\"},\"isPartOf\":{\"@id\":\"https://arize.com/docs#website\"}}]}\u003c/script>","agentPointerHref":"/docs/ax/integrations/evaluation-integrations/phoenix-evals.md","siteName":"Arize AX Docs","nav":{"global":null,"tabs":[{"tab":"Get Started","pages":[{"title":"Get Started","href":"/ax"}]},{"tab":"Guides","pages":[{"title":"Guides","href":"/ax/cookbooks"}]},{"tab":"Integrations","pages":[{"title":"Integrations","sidebarTitle":"Overview","href":"/ax/integrations"},{"group":"OpenTelemetry","pages":[{"title":"Overview","href":"/ax/integrations/opentelemetry/overview"},{"title":"OpenTelemetry (arize-otel)","href":"/ax/integrations/opentelemetry/opentelemetry-arize-otel"},{"title":"OpenLLMetry \u0026 Traceloop","href":"/ax/integrations/opentelemetry/openllmetry-traceloop"},{"title":"OpenLIT","href":"/ax/integrations/opentelemetry/openlit"}]},{"group":"LLM Providers","pages":[{"title":"Amazon Bedrock","href":"/ax/integrations/llm-providers/amazon-bedrock/amazon-bedrock-tracing"},{"title":"Anthropic","href":"/ax/integrations/llm-providers/anthropic/anthropic-tracing"},{"title":"Baseten","href":"/ax/integrations/llm-providers/baseten/baseten-tracing"},{"title":"Cerebras","href":"/ax/integrations/llm-providers/cerebras/cerebras-tracing"},{"title":"Cloudflare AI Gateway","href":"/ax/integrations/llm-providers/cloudflare-ai-gateway/cloudflare-ai-gateway-tracing"},{"title":"Cohere","href":"/ax/integrations/llm-providers/cohere/cohere-tracing"},{"title":"DeepSeek","href":"/ax/integrations/llm-providers/deepseek/deepseek-tracing"},{"title":"Doubleword","href":"/ax/integrations/llm-providers/doubleword/doubleword-tracing"},{"title":"Fireworks AI","href":"/ax/integrations/llm-providers/fireworks-ai/fireworks-ai-tracing"},{"title":"Google GenAI","href":"/ax/integrations/llm-providers/google-gen-ai/google-genai-tracing"},{"title":"Gravitee","href":"/ax/integrations/llm-providers/gravitee/gravitee-tracing"},{"title":"Groq","href":"/ax/integrations/llm-providers/groq/groq-tracing"},{"title":"Hugging Face Inference Providers","href":"/ax/integrations/llm-providers/hugging-face/hugging-face-inference-providers-tracing"},{"title":"LiteLLM","href":"/ax/integrations/llm-providers/litellm/litellm-tracing"},{"title":"Mistral AI","href":"/ax/integrations/llm-providers/mistralai/mistralai-tracing"},{"title":"Novita AI","href":"/ax/integrations/llm-providers/novita-ai/novita-ai-tracing"},{"title":"Ollama","href":"/ax/integrations/llm-providers/llama/ollama-tracing"},{"title":"OpenAI","href":"/ax/integrations/llm-providers/openai/openai-tracing"},{"title":"OpenRouter","href":"/ax/integrations/llm-providers/openrouter/openrouter-tracing"},{"title":"OrcaRouter","href":"/ax/integrations/llm-providers/orcarouter/orcarouter-tracing"},{"title":"PrismML","href":"/ax/integrations/llm-providers/prismml/prismml-tracing"},{"title":"Qwen","href":"/ax/integrations/llm-providers/qwen/qwen-tracing"},{"title":"TrueFoundry","href":"/ax/integrations/llm-providers/truefoundry/truefoundry-tracing"},{"title":"Together AI","href":"/ax/integrations/llm-providers/together-ai/together-ai-tracing"},{"title":"Vertex AI","href":"/ax/integrations/llm-providers/vertexai/vertexai-tracing"},{"title":"vLLM","href":"/ax/integrations/llm-providers/vllm/vllm-tracing"},{"title":"xAI Grok","href":"/ax/integrations/llm-providers/xai/xai-grok-tracing"},{"title":"Z.ai","href":"/ax/integrations/llm-providers/z-ai/z-ai-tracing"}]},{"group":"Agent Frameworks","pages":[{"group":"Python","pages":[{"title":"Agno","href":"/ax/integrations/python-agent-frameworks/agno/agno-tracing"},{"title":"Atomic Agents","href":"/ax/integrations/python-agent-frameworks/atomic-agents/atomic-agents-tracing"},{"title":"Amazon Bedrock Agents","href":"/ax/integrations/llm-providers/amazon-bedrock/amazon-bedrock-agents-tracing"},{"title":"AutoGen (legacy API)","href":"/ax/integrations/python-agent-frameworks/autogen/autogen-tracing"},{"title":"AutoGen AgentChat","href":"/ax/integrations/python-agent-frameworks/autogen/autogen-agentchat-tracing"},{"title":"AG2","href":"/ax/integrations/python-agent-frameworks/ag2/ag2-tracing"},{"title":"Bedrock AgentCore","href":"/ax/integrations/python-agent-frameworks/aws-strands/bedrock-agentcore"},{"title":"BeeAI","href":"/ax/integrations/python-agent-frameworks/beeai/beeai-tracing-python"},{"title":"Claude Agent SDK","href":"/ax/integrations/python-agent-frameworks/claude-agent-sdk/claude-agent-sdk-tracing"},{"title":"CrewAI","href":"/ax/integrations/python-agent-frameworks/crewai/crewai-tracing"},{"title":"DSPy","href":"/ax/integrations/python-agent-frameworks/dspy/dspy-tracing"},{"title":"Google ADK","href":"/ax/integrations/python-agent-frameworks/google-adk/google-adk-tracing"},{"title":"Graphite","href":"/ax/integrations/python-agent-frameworks/graphite/graphite-tracing"},{"title":"Guardrails AI","href":"/ax/integrations/python-agent-frameworks/guardrails-ai/guardrails-ai-tracing"},{"title":"Haystack","href":"/ax/integrations/python-agent-frameworks/haystack/haystack-tracing"},{"title":"Instructor","href":"/ax/integrations/python-agent-frameworks/instructor/instructor-tracing"},{"title":"LangChain","href":"/ax/integrations/python-agent-frameworks/langchain/langchain-tracing"},{"title":"LangGraph","href":"/ax/integrations/python-agent-frameworks/langgraph/langgraph-tracing"},{"title":"LiveKit Agents","href":"/ax/integrations/python-agent-frameworks/livekit/livekit-agents-tracing"},{"title":"LlamaIndex","href":"/ax/integrations/python-agent-frameworks/llamaindex/llamaindex-tracing"},{"title":"LlamaIndex Workflows","href":"/ax/integrations/python-agent-frameworks/llamaindex/llamaindex-workflows-tracing"},{"title":"MCP","href":"/ax/integrations/python-agent-frameworks/model-context-protocol/mcp-tracing"},{"title":"Microsoft Agent Framework","href":"/ax/integrations/python-agent-frameworks/microsoft/microsoft-agent-framework"},{"title":"NVIDIA NeMo Agent Toolkit","href":"/ax/integrations/python-agent-frameworks/nvidia/nemo-agent-toolkit-tracing"},{"title":"Open Agent Spec","href":"/ax/integrations/python-agent-frameworks/agentspec/agentspec-tracing"},{"title":"OpenAI Agents SDK","href":"/ax/integrations/python-agent-frameworks/openai-agents/openai-agents-sdk-tracing"},{"title":"Pipecat","href":"/ax/integrations/python-agent-frameworks/pipecat/pipecat-tracing"},{"title":"Portkey","href":"/ax/integrations/python-agent-frameworks/portkey/portkey-tracing"},{"title":"Pydantic AI","href":"/ax/integrations/python-agent-frameworks/pydantic/pydantic-ai-tracing"},{"title":"Restate","href":"/ax/integrations/python-agent-frameworks/restate/restate-tracing"},{"title":"Semantic Kernel","href":"/ax/integrations/python-agent-frameworks/semantic-kernel/semantic-kernel-tracing"},{"title":"Temporal","href":"/ax/integrations/python-agent-frameworks/temporal/temporal-tracing"},{"title":"TypeSafe AI","href":"/ax/integrations/python-agent-frameworks/typesafe/typesafe-tracing"},{"title":"Smolagents","href":"/ax/integrations/python-agent-frameworks/hugging-face-smolagents/smolagents-tracing"},{"title":"Strands Agents SDK","href":"/ax/integrations/python-agent-frameworks/aws-strands/aws-strands-tracing"}]},{"group":"TypeScript / JavaScript","pages":[{"title":"Amazon Bedrock Agents JS","href":"/ax/integrations/ts-js-agent-frameworks/amazon-bedrock/amazon-bedrock-agents-js"},{"title":"BeeAI","href":"/ax/integrations/ts-js-agent-frameworks/beeai/beeai-tracing-js"},{"title":"Claude Agent SDK JS","href":"/ax/integrations/ts-js-agent-frameworks/claude-agent-sdk/claude-agent-sdk-js"},{"title":"LangChain.js","href":"/ax/integrations/ts-js-agent-frameworks/langchain/langchain-js"},{"title":"Mastra","href":"/ax/integrations/ts-js-agent-frameworks/mastra/mastra-tracing"},{"title":"MCP JS","href":"/ax/integrations/ts-js-agent-frameworks/model-context-protocol/mcp-tracing-js"},{"title":"OpenAI Agents JS","href":"/ax/integrations/ts-js-agent-frameworks/openai-agents/openai-agents-js"},{"title":"TanStack AI","href":"/ax/integrations/ts-js-agent-frameworks/tanstack/tanstack-ai-tracing"},{"title":"TypeSafe AI","href":"/ax/integrations/ts-js-agent-frameworks/typesafe/typesafe-tracing-js"},{"title":"Vercel AI SDK (v6 and earlier)","href":"/ax/integrations/ts-js-agent-frameworks/vercel/vercel-ai-sdk-v6-tracing"},{"title":"Vercel AI SDK v7","href":"/ax/integrations/ts-js-agent-frameworks/vercel/vercel-ai-sdk-v7-tracing"},{"title":"Vercel Eve","href":"/ax/integrations/ts-js-agent-frameworks/vercel/eve-tracing"}]},{"group":"Java","pages":[{"title":"Annotations","href":"/ax/integrations/java/annotation/annotation-tracing"},{"title":"Arconia","href":"/ax/integrations/java/arconia/arconia-tracing"},{"title":"Google ADK for Java","href":"/ax/integrations/java/google-adk/google-adk-tracing"},{"title":"LangChain4j","href":"/ax/integrations/java/langchain4j/langchain4j-tracing"},{"title":"Spring AI","href":"/ax/integrations/java/spring-ai/spring-ai-tracing"}]},{"group":".NET / C#","pages":[{"title":"Microsoft Agent Framework for C#","href":"/ax/integrations/dotnet-agent-frameworks/microsoft/microsoft-agent-framework"}]}]},{"group":"Coding Agents","pages":[{"title":"Antigravity","href":"/ax/integrations/platforms/antigravity/antigravity-tracing"},{"title":"Claude Code","href":"/ax/integrations/platforms/claude-code/claude-code-tracing"},{"title":"Codex","href":"/ax/integrations/platforms/codex/codex-tracing"},{"title":"GitHub Copilot","href":"/ax/integrations/platforms/copilot/copilot-tracing"},{"title":"Cursor","href":"/ax/integrations/platforms/cursor/cursor-tracing"},{"title":"Devin","href":"/ax/integrations/platforms/devin/devin-tracing"},{"title":"Gemini CLI","href":"/ax/integrations/platforms/gemini/gemini-tracing"},{"title":"Kiro","href":"/ax/integrations/platforms/kiro/kiro-tracing"},{"title":"Oh My Pi","href":"/ax/integrations/platforms/oh-my-pi/oh-my-pi-tracing"},{"title":"OpenCode","href":"/ax/integrations/platforms/opencode/opencode-tracing"}]},{"group":"Platforms","pages":[{"title":"Claude Managed Agents","href":"/ax/integrations/platforms/claude-managed-agents/claude-managed-agents-tracing"},{"title":"Dify","href":"/ax/integrations/platforms/dify/dify-tracing"},{"title":"Flowise","href":"/ax/integrations/platforms/flowise/flowise-tracing"},{"title":"LangFlow","href":"/ax/integrations/platforms/langflow/langflow-tracing"},{"title":"n8n","href":"/ax/integrations/platforms/n8n/n8n-tracing"},{"title":"OpenRouter","href":"/ax/integrations/platforms/openrouter/openrouter-tracing"},{"title":"Open WebUI","href":"/ax/integrations/platforms/openwebui/openwebui-tracing"},{"title":"Prompt flow","href":"/ax/integrations/platforms/prompt-flow/prompt-flow-tracing"},{"title":"Scalekit AgentKit Tracing","href":"/ax/integrations/platforms/scalekit/scalekit-tracing"},{"title":"TrueFoundry","href":"/ax/integrations/platforms/truefoundry/truefoundry-tracing"}]},{"group":"Orchestration","pages":[{"group":"Apache Airflow","pages":[{"title":"Apache Airflow Provider","href":"/ax/integrations/orchestration/airflow/airflow-provider"},{"title":"Operators, Sensors \u0026 Hooks","href":"/ax/integrations/orchestration/airflow/airflow-operators"}]}]},{"group":"Vector Databases","pages":[{"title":"Qdrant","href":"/ax/integrations/vector-databases/qdrant/qdrant-tracing"}]},{"group":"Evaluation","pages":[{"title":"Amazon Bedrock","href":"/ax/integrations/llm-providers/amazon-bedrock/amazon-bedrock-evals"},{"title":"Anthropic","href":"/ax/integrations/llm-providers/anthropic/anthropic-evals"},{"title":"Cleanlab","href":"/ax/integrations/evaluation-integrations/cleanlab"},{"title":"Gemini","href":"/ax/integrations/llm-providers/google-gen-ai/gemini-evals"},{"title":"LiteLLM","href":"/ax/integrations/llm-providers/litellm/litellm-evals"},{"title":"Microsoft Azure AI Evaluation","href":"/ax/integrations/evaluation-integrations/microsoft"},{"title":"Mistral AI","href":"/ax/integrations/llm-providers/mistralai/mistralai-evals"},{"title":"MLflow","href":"/ax/integrations/evaluation-integrations/mlflow"},{"title":"OpenAI","description":"Use OpenAI as the judge LLM for evaluations with the arize-phoenix-evals library.","href":"/ax/integrations/llm-providers/openai/openai-evals"},{"title":"Phoenix Evals","href":"/ax/integrations/evaluation-integrations/phoenix-evals"},{"title":"Pydantic Evals","description":"Use Pydantic Evals LLMJudge evaluators to grade Arize AX traces and as evaluators in Arize AX experiments.","href":"/ax/integrations/evaluation-integrations/pydantic-evals"},{"title":"Ragas","href":"/ax/integrations/evaluation-integrations/ragas"},{"title":"NVIDIA RAG Metrics via Ragas","href":"/ax/integrations/evaluation-integrations/ragas-nvidia-rag-metrics"},{"title":"UQLM","href":"/ax/integrations/evaluation-integrations/uqlm"},{"title":"Vertex AI","href":"/ax/integrations/llm-providers/vertexai/vertexai-evals"}]}]},{"tab":"APIs, SDKs, Skills \u0026 MCP","pages":[{"title":"APIs, SDKs, Skills \u0026 MCP","href":"/api-reference/overview"}]},{"tab":"Courses","pages":[{"title":"Courses","href":"/ax/learn/courses/overview"}]},{"tab":"Concepts","pages":[{"title":"Concepts","href":"/ax/concepts/otel-openinference/overview"}]},{"tab":"Self-Hosting","pages":[{"title":"Self-Hosting","href":"/ax/selfhosting"}]},{"tab":"Changelog","pages":[{"title":"Changelog","href":"/ax/release-notes"}]}]},"basePath":"","title":"Phoenix Evals - Arize AX Docs","buildId":"dpl_Aw98ji4XvBsgtZwL1VrkUF9ehFqz","deploymentId":"dpl_Aw98ji4XvBsgtZwL1VrkUF9ehFqz","dynamics":[],"chunks":[]}