Build a Prompt

In Arize, a prompt is managed as a Prompt Object — messages, model settings, tools/functions, and response format together as one versioned artifact. For the conceptual breakdown of what’s in a Prompt Object and why everything travels together, see The Prompt Object.

Let Alyx build it for you. Press Cmd+L (macOS) or Ctrl+L (Windows/Linux) to open Alyx and try: “Create three prompt variations to compare”

Build and improve a prompt

Start in Playground with a draft prompt, then refine it for clarity, correctness, and brevity.

Create a prompt

Navigate to Playgrounds and click New Playground, or start from the home page and have Alyx create a playground — Open Playgrounds and create a new playground

By Alyx
By UI
By Code

Create a new playground

Navigate to Playgrounds and click New Playground to create a new playground. You can also start from the home page, and Alyx will create a playground for you.

Choose a provider or model

If you have not already added one, select your LLM provider and model in the Playground before you draft or run. Options include OpenAI, Azure, Vertex AI, AWS Bedrock, and others.

LLM provider and model dropdowns in the Prompt Playground — Choose provider and model in the Playground

Draft and refine your prompt with Alyx

Describe your use case in plain language in the Alyx panel on the right. Include details about what your prompt should capture, desired output format, tone, length, etc. Alyx will wire them in automatically.For example, you could say:

Create a prompt for a customer support agent that handles returns and escalations. Use {customer_input} and {order_id} as variables.

Give feedback on Alyx’s drafts and keep iterating until you are satisfied with the prompt.

Use Alyx in the Prompt Playground to draft prompts

Try the prompt in the Playground

If you added a dataset or test cases, run the prompt over rows to check outputs. If you skipped that step, exercise the prompt with manual inputs or keep iterating with Alyx. For a fuller testing workflow, see Test a prompt.

Create a new playground

Navigate to Playgrounds and click New Playground to create a new playground.

Choose provider and model

Select your LLM provider and model from OpenAI, Azure, Vertex AI, AWS Bedrock, and others.

Add prompt messages

Add any system, user, or assistant messages. Use curly braces around names (for example, {destination}) for template variables filled dynamically at run time.

Define system and user messages with template variables

Add function calling (optional)

Click Functions and paste tool definitions as JSON. Function descriptions must be in JSON format. To add multiple functions, add entries to the JSON list with the proper attributes. Set Function Selection to give the model instructions on when to invoke each function.

Adding function definitions and function selection in the Prompt Playground — Configure tools and function calling in the Playground

Set invocation parameters

Click Params to configure temperature, max completion tokens, frequency penalty, and more. Different LLM providers expose different hyperparameters that affect the output.

Invocation parameters in the Playground

Create a prompt and initial version from code. Use the reference for your stack:

Python SDK
TypeScript SDK
CLI

Full options (providers, invocation params, tools): Create a Prompt in the Python v8 Prompts API.

from arize.prompts.types import InputVariableFormat, LlmProvider, LLMMessage, MessageRole

prompt = client.prompts.create(
    space="your-space-name-or-id",
    name="customer-support-agent",
    description="Handles returns and escalations",
    commit_message="Initial version",
    input_variable_format=InputVariableFormat.F_STRING,
    provider=LlmProvider.OPEN_AI,
    model="gpt-4o",
    messages=[
        LLMMessage(role=MessageRole.SYSTEM, content="You are a support agent for {company}."),
        LLMMessage(role=MessageRole.USER, content="{ticket_text}"),
    ],
)

print(prompt.id, prompt.name)

Full options: Create a Prompt in the TypeScript Prompts API.

import { createPrompt } from "@arizeai/ax-client";

const prompt = await createPrompt({
  space: "my-space",
  name: "customer-support",
  description: "Handles returns and escalations",
  version: {
    commitMessage: "Initial version",
    inputVariableFormat: "f_string",
    provider: "openAI",
    model: "gpt-4o",
    messages: [
      { role: "system", content: "You are a support agent for {company_name}." },
      { role: "user", content: "{ticket_text}" },
    ],
  },
});

Flags and message JSON shape: ax prompts create in the CLI reference.

ax prompts create \
  --name "customer-support-agent" \
  --space sp_abc123 \
  --provider openAI \
  --input-variable-format f_string \
  --messages '[{"role":"system","content":"You are a support agent for {company}."},{"role":"user","content":"{ticket_text}"}]' \
  --model gpt-4o \
  --description "Handles returns and escalations" \
  --commit-message "Initial version"

Test a prompt

Test your prompt against real examples before shipping.

Test prompts in Playground against datasets, traces, or multiple versions

By Alyx
By UI
By Code

Load or generate data

Open Alyx and load your data, or ask it to generate examples from your use case.

Load your prompt

Ask Alyx to load a prompt from Prompt Hub and align {variables} with your dataset columns.

Create evaluator and run

Ask Alyx to draft an evaluator, then run the experiment.

Review results

Open View Experiment to compare outputs, latency, tokens, and evaluator labels.

Select a dataset

Choose your dataset in the Playground and confirm variable mapping.

Load or write your prompt

Load from Prompt Hub or draft a new prompt with {variables} from your dataset.

Attach evaluators

Add evaluators to score each row.

Run and compare

Click Run, then open View Experiment to inspect regressions and wins.

Compare Experiments view with Diff Output Mode enabled, highlighting insertions and deletions against a baseline run — Diff Output Mode highlights text changes against a baseline

For full run-and-compare workflow details, go to Experiment in Playground.

Save and version prompts

When a variant wins, save it to Prompt Hub so your team can track and deploy it safely.

Prompt Hub in Arize AX showing prompts list and version management — Prompt Hub: prompts list, versions, and labels

By Alyx
By UI
By Code

In the Prompt Playground, you can ask Alyx to persist the prompt you have been editing, for example:

Save this prompt to Prompt Hub as support-agent-v2 with a description that we tightened escalation rules.

Alyx can help you refine messages, parameters, and tools first; when you are ready, ask it to save (or click Save Prompt yourself) so a new immutable version is written to Prompt Hub with the name, description, and tags you want.

Step	Python	TypeScript	CLI
List versions	List versions	List versions	`ax prompts list-versions`
Create a new version	Create a new version	Create a new version	`ax prompts create-version`

Browse and compare prompt version history

For deeper version-control and SDK workflows, see Save and version prompts.

Quickstart

Instrument

Observe

Evaluate

Improve

Agents

Machine Learning

Settings

Security

Build and improve a prompt

Create a prompt

Test a prompt

Save and version prompts

​Build and improve a prompt

​Create a prompt

​Test a prompt

​Save and version prompts

Build and improve a prompt

Create a prompt

Test a prompt

Save and version prompts