Skip to main content
Red teaming sends adversarial prompts, such as jailbreak attempts and prompt injections, to your AI agent so you find the responses that go along with them before a user does. In Arize AX you run it with the same datasets, experiments, evaluators, and labeling queues you use for any other test, so the prompt set stays versioned and you can rerun it after every change to the agent.
  1. Create a dataset of adversarial prompts.
  2. Run an experiment that sends each prompt to your AI agent.
  3. Score each response with an evaluator.
  4. Send failing responses to a labeling queue for review.
For worked examples, see the Jailbreak and prompt injection defense and Foundry red team cookbooks.