Documentation Index
Fetch the complete documentation index at: https://arizeai-433a7140.mintlify.app/llms.txt
Use this file to discover all available pages before exploring further.
Design Considerations and Limitations
While the API is minimal—centered onAgent, Task, and Tool—there are important tradeoffs and design constraints to be aware of.
| Design Considerations | Limitations |
|---|---|
API centered on Agent, Task, and Tool | Tools are just Python functions decorated with @tool. There’s no centralized registry or schema enforcement, so developers must define conventions and structure on their own. |
| Provides flexibility for orchestration | No retry mechanism or built-in workflow engine |
| Supports evaluator-optimizer loops, routing, and fan-out/fan-in | |
| Agents are composed, not built-in abstractions | Must implement orchestration logic |
| Multi-Agent support | No built-in support for collaboration structures like voting, planning, or debate. |
| Token-level streaming is not supported | |
| No state or memory management out of the box. Applications that require persistent state—such as conversations or multi-turn workflows—will need to integrate external storage (e.g., a vector database or key-value store). | |
| There’s no native memory or “trajectory” tracking between agents. Handoffs between tasks are manual. This is workable in small systems, but may require structure in more complex workflows. |
Prompt Chaining
This workflow breaks a task into smaller steps, where the output of one agent becomes the input to another. It’s useful when a single prompt can’t reliably handle the full complexity or when you want clarity in intermediate reasoning. Notebook: Prompt Chaining with Keyword Extraction + Summarization The agent first extracts keywords from a resume, then summarizes what those keywords suggest. How to evaluate: Check whether each step performs its function correctly and whether the final result meaningfully depends on the intermediate output (e.g., do summaries reflect the extracted keywords?)- Check if the intermediate step (e.g. keyword extraction) is meaningful and accurate
- Ensure the final output reflects or builds on the intermediate output
- Compare chained vs. single-step prompting to see if chaining improves quality or structure
Orchestrator + Worker Pattern
In this approach, a central agent coordinates multiple agents, each with a specialized role. It’s helpful when tasks can be broken down and assigned to domain-specific workers. Notebook: Recruiting Evaluator Orchestrator The orchestrator delegates resume review, culture fit assessment, and decision-making to different agents, then composes a final recommendation. How to evaluate: Assess consistency between subtasks and whether the final output reflects the combined evaluations (e.g., does the final recommendation align with the inputs from each worker agent?)- Ensure each worker agent completes its role accurately and in isolation
- Check if the orchestrator integrates worker outputs into a consistent final result
- Look for agreement or contradictions between components (e.g., technical fit vs. recommendation)