07.29.2025: Google GenAI Evals 🌐

We’ve added support for the GoogleGenAIModel in phoenix-evals, enabling direct access to Google's Gemini models through the official Google GenAI SDK. As of late 2024, this is the recommended approach for working with Gemini, offering a unified interface across both the Developer API and VertexAI.

🚀 Key Features

• Multimodal Support Run evaluations on text, image, and audio inputs using Gemini’s multimodal capabilities.

• Async-Ready Optimized for high-throughput evals with full async compatibility.

• Flexible Authentication Supports both API key and VertexAI-based authentication methods.

• Dynamic Rate Limiting Built-in rate limiter with automatic adjustment based on API feedback and usage patterns.

This integration makes it easier to run robust, scalable evaluations using Gemini models directly within your phoenix-evals workflows.

Huge shoutout to Siddharth Sahu for this contribution!

More Information in our docs:

Google Gen AI Evals | Integrations | Arize Phoenixarize.com