Connect any OpenAI-compatible custom model endpoint to Arize AX
Add a Custom Model Endpoint to access your model in Arize AX’s prompt playground. Arize AX uses the OpenAI client to make calls to these OpenAI-compatible endpoints.
By adding this integration, your data may be sent to your custom LLM deployment for certain actions within Arize AX (e.g., prompt playground) and your account may be billed for usage.
Give your integration a name, then set the API Base URL for your deployment.
Enter the API Base URL including the version path (for example, https://my-hosted-model/openai/v1). Do not include endpoint paths such as /chat/completions; Arize AX appends them automatically.
Choose one of the following authentication methods.
API Key
OAuth 2.0
Enter the API key for your deployment.
The Custom Model Endpoint API-key configuration pane
Select OAuth 2.0 to authenticate with the OAuth 2.0 client-credentials grant. Arize AX retrieves the short-lived access token from your authorization server and refreshes it as needed.Provide the following required values:
Token URL — the OAuth server endpoint that issues access tokens.
Client ID — the client identifier issued by your authorization server.
Client Secret — the client secret issued for that client.
You can also provide optional, space-separated Scopes when your authorization server requires them. Under Optional OAuth settings, provide an Audience only when your authorization server requires an audience value for the access token; otherwise, leave it blank.
The Custom Model Endpoint OAuth 2.0 configuration pane
After configuring either authentication method, set any extra headers that should be sent with every request.Then define the available model deployments. If all default OpenAI models are available, enable OpenAI default models; otherwise, add each model as a custom model. Function calling and authorization settings are also available in Advanced Settings.
