> ## Documentation Index
> Fetch the complete documentation index at: https://arize-ax.mintlify.site/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Instrument Groq agents
> Trace a Groq agent in Arize AX: auto-instrument Llama chat completions and add manual agent and tool spans for a complete agent trace tree.
[Groq](https://console.groq.com/docs) provides fast LLM inference for openly-available models such as Llama. Learn how to instrument a Groq agent to capture every agent run, model call, and tool call so you can observe, evaluate, and improve quality. This guide attaches the OpenInference Groq instrumentor to a Groq app and configures the `arize-otel` exporter to send its traces to Arize AX.
## Overview
In this guide, you will learn how to attach the OpenInference Groq instrumentor and the `arize-otel` exporter to a Groq agent, run the agent, and read the resulting trace tree in Arize AX. The agent calls Groq's chat completions endpoint, its general-purpose interface for text generation. This setup applies to any Groq agent, whether you run it as a script, inside a web service such as FastAPI, or as a background worker.
This guide assumes you have familiarity with:
* Python
* The Groq chat completions API and the tool-use loop (`chat.completions.create`, `tool_calls`, `role: "tool"` messages)
* Environment variables and package installation
By the end of this guide, you will be able to:
* Auto-instrument Groq SDK calls so each model request is captured as an LLM span
* Add manual AGENT and TOOL spans so the loop and tool executions appear in the trace
* Group traces by conversation using session IDs
* Verify and read the full agent trace tree in Arize AX
## Before you start
You need:
* An existing agent built with Groq
* An Arize AX account ([sign up](https://arize.com/sign-up/))
* A `GROQ_API_KEY` from the [Groq Console](https://console.groq.com/keys)
* Python 3.10 or later
### Get your Arize AX credentials
Sign in to your [Arize AX account](https://app.arize.com/) and create a tracing project from **Projects → New Tracing Project**. The setup page shows the credentials you need: copy your **Space ID**, then click **Create API Key** to generate a key and save it somewhere safe. You set them as `ARIZE_SPACE_ID` and `ARIZE_API_KEY` when you configure your environment below, and they route your traces to the correct space.
The steps below instrument a small example agent, a nutrition agent with a single tool, so the trace shows the AGENT → LLM → TOOL shape end to end. The agent is a fixture; the instrumentation is the point. Attach the Groq instrumentor once at startup, then wrap your agent loop and each tool call in manual spans.
Install the Groq instrumentor, the OpenInference helpers for manual spans, and the `groq` SDK.
```bash theme={null}
pip install arize-otel \
openinference-instrumentation-groq \
openinference-instrumentation \
groq
```
Set your Arize AX and Groq credentials as environment variables. The tracing setup reads them at startup.
```bash theme={null}
export ARIZE_SPACE_ID=""
export ARIZE_API_KEY=""
export ARIZE_PROJECT_NAME="groq-agent"
export GROQ_API_KEY=""
```
`ARIZE_PROJECT_NAME` is the project your traces land in. Name it for the app so runs stay grouped where you expect them.
Set up tracing in a dedicated module so it runs once, before your agent uses the SDK. This registers a tracer provider with your Arize credentials and attaches the Groq instrumentor, which captures every `chat.completions.create` call as an LLM span.
```python instrumentation.py theme={null}
import os
from arize.otel import register
from openinference.instrumentation.groq import GroqInstrumentor
# register() reads ARIZE_SPACE_ID and ARIZE_API_KEY from the environment
# when the arguments are omitted. Pass them explicitly if you prefer.
tracer_provider = register(
space_id=os.environ["ARIZE_SPACE_ID"],
api_key=os.environ["ARIZE_API_KEY"],
project_name=os.environ.get("ARIZE_PROJECT_NAME", "groq-agent"),
)
# Auto-instrument the Groq SDK: each chat.completions.create call becomes an LLM span.
GroqInstrumentor().instrument(tracer_provider=tracer_provider)
```
In the example below, the tool definition, `run_tool`, and the `while` loop are ordinary Groq agent code. The instrumentation is the tracer and the two `start_as_current_span` blocks that wrap the loop in an AGENT span and each tool execution in a TOOL span. The `chat.completions.create` calls inside the AGENT span are auto-instrumented, so they nest underneath it as LLM spans. The OpenInference attributes are set as string keys (`"openinference.span.kind"`, `"input.value"`).
```python theme={null}
import json
from groq import Groq
from opentelemetry import trace
from opentelemetry.trace import Status, StatusCode
import instrumentation # noqa: F401 - runs the tracing setup first
client = Groq()
tracer = trace.get_tracer(__name__)
MODEL = "llama-3.3-70b-versatile"
TOOLS = [
{
"type": "function",
"function": {
"name": "get_daily_calories",
"description": "Recommended daily calorie intake for an age and activity level.",
"parameters": {
"type": "object",
"properties": {
"age": {"type": "integer"},
"activity_level": {
"type": "string",
"enum": ["low", "moderate", "high"],
},
},
"required": ["age", "activity_level"],
},
},
}
]
def run_tool(name: str, args: dict) -> str:
# Replace with your real tool implementation.
if name == "get_daily_calories":
base = {"low": 1800, "moderate": 2200, "high": 2600}[args["activity_level"]]
return json.dumps({"recommended_calories": base})
return json.dumps({"error": f"unknown tool {name}"})
def run_agent(user_input: str) -> str:
with tracer.start_as_current_span("nutrition-agent") as agent_span:
agent_span.set_attribute("openinference.span.kind", "AGENT")
agent_span.set_attribute("input.value", user_input)
messages = [{"role": "user", "content": user_input}]
while True:
# Auto-instrumented: each call appears as a child LLM span.
response = client.chat.completions.create(
model=MODEL,
messages=messages,
tools=TOOLS,
)
message = response.choices[0].message
messages.append(message)
if not message.tool_calls:
break
for tool_call in message.tool_calls:
name = tool_call.function.name
args = json.loads(tool_call.function.arguments)
# Manual TOOL span for each tool execution.
with tracer.start_as_current_span(name) as tool_span:
tool_span.set_attribute("openinference.span.kind", "TOOL")
tool_span.set_attribute("tool.name", name)
tool_span.set_attribute("input.value", json.dumps(args))
result = run_tool(name, args)
tool_span.set_attribute("output.value", result)
messages.append(
{
"role": "tool",
"tool_call_id": tool_call.id,
"content": result,
}
)
final_text = message.content or ""
agent_span.set_attribute("output.value", final_text)
agent_span.set_status(Status(StatusCode.OK))
return final_text
def handle_turn(session_id: str, user_input: str) -> str:
# One entry point per turn. session_id is used once you enable sessions below.
return run_agent(user_input)
print(handle_turn("user-123", "I'm 34 and moderately active. What's my daily calorie target?"))
# Short-lived script: flush batched spans before the process exits.
instrumentation.tracer_provider.force_flush()
```
To follow a multi-turn conversation as a single thread, tag its runs with a shared session ID that is stable across the conversation, such as the user or thread ID. Wrapping the body of `handle_turn` in the `using_session()` context manager is the only change; every span emitted inside, manual and auto, then carries the same `session.id`.
```python theme={null}
from openinference.instrumentation import using_session
def handle_turn(session_id: str, user_input: str) -> str:
with using_session(session_id): # the only added line
return run_agent(user_input)
```
Arize AX groups the tagged runs together, so you can replay a full conversation.
Open your Arize AX space and select the project you set in `ARIZE_PROJECT_NAME`. Within about 30 seconds of a run, a new trace appears with this shape:
* An **AGENT** root span (`nutrition-agent`) carrying the user input and final answer.
* One or more **LLM** child spans, one per `chat.completions.create` call, with the prompt, response, model name, and token counts filled in automatically by the Groq instrumentor.
* A **TOOL** child span for each tool the agent ran, with the tool inputs and outputs.
If no traces appear, see [Troubleshooting](#troubleshooting).
## What gets captured
The two layers combine into one trace:
* **LLM spans (automatic).** The Groq instrumentor records each `chat.completions.create` call: prompt, response, the `tool_calls` the model emits, the model name, and token usage.
* **AGENT and TOOL spans (manual).** Your spans record the loop as a whole and each tool execution, so the trace shows what the agent decided to do and what each tool returned, not just the raw model calls.
* **Session grouping.** The `session.id` you set threads related runs into one conversation.
Because the manual and auto spans share the same tracer provider, the LLM spans nest under your AGENT span automatically as long as the `chat.completions.create` calls run inside it.
Groq's [compound models](https://console.groq.com/docs/compound) (`groq/compound` and `groq/compound-mini`) run built-in tools such as web search and code execution on Groq's own infrastructure. Those executions happen server-side, so the instrumentor cannot break them out into separate TOOL spans. They are captured as part of the LLM span for the call. Client-side tools that you execute in your own loop, as shown above, are what produce distinct TOOL spans.
## Troubleshooting
* **No traces in Arize AX.** Confirm `ARIZE_SPACE_ID` and `ARIZE_API_KEY` are set in the same shell that runs your app. Enable OpenTelemetry debug logs with `export OTEL_LOG_LEVEL=debug` and rerun.
* **LLM spans missing.** `GroqInstrumentor().instrument(...)` must run before the first `chat.completions.create` call. Import the tracing module first.
* **LLM spans not nested under the agent span.** The `chat.completions.create` call must run inside the active AGENT span. Keep it inside the `with tracer.start_as_current_span(...)` block.
* **`401` or `404` from Groq.** Verify `GROQ_API_KEY` is set and valid, and that the model in the example is available to your account. Swap `llama-3.3-70b-versatile` for another [supported model](https://console.groq.com/docs/models) if needed.
* **A short-lived script exports nothing.** Spans are batched by default, so a script that exits right after the run can quit before they flush. Call `tracer_provider.force_flush()` before the process exits. Long-running apps and servers flush on their own.
* **`wrap_function_wrapper() got an unexpected keyword argument 'module'`.** A `wrapt` 2.x release changed that function's signature, which older instrumentor pins do not yet account for. Pin `wrapt` below 2 until the pins catch up: `pip install "wrapt<2"`.
## Next steps
The general pattern for mixing auto LLM spans with manual spans.
Render the agent's execution as an interactive graph.
Group multi-turn runs into conversations with session and user IDs.
The same auto-plus-manual pattern for an agent built on the Anthropic SDK.