GitHub Copilot

Trace GitHub Copilot sessions in VS Code and the Copilot CLI in Arize AX for full observability.

GitHub Copilot is GitHub’s AI pair programmer for VS Code and the command line. The Arize Coding Harness Tracing instruments both Copilot surfaces and exports OpenInference spans to Arize AX. VS Code uses per-event hook files; the CLI uses a single hooks.json.

Launch Arize AX

To get started, sign up for a free Arize AX account and get your Space ID and API Key:

Log in at app.arize.com
Click Settings and copy the Space ID
Open the API Keys tab and create or copy an API key

Install

Copilot hooks are installed at the project level under .github/hooks/. Run the installer from the root of each repository where you want tracing.

Curl installer

macOS / Linux:

cd /path/to/your/project
curl -sSL https://raw.githubusercontent.com/Arize-ai/coding-harness-tracing/main/install.sh | bash -s -- copilot

Windows (PowerShell):

cd C:\path\to\your\project
iwr -useb https://raw.githubusercontent.com/Arize-ai/coding-harness-tracing/main/install.bat -OutFile $env:TEMP\install.bat
& $env:TEMP\install.bat copilot

Local clone

git clone https://github.com/Arize-ai/coding-harness-tracing.git
cd /path/to/your/project
/path/to/coding-harness-tracing/install.sh copilot         # macOS / Linux
\path\to\coding-harness-tracing\install.bat copilot        # Windows

The installer prompts for your Arize AX credentials and project name, writes them to ~/.arize/harness/config.json, and registers VS Code (.github/hooks/*.json) and Copilot CLI (.github/hooks/hooks.json) hooks in the current project.

Configuration

Credentials live in ~/.arize/harness/config.json and apply across all projects. Per-project environment variables override config.json and can be set in your shell profile.

export ARIZE_API_KEY="<your-api-key>"
export ARIZE_SPACE_ID="<your-space-id>"
export ARIZE_PROJECT_NAME="copilot"
export ARIZE_TRACE_ENABLED="true"

Redaction controls

Each ARIZE_LOG_* flag accepts "true" or "false" and defaults to "true". Set to "false" to opt out per category:

export ARIZE_LOG_PROMPTS="false"
export ARIZE_LOG_TOOL_DETAILS="false"
export ARIZE_LOG_TOOL_CONTENT="false"

Flag	Redacts
`ARIZE_LOG_PROMPTS`	User prompt and assistant response text
`ARIZE_LOG_TOOL_DETAILS`	Tool names and arguments
`ARIZE_LOG_TOOL_CONTENT`	Tool call output content

Observe

Once tracing is enabled, Copilot activity from both VS Code and the CLI streams to Arize AX. You’ll see:

Turn traces — each prompt → response cycle as a parent span
Tool spans — nested spans for each tool call with inputs, outputs, and duration
Subagent spans — activity from any subagents Copilot spawns (VS Code)
Error spans — errorOccurred events from the CLI
Session grouping — all turns from the same session grouped by session_id

Hooks Captured

Copilot in VS Code and on the CLI use different event names. The Agent Kit handles both with one install command.

Surface	Hook File	Events
VS Code	`.github/hooks/session-start.json`	`SessionStart`
VS Code	`.github/hooks/user-prompt.json`	`UserPromptSubmit`
VS Code	`.github/hooks/pre-tool.json`	`PreToolUse`
VS Code	`.github/hooks/post-tool.json`	`PostToolUse`
VS Code	`.github/hooks/stop.json`	`Stop`
VS Code	`.github/hooks/subagent-stop.json`	`SubagentStop`
CLI	`.github/hooks/hooks.json`	`sessionStart`, `userPromptSubmitted`, `preToolUse`, `postToolUse`, `errorOccurred`, `sessionEnd`

Reference

For the full list of environment variables, default file paths, and troubleshooting steps, see the Copilot tracing README.

Uninstall

Run from the project root where you installed the hooks:

curl -sSL https://raw.githubusercontent.com/Arize-ai/coding-harness-tracing/main/install.sh | bash -s -- uninstall copilot

OpenTelemetry

LLM Providers

Agent Frameworks

Coding Agents

Platforms

Orchestration

Evaluation

Launch Arize AX

Install

Curl installer

Local clone

Configuration

Redaction controls

Observe

Hooks Captured

Reference

Uninstall

Resources

Arize Coding Harness Tracing

OpenInference

GitHub Copilot

​Launch Arize AX

​Install

​Curl installer

​Local clone

​Configuration

​Redaction controls

​Observe

​Hooks Captured

​Reference

​Uninstall

​Resources

Arize Coding Harness Tracing

OpenInference

GitHub Copilot

Launch Arize AX

Install

Curl installer

Local clone

Configuration

Redaction controls

Observe

Hooks Captured

Reference

Uninstall

Resources