How sync works
- You need write access to the target bucket and at least one tracing project in your space.
- A connector points at one file path in your bucket and syncs any number of projects.
- Sync runs every 60 minutes. The interval cannot be changed in the UI. The first sync includes all historical data.
- Project syncs can be paused, resumed, or deleted. Deleting a synced project stops its sync and leaves written data in your storage.
- Tables land at
<path>/<namespace>/<project_name>/. The current Iceberg metadata file is<namespace>/<project_name>/metadata/latest.metadata.json.
Create a connector
- Go to Settings → Space Settings → Data Fabric in your space and click New Connector.
- Enter a Connector Name, then Select Project(s) to sync.
- Choose the Table Format: Apache Iceberg or Delta Lake.
- Choose the Select Data Storage provider, then enter the File Path to Write Data To.
- Google Cloud Storage
- Amazon S3
- Azure Blob Storage
Enter the bucket and prefix, for example
my-data-bucket/arize-sync/production.Set up permissions
Each bucket needs a tag, label, or metadata value set to the ingestion key shown in the setup dialog. Arize uses it to confirm that you own the bucket. The key name isarize-ingestion-key on Google Cloud Storage and Amazon S3, and arize_ingestion_key on Azure.
- Google Cloud Storage
- Amazon S3
- Azure Blob Storage
- Add a bucket label with key
arize-ingestion-keyand the value from the setup dialog. - Create the custom IAM role from the setup dialog:
- Grant the role to the Arize service account on your bucket:
Set up query tables
After the first sync finishes, create one external table per synced project.- BigQuery
- Databricks
- Snowflake