Set up a Data Server

With your Icebreaker Agent connected, Icebreaker is ready to manage resources in your cluster. Three declarations make queries possible:

1. Data Catalog — your existing Iceberg catalog connection. Icebreaker reads metadata from it; your catalog stays the source of truth for what data exists.
2. Capacity Policy — the EC2 instance policy that governs how Karpenter provisions capacity for query execution.
3. Data Server — the Arrow Flight SQL endpoint your SQL clients will connect to.

This declarative model is what keeps Icebreaker serverless from your side: you set the boundaries, Icebreaker picks the resources per query. No warehouses to size, no clusters to pre-allocate.

Icebreaker is validated end-to-end with Lakekeeper. AWS Glue, Apache Polaris, Snowflake Open Catalog, and other Iceberg-REST-compatible catalogs are expected to work.

1. Navigate to Data Catalogs under your Account.
2. Click Add Data Catalog.
3. Provide the catalog endpoint, credentials, and warehouse identifier in JSON format. See Catalog integration for the exact values to enter for your catalog type (Lakekeeper, Glue, or Polaris).
4. Click Create.

A Capacity Policy defines which EC2 instance types and capacity types (on-demand, Spot, Reserved Instances) Karpenter can provision when query executor pods need capacity, and optionally caps the total CPU, memory, or number of concurrent instances.

When you create a Data Server and associate it with a Capacity Policy, the Icebreaker Agent deploys both into your cluster: the Capacity Policy as a Karpenter NodePool and the Data Server as a Kubernetes Deployment. For each incoming query, the Data Server dispatches a query executor pod into the NodePool the Policy defines, where it runs within the Policy’s capacity bounds — its eligible instance families, capacity types, and quotas.

One Icebreaker Agent can hold multiple Capacity Policies: a broad default, a high-memory policy for analytics workloads, a tighter-quota policy for a specific team. For a first deployment, one Capacity Policy is sufficient. Start with a broad one that reflects what your FinOps team has approved — see Demand Shaping for the mental model.

1. Navigate to your Icebreaker Agent.
2. Click Compute Specs in the left navigation.
3. Click Add Compute Spec.
4. Provide:
   • A name (e.g., default).
   • Instance types (e.g., c7a.medium, c7a.large, c6a.medium, c6a.large) — limits which EC2 instances Karpenter can provision.
   • Capacity types — pick from on-demand, spot, or both.
5. Click Create.

A Data Server is the Arrow Flight SQL endpoint your SQL clients connect to. It accepts queries and dispatches them to per-query executor pods that run inside your cluster. The Data Server itself runs as a Kubernetes Deployment inside your cluster, with one or more pod replicas for horizontally scaling incoming connections.

1. Navigate to your Icebreaker Agent.

2. Click Data Servers in the left navigation.

3. Click Add Data Server.

4. Provide:

   • A name (e.g., eval, test, or prod).
   • Data Catalog — select from the dropdown.
   • Compute Spec — select the Capacity Policy to govern query execution capacity.

5. Click Create. The Data Server is created in the Admin console; the Icebreaker Agent immediately deploys it into your cluster.

6. Wait for the Data Server status to change to ready (typically within a minute, depending on whether your Capacity Policy has warm capacity).

7. Click the Data Server to open its info modal. Leave it open — you will need the connection values in the next step.

• One Data Catalog is configured in your Account.
• One Capacity Policy is defined for your Icebreaker Agent.
• One Data Server is running in your cluster, with a JDBC URL ready to copy.

Continue to Run your first query.