Query Isolation

In traditional data warehouses and Spark-style clusters, multiple queries share a long-running process — a JVM heap, a thread pool, a memory cache. Right-sizing in that world is aggregate sizing: pick a cluster size big enough for the combined peak workload, and accept that any one query can disrupt the others.

Icebreaker does not share. Every query runs in its own pod, in its own kernel-isolated environment, with its own resources. There is no shared executor, no shared memory, no noisy-neighbor problem to engineer around.

For the business framing, see Query Isolation on the Icebreaker site. This page covers what isolation means concretely.

Each query gets:

• Its own Kubernetes pod. The pod is created when the query arrives and destroyed when it finishes. No state survives between queries.
• Its own DataFusion engine. A Rust-native binary, not a JVM. Cold start is sub-second, so there is no penalty for spinning up a fresh process per query.
• Kernel-enforced CPU and memory limits. The pod is sized to the specific query. If the query exceeds its limits, the kernel terminates the pod — the rest of the fleet is not affected. Icebreaker then retries the query on a right-sized pod.
• No shared memory or caches with other queries. Source data is read from S3 directly. Results are written to S3.
• A namespace per Capacity Policy. Pods for different Capacity Policies land in different Kubernetes namespaces, isolated by network policy and quota.

A misbehaving query — a runaway join, a chatty AI agent submitting thousands of small queries, an analyst running an unoptimized aggregation — cannot consume the resources of another query. It hits its own pod’s limits and stops there.

Unlike shared-process query engines — where tenants’ queries share a memory heap and isolation depends on software-level access controls — Icebreaker puts the tenant boundary in the Kubernetes pod sandbox. When you serve multiple tenants (SaaS customers, business units, regulated workloads), each gets separate Capacity Policies, separate namespaces, separate pods, and separate kernel-enforced limits. Failed queries cannot leak data across tenants because there is no shared memory to leak through.

Because each pod is sized to a single query, you can attribute compute cost directly to that query — by tenant, by workload, by team. Aggregate cluster sizing forces you to estimate share; pod-per-query lets you measure it.

You do not share locks or pools across queries, so concurrency scales with available capacity. A burst of 200 small queries can run as 200 small pods bin-packed onto existing nodes, without queueing on a shared executor. When existing capacity is saturated, Icebreaker queues the remainder; if the queue is large enough, it provisions a new node to absorb it.

• No warm cache inside the executor. A long-running query engine can keep recently-used plans, compiled code, and data blocks in memory so subsequent queries run faster. Icebreaker’s query executor pods start fresh each time, but caching happens above the query executor pod: the planner can serve previously-computed results directly or dispatch only the uncached portion of a query to a fresh query executor pod.
• A cold pod per query. Sub-second to start, but not zero. For interactive sub-second SLAs, you may want pre-warmed capacity (see Manage Capacity Policies).
• Network round trips to storage. Each pod reads from S3 rather than from a shared local cache. S3 throughput is generally sufficient; pathological queries doing huge scans will benefit from partition pruning at the catalog level.