Caching and checkpointing

Most query engines cache inside long-running processes — compiled plans, recently-read data blocks, warm memory heaps. That model assumes compute is a fixed, always-on resource. Icebreaker’s compute is ephemeral and stateless by design, so it needs a fundamentally different approach.

Icebreaker’s planner treats intermediate result fragments as durable checkpoints rather than transient in-memory state. The mechanism is not just a performance optimization — it is the reason stateless, Spot-backed compute works reliably and cost-effectively at scale.

Before dispatching any query executor pod, the planner walks the full tree of query operations. At each node in the tree, it evaluates the cost of computing that fragment versus retrieving it from cache. Unlike a traditional database — which optimizes assuming a static set of always-available resources — the planner’s cost model is cloud-native:

• S3 scan cost. How many bytes would an executor need to read from S3 to produce this fragment? What does that scan cost in time and money at current data volume and partition layout?
• Network and transfer cost. What is the cost of materializing and storing this fragment versus re-fetching the source data?
• Compute cost. How expensive is this operation relative to the I/O cost of the alternative?

Based on that evaluation, the planner proactively decides which fragments to preserve — not reactively after a cache miss, but as part of the execution plan itself. Fragments that are expensive to recompute relative to their storage cost are checkpointed before execution begins.

The practical effect is that a query’s execution plan has explicit preservation points built into it — similar in spirit to Spark’s stage boundaries, but without Spark’s serialization overhead or the latency of staged shuffle operations. The planner selects checkpoint locations based on cost, not on shuffle topology.

When a subsequent query overlaps with a preserved fragment:

1. The planner identifies which sub-trees of the new query can be satisfied by existing fragments.
2. Those sub-trees are served directly — no query executor pod, no S3 scan.
3. Only the genuinely new portions of the query are dispatched to fresh query executor pods.
4. Results are composed and returned.

A query that largely overlaps prior work may require no query executor pod at all.

This is where the checkpointing model has its most significant operational impact.

Spot instances can be interrupted with two minutes’ notice. In a traditional query engine — or in Spark without explicit checkpointing — a Spot interruption means restarting the query from scratch, re-scanning all source data from S3. The cost of the interrupted work is lost.

With Icebreaker’s planner-level checkpointing, the preserved fragments from an interrupted executor remain valid. When the query is retried — on a new Spot instance, on on-demand, or after Karpenter provisions a replacement — execution resumes from the last checkpoint rather than from S3. The cost of the scan work already done is not lost.

This changes the economics of Spot: the interruption risk that makes Spot unsuitable for long-running queries in conventional systems is substantially mitigated. You get Spot pricing on workloads that would otherwise require on-demand.

Icebreaker’s query executor pods are fully stateless — no warm process, no shared memory, no persistent heap. They start, execute one query fragment, and terminate. This is what allows them to scale to zero between query bursts with no idle cost.

The checkpointing layer is how Icebreaker reconciles stateless compute with the benefits that stateful systems usually claim: reusability, cost avoidance from not reprocessing data, and resilience against interruption. The state lives in the cache layer — durable, Iceberg-snapshot-aware, independent of any executor’s lifecycle — rather than inside a long-running process.

Iceberg’s snapshot model gives Icebreaker a precise and automatic staleness signal. Every cached fragment is associated with the Iceberg table snapshot it was computed from. When a write to a table produces a new snapshot, any fragment derived from an older snapshot of that table is invalidated. Fragments derived from tables that have not changed remain valid regardless.

This means cache invalidation is not a TTL or a manual flush — it is automatic and exact, driven by the same versioning that Iceberg uses for time travel and transactional consistency.

Suppose a query returns daily sales aggregates for the last quarter — 90 rows of computed data. The planner has checkpointed those aggregates.

A second query asks: “How does this quarter’s total compare to the same quarter last year?”

The planner:

• Serves this quarter’s aggregates directly from the checkpoint — no S3 scan, no query executor pod for that sub-tree.
• Dispatches only the last-year computation to a fresh query executor pod.
• Composes the two results and returns the comparison.

If the executor for the last-year computation is interrupted by a Spot reclaim mid-execution, the re-dispatched query executor pod resumes from any intermediate checkpoints the planner placed within that sub-tree — it does not restart the S3 scan from the beginning.

• Lower cost per query. S3 scans are the dominant cost driver for most workloads. Checkpointing eliminates redundant scans across overlapping queries and across retries.
• Spot viability for long queries. Interrupted work is not lost. Spot interruptions become a minor delay rather than a full restart, making Spot practical for workloads that would otherwise require on-demand.
• Stateless scaling without the usual penalty. Query executor pods scale to zero with no warm-up cost, and restarting is cheap because preserved fragments eliminate the need to rebuild state from source.
• Predictable performance for repeated patterns. Dashboards, scheduled reports, and agentic AI workloads that repeatedly explore similar data ranges benefit the most — each pass makes subsequent passes cheaper.
• No manual checkpointing configuration. The planner decides what to preserve based on cost, not on developer-defined stage boundaries. There is nothing to configure.