Evaluate Icebreaker with Lakekeeper

This guide walks you through deploying Lakekeeper as your Iceberg REST catalog and connecting it to Icebreaker. Follow the steps in order — each step builds on the previous one.

Before you start, confirm you have:

• EKS cluster running with kubectl and helm configured (prerequisites step 3, step 4)
• The S3 bucket where your Iceberg source data lives (prerequisites step 2)
• The domain and ACM certificate from prerequisites step 5 — reuse the same certificate here
• A terminal authenticated to your AWS account — AWS CloudShell is the lightest-weight option (read through the CloudShell notes under Configure the cluster)

Lakekeeper authenticates to S3 as an IAM user with a static access key pair. This key pair is also needed for step 4, below.

The pattern below mirrors Lakekeeper’s AWS storage guidance, with the action list tightened to the operations Iceberg actually performs.

AWS Console: IAM → Policies → Create policy → JSON tab → paste → Next → name lakekeeper-metadata → Create policy.

Replace <your-source-bucket> with the bucket from prerequisites step 2.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "LakekeeperListBuckets",
      "Effect": "Allow",
      "Action": [
        "s3:ListAllMyBuckets",
        "s3:GetBucketLocation"
      ],
      "Resource": "*"
    },
    {
      "Sid": "BucketAccess",
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::<your-source-bucket>"
    },
    {
      "Sid": "IcebergObjectAccess",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject",
        "s3:GetObjectVersion",
        "s3:GetObjectTagging",
        "s3:PutObjectTagging",
        "s3:ListMultipartUploadParts",
        "s3:AbortMultipartUpload"
      ],
      "Resource": "arn:aws:s3:::<your-source-bucket>/*"
    }
  ]
}

The LakekeeperListBuckets statement lets the Lakekeeper UI populate its bucket selector. For a tighter scope, drop it and type the bucket name directly when creating the warehouse.

AWS Console: IAM → Users → Create user → name lakekeeper-user (leave “Provide user access to the AWS Management Console” unchecked) → Next → Attach policies directly → select lakekeeper-metadata → Next → Create user.

Then click the user’s page: Security credentials → Create access key → Application running on an AWS compute service. AWS may warn that an IAM role is preferred over static keys for compute workloads — check the confirmation box and click Next.

Set the description tag to Lakekeeper S3 access for Icebreaker evaluation and click Create access key.

Click Show. Store both the Access key and Secret access key in a secure location like a password or secrets manager. Click Done.

Lakekeeper uses STS to vend short-lived S3 credentials to Icebreaker’s query executor pods instead of passing the long-lived access keys directly. This requires an IAM role that Lakekeeper can assume.

1. Create IAM role: lakekeeper-sts-vending

• AWS Console: IAM → Roles → Create role → Custom trust policy → paste the policy below, replacing <your-account-id> with your AWS account ID → Next → skip permissions → name lakekeeper-sts-vending, description Lakekeeper assumes this role to vend short-lived S3 credentials to Icebreaker query executors → Create role.

  {
    "Version": "2012-10-17",
    "Statement": [
      {
        "Effect": "Allow",
        "Principal": {
          "AWS": "arn:aws:iam::<your-account-id>:user/lakekeeper-user"
        },
        "Action": "sts:AssumeRole"
      }
    ]
  }

2. Attach lakekeeper-metadata policy to lakekeeper-sts-vending role

• On the lakekeeper-sts-vending role page: Permissions → Add permissions → Attach policies → select lakekeeper-metadata → Add permissions.

3. Allow lakekeeper-user to assume lakekeeper-sts-vending role

• IAM → Policies → Create policy → JSON tab → paste → Next → name lakekeeper-sts-assume → Create policy.

  {
    "Version": "2012-10-17",
    "Statement": [
      {
        "Sid": "LakekeeperSTSAssume",
        "Effect": "Allow",
        "Action": "sts:AssumeRole",
        "Resource": "arn:aws:iam::<your-account-id>:role/lakekeeper-sts-vending"
      }
    ]
  }

• Attach it to lakekeeper-user: IAM → Users → lakekeeper-user → Permissions → Add permissions → Attach policies → select lakekeeper-sts-assume.

• Copy the lakekeeper-sts-vending role ARN from the role summary page — you’ll paste it into the Lakekeeper warehouse form in step 3.

Continue in CloudShell or your preferred terminal.

helm repo add lakekeeper https://lakekeeper.github.io/lakekeeper-charts/
helm repo update
kubectl create namespace lakekeeper

Create lakekeeper-values.yaml in your home directory with the following content. This exposes Lakekeeper’s UI and API via an AWS Network Load Balancer, with TLS terminated at the NLB using the ACM certificate from Before you start — substitute its ARN below. Decide on the subdomain you’ll use for Lakekeeper now (e.g. lakekeeper.yourdomain.com, covered by your wildcard certificate) — you’ll point DNS at it once the NLB exists, but the hostname itself needs to go into this values file already.

# Embedded PostgreSQL — zero extra infrastructure for evaluation.
# Switch to an external database for production.
postgresql:
  enabled: true

# Disable authorization for evaluation — any caller that can reach
# Lakekeeper can use the catalog. Set authz.backend to openfga or cedar
# and configure an OIDC IdP for production.
authz:
  backend: "allowall"

catalog:
  # Tell Lakekeeper its own externally-reachable URL. An NLB operates at
  # Layer 4 — it terminates TLS but, unlike an ALB or nginx, does not
  # inject X-Forwarded-Proto/X-Forwarded-Host headers, so Lakekeeper has
  # no way to detect that traffic outside the cluster is HTTPS. Without
  # this, it generates http:// links in its own UI/API responses, which
  # browsers block as mixed content once the page itself loads over
  # https:// — surfacing as "failed to fetch" in the Lakekeeper UI.
  config:
    LAKEKEEPER__BASE_URI: "https://<your-lakekeeper-host>:8181"

  # Expose Lakekeeper's UI and API via an AWS Network Load Balancer, with
  # TLS terminated at the NLB so you can reach it from a browser and from
  # curl without port-forwarding.
  service:
    type: LoadBalancer
    annotations:
      service.beta.kubernetes.io/aws-load-balancer-type: "external"
      service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "ip"
      service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing"
      service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "<your-acm-certificate-arn>"
      service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "8181"

Lakekeeper’s embedded PostgreSQL needs persistent storage. EKS Auto Mode clusters only have the legacy gp2 StorageClass by default, which its control plane cannot provision. Create a CSI-backed one before installing:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ebs-csi
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: ebs.csi.eks.amazonaws.com
volumeBindingMode: WaitForFirstConsumer
reclaimPolicy: Delete
allowVolumeExpansion: true

kubectl apply -f ebs-storageclass.yaml
kubectl get storageclass

ebs-csi should appear with (default). If gp2 also shows (default), remove it:

kubectl patch storageclass gp2 -p '{"metadata":{"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'

In one terminal, install Lakekeeper:

helm install icebreaker-lakekeeper lakekeeper/lakekeeper --namespace lakekeeper --values lakekeeper-values.yaml

Optional: in a second terminal, watch the deployment. This can be helpful for troubleshooting.

kubectl get pods -n lakekeeper -w

Wait until the status of the lakekeeper-pg (PostgresDB) and lakekeeper-pods show Running, and a migration job is Completed. Press Ctrl+C to exit watch mode.

The NLB hostname takes 1–2 minutes after install to populate:

kubectl get svc icebreaker-lakekeeper -n lakekeeper -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'

It will look similar to k8s-lakekeep-icebreak-<id>.elb.<region>.amazonaws.com. Create a CNAME record from the hostname you already chose above (the one you set as LAKEKEEPER__BASE_URI) to this NLB hostname — the certificate is valid for your domain, not for the NLB’s own hostname, so browsers and curl need to reach it by the domain name. Once the CNAME resolves, that same hostname is <your-lakekeeper-host> for the rest of this guide.

curl -X POST https://<your-lakekeeper-host>:8181/management/v1/bootstrap -H "Content-Type: application/json" -d '{"accept-terms-of-use": true}'

An empty or 200 response confirms Lakekeeper is ready.

Create a warehouse pointed at your source-data bucket through the Lakekeeper UI at https://<your-lakekeeper-host>:8181/ui/ or through the management API. Either way, the settings that matter for Icebreaker are:

• Storage type: s3, flavor aws
• Storage credential: type s3, credential-type access-key — paste the access key ID and secret from step 1.
• STS enabled: true. Paste the lakekeeper-sts-vending role ARN from step 1. Lakekeeper assumes this role to obtain short-lived S3 credentials, which it returns to Icebreaker’s query executor pods on each catalog request — the long-lived access keys never leave Lakekeeper.

After creation, capture the Warehouse ID that Lakekeeper assigns. This is the value Icebreaker needs — not the warehouse name. The Warehouse ID appears in the warehouse Details tab.

Register your Iceberg tables in the new warehouse through the Lakekeeper UI or the Iceberg REST API. Icebreaker does not register tables — that’s the catalog’s responsibility.

Via the UI:

1. Open the warehouse and create a Namespace if you don’t already have one to register into.
2. Open the namespace, go to the Tables tab, and click Register Tables.
3. Enter a Table Name and the table’s existing Metadata Location — the full s3:// path to its current metadata/<version>.metadata.json file (find the highest-numbered one if there are several).
4. Confirm. Repeat Add Table for additional tables, or use Import from CSV / JSON to register several at once.

This is also how you recover tables whose data already exists in S3 from a prior installation — you’re pointing a new catalog at existing metadata rather than creating anything new.

Lakekeeper’s UI includes a built-in SQL query editor that reads and writes Iceberg data directly from your browser to S3. It needs a CORS policy on the storage bucket backing this warehouse to allow that direct, cross-origin access.

1. Open the storage bucket in the AWS Console → Permissions tab → Cross-origin resource sharing (CORS) → Edit.

2. Paste the following, substituting your own domain from step 2 (the console’s CORS box expects the bare array, no wrapping key):

   [
     {
       "AllowedOrigins": [
         "https://<your-lakekeeper-host>:8181"
       ],
       "AllowedMethods": ["GET", "HEAD", "POST", "PUT", "DELETE"],
       "AllowedHeaders": ["*"],
       "ExposeHeaders": ["ETag", "x-amz-version-id"],
       "MaxAgeSeconds": 3000
     }
   ]

3. Click Save changes — it takes effect immediately, no propagation delay.

Using the credentials provided to you during onboarding, open the Icebreaker console.

Navigate to Data Catalogs → New Data Catalog, give it a name (e.g., eval-catalog), and paste the following into the Configuration (JSON) field. Substitute your source and results bucket names (see prerequisites step 2), region, the access key pair from step 1, and the warehouse UUID from step 3:

{
  "catalog": {
    "id": "lakekeeper",
    "name": "Lakekeeper",
    "uri": "http://icebreaker-lakekeeper.lakekeeper.svc.cluster.local:8181/catalog",
    "warehouse": "<your-warehouse-uuid>",
    "properties": {
      "s3.region": "<your-aws-region>",
      "s3.endpoint": "https://s3.<your-aws-region>.amazonaws.com",
      "s3.access-key-id": "<your-access-key-id>",
      "s3.secret-access-key": "<your-secret-access-key>",
      "s3.allow_http": "false"
    }
  },
  "storage": {
    "format": "parquet",
    "compression": "snappy",
    "path_prefix": "results",
    "properties": {
      "bucket_name": "<your-bucket>",
      "region": "<your-aws-region>",
      "endpoint": "https://s3.<your-aws-region>.amazonaws.com",
      "access_key_id": "<your-access-key-id>",
      "secret_access_key": "<your-secret-access-key>"
    }
  }
}

The uri uses Lakekeeper’s cluster-internal Service DNS name over plain HTTP — Icebreaker’s query executor pods reach Lakekeeper over the pod network, not through the TLS-terminated NLB, so no ingress or certificate is involved here. This is the simplest reachable-from-the-cluster path for an in-cluster Lakekeeper deployment.

See Catalog integration > Configuration schema for the full field reference.