IceGuard

A modern web console to manage, inspect and maintain Apache Iceberg™ tables across multiple catalogs.

🌐 adelfardi.github.io/iceguard-site

_{Dashboard → a table's branches & tags graph, storage health down to each partition, timeline and schema evolution → a Spark maintenance action with its result + logs → catalogs, wizards and pipelines. ▶ MP4 version · live demo}

⚠️ Independent community project — not affiliated with or endorsed by the Apache Software Foundation. "Apache", "Apache Iceberg", "Iceberg" and "Apache Polaris" are trademarks of the ASF.

---

What is IceGuard?

IceGuard is an admin & maintenance console for the Apache Iceberg lakehouse. Point it at one or more catalogs (REST, Nessie, Polaris) and get a single UI to browse namespaces and tables, evolve schemas and partitions, edit properties, sample data, inspect storage layout, view lineage, and run (or schedule) maintenance like snapshot expiration and data-file compaction.

It is not a catalog — it sits on top of your existing Iceberg catalogs.

What IceGuard actually is

IceGuard is two components:

1. Frontend — the React/TypeScript web UI.
2. Backend — the Quarkus REST API.

It has one required dependency: a PostgreSQL database (you provide it), where the backend stores IceGuard's own state — registered catalogs, pipelines, schedules, alerts and execution history. Postgres is not part of IceGuard; it's infrastructure IceGuard needs to run, like a JVM.

IceGuard then connects to your own Iceberg catalog(s) and object store.

The docker-compose.yml in this repo (MinIO, an Iceberg REST Catalog, Nessie, Apache Polaris, optionally Spark) is only a local test sandbox so you can try IceGuard end-to-end without bringing your own infrastructure. It is not part of the product — in production you point IceGuard at your existing catalogs and storage.

Features

• Multi-catalog — register and switch between REST, Nessie, Polaris and Unity Catalog (None/Bearer/OAuth2 supported). Catalogs can be tagged and filtered, and edited on a dedicated page (credentials, arbitrary properties, S3 access modes).
• Dashboard — configurable, live widgets (storage, maintenance reliability, recently active partitions, file sizes, catalogs, executions): add them from a gallery, pin them from a table, drag to reorder.
• Namespaces & tables — browse the tree, create namespaces, create/drop/rename tables, insert sample rows.
• Schema editor — add / rename / retype / re-doc / drop multiple columns, applied in a single commit.
• Properties editor — add / update / remove table properties in a single commit.
• Partition evolution — add or drop partition fields (identity, bucket, truncate, year/month/day/hour).
• Versioning — a git-style graph of the snapshot history with one lane per branch and tag; branch, tag (with retention policy) and snapshot lists, snapshot details and rollback. Nessie history is rebuilt from its commit log.
• Storage & health — point-in-time totals, file-size histogram and per-partition aggregates (server-side paginated, typed filters), with a health grade for the table and for every partition (file size vs target, small files, delete files; configurable thresholds). Drill down to a partition's files and to a file's rows, data and delete files alike. See docs/STORAGE_HEALTH.md.
• Evolution — schema-version history with column diffs, and a visual snapshot-to-snapshot diff.
• Timeline — snapshots + executions on one timeline, click any item for its output/logs.
• Maintenance — expire snapshots, rewrite data files (strategy, sort order, where, options; whole table or selected partitions), rewrite manifests, rewrite position / equality delete files, remove orphan files, rollback. Pluggable executors: a Java API executor (real compaction and orphan removal on small, append-only tables) and a Spark executor (local local[*] with tunable resources, or a remote Spark cluster).
• Commit activity — the quietest window of the day, detected from snapshot timestamps, to schedule maintenance off-peak.
• Pipelines — chain maintenance actions with per-task parameters, retries and delay, and a cron schedule built graphically; Airflow-style run view with rerun and retry of a failed task.
• Alerts — threshold rules on table metrics with optional SMTP email notifications.
• Public demo mode — ICEGUARD_DEMO_MODE=true on the frontend serves a read-only UI (banner, writes refused with a clear message), as on demo.iceguard.cloud.

Supported catalogs

Any Iceberg REST-compatible catalog works. The Add Catalog wizard shows a card per engine, and the type is auto-detected from the catalog name/URI:

	Catalog	Typical URI	Auth
	Iceberg REST	http://host:8181	None / Bearer
	Nessie	http://host:19120/iceberg	None
	Apache Polaris	http://host:8182/api/catalog	OAuth2
	Unity Catalog	https://<workspace>/api/2.1/unity-catalog/iceberg-rest	Bearer (PAT) / OAuth2

_{Unity Catalog (Databricks or the OSS unitycatalog.io) is reached through its Iceberg REST Catalog API — browse works today; writes depend on the endpoint (Databricks managed-Iceberg read/write is in preview, requires EXTERNAL USE SCHEMA + a PAT/OAuth token).}

Nessie has a few specifics (snapshot history rebuilt from its commit log, S3 region requirement, flaky-remote handling) — see docs/NESSIE.md.

Stack

Layer	Tech
Frontend (product)	React, TypeScript, Vite, Tailwind CSS, shadcn/ui, TanStack Query, Zustand, Recharts
Backend (product)	Java 21, Quarkus 3.17, RESTEasy Reactive, Hibernate Panache, Flyway, Apache Iceberg 1.11
Required dependency	PostgreSQL (backend state)
Test sandbox only	Docker Compose: MinIO (S3), Iceberg REST Catalog, Nessie, Apache Polaris, (optional) Spark

Quick start

Requirements: Docker + Docker Compose. (For hot-reload development: JDK 21 & Maven, Node 20+.)

The app is two services — frontend + backend — plus a required Postgres. docker-compose.yml bundles them in three tiers via Compose profiles:

Level	Command	Services	When
1 — app (official)	docker compose up -d --build	frontend + backend	Bring your own Postgres (ICEGUARD_DB_URL in .env)
2 — + database	docker compose --profile db up -d --build	+ Postgres	Self-contained app + its required DB
3 — + test sandbox	docker compose --profile db --profile sandbox up -d --build	+ MinIO + REST catalog	Try everything locally

No local build? Use the published images from GHCR instead — same levels, just point at the images file (no --build):

docker compose -f docker-compose.images.yml --profile db --profile sandbox up -d

Defaults to :latest; pin a release with TAG=0.1.0 docker compose -f docker-compose.images.yml ….

For the quickest tour, use Level 3 and open http://localhost:8090.

Add your first catalog (sandbox)

In the UI, go to Catalogs → Add Catalog and register the bundled Iceberg REST catalog:

Field	Value
Catalog type	Iceberg REST
URI	http://rest-catalog:8181 (Docker service name — not localhost; the backend reaches it over the Compose network)
Warehouse	s3://warehouse/rest/
Authentication	None
S3 / Storage access	Static keys
• Endpoint	http://minio:9000
• Access key / Secret key	minioadmin / minioadmin
• Region	us-east-1
• Path-style access	enabled ✅

The S3 settings let the backend read/write data files on the sandbox MinIO. Once added, create a namespace and a table to see browsing, schema, storage and maintenance in action.

Tear down with docker compose --profile db --profile sandbox down (add -v to wipe data) — use -f docker-compose.images.yml in the same command if you started from the images file.

Service	URL	Notes
Frontend (UI)	http://localhost:8090	nginx; proxies /api to the backend
Backend API	http://localhost:8080	Swagger at /q/swagger-ui
MinIO console	http://localhost:9001	minioadmin / minioadmin (sandbox profile)
Iceberg REST catalog	http://localhost:8181	sandbox profile; register it as http://rest-catalog:8181

Develop (hot reload)

Run the dependencies in Docker and the app from source:

# deps only (Postgres :5433, REST catalog :8181, Nessie :19120, MinIO) — not the backend container
docker compose -f docker-compose.dev.yml up -d postgres rest-catalog nessie minio minio-init
cd backend && mvn quarkus:dev -Dquarkus.profile=docker   # :8080  (Swagger: /q/swagger-ui)
cd frontend && npm install && npm run dev                 # :5173  (proxies /api to :8080)

Advanced: full multi-catalog sandbox

For every catalog type at once (REST + Nessie + Polaris) and real Spark compaction:

docker compose -f docker-compose.dev.yml up -d
./scripts/seed-catalog.sh        # after the backend is up

The REST Catalog and Nessie demo catalogs work out of the box on MinIO. Polaris writes require real AWS S3 — copy .env.example to .env.

Kubernetes (Helm)

A chart lives in charts/iceguard. Like Level 1 above it deploys the two IceGuard services only — frontend + backend — against a PostgreSQL you provide (images default to the published ghcr.io/adelfardi/iceguard-{frontend,backend}:0.3.0):

helm install iceguard ./charts/iceguard -n iceguard --create-namespace \
  --set database.host=pg.internal \
  --set database.existingSecret=iceguard-db-credentials    # key: password

kubectl port-forward -n iceguard svc/iceguard-frontend 8090:80    # then http://localhost:8090

The schema must be empty on first install — Flyway owns it and migrates at startup. A managed database usually wants the full URL instead: --set database.jdbcUrl='jdbc:postgresql://…?sslmode=require'.

Prefer plain kubectl? Render the manifests and apply them — Helm is only needed to produce them:

helm template iceguard ./charts/iceguard -n iceguard --skip-tests \
  --set database.host=pg.internal --set database.existingSecret=iceguard-db-credentials \
  > iceguard.yaml
kubectl create namespace iceguard && kubectl apply -n iceguard -f iceguard.yaml

Ingress, network policies, OIDC, replicas/PDBs and the full values reference are documented in charts/iceguard/README.md.

Using IceGuard for real

Deploy the frontend and backend, point the backend at a PostgreSQL you provide (QUARKUS_DATASOURCE_JDBC_URL / username / password), then add your own catalog(s) from the UI (Catalogs → Add Catalog): REST / Nessie / Polaris URI, warehouse, auth (None / Bearer / OAuth2) and, if needed, S3 credentials. No MinIO/Spark/sandbox containers required.

Architecture

        IceGuard (the product)            required dep            your existing systems
   ┌───────────────────────────────┐
   │ React + Vite (UI, :5173)       │
   │           │                    │
   │ Quarkus REST API (:8080) ──────┼──> PostgreSQL ─────────┐
   └───────────────────────────────┘   (IceGuard state)     │
                   │                                         │
                   └──────────────────> Iceberg catalogs (REST / Nessie / Polaris)
                                                  └──> S3 / object store (data & metadata)

IceGuard's own state (registered catalogs, pipelines, schedules, alerts, execution history) lives in the PostgreSQL you provide; table data & metadata live in your object store via your Iceberg catalogs.

Project layout

backend/                 Quarkus REST API (com.iceguard.*)
frontend/                React + TypeScript SPA
charts/iceguard/         Helm chart (Kubernetes deployment)
scripts/                 seed + helper scripts
docker-compose.yml       app stack (frontend + backend; profiles: db / sandbox)
docker-compose.dev.yml   advanced multi-catalog sandbox (Nessie, Polaris, Spark)

Known limitations

• The Java executor really runs compaction & orphan-removal, but only on small, append-only tables (hard size/file-count limits; refuses above them) — use the Spark executor for large or merge-on-read tables.
• Polaris + MinIO: browsing works, but writes fail (Polaris ignores the S3-compatible endpoint) — use real AWS S3.
• Catalog credentials are stored in PostgreSQL (not encrypted at rest) — treat the DB as sensitive.
• Test coverage is early-stage. Contributions very welcome 🙂

Contributing

Contributions, issues and ideas are welcome! Please read CONTRIBUTING.md and our Code of Conduct. Good first issues are labelled good first issue.

How the pipelines and automation work (CI, releases, Dependabot, CodeQL, Trivy) is documented in docs/CI.md.

License

Apache License 2.0.