Skip to content

feat(db): Redis and MinIO snapshot/restore dumpers (spec 15) - #105

Merged
gustavobertoi merged 1 commit into
mainfrom
feat/db-redis-minio-dumpers
Jul 1, 2026
Merged

feat(db): Redis and MinIO snapshot/restore dumpers (spec 15)#105
gustavobertoi merged 1 commit into
mainfrom
feat/db-redis-minio-dumpers

Conversation

@gustavobertoi

Copy link
Copy Markdown
Contributor

What

Extends the spec-15 db snapshot/db restore data-lifecycle verbs beyond Postgres to the other stateful shared engines, behind the existing internal/db Dumper seam. Today only Postgres has a dumper; this adds Redis and MinIO and wires engine selection.

RedisDumper (internal/db/redis.go)

  • Shells redis-cli behind the injectable Runner (mirrors PgDumper).
  • Snapshot: redis-cli --rdb <out> — a whole-instance RDB via replication SYNC; does not stop the shared server, so other tenants are undisturbed.
  • Restore: streams the artifact back through redis-cli's mass-insert pipe (sh -c "redis-cli … --pipe < dump", because the shared Runner has no stdin channel). Never restarts/bounces the container.
  • Password rides REDISCLI_AUTH in the process env, never on argv. Preflight degrades the db verbs only, with a one-line redis-cli not found remediation.
  • Best-effort (documented): a host client cannot cheaply carve one logical DB out of a live shared Redis, and a byte-faithful whole-RDB reload needs a controlled restart the never-recreate guard forbids. Redis is exactly the engine spec 15 flags as best-effort for the shared model.

MinioDumper (internal/db/minio.go)

  • Uses the already-vendored pure-Go aws-sdk-go-v2 S3 client (no new external binary) behind an S3Factory seam.
  • Snapshot: list + get the tenant bucket's objects into a deterministic (key-sorted) tar under the content-addressed store. Only the tenant bucket is read → project A can never capture project B.
  • Restore: read the tar, put each object back into the same bucket (bucket names are globally unique per instance).

Wiring (internal/orchestrate/snapshot.go, internal/cli/db.go)

  • resolveTenant now derives the per-engine tenant namespace (pg db / redis index / minio bucket); tenantConn builds the right ConnInfo (redis auth-less by default); SelectDumper returns the correct dumper by --kind.
  • Dump file extension + SnapshotMeta.Kind follow the engine (.dump/.rdb/.tar).
  • db snapshot/db restore gain --kind pg|redis|minio.
  • Host-port overlay reachability pattern and lock-first ledger writes are unchanged (streaming outside the flock, row writes inside).

Why

Redis/MinIO tenants deserve the same per-project snapshot/restore lifecycle as Postgres; the Dumper seam was designed for exactly this extension.

How tested

CGO_ENABLED=1 go test ./internal/... (+ -race on db/orchestrate/cli) all green; CGO_ENABLED=0 go build ./..., go vet ./..., gofmt -l clean.

  • Redis: snapshot/restore argv + auth-in-env-not-argv + no-auth/no-index cases + IsEmpty via mock runner.
  • MinIO: snapshot/restore round-trip, determinism, tenant isolation, IsEmpty via a mock in-memory S3.
  • Dumper-selection-by-engine; full-flow redis/minio Snapshot/Restore over the host-port overlay with the mock docker client.

No changes to internal/generate or templates/, so generated-artifact determinism is unaffected.

🤖 Generated with Claude Code

Extend the spec-15 data-lifecycle verbs beyond Postgres to the other
stateful shared engines, behind the existing db.Dumper seam.

- RedisDumper: shells `redis-cli` behind the injectable Runner. Snapshot
  captures a whole-instance RDB via `redis-cli --rdb` (SYNC, no server
  stop → other tenants undisturbed); restore streams it back through the
  mass-insert pipe (`sh -c "redis-cli … --pipe < dump"`, since the Runner
  has no stdin channel). Password rides REDISCLI_AUTH in the env, never
  argv. Preflight gives a 'redis-cli not found' remediation. Redis is the
  documented best-effort engine: a host client cannot cheaply carve one
  logical DB out of a live shared instance, and a byte-faithful whole-RDB
  reload would need a controlled restart the never-recreate guard forbids.

- MinioDumper: uses the already-vendored pure-Go aws-sdk-go-v2 S3 client
  (no new external binary) behind an S3Factory seam. Snapshot lists+gets
  the tenant bucket's objects into a deterministic (key-sorted) tar under
  the content-addressed store; restore puts them back into the same bucket.
  Only the tenant bucket is read → project A can never capture project B.

- snapshot.go: engine selection by kind (pg|redis|minio) — resolveTenant
  now derives the per-engine tenant namespace (pg db / redis index / minio
  bucket), tenantConn builds the right ConnInfo (redis auth-less by
  default), and SelectDumper wires the correct dumper. Dump file extension
  and SnapshotMeta.Kind follow the engine (.dump/.rdb/.tar).

- cli/db.go: `db snapshot`/`db restore` gain `--kind pg|redis|minio` and
  select the dumper via orchestrate.SelectDumper.

Tests: redis snapshot/restore argv + auth/env via a mock runner; minio
snapshot/restore round-trip + determinism + tenant isolation via a mock
S3; dumper-selection-by-engine; full-flow redis/minio Snapshot/Restore
over the host-port overlay with the mock docker client. Host-port overlay
reachability pattern and lock-first ledger writes are unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@gustavobertoi
gustavobertoi merged commit 38b68ad into main Jul 1, 2026
4 checks passed
@gustavobertoi
gustavobertoi deleted the feat/db-redis-minio-dumpers branch July 1, 2026 15:52
gustavobertoi added a commit that referenced this pull request Jul 1, 2026
#103 (db reset/pull) and #105 (redis/minio dumpers) each passed CI alone but
broke `main` when both merged — a semantic conflict git didn't flag:

- #105 changed `resolveTenant` to take a `kind` and return `engine,label`
  (5 args / 6 returns); #103's `reset.go` still called the 4-arg form.
- #105 replaced the `defaultPgDumper()` helper with `SelectDumper(d, kind)`;
  #103's `db pull` still called the removed helper.

Both call sites now use the merged signatures (pg kind, extra returns discarded).
Build, `go vet`, full `go test ./internal/...`, and `make determinism` all green.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant