feat: standard DB host ports, expose-by-default, unified provisioning + up/down fixes - #123
Merged
Merged
Conversation
…rangoDB engines Publish shared engines on their WELL-KNOWN host ports instead of a 5xxxx offset band, so `defaultPort` (in-network) and the exposed host port match and a GUI client's defaults just work: postgres 5432, mysql/mariadb 3306, mongodb 27017, cassandra 9042, arangodb 8529, redis 6379, minio 9000/9001. The standard port becomes the FreeHostPort search base. This is safe because AllocatePort skips every already-allocated port across all owners, so a lone engine lands on the standard port, a host-native server holding it makes the allocator fall back to the next free port, and protocol twins (mysql+mariadb on 3306, localstack+ministack on 4566) deconflict to base/base+1. Kafka keeps its fixed advertised 49092. Add five image-based shared database engines (built-in templates), each reachable at shared-<name>, exposable, and with a client-ready connection URL: mysql, mariadb, mongodb, cassandra, arangodb. Wire the devdock-import engine map and the templates README. Determinism, vet, gofmt, and the race detector on orchestrate/generate pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ix up/down There were TWO host-port overlays fighting over the same shared containers' `ports:` — the expose overlay (standard ports) and a separate provisioning band (45xxx via writeProvisionOverlay/engineTarget). Every provision, `db reset`, snapshot, resource op and re-up applied a DIFFERENT subset, so compose kept recreating the shared container with a different port set. That churn is the root of "port already allocated", the connection-reset races, and re-up flakiness. Collapse the two into ONE unified overlay published on the engine's STANDARD 127.0.0.1 port: - ensureExposed / primaryExposePort (expose.go): the single host-reachability primitive. Provisioning, reset, snapshot and resource ops all resolve their admin endpoint from it, so there is exactly one host port per engine — the well-known one — never a separate band. Idempotent: same ports + same overlay bytes → no container recreate. - engineTarget, the resources phase, and the provision phase now go through ensureExposed. Deleted engineOverlays/perEngineOverlay, writeProvisionOverlay, and the provisionPortBase/provisionPurpose (pg-provision → pg-expose) band. - Expose-by-default: `up` auto-publishes every exposable shared engine on its standard 127.0.0.1 port (loopback-only), so a GUI client's defaults just work and provisioning has one port to dial. Opt out with `up --no-expose`; skipped on a remote backend. - `down` now actually brings things DOWN: after dropping a project's refs it stops every shared engine that fell to zero refs (Manager.GC stop=true), respecting cross-workspace ref-counting, and CLEARS the saga phase records for what it stopped (compose-up per project; shared/provision/resources when the shared stack is stopped) so the next `up` restarts them instead of skipping on a stale fingerprint — the "re-up after down is a no-op" bug. Verified end-to-end against a real Docker daemon (up → provision on 127.0.0.1:5432 → down stops shared → re-up restarts and re-publishes): standard ports throughout, no collisions across the cycle. Unit suite + race + determinism pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Builds on #122. Makes shared services reachable on the ports developers already expect, adds the common databases as first-class engines, and fixes the
up/downreliability issues rooted in the old dual host-port design.1. One standard host port per engine — no separate bands
Previously there were two host-port overlays fighting over the same shared containers'
ports::45xxxviawriteProvisionOverlay/engineTarget).Every
provision,db reset, snapshot, resource op and re-up applied a different subset, so compose kept recreating the shared container with a different port set — the root of "port already allocated", connection-reset races, and re-up flakiness.Now collapsed into one overlay published on the engine's standard
127.0.0.1port (ensureExposed/primaryExposePort). Provisioning, reset, snapshot and resource ops all resolve their admin endpoint from it — exactly one host port per engine, the well-known one. Idempotent (same ports + same overlay bytes → no recreate). DeletedengineOverlays,writeProvisionOverlay, and thepg-provision/45xxxband.*kafka keeps its fixed advertised external listener.
2. Expose by default on
upupauto-publishes every exposable shared engine on its standard loopback port, so GUI clients' defaults just work and provisioning has one port to dial. Opt out withup --no-expose; skipped on remote backends.3.
downactually brings things down + re-up worksdownnow stops every shared engine that fell to zero refs (Manager.GC), respecting cross-workspace ref-counting — no more warm engines left running.compose-upper project;shared/provision/resourceswhen the shared stack stops), so the nextuprestarts them instead of skipping on a stale fingerprint — fixes "re-up after down is a no-op".4. Five new shared database engines
Built-in templates, each reachable at
shared-<name>, exposable, with a connection URL:mysql,mariadb,mongodb,cassandra,arangodb. Plus the devdock-import map and README.Verification
up→ auto-expose on127.0.0.1:5432→ provision role/db (psql confirmed) →downstops shared → re-up restarts shared+provision+app and re-publishes ports. Standard ports throughout, zero collisions across the cycle.${ref}resolves toshared-mysql/3306etc.go vet,gofmt,make determinism, and the race detector onorchestrate/clipass.Notes
fix/expose-wsl2-excluded-ports(fix(expose): skip Windows/Hyper-V excluded port ranges on WSL2 #122) so the diff is scoped — retarget tomainonce fix(expose): skip Windows/Hyper-V excluded port ranges on WSL2 #122 merges.TestRunUninstallSequence) is env leakage from a real~/.local/bin/dssymlink on the dev machine, unrelated to this change.🤖 Generated with Claude Code