Skip to content

docs(cluster): "Running HFS in a Cluster" operator chapter and cluster-testing skill section (Phase 0.5) - #909

Open
aacruzgon wants to merge 1 commit into
mainfrom
cluster/p0.5-ch15
Open

docs(cluster): "Running HFS in a Cluster" operator chapter and cluster-testing skill section (Phase 0.5)#909
aacruzgon wants to merge 1 commit into
mainfrom
cluster/p0.5-ch15

Conversation

@aacruzgon

Copy link
Copy Markdown
Contributor

Summary

Phase 0.5 of the cluster-capable-state rebuild (discussion #223): the operator chapter "Running HFS in a Cluster" (book ch. 15) and the cluster-testing section of the test-hfs skill. Documentation only.

Branched from main; the chapter describes the refusal and warning tables that #906 implements and the SIGTERM drain that #907 implements, so it reads best once those merge, but it does not conflict with them (different files).

Changes

  • book/src/ch15-cluster-deployment.md (new) + SUMMARY.md entry: prerequisites; the HFS_CLUSTER=true one-switch setup with the refusal table (SQLite primary, local-fs bulk output, bulk export on a MongoDB/S3 primary, file audit, explicit memory job store) and the two warnings ($sql-export per-instance job state, HFS_EXPORT_SINK=fs); a per-subsystem 1-vs-N matrix that states plainly what is safe today and what is still per-instance and planned; load-balancer notes (sticky sessions required today for async exports and Subscriptions, and why sticky is not sufficient for Subscriptions); how to verify a cluster with the smoke harness or by hand.
  • test-hfs skill (both .claude/skills and .agents/skills): the T1/T2/T3 tiers, the two-handle harness and its definition-of-done rows, the commands for the T2 suites that exist today, and how to dispatch the smoke workflow on a branch.

Testing

  • mdbook build book succeeds; ch15-cluster-deployment.html renders with the summary link.

Notes

  • The matrix is meant to be edited by every later cluster PR as its subsystem flips from planned to safe.
  • The frozen feat(cluster): make HFS cluster-capable behind a load balancer #269 chapter's architecture diagram (vendored mermaid) and its "landed" sections for later phases were deliberately not carried over; those return with the subsystems they describe.

There was no operator-facing description of what happens when N hfs
instances share a database: which subsystems are safe, which still need a
pinned instance, what HFS_CLUSTER refuses, and how a cluster is verified.

Book chapter 15 covers prerequisites, the one-switch setup with the refusal
and warning tables as implemented, a per-subsystem 1-vs-N matrix that names
what is safe today (CRUD with compare-and-swap version bumps, schema init,
bulk data on PostgreSQL, SIGTERM drain, JWT validation) and what is still
per-instance and planned (SQL-on-FHIR async export, reindex, subscriptions,
HTS caches and bootstrap, composite async sync), load-balancer notes
(sticky sessions required today for exports and subscriptions, and why
they are not sufficient for subscriptions), and the two-instance smoke
harness. The test-hfs skill gains the T1/T2/T3 cluster testing section in
both skill trees.

Tests: mdbook build succeeds and the chapter renders; no code changes.
@codecov

codecov Bot commented Sep 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant