Have /health report the services behind the site - #302
Merged
Merged
Conversation
Every deployment fault on this box has had one shape: something merged,
something else kept running the version from before it, and nothing said so.
Three times in a day, the last one a live 404 -- an nginx route added to
content-node while content-node still ran the image built before those
endpoints existed.
Each service already answered its own health check. Nobody polls three URLs, so
the drift was only ever found by someone tripping over it. `/health` now asks
them and reports what each one *is*: the build it is running, the release it
holds, how many endpoints it serves. Two services disagreeing about the release
is the thing worth seeing, and it is invisible when all you have is a green
tick each.
"services": {
"content-node": { "up": true, "build": "8e87ba1d", "release": "97", "endpoints": 12 },
"render": { "up": true, "build": "d4b04cde" },
"mcp": { "up": true, "build": "1.4.0" }
}
`ok` stays the *site's* answer and not the estate's. The site renders without
the render service, so a monitor asking "is the website up" must not be told no
because a sibling is down -- that pages somebody for the wrong thing. What is
wrong is named in `services`.
Asked in parallel with a two-second budget, and a service that is down is a
fact rather than an error: measured at 11ms with everything up, and 11ms with
mcp stopped.
The addresses are overridable rather than hardcoded. That is what makes the
case worth testing testable -- on this box those services are genuinely
running, so a test asserting "down" would have been green in CI and red
locally, for a reason unrelated to the code. The spec points them at a closed
port and asserts each reports its own state while the site stays ok.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every deployment fault on this box has had one shape: something merged, something else kept running the version from before it, and nothing said so. Three times in one day, the last a live 404 — an nginx route added to
content-nodewhilecontent-nodestill ran the image built before those endpoints existed.Each service already answered its own health check. Nobody polls three URLs, so the drift was only ever found by someone tripping over it.
What it reports now
Not just that each answered — what each one is. Two services disagreeing about the release is the thing worth seeing, and it is invisible when all you have is a green tick each.
endpoints: 12is exactly what would have shown yesterday's stale content-node at a glance: it was serving 10.okstays the site's answerThe site renders without the render service. A monitor asking "is the website up" must not be told no because a sibling is down — that pages somebody for the wrong thing. What is wrong is named in
services.Cost
Asked in parallel with a 2s budget. A service being down is a fact, not an error: 11ms with everything up, 11ms with mcp stopped (measured by stopping the container).
The addresses are overridable, and that is what made it testable
On this box those services are genuinely running, so a test asserting "down" would have been green in CI and red locally, for a reason unrelated to the code. The spec points them at a closed port and asserts each reports its own state while the site stays
ok.457 tests green; lint 652 and dead-code 145 unchanged from baseline.
🤖 Generated with Claude Code