Expose metrics and a health check - #64
Merged
Merged
Conversation
modbot had no way to answer "is it alive right now" other than typing in chat and waiting. Add /metrics and /healthz on :9090, a keepalive PING to drive connection state, and a container HEALTHCHECK so a wedged bot gets restarted instead of sitting there. Health is the websocket being up, not chat being busy. Gating on time since the last message would restart the bot for being quiet at 4am. SendPing fails when the socket is gone, so a ping write landing is the signal; the PONG is never read, which also sidesteps dggchat's parsePing base64-decoding the payload and listen() silently skipping the handler when that fails. An expired cookie is still caught: the server closes the socket, the read errors, onSocketError fires, modbot_connected drops and ws_reconnects climbs. Four series, deliberately: connected, last_message_received, ws_reconnects, command_persist_failures. Nothing labelled by nick -- unbounded cardinality, and it would make the metrics store a record of user behaviour. The scratch image has no shell or curl, so HEALTHCHECK runs the binary's own -healthcheck mode. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
docker records the check command's output in .State.Health.Log, so the reason is worth passing along -- "healthz 503 Service Unavailable: websocket down" beats a bare status code when you are reading docker inspect at 3am. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
modbot had no way to answer "is it alive right now" other than typing in chat and waiting. This adds
/metricsand/healthzon:9090, a keepalive PING to drive connection state, and a containerHEALTHCHECKso a wedged bot gets restarted instead of sitting there.Four metrics, deliberately:
modbot_connectedmodbot_last_message_received_timestamp_secondsmodbot_ws_reconnects_totalmodbot_command_persist_failures_total!addcommandacknowledged and then lostPlus
go_*/process_*free from client_golang. Nothing is labelled by nick — unbounded cardinality, and it would turn the metrics store into a record of user behaviour.Health is the socket being up, not chat being busy
Gating on time-since-last-message would restart the bot for being quiet at 4am.
SendPingfails when the socket is gone, so a ping write landing is the signal. The PONG is never read, which also sidestepsdggchat'sparsePingbase64-decoding the payload andlisten()silently skipping the handler when that fails.An expired cookie is still caught: the server closes the socket, the read errors,
onSocketErrorfires,modbot_connecteddrops andws_reconnectsclimbs.Notes
FROM scratch, soHEALTHCHECKruns the binary's own-healthcheckmode. Verified baked into the image as["CMD","/modbot","-healthcheck"].strimsdocker network, sohooks/redeployneeds no change.docker psstarts showing a health status. The collector stack is the next piece of work.modbot-cd.yamldeploys on push tomaster, so merging this ships it.