Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 31 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,9 @@ env:
# Image versions are NOT duplicated here. They are resolved from compose.yaml
# at run time by scripts/image-for.sh, because Dependabot only updates
# compose.yaml — hardcoded copies went stale silently and CI ended up
# validating v3.1.0 configs against a stack running v3.13.2.
GITLEAKS_IMAGE: zricethezav/gitleaks:v8.24.0
# validating v3.1.0 configs against a stack running v3.13.2. gitleaks was the
# last exception to that and is now a profile-gated service in compose.yaml
# like every other image (#65).

jobs:
# ---------------------------------------------------------------------------
Expand Down Expand Up @@ -77,9 +78,14 @@ jobs:

# Guard against the duplication coming back. Any image: pin outside
# compose.yaml is drift waiting to happen, since Dependabot cannot see it.
#
# This matched only prom/ and grafana/ until #65, which is how
# zricethezav/gitleaks sat pinned in this file's own env: block. Any
# repo/name:version fails now. The illustrative version in the header of
# scripts/pin-digests.sh is a comment and is filtered out below.
- name: Verify image versions are not duplicated outside compose.yaml
run: |
if grep -rnE '(prom|grafana)/[a-z-]+:v?[0-9]+\.[0-9]+' \
if grep -rnE '(^|[^a-zA-Z0-9._/-])[a-z0-9][a-z0-9._-]*/[a-z0-9][a-z0-9._-]*:v?[0-9]+\.[0-9]+' \
--include='*.sh' --include='*.yml' --include='Makefile' \
scripts .github Makefile 2>/dev/null \
| grep -vE '^[^:]+:[0-9]+:[[:space:]]*#' ; then
Expand Down Expand Up @@ -116,6 +122,19 @@ jobs:
done < <(awk '$1 == "image:" { print $2 }' "$STACK/compose.yaml")
exit "$missing"

# The three checks above are all pattern matches, and #65 walked past all
# three: `make backup` ran a bare `alpine` — no `:latest` literal, no
# prom/ or grafana/ prefix, and not an `image:` line in compose.yaml. A
# grep cannot match a pin that is simply absent. This parses every docker
# run/pull/create in the Makefile, the scripts, this workflow and the
# runbooks, and requires each one's image to resolve through
# scripts/image-for.sh — so an image that is not in compose.yaml cannot be
# run at all, whatever it is called. That is also what makes the digest
# check above complete: it reads compose.yaml alone, and compose.yaml is
# now the only place an image can come from.
- name: Verify every docker image comes from compose.yaml
run: python3 scripts/check_image_pins.py

- name: promtool check config
run: |
docker run --rm --entrypoint promtool \
Expand Down Expand Up @@ -239,6 +258,15 @@ jobs:
with:
fetch-depth: 0

# Same single source of truth as the validate job: whatever compose.yaml
# pins is what runs. Repeated here rather than shared because jobs do not
# share $GITHUB_ENV — it is the resolution that is duplicated, not the
# version.
- name: Resolve the pinned gitleaks image from compose.yaml
run: |
echo "GITLEAKS_IMAGE=$(./scripts/image-for.sh gitleaks)" >> "$GITHUB_ENV"
./scripts/image-for.sh gitleaks

# Two scans, because they answer different questions.
#
# The working-tree scan asks "is there a secret in the code as it stands"
Expand Down
4 changes: 4 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,10 @@ check-compose-health: ## Verify compose health dependencies can be satisfied
check-loki-rules: ## Validate Loki (LogQL) alerting rules
./scripts/check_loki_rules.sh

.PHONY: check-image-pins
check-image-pins: ## Verify every docker image comes from compose.yaml
python3 scripts/check_image_pins.py

.PHONY: pin-digests
pin-digests: ## Re-resolve image digests in compose.yaml (--write applies)
./scripts/pin-digests.sh --write
Expand Down
12 changes: 11 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,10 @@ incident.
the build.
- **Supply chain pinned by digest.** Every image carries both a tag and a
`sha256:` digest, so a moved tag cannot change what deploys. CI enforces it;
`make pin-digests` re-resolves them from the registry.
`make pin-digests` re-resolves them from the registry. Every `docker run` in
the Makefile, the scripts, the workflow and the runbooks resolves its image
from `compose.yaml` too, so an image that is not pinned there cannot be run
at all.
- **Documented decisions and runbooks.** Eleven ADRs covering what was chosen
and what was rejected — including the costs accepted knowingly; twelve
runbooks for the operations that are easy to get wrong at 1am.
Expand Down Expand Up @@ -251,6 +254,13 @@ Container images are pinned by **tag and digest**. A tag is a mutable pointer; a
digest is the content hash, so a moved tag cannot change what gets deployed. CI
enforces it, and `make pin-digests` re-resolves them.

`compose.yaml` is the only place an image may be named, including images no
service runs — the tar that takes backups and the scanner CI runs are both
profile-gated entries there. CI parses every `docker run`, `pull` and `create`
in the repository and requires each to resolve its image through
`scripts/image-for.sh`, because the pin that caused this rule was not a wrong
one but a missing one, and no amount of grepping finds those.

## Roadmap

Open work is tracked in
Expand Down
3 changes: 3 additions & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,9 @@ Every push and pull request runs:
`.purge-secrets.txt` — is ever a tracked file.
- Verification that every container image is pinned by **tag *and* digest**, so a
moved tag cannot silently change what is deployed.
- Verification that every image any script, recipe, workflow step or runbook
runs is resolved from `compose.yaml`, so a container cannot be started from an
image the digest check never saw.

See [`docs/security.md`](docs/security.md) for the threat model and segmentation
rationale.
10 changes: 9 additions & 1 deletion docs/runbooks/add-monitored-device.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ sudo docker run -d \
-v /var/log:/var/log:ro \
-v /:/rootfs:ro \
-p 127.0.0.1:12345:12345 \
grafana/alloy:v1.18.1 \
"$(/path/to/HomeLab/scripts/image-for.sh alloy)" \
run --server.http.listen-addr=0.0.0.0:12345 \
--storage.path=/var/lib/alloy/data \
/etc/alloy/config.alloy
Expand All @@ -35,6 +35,14 @@ sudo docker run -d \
The two `*_URL` variables are the only difference from the monitoring host's own
agent — inside the compose stack they default to service names.

The image comes from `compose.yaml` rather than being written out here, so a new
host starts on the same Alloy — tag *and* digest — that the monitoring host runs,
and keeps doing so after Dependabot bumps it. A version copied into this runbook
would be stale from the next bump onward, which is the whole argument in the
header of `scripts/image-for.sh`. If the repository is not on the new host, run
`./scripts/image-for.sh alloy` on the monitoring host and paste the reference it
prints.

`--hostname` is not optional. Alloy labels everything it produces with
`constants.hostname`, which inside a container is the **container ID** unless one
is set — a hex string that identifies nothing and changes every time the
Expand Down
Loading
Loading