From df36fc8dc47a7b46b2a4e944271394b117fe37fb Mon Sep 17 00:00:00 2001 From: Derek Date: Sat, 29 Aug 2026 10:58:42 +1000 Subject: [PATCH] fix: install git-scrub with the contributor toolchain We use git-scrub to rewrite AI residue and leaked secrets out of git history, but nothing in the installer put it on a box -- everyone was fetching it by hand. It goes in contributor rather than soe because it is what clears a gitleaks failure, and gitleaks scans full history: a secret removed from HEAD still fails the gate. soe inherits it through meta/dependencies. Snipeable on its own with --tags git-scrub. GitHub release tarball on every platform. That is the only rung going -- it is not on crates.io, downloads.hyperi.io serves no git-scrub path, and the release ships a git-scrub.rb that is not in our tap yet, so macOS takes the darwin asset until someone taps it. The tarball is kept rather than deleted, and that is what makes the role idempotent: get_url re-reports ok for an unchanged asset so the extract and install stay skipped on a converged box. We cannot ask the binary its version instead -- v1.0.2 reports itself as "git-scrub 1.0.1". --- README.md | 4 +- ansible/roles/contributor/tasks/git_scrub.yml | 110 ++++++++++++++++++ ansible/roles/contributor/tasks/main.yml | 7 ++ ansible/roles/contributor/tasks/verify.yml | 11 ++ docs/install-matrix.md | 10 +- 5 files changed, 137 insertions(+), 5 deletions(-) create mode 100644 ansible/roles/contributor/tasks/git_scrub.yml diff --git a/README.md b/README.md index 8a2fb9b..3d60e8d 100644 --- a/README.md +++ b/README.md @@ -78,7 +78,7 @@ flowchart TD | `developer-gui` | VS Code, Ghostty, DBeaver. Privacy + AI-upsell de-nag profile for VSCode/VSCodium/Cursor off unless `-e vscode_privacy_enabled=true` | | `developer-rust` / `-go` / `-python` / `-node` / `-typescript` / `-c` | Language toolchains | | `infrastructure` | OpenTofu, OpenBao, AWS CLI, helm, terraform-docs, `k8s` (kubectl, kubectx, kubens, k9s, kind, argocd, dive, kustomize, kubeconform, kube-linter), `data` (clickhouse-client, rpk, valkey-cli, vector), `cloudflare` (flarectl, wrangler) | -| `contributor` | hyperi-ci + its check tools (semgrep, alint), gitleaks, trivy, hadolint, pip-audit, yamllint, ansible-lint, pre-commit, act | +| `contributor` | hyperi-ci + its check tools (semgrep, alint), gitleaks, trivy, hadolint, pip-audit, yamllint, ansible-lint, pre-commit, act, git-scrub | | `soe` / `soe-gui` | HyperI org policy (opt-in) | | `--full-stack` / `--infra` / `--languages [list]` | Persona bundles (see `--help`) | | `winlike` / `maclike` | GNOME taskbar (winlike) or dock (maclike), winlike wins if both | @@ -172,7 +172,7 @@ reports success -- the settings simply land where nobody sees them. - `vscode-privacy` (off by default): strips the Copilot/AI upsell UI and the telemetry that stock VSCode ships enabled, across VSCode, VSCodium and Cursor. Enable with `-e vscode_privacy_enabled=true`. It merges one marked block into `settings.json` and never touches a comment or a key it does not manage, backs the file up before its first write, and `-e vscode_privacy_uninstall=true` takes only its own keys back out. Where you have set one of those keys yourself further down the file, yours wins and the run tells you which ones -- so it cannot look applied while changing nothing - Languages: Rust, Go, Python, C/C++, Node.js, TypeScript (the Astral suite -- uv, ruff, ty -- ships in the base, as does Node.js: it is core tooling that semantic-release and CI need) - `infrastructure`: OpenTofu + OpenBao (the OSS forks, no HashiCorp BUSL tools), AWS CLI v2, checkov, and terraform-docs for generating IaC module reference docs (engine-agnostic -- it reads `.tf` whichever binary runs it, and OpenTofu has no native `tofu docs`). Under `k8s`: kubectl + kubectx + kubens + k9s + kind + argocd + dive + kustomize + kubeconform + kube-linter. helm is NOT in that group -- it sits in `cloud`, so plain `--tags infrastructure` gets it whether or not you select `k8s`. The `data` group: clickhouse-client, rpk, valkey-cli, vector. The `cloudflare` group: flarectl + wrangler (flarectl builds from source on both platforms -- Cloudflare ships no binary -- so Linux needs `developer-go`) -- `contributor`: hyperi-ci and the tools its checks drive (semgrep, alint), gitleaks, trivy, hadolint, pip-audit, ansible-lint, pre-commit, act +- `contributor`: hyperi-ci and the tools its checks drive (semgrep, alint), gitleaks, trivy, hadolint, pip-audit, ansible-lint, pre-commit, act, and git-scrub for rewriting AI residue or a leaked secret out of git history -- gitleaks scans FULL history, so a secret removed from HEAD still fails the gate - `soe` / `soe-gui`: HyperI org policy: VPN clients, Claude Code, Slack, LibreOffice, RDP client, telemetry-disable, auto-updates, GNOME taskbar - `power-profile` (off by default, and deliberately not in `soe`): sleep, idle and lid policy, selected per machine. `always-on` (the default profile) never idle-suspends on mains power and does not sleep when the lid shuts -- for a repurposed laptop doing build work, or a desktop that has to answer ssh. `vm` never sleeps or suspends at all, for an unattended RDP guest that nobody can walk over and wake. Battery behaviour stays stock under `always-on`, because a machine that will not sleep in a bag cooks itself. Profiles are data files, so adding one is adding a file -- see [roles/power-profile/README.md](ansible/roles/power-profile/README.md) - `arcane` (off by default): [Arcane](https://getarcane.app), a web UI for the containers on the box. Enable it with `-e soe_arcane_enabled=true` and you get a daemon on `http://localhost:3552` that comes back after a reboot and keeps itself updated. Works against docker-ce on Linux and colima on macOS. Bound to loopback because it holds the Docker socket, so whatever reaches that port owns the machine. Login is whatever Arcane seeds -- `arcane` / `arcane-admin` as upstream documents it. The role sets neither, and only clears the forced first-login password prompt, which it does by re-submitting that seeded password so the credentials stay unchanged. That needs the password policy relaxed to `basic` (`soe_arcane_password_policy`), because upstream's default `strong` policy rejects its own seeded password. There is still a login -- auto-login sits behind a `buildables` Go build tag that no published image is compiled with, so zero-auth is not available without building your own image diff --git a/ansible/roles/contributor/tasks/git_scrub.yml b/ansible/roles/contributor/tasks/git_scrub.yml new file mode 100644 index 0000000..7b21b3d --- /dev/null +++ b/ansible/roles/contributor/tasks/git_scrub.yml @@ -0,0 +1,110 @@ +--- +# git-scrub - rewrites a git repository's whole history to strip AI coding-agent +# residue (artefact files, `Co-Authored-By` trailers, agent-credited identities) +# and redact secret/PII shapes out of blobs. +# +# In `contributor` rather than `soe` because it is what clears a gitleaks +# failure: gitleaks scans FULL history, so a secret removed from HEAD still +# fails `hyperi-ci check`. soe inherits it through meta/dependencies. +# +# GitHub release tarball on every platform (Tier 3 -- hyperi-update pulls the +# latest on each run). It is the only rung: not on crates.io, no git-scrub path +# on downloads.hyperi.io, and the release's git-scrub.rb is not in the hyperi-io +# tap. Move macOS to community.general.homebrew once that formula is tapped. +# +# The asset unpacks into a directory named after itself, so the extracted binary +# path carries the version. The tag has a leading `v`; the filename does not. + +- name: Install git-scrub (re-fetched GitHub release, Tier 3) + block: + # --pinned takes the CI-exact tag from group_vars, latest otherwise, so a + # pinned install never depends on the GitHub API. + # + # hyperi-ci carries no git-scrub pin, so --pinned falls through to latest. + # Adding an entry here without the matching hyperi-ci pin turns the build + # red via check_version_pins.py. + - name: Get latest git-scrub version from GitHub API + ansible.builtin.uri: + url: https://api.github.com/repos/hyperi-io/git-scrub/releases/latest + return_content: true + headers: "{{ hyperi_github_headers }}" + register: contributor_git_scrub_release + check_mode: false + when: not (hyperi_pinned | default(false) and 'git-scrub' in (hyperi_versions | default({}))) + + - name: Resolve the git-scrub tag (pinned or latest) + ansible.builtin.set_fact: + contributor_git_scrub_ref: >- + {{ hyperi_versions['git-scrub'] + if (hyperi_pinned | default(false) and 'git-scrub' in (hyperi_versions | default({}))) + else contributor_git_scrub_release.json.tag_name }} + + # hyperi_arch_deb already spells the architecture the way this release does. + - name: Build the git-scrub asset name + ansible.builtin.set_fact: + contributor_git_scrub_stem: >- + git-scrub-{{ contributor_git_scrub_ref | regex_replace('^v', '') }}-{{ + 'darwin' if ansible_facts['distribution'] == 'MacOSX' else 'linux' }}-{{ hyperi_arch_deb }} + + # The tarball is KEPT, and that is what makes the role idempotent: get_url + # re-reports ok for an unchanged asset, so the extract and install below + # stay skipped on a converged box. Deleting it would re-download every run + # and report changed forever. + # + # The upstream binary cannot be asked instead -- v1.0.2 reports itself as + # "git-scrub 1.0.1", so a version comparison would reinstall on every run. + - name: Download the git-scrub tarball + ansible.builtin.get_url: + url: >- + https://github.com/hyperi-io/git-scrub/releases/download/{{ contributor_git_scrub_ref }}/{{ contributor_git_scrub_stem }}.tar.gz + dest: "/tmp/{{ contributor_git_scrub_stem }}.tar.gz" + mode: '0644' + register: contributor_git_scrub_download + + - name: Check whether git-scrub is already installed + ansible.builtin.stat: + path: /usr/local/bin/git-scrub + register: contributor_git_scrub_installed + + # Root-owned on Linux, absent on a fresh Apple Silicon box. + - name: Ensure /usr/local/bin exists + ansible.builtin.file: + path: /usr/local/bin + state: directory + mode: '0755' + become: true + + # A new asset OR a missing binary: the second repairs drift on a box where + # the tarball is still cached but the binary was removed. + - name: Install git-scrub + when: + - not ansible_check_mode + - contributor_git_scrub_download.changed or not contributor_git_scrub_installed.stat.exists + block: + - name: Extract the git-scrub tarball + ansible.builtin.unarchive: + src: "/tmp/{{ contributor_git_scrub_stem }}.tar.gz" + dest: /tmp + remote_src: true + + - name: Install the git-scrub binary + ansible.builtin.copy: + src: "/tmp/{{ contributor_git_scrub_stem }}/git-scrub" + dest: /usr/local/bin/git-scrub + mode: '0755' + remote_src: true + become: true + + - name: Remove the extracted git-scrub directory + ansible.builtin.file: + path: "/tmp/{{ contributor_git_scrub_stem }}" + state: absent + + rescue: + - name: Record that git-scrub did not install + # noqa: var-naming[no-role-prefix] -- deploy_warnings is the shared + # accumulator reported by playbooks/main.yml post_tasks. + ansible.builtin.set_fact: + deploy_warnings: >- + {{ deploy_warnings | default([]) + + ['git-scrub: ' ~ (ansible_failed_result.msg | default('download failed'))] }} diff --git a/ansible/roles/contributor/tasks/main.yml b/ansible/roles/contributor/tasks/main.yml index 3c877b1..7922aae 100644 --- a/ansible/roles/contributor/tasks/main.yml +++ b/ansible/roles/contributor/tasks/main.yml @@ -120,6 +120,13 @@ tags: ['contributor', 'maid'] tags: ['contributor', 'maid'] +- name: Install git-scrub (git-history scrubber) + ansible.builtin.include_tasks: + file: git_scrub.yml + apply: + tags: ['contributor', 'git-scrub'] + tags: ['contributor', 'git-scrub'] + - name: Verify the contributor toolchain ansible.builtin.include_tasks: file: verify.yml diff --git a/ansible/roles/contributor/tasks/verify.yml b/ansible/roles/contributor/tasks/verify.yml index 63d19a9..72cb088 100644 --- a/ansible/roles/contributor/tasks/verify.yml +++ b/ansible/roles/contributor/tasks/verify.yml @@ -41,6 +41,16 @@ become: "{{ ansible_facts['distribution'] != 'MacOSX' }}" become_user: "{{ actual_user if ansible_facts['distribution'] != 'MacOSX' else omit }}" +# Non-blocking: nothing in `hyperi-ci check` drives it, and its install already +# records a warning on failure. +- name: Check git-scrub + ansible.builtin.command: git-scrub --version + environment: + PATH: "/opt/homebrew/bin:/usr/local/bin:{{ user_home }}/.local/bin:{{ ansible_facts['env'].PATH }}" + register: verify_git_scrub + changed_when: false + failed_when: false + - name: Display contributor tools verification ansible.builtin.debug: msg: | @@ -50,3 +60,4 @@ {% for r in verify_ci_tools.results %} - {{ r.item }}: {{ r.stdout_lines[0] | default('NOT AVAILABLE') if r.rc == 0 else 'NOT AVAILABLE -- hyperi-ci check will skip its pass' }} {% endfor %} + - git-scrub: {{ verify_git_scrub.stdout | default('NOT AVAILABLE') if verify_git_scrub.rc == 0 else 'NOT AVAILABLE' }} diff --git a/docs/install-matrix.md b/docs/install-matrix.md index 2c306c1..562ea6c 100644 --- a/docs/install-matrix.md +++ b/docs/install-matrix.md @@ -291,9 +291,12 @@ The base ships the Astral suite (uv, ruff, ty) and `uv` bundles `uv audit` / | wrangler (the `cloudflare` group) | all | npm-global / brew | version | | flarectl (the `cloudflare` group) | all | `go install` from source / brew | source tag | -Every macOS path resolves to brew or a cask. The language managers that remain -there carry no formula at all: `alint` and `maid` have none, and semantic-release -needs its plugin set installed alongside it, which only npm gives. +Almost every macOS path resolves to brew or a cask. The language managers that +remain there carry no formula at all: `alint` and `maid` have none, and +semantic-release needs its plugin set installed alongside it, which only npm +gives. `git-scrub` is the one release-tarball exception -- its formula exists in +the release but is not in the hyperi-io tap, so macOS takes the darwin asset +until it is tapped. Cloudflare publishes no flarectl binary and no distro packages it, so both platforms build it from source. It also lives on cloudflare-go's `v0` branch -- @@ -321,6 +324,7 @@ warning and continues. | vulture | all | Ubuntu apt / Fedora uv-tool (Tier 2) / brew | version | | typos | all | cargo (Tier 2) / brew | version | | maid (mermaid validator, used by `/docs`) | all | npm global (Tier 2) | n/a | +| git-scrub (git-history scrubber) | all | github-binary (Tier 3: re-fetch) | version | `hyperi-ci` is a Python tool from PyPI, installed via `uv tool` and refreshed to the latest release on every run (upgrade-if-present, not install-once). soe