diff --git a/README.md b/README.md index 8a2fb9b..09ce0cd 100644 --- a/README.md +++ b/README.md @@ -78,7 +78,7 @@ flowchart TD | `developer-gui` | VS Code, Ghostty, DBeaver. Privacy + AI-upsell de-nag profile for VSCode/VSCodium/Cursor off unless `-e vscode_privacy_enabled=true` | | `developer-rust` / `-go` / `-python` / `-node` / `-typescript` / `-c` | Language toolchains | | `infrastructure` | OpenTofu, OpenBao, AWS CLI, helm, terraform-docs, `k8s` (kubectl, kubectx, kubens, k9s, kind, argocd, dive, kustomize, kubeconform, kube-linter), `data` (clickhouse-client, rpk, valkey-cli, vector), `cloudflare` (flarectl, wrangler) | -| `contributor` | hyperi-ci + its check tools (semgrep, alint), gitleaks, trivy, hadolint, pip-audit, yamllint, ansible-lint, pre-commit, act | +| `contributor` | hyperi-ci + its check tools (semgrep, alint), gitleaks, trivy, hadolint, pip-audit, yamllint, ansible-lint, pre-commit, act, git-scrub, macbash | | `soe` / `soe-gui` | HyperI org policy (opt-in) | | `--full-stack` / `--infra` / `--languages [list]` | Persona bundles (see `--help`) | | `winlike` / `maclike` | GNOME taskbar (winlike) or dock (maclike), winlike wins if both | @@ -172,7 +172,7 @@ reports success -- the settings simply land where nobody sees them. - `vscode-privacy` (off by default): strips the Copilot/AI upsell UI and the telemetry that stock VSCode ships enabled, across VSCode, VSCodium and Cursor. Enable with `-e vscode_privacy_enabled=true`. It merges one marked block into `settings.json` and never touches a comment or a key it does not manage, backs the file up before its first write, and `-e vscode_privacy_uninstall=true` takes only its own keys back out. Where you have set one of those keys yourself further down the file, yours wins and the run tells you which ones -- so it cannot look applied while changing nothing - Languages: Rust, Go, Python, C/C++, Node.js, TypeScript (the Astral suite -- uv, ruff, ty -- ships in the base, as does Node.js: it is core tooling that semantic-release and CI need) - `infrastructure`: OpenTofu + OpenBao (the OSS forks, no HashiCorp BUSL tools), AWS CLI v2, checkov, and terraform-docs for generating IaC module reference docs (engine-agnostic -- it reads `.tf` whichever binary runs it, and OpenTofu has no native `tofu docs`). Under `k8s`: kubectl + kubectx + kubens + k9s + kind + argocd + dive + kustomize + kubeconform + kube-linter. helm is NOT in that group -- it sits in `cloud`, so plain `--tags infrastructure` gets it whether or not you select `k8s`. The `data` group: clickhouse-client, rpk, valkey-cli, vector. The `cloudflare` group: flarectl + wrangler (flarectl builds from source on both platforms -- Cloudflare ships no binary -- so Linux needs `developer-go`) -- `contributor`: hyperi-ci and the tools its checks drive (semgrep, alint), gitleaks, trivy, hadolint, pip-audit, ansible-lint, pre-commit, act +- `contributor`: hyperi-ci and the tools its checks drive (semgrep, alint), gitleaks, trivy, hadolint, pip-audit, ansible-lint, pre-commit, act, and git-scrub for rewriting AI residue or a leaked secret out of git history -- gitleaks scans FULL history, so a secret removed from HEAD still fails the gate. macbash is here too: it flags the GNU-only bash constructs that break on macOS, and CONTRIBUTING.md asks for it on every shell change - `soe` / `soe-gui`: HyperI org policy: VPN clients, Claude Code, Slack, LibreOffice, RDP client, telemetry-disable, auto-updates, GNOME taskbar - `power-profile` (off by default, and deliberately not in `soe`): sleep, idle and lid policy, selected per machine. `always-on` (the default profile) never idle-suspends on mains power and does not sleep when the lid shuts -- for a repurposed laptop doing build work, or a desktop that has to answer ssh. `vm` never sleeps or suspends at all, for an unattended RDP guest that nobody can walk over and wake. Battery behaviour stays stock under `always-on`, because a machine that will not sleep in a bag cooks itself. Profiles are data files, so adding one is adding a file -- see [roles/power-profile/README.md](ansible/roles/power-profile/README.md) - `arcane` (off by default): [Arcane](https://getarcane.app), a web UI for the containers on the box. Enable it with `-e soe_arcane_enabled=true` and you get a daemon on `http://localhost:3552` that comes back after a reboot and keeps itself updated. Works against docker-ce on Linux and colima on macOS. Bound to loopback because it holds the Docker socket, so whatever reaches that port owns the machine. Login is whatever Arcane seeds -- `arcane` / `arcane-admin` as upstream documents it. The role sets neither, and only clears the forced first-login password prompt, which it does by re-submitting that seeded password so the credentials stay unchanged. That needs the password policy relaxed to `basic` (`soe_arcane_password_policy`), because upstream's default `strong` policy rejects its own seeded password. There is still a login -- auto-login sits behind a `buildables` Go build tag that no published image is compiled with, so zero-auth is not available without building your own image diff --git a/ansible/inventories/localhost/group_vars/all.yml b/ansible/inventories/localhost/group_vars/all.yml index c0d6c77..124dc69 100644 --- a/ansible/inventories/localhost/group_vars/all.yml +++ b/ansible/inventories/localhost/group_vars/all.yml @@ -34,6 +34,12 @@ hyperi_github_headers: >- {{ {'Authorization': 'Bearer ' ~ hyperi_github_token} if hyperi_github_token else {} }} +# The same token as a process environment, for tasks shelling out to a script +# that does its own release lookups. An exported token does not survive `become` +# to a different uid, so a task `environment:` is the only way it reaches them. +hyperi_github_env: >- + {{ {'GITHUB_TOKEN': hyperi_github_token} if hyperi_github_token else {} }} + hyperi_versions: alint: v0.14.1 gitleaks: v8.30.1 diff --git a/ansible/roles/contributor/tasks/git_scrub.yml b/ansible/roles/contributor/tasks/git_scrub.yml new file mode 100644 index 0000000..7b21b3d --- /dev/null +++ b/ansible/roles/contributor/tasks/git_scrub.yml @@ -0,0 +1,110 @@ +--- +# git-scrub - rewrites a git repository's whole history to strip AI coding-agent +# residue (artefact files, `Co-Authored-By` trailers, agent-credited identities) +# and redact secret/PII shapes out of blobs. +# +# In `contributor` rather than `soe` because it is what clears a gitleaks +# failure: gitleaks scans FULL history, so a secret removed from HEAD still +# fails `hyperi-ci check`. soe inherits it through meta/dependencies. +# +# GitHub release tarball on every platform (Tier 3 -- hyperi-update pulls the +# latest on each run). It is the only rung: not on crates.io, no git-scrub path +# on downloads.hyperi.io, and the release's git-scrub.rb is not in the hyperi-io +# tap. Move macOS to community.general.homebrew once that formula is tapped. +# +# The asset unpacks into a directory named after itself, so the extracted binary +# path carries the version. The tag has a leading `v`; the filename does not. + +- name: Install git-scrub (re-fetched GitHub release, Tier 3) + block: + # --pinned takes the CI-exact tag from group_vars, latest otherwise, so a + # pinned install never depends on the GitHub API. + # + # hyperi-ci carries no git-scrub pin, so --pinned falls through to latest. + # Adding an entry here without the matching hyperi-ci pin turns the build + # red via check_version_pins.py. + - name: Get latest git-scrub version from GitHub API + ansible.builtin.uri: + url: https://api.github.com/repos/hyperi-io/git-scrub/releases/latest + return_content: true + headers: "{{ hyperi_github_headers }}" + register: contributor_git_scrub_release + check_mode: false + when: not (hyperi_pinned | default(false) and 'git-scrub' in (hyperi_versions | default({}))) + + - name: Resolve the git-scrub tag (pinned or latest) + ansible.builtin.set_fact: + contributor_git_scrub_ref: >- + {{ hyperi_versions['git-scrub'] + if (hyperi_pinned | default(false) and 'git-scrub' in (hyperi_versions | default({}))) + else contributor_git_scrub_release.json.tag_name }} + + # hyperi_arch_deb already spells the architecture the way this release does. + - name: Build the git-scrub asset name + ansible.builtin.set_fact: + contributor_git_scrub_stem: >- + git-scrub-{{ contributor_git_scrub_ref | regex_replace('^v', '') }}-{{ + 'darwin' if ansible_facts['distribution'] == 'MacOSX' else 'linux' }}-{{ hyperi_arch_deb }} + + # The tarball is KEPT, and that is what makes the role idempotent: get_url + # re-reports ok for an unchanged asset, so the extract and install below + # stay skipped on a converged box. Deleting it would re-download every run + # and report changed forever. + # + # The upstream binary cannot be asked instead -- v1.0.2 reports itself as + # "git-scrub 1.0.1", so a version comparison would reinstall on every run. + - name: Download the git-scrub tarball + ansible.builtin.get_url: + url: >- + https://github.com/hyperi-io/git-scrub/releases/download/{{ contributor_git_scrub_ref }}/{{ contributor_git_scrub_stem }}.tar.gz + dest: "/tmp/{{ contributor_git_scrub_stem }}.tar.gz" + mode: '0644' + register: contributor_git_scrub_download + + - name: Check whether git-scrub is already installed + ansible.builtin.stat: + path: /usr/local/bin/git-scrub + register: contributor_git_scrub_installed + + # Root-owned on Linux, absent on a fresh Apple Silicon box. + - name: Ensure /usr/local/bin exists + ansible.builtin.file: + path: /usr/local/bin + state: directory + mode: '0755' + become: true + + # A new asset OR a missing binary: the second repairs drift on a box where + # the tarball is still cached but the binary was removed. + - name: Install git-scrub + when: + - not ansible_check_mode + - contributor_git_scrub_download.changed or not contributor_git_scrub_installed.stat.exists + block: + - name: Extract the git-scrub tarball + ansible.builtin.unarchive: + src: "/tmp/{{ contributor_git_scrub_stem }}.tar.gz" + dest: /tmp + remote_src: true + + - name: Install the git-scrub binary + ansible.builtin.copy: + src: "/tmp/{{ contributor_git_scrub_stem }}/git-scrub" + dest: /usr/local/bin/git-scrub + mode: '0755' + remote_src: true + become: true + + - name: Remove the extracted git-scrub directory + ansible.builtin.file: + path: "/tmp/{{ contributor_git_scrub_stem }}" + state: absent + + rescue: + - name: Record that git-scrub did not install + # noqa: var-naming[no-role-prefix] -- deploy_warnings is the shared + # accumulator reported by playbooks/main.yml post_tasks. + ansible.builtin.set_fact: + deploy_warnings: >- + {{ deploy_warnings | default([]) + + ['git-scrub: ' ~ (ansible_failed_result.msg | default('download failed'))] }} diff --git a/ansible/roles/contributor/tasks/hyperi_ci.yml b/ansible/roles/contributor/tasks/hyperi_ci.yml index 85150ac..66ede3f 100644 --- a/ansible/roles/contributor/tasks/hyperi_ci.yml +++ b/ansible/roles/contributor/tasks/hyperi_ci.yml @@ -116,7 +116,12 @@ become: "{{ ansible_facts['distribution'] != 'MacOSX' }}" become_user: "{{ actual_user if ansible_facts['distribution'] != 'MacOSX' else omit }}" environment: - PATH: "{{ user_home }}/.cargo/bin:/opt/homebrew/bin:/usr/local/bin:{{ ansible_facts['env'].PATH }}" + # /opt/homebrew/opt/rustup/bin: brew's rustup never links its shims into + # /opt/homebrew/bin, so without it no cargo resolves on a rustup Mac. This + # task has no distribution gate, so it runs there too. + PATH: >- + {{ user_home }}/.cargo/bin:/opt/homebrew/opt/rustup/bin:/opt/homebrew/bin:/usr/local/bin:{{ + ansible_facts['env'].PATH }} register: contributor_alint changed_when: "'Installed package' in contributor_alint.stdout | default('')" failed_when: false diff --git a/ansible/roles/contributor/tasks/macbash.yml b/ansible/roles/contributor/tasks/macbash.yml new file mode 100644 index 0000000..4032af7 --- /dev/null +++ b/ansible/roles/contributor/tasks/macbash.yml @@ -0,0 +1,83 @@ +--- +# macbash - flags GNU/Linux-only bash constructs that break on macOS (BSD), and +# fixes many in place. +# +# In `contributor` because CONTRIBUTING.md requires it for every shell change in +# this repo, so an outside contributor needs it to pass review. soe inherits it +# through meta/dependencies. +# +# Linux takes the bare binary from downloads.hyperi.io rather than the .deb/.rpm +# beside it, because those carry the version in their filename and would need a +# release lookup that this path avoids entirely. +# +# Upstream publishes a .sha256 next to each asset, so this fetch verifies a +# digest instead of trusting the transport. + +- name: Install macbash (macOS) + when: ansible_facts['distribution'] == 'MacOSX' + block: + - name: Tap the hyperi-io formulae (macOS) + community.general.homebrew_tap: + name: hyperi-io/tap + state: present + become: false + environment: "{{ homebrew_env }}" + + - name: Install macbash via Homebrew (macOS) + community.general.homebrew: + name: macbash + state: present + become: false + environment: "{{ homebrew_env }}" + + rescue: + - name: Record that macbash did not install (macOS) + # noqa: var-naming[no-role-prefix] -- deploy_warnings is the shared + # accumulator reported by playbooks/main.yml post_tasks. + ansible.builtin.set_fact: + deploy_warnings: >- + {{ deploy_warnings | default([]) + + ['macbash: ' ~ (ansible_failed_result.msg | default('brew install failed'))] }} + +- name: Install macbash (Linux) + when: ansible_facts['distribution'] in ['Ubuntu', 'Fedora'] + block: + - name: Build the macbash asset name + ansible.builtin.set_fact: + contributor_macbash_asset: "macbash-linux-{{ hyperi_arch_deb }}" + + - name: Fetch the published macbash checksum + ansible.builtin.uri: + url: https://downloads.hyperi.io/macbash/latest/{{ contributor_macbash_asset }}.sha256 + return_content: true + register: contributor_macbash_sha_file + check_mode: false + + # The file lists the bare binary and the tarball, so anchor on the exact + # asset name or the tarball's digest is picked up instead. + - name: Resolve the macbash checksum + ansible.builtin.set_fact: + contributor_macbash_sha: >- + {{ (contributor_macbash_sha_file.content.splitlines() + | select('match', '^[0-9a-f]+ +' ~ contributor_macbash_asset ~ '$') + | first).split() | first }} + + # The digest comparison is what makes this idempotent: the asset name carries + # no version, so nothing else would notice a republished binary. + - name: Install the macbash binary (Linux) + ansible.builtin.get_url: + url: https://downloads.hyperi.io/macbash/latest/{{ contributor_macbash_asset }} + dest: /usr/local/bin/macbash + checksum: "sha256:{{ contributor_macbash_sha }}" + owner: root + group: root + mode: '0755' + become: true + + rescue: + - name: Record that macbash did not install (Linux) + # noqa: var-naming[no-role-prefix] -- see the note above. + ansible.builtin.set_fact: + deploy_warnings: >- + {{ deploy_warnings | default([]) + + ['macbash: ' ~ (ansible_failed_result.msg | default('download failed'))] }} diff --git a/ansible/roles/contributor/tasks/main.yml b/ansible/roles/contributor/tasks/main.yml index 3c877b1..8e847b9 100644 --- a/ansible/roles/contributor/tasks/main.yml +++ b/ansible/roles/contributor/tasks/main.yml @@ -120,6 +120,20 @@ tags: ['contributor', 'maid'] tags: ['contributor', 'maid'] +- name: Install git-scrub (git-history scrubber) + ansible.builtin.include_tasks: + file: git_scrub.yml + apply: + tags: ['contributor', 'git-scrub'] + tags: ['contributor', 'git-scrub'] + +- name: Install macbash (macOS bash portability checker) + ansible.builtin.include_tasks: + file: macbash.yml + apply: + tags: ['contributor', 'macbash'] + tags: ['contributor', 'macbash'] + - name: Verify the contributor toolchain ansible.builtin.include_tasks: file: verify.yml diff --git a/ansible/roles/contributor/tasks/verify.yml b/ansible/roles/contributor/tasks/verify.yml index 63d19a9..fd23f4d 100644 --- a/ansible/roles/contributor/tasks/verify.yml +++ b/ansible/roles/contributor/tasks/verify.yml @@ -41,6 +41,19 @@ become: "{{ ansible_facts['distribution'] != 'MacOSX' }}" become_user: "{{ actual_user if ansible_facts['distribution'] != 'MacOSX' else omit }}" +# Non-blocking: `hyperi-ci check` drives neither, and both installs already +# record a warning on failure. +- name: Check the HyperI developer tools + ansible.builtin.command: "{{ item }} --version" + environment: + PATH: "/opt/homebrew/bin:/usr/local/bin:{{ user_home }}/.local/bin:{{ ansible_facts['env'].PATH }}" + loop: + - git-scrub + - macbash + register: verify_hyperi_tools + changed_when: false + failed_when: false + - name: Display contributor tools verification ansible.builtin.debug: msg: | @@ -50,3 +63,6 @@ {% for r in verify_ci_tools.results %} - {{ r.item }}: {{ r.stdout_lines[0] | default('NOT AVAILABLE') if r.rc == 0 else 'NOT AVAILABLE -- hyperi-ci check will skip its pass' }} {% endfor %} + {% for r in verify_hyperi_tools.results %} + - {{ r.item }}: {{ r.stdout_lines[0] | default('NOT AVAILABLE') if r.rc == 0 else 'NOT AVAILABLE' }} + {% endfor %} diff --git a/ansible/roles/developer-rust/defaults/main.yml b/ansible/roles/developer-rust/defaults/main.yml index bab1d79..d3023e2 100644 --- a/ansible/roles/developer-rust/defaults/main.yml +++ b/ansible/roles/developer-rust/defaults/main.yml @@ -64,6 +64,11 @@ rust_governor_enabled: true # One rust build at a time per user; later invocations wait for the lock. rust_governor_serialize: true +# A daemon started inside a governed build inherits the lock and holds it for +# its own lifetime, which would otherwise hang every later cargo command. After +# this many seconds the build proceeds unserialised and says so. +rust_governor_lock_wait_seconds: 1800 + # Percentages of the HOST's physical RAM (systemd resolves them all against # RAM, the swap cap included), so the same numbers ship everywhere. High # throttles and reclaims; Max kills the build -- the build dies, never the diff --git a/ansible/roles/developer-rust/files/hyperi-rust-govern b/ansible/roles/developer-rust/files/hyperi-rust-govern index 84a6544..efdc4d8 100644 --- a/ansible/roles/developer-rust/files/hyperi-rust-govern +++ b/ansible/roles/developer-rust/files/hyperi-rust-govern @@ -25,6 +25,7 @@ CONF="${XDG_CONFIG_HOME:-$HOME/.config}/hyperi/rust-governor.conf" SERIALIZE="${HYPERI_RUST_GOVERN_SERIALIZE:-1}" RESERVE="${HYPERI_RUST_GOVERN_CPU_RESERVE:-2}" QOS="${HYPERI_RUST_GOVERN_MACOS_QOS:-utility}" +LOCK_WAIT="${HYPERI_RUST_GOVERN_LOCK_WAIT:-1800}" TOOL="$(basename "$0")" if [ "$TOOL" = "hyperi-rust-govern" ]; then @@ -82,6 +83,9 @@ esac case "$RESERVE" in ''|*[!0-9]*) RESERVE=2 ;; esac +case "$LOCK_WAIT" in + ''|*[!0-9]*) LOCK_WAIT=1800 ;; +esac # The cgroup quota is invisible to rustc, which sizes its parallelism off the # visible core count and thrashes inside the cap -- so the job count is @@ -101,34 +105,72 @@ notify_wait() { echo "hyperi-rust-govern: another rust build holds the lock -- waiting (HYPERI_RUST_GOVERNOR=off to bypass)" >&2 } +# The same fd inheritance that makes the lock outlive a crash also hands it to +# any daemon the build starts, which then holds it for its own lifetime. +notify_timeout() { + if [ -n "${1:-}" ]; then + echo "hyperi-rust-govern: lock held by PID(s) $1 for over ${LOCK_WAIT}s -- building unserialised (HYPERI_RUST_GOVERNOR=off to silence)" >&2 + else + echo "hyperi-rust-govern: lock held for over ${LOCK_WAIT}s -- building unserialised (HYPERI_RUST_GOVERNOR=off to silence)" >&2 + fi +} + # Linux: flock on fd 9. The fd survives exec, so the lock lives exactly as # long as the build and releases on any exit, crash included. if [ "$SERIALIZE" = "1" ] && command -v flock >/dev/null 2>&1; then - exec 9>"$LOCK" - if ! flock -n 9; then + # An XDG_RUNTIME_DIR that is set but absent leaves fd 9 unopened, and every + # flock below then fails instantly -- which is not the same as contention. + # No 2>/dev/null here: on success that redirect would persist and silence + # the build's own stderr. + if ! exec 9>"$LOCK"; then + echo "hyperi-rust-govern: cannot open $LOCK -- building unserialised" >&2 + elif ! flock -n 9; then notify_wait - flock 9 + # Serialisation is an optimisation, so a stuck holder degrades the build + # rather than hanging it. + if ! flock -w "$LOCK_WAIT" 9; then + # fuser space-separates several holders, so squeeze rather than + # delete or they concatenate into one meaningless number. + lock_holder="" + command -v fuser >/dev/null 2>&1 && + lock_holder="$(fuser "$LOCK" 2>/dev/null | tr -s '[:space:]' ' ')" + lock_holder="${lock_holder# }" + notify_timeout "${lock_holder% }" + fi fi elif [ "$SERIALIZE" = "1" ]; then # macOS ships no flock(1). A pid-stamped mkdir lock is atomic everywhere; # a holder that died without cleaning up is detected and cleared. LOCK_D="$LOCK.d" notified=0 - while ! mkdir "$LOCK_D" 2>/dev/null; do + waited=0 + holder="" + locked=0 + while [ "$waited" -lt "$LOCK_WAIT" ]; do + if mkdir "$LOCK_D" 2>/dev/null; then + locked=1 + break + fi holder="$(cat "$LOCK_D/pid" 2>/dev/null || true)" + # A failed clear falls through to the wait, so the loop always advances. if [ -n "$holder" ] && ! kill -0 "$holder" 2>/dev/null; then - rm -rf "$LOCK_D" - continue + rm -rf "$LOCK_D" && continue fi if [ "$notified" -eq 0 ]; then notify_wait notified=1 fi sleep 2 + waited=$((waited + 2)) done - printf '%s\n' "$$" >"$LOCK_D/pid" - # Single quotes: expanded when the trap fires, not here. - trap 'rm -rf "$LOCK_D"' EXIT INT TERM + if [ "$locked" -eq 1 ]; then + printf '%s\n' "$$" >"$LOCK_D/pid" + # Single quotes: expanded when the trap fires, not here. + trap 'rm -rf "$LOCK_D"' EXIT INT TERM + else + # No trap: the directory belongs to whoever still holds it. + notify_timeout "$holder" + fi fi # Linux with a live user manager: the scope joins rust-build.slice, whose diff --git a/ansible/roles/developer-rust/files/hyperi-rust-setup b/ansible/roles/developer-rust/files/hyperi-rust-setup index a278a98..e8af4cf 100644 --- a/ansible/roles/developer-rust/files/hyperi-rust-setup +++ b/ansible/roles/developer-rust/files/hyperi-rust-setup @@ -314,6 +314,20 @@ def _sccache_target() -> str | None: }.get(platform.machine()) +def _github_request(url: str) -> urllib.request.Request: + """A GitHub API request carrying a token when the environment holds one. + + api.github.com allows 60 requests an hour per IP anonymously, and a shared + egress address exhausts that mid-run. The failure arrives as a 403 with the + rate-limit headers rather than as anything named "rate limit". + """ + request = urllib.request.Request(url) + token = os.environ.get("GITHUB_TOKEN") or os.environ.get("GH_TOKEN") + if token: + request.add_header("Authorization", f"Bearer {token}") + return request + + def _installed_sccache_version() -> str | None: """The version of the sccache this script manages, or None if absent.""" if not SCCACHE_BIN.is_file(): @@ -339,7 +353,7 @@ def install_sccache_linux(dry_run: bool, rep: Reporter) -> None: url = f"https://api.github.com/repos/{SCCACHE_REPO}/releases/latest" try: - with urllib.request.urlopen(url, timeout=60) as response: + with urllib.request.urlopen(_github_request(url), timeout=60) as response: latest = json.load(response)["tag_name"] except (OSError, ValueError, KeyError) as exc: rep.warn(f"sccache: could not read the latest release ({exc}) -- leaving it alone") diff --git a/ansible/roles/developer-rust/tasks/governor.yml b/ansible/roles/developer-rust/tasks/governor.yml index dbba594..42e1120 100644 --- a/ansible/roles/developer-rust/tasks/governor.yml +++ b/ansible/roles/developer-rust/tasks/governor.yml @@ -104,7 +104,7 @@ become: true become_user: "{{ actual_user }}" environment: - XDG_RUNTIME_DIR: "/run/user/{{ ansible_facts['user_uid'] | default(1000) }}" + XDG_RUNTIME_DIR: "/run/user/{{ actual_user_uid | default(ansible_facts['user_uid']) }}" when: - ansible_facts['system'] == 'Linux' - not ansible_check_mode @@ -180,7 +180,7 @@ become: true become_user: "{{ actual_user }}" environment: - XDG_RUNTIME_DIR: "/run/user/{{ ansible_facts['user_uid'] | default(1000) }}" + XDG_RUNTIME_DIR: "/run/user/{{ actual_user_uid | default(ansible_facts['user_uid']) }}" when: - ansible_facts['system'] == 'Linux' - not ansible_check_mode diff --git a/ansible/roles/developer-rust/tasks/rust.yml b/ansible/roles/developer-rust/tasks/rust.yml index 8914f5b..c0f0285 100644 --- a/ansible/roles/developer-rust/tasks/rust.yml +++ b/ansible/roles/developer-rust/tasks/rust.yml @@ -453,7 +453,9 @@ {{ '' if rust_cache_central_build_dir else '--no-build-dir' }} become: "{{ ansible_facts['distribution'] != 'MacOSX' }}" become_user: "{{ actual_user if ansible_facts['distribution'] != 'MacOSX' else omit }}" - environment: "{{ cargo_env }}" + # The script looks up the latest sccache release itself, so it needs the token + # too -- it is the one GitHub caller the uri module's headers cannot cover. + environment: "{{ cargo_env | combine(hyperi_github_env) }}" register: developer_rust_setup changed_when: "'CHANGED' in developer_rust_setup.stdout" # An optional build accelerator must never abort the run (see S1.T11). diff --git a/ansible/roles/developer-rust/tasks/sccache_service.yml b/ansible/roles/developer-rust/tasks/sccache_service.yml index 0661f00..8138748 100644 --- a/ansible/roles/developer-rust/tasks/sccache_service.yml +++ b/ansible/roles/developer-rust/tasks/sccache_service.yml @@ -44,10 +44,25 @@ become: true become_user: "{{ actual_user }}" environment: - XDG_RUNTIME_DIR: "/run/user/{{ ansible_facts['user_uid'] | default(1000) }}" + XDG_RUNTIME_DIR: "/run/user/{{ actual_user_uid | default(ansible_facts['user_uid']) }}" when: not ansible_check_mode + register: developer_rust_sccache_enable failed_when: false +- name: Warn if the sccache unit did not enable + # noqa: var-naming[no-role-prefix] -- deploy_warnings is the shared + # accumulator reported by playbooks/main.yml post_tasks. + ansible.builtin.set_fact: + deploy_warnings: >- + {{ deploy_warnings | default([]) + + ['sccache: enabling hyperi-sccache.service failed -- ' + ~ (developer_rust_sccache_enable.msg | default('no message')) + ~ '. Builds still work, without the shared compile cache.'] }} + when: + - not ansible_check_mode + - developer_rust_sccache_enable is failed + or developer_rust_sccache_enable.rc | default(0) != 0 + # Starting it while an unmanaged server holds the port would fail with # "Address in use", and stopping that server first would kill any compile # running through it. The unit takes over at the next login instead. @@ -59,12 +74,27 @@ become: true become_user: "{{ actual_user }}" environment: - XDG_RUNTIME_DIR: "/run/user/{{ ansible_facts['user_uid'] | default(1000) }}" + XDG_RUNTIME_DIR: "/run/user/{{ actual_user_uid | default(ansible_facts['user_uid']) }}" when: - not ansible_check_mode - developer_rust_sccache_running.rc != 0 + register: developer_rust_sccache_start failed_when: false +- name: Warn if the sccache unit did not start + # noqa: var-naming[no-role-prefix] -- see the note above. + ansible.builtin.set_fact: + deploy_warnings: >- + {{ deploy_warnings | default([]) + + ['sccache: starting hyperi-sccache.service failed -- ' + ~ (developer_rust_sccache_start.msg | default('no message')) + ~ '. It retries at the next login.'] }} + when: + - not ansible_check_mode + - developer_rust_sccache_running.rc != 0 + - developer_rust_sccache_start is failed + or developer_rust_sccache_start.rc | default(0) != 0 + - name: Note that the sccache unit takes over at the next login ansible.builtin.set_fact: deploy_warnings: >- diff --git a/ansible/roles/developer-rust/templates/rust-governor.conf.j2 b/ansible/roles/developer-rust/templates/rust-governor.conf.j2 index c301a34..f413d1b 100644 --- a/ansible/roles/developer-rust/templates/rust-governor.conf.j2 +++ b/ansible/roles/developer-rust/templates/rust-governor.conf.j2 @@ -3,3 +3,5 @@ HYPERI_RUST_GOVERN_SERIALIZE={{ '1' if rust_governor_serialize else '0' }} HYPERI_RUST_GOVERN_CPU_RESERVE={{ rust_governor_cpu_reserve_cores }} HYPERI_RUST_GOVERN_MACOS_QOS={{ rust_governor_macos_qos }} +# Seconds to wait for the build lock before giving up and building unserialised. +HYPERI_RUST_GOVERN_LOCK_WAIT={{ rust_governor_lock_wait_seconds }} diff --git a/ansible/roles/developer/tasks/init.yml b/ansible/roles/developer/tasks/init.yml index a53665a..fe648b5 100644 --- a/ansible/roles/developer/tasks/init.yml +++ b/ansible/roles/developer/tasks/init.yml @@ -42,10 +42,50 @@ user_home: "{{ '/Users/' + actual_user if actual_user != 'root' else '/var/root' }}" when: ansible_facts['distribution'] == 'MacOSX' +# A systemd --user instance lives at /run/user/ of the DESKTOP user, and +# ansible_facts['user_uid'] is the CONNECTING user's -- a different account on a +# fleet box. Linux only: macOS has no getent and no systemd --user. +- name: Look up the target user's uid (Linux) + ansible.builtin.getent: + database: passwd + key: "{{ actual_user }}" + failed_when: false + changed_when: false + when: ansible_facts['distribution'] != 'MacOSX' + +- name: Set the target user's uid (Linux) + # noqa: var-naming[no-role-prefix] -- consumed by developer-rust, so it is + # deliberately play-scoped, not role-scoped. + ansible.builtin.set_fact: + actual_user_uid: >- + {{ ansible_facts.getent_passwd[actual_user][1] + if (ansible_facts.getent_passwd is defined + and actual_user in ansible_facts.getent_passwd) + else ansible_facts['user_uid'] }} + when: ansible_facts['distribution'] != 'MacOSX' + - name: Display user information ansible.builtin.debug: msg: "Installing for user: {{ actual_user }} (home: {{ user_home }})" +# Here rather than beside the pnpm install because tag selection can reach a +# consumer without its producer -- `--tags typescript` picks the TypeScript task +# and not the Node one. This file carries `tags: ['always']`. +- name: Set the pnpm environment + # noqa: var-naming[no-role-prefix] -- consumed by developer-node and + # developer-typescript, so it is deliberately play-scoped, not role-scoped. + ansible.builtin.set_fact: + pnpm_env: + PNPM_HOME: "{{ user_home }}/.local/share/pnpm" + # BOTH PNPM_HOME and PNPM_HOME/bin: pnpm reports its global bin directory + # as the `bin` subdirectory and checks THAT is on PATH, while its own + # binary lives in PNPM_HOME itself. + PATH: >- + {{ user_home }}/.local/share/pnpm:{{ user_home }}/.local/share/pnpm/bin:{{ + user_home }}/.npm-global/bin:{{ + '/opt/homebrew/bin:/usr/local/bin:' if ansible_facts['distribution'] == 'MacOSX' else '' + }}{{ ansible_facts['env'].PATH }} + - name: Load macOS-specific variables ansible.builtin.include_vars: file: macos.yml diff --git a/ansible/roles/developer/tasks/main.yml b/ansible/roles/developer/tasks/main.yml index 7716872..024a494 100644 --- a/ansible/roles/developer/tasks/main.yml +++ b/ansible/roles/developer/tasks/main.yml @@ -4,9 +4,13 @@ # opinions. GUI tooling lives in developer-gui; languages in developer-; # IaC in infrastructure; org-specific in soe / soe-gui. +# `apply` as well as the tag: without it the include runs but its tasks keep no +# tags of their own, so a narrow --tags run skips every fact they set. - name: Initialize - Setup common variables ansible.builtin.include_tasks: file: init.yml + apply: + tags: ['always'] tags: ['always'] # Fix AppArmor user namespace restriction (Ubuntu 23.10+) diff --git a/ansible/roles/developer/tasks/nodejs.yml b/ansible/roles/developer/tasks/nodejs.yml index 0ed28be..2361e73 100644 --- a/ansible/roles/developer/tasks/nodejs.yml +++ b/ansible/roles/developer/tasks/nodejs.yml @@ -416,20 +416,8 @@ group: "{{ actual_user if ansible_facts['distribution'] != 'MacOSX' else omit }}" become: "{{ ansible_facts['distribution'] != 'MacOSX' }}" -- name: Set the pnpm environment - # noqa: var-naming[no-role-prefix] -- consumed by developer-node and - # developer-typescript, so it is deliberately play-scoped, not role-scoped. - ansible.builtin.set_fact: - pnpm_env: - PNPM_HOME: "{{ user_home }}/.local/share/pnpm" - # BOTH PNPM_HOME and PNPM_HOME/bin: pnpm reports its global bin directory - # as the `bin` subdirectory and checks THAT is on PATH, while its own - # binary lives in PNPM_HOME itself. - PATH: >- - {{ user_home }}/.local/share/pnpm:{{ user_home }}/.local/share/pnpm/bin:{{ - user_home }}/.npm-global/bin:{{ - '/opt/homebrew/bin:/usr/local/bin:' if ansible_facts['distribution'] == 'MacOSX' else '' - }}{{ ansible_facts['env'].PATH }} +# pnpm_env is set in the role's init.yml, which runs under `tags: ['always']` so +# a consumer selected without this file still resolves it. # PNPM_HOME is an environment variable pnpm itself reads, so it has to be set # for non-interactive shells too -- ~/.profile, not ~/.bashrc. diff --git a/docs/install-matrix.md b/docs/install-matrix.md index 2c306c1..2ddb56e 100644 --- a/docs/install-matrix.md +++ b/docs/install-matrix.md @@ -136,9 +136,10 @@ Pinning is by tag today across the board, and digest verification at download is still the planned hardening (see Auto-update below, and hyperi-ci #66). A tag can be force-moved, so those rows currently rest on HTTPS and the tag alone. -Three fetches do verify a digest today, and they are the ones whose SHA is held -in the repo or read from a vendor manifest: the Go toolchain, rustup-init, and -the Claude Code binary. +Four fetches do verify a digest today, and they are the ones whose SHA is held +in the repo or read from a published manifest: the Go toolchain, rustup-init, +the Claude Code binary, and macbash (whose `.sha256` sits beside the asset on +downloads.hyperi.io, so the fetch also re-pulls a republished binary). ### developer (base, additive - runs bare) @@ -291,9 +292,12 @@ The base ships the Astral suite (uv, ruff, ty) and `uv` bundles `uv audit` / | wrangler (the `cloudflare` group) | all | npm-global / brew | version | | flarectl (the `cloudflare` group) | all | `go install` from source / brew | source tag | -Every macOS path resolves to brew or a cask. The language managers that remain -there carry no formula at all: `alint` and `maid` have none, and semantic-release -needs its plugin set installed alongside it, which only npm gives. +Almost every macOS path resolves to brew or a cask. The language managers that +remain there carry no formula at all: `alint` and `maid` have none, and +semantic-release needs its plugin set installed alongside it, which only npm +gives. `git-scrub` is the one release-tarball exception -- its formula exists in +the release but is not in the hyperi-io tap, so macOS takes the darwin asset +until it is tapped. Cloudflare publishes no flarectl binary and no distro packages it, so both platforms build it from source. It also lives on cloudflare-go's `v0` branch -- @@ -321,6 +325,8 @@ warning and continues. | vulture | all | Ubuntu apt / Fedora uv-tool (Tier 2) / brew | version | | typos | all | cargo (Tier 2) / brew | version | | maid (mermaid validator, used by `/docs`) | all | npm global (Tier 2) | n/a | +| git-scrub (git-history scrubber) | all | github-binary (Tier 3: re-fetch) | version | +| macbash (macOS bash portability checker) | all | Linux downloads.hyperi.io binary (Tier 3, digest-verified) / brew tap | SHA256 | `hyperi-ci` is a Python tool from PyPI, installed via `uv tool` and refreshed to the latest release on every run (upgrade-if-present, not install-once). soe