Skip to content

feat(ops): run test suites in WSL2 from a native-Windows checkout, CC-638 - #683

Merged
screenleon merged 5 commits into
mainfrom
feat/cc638-run-tests-in-wsl
Oct 6, 2026
Merged

screenleon merged 5 commits into
mainfrom
feat/cc638-run-tests-in-wsl

Conversation

@screenleon

@screenleon screenleon commented Oct 6, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • New developer aid ops/diagnostics/run-tests-in-wsl.sh: from a native-Windows Git Bash checkout it copies the working tree (tracked plus untracked-not-ignored, uncommitted edits included) into its own scratch directory in WSL2, clears the fake mode bits and restores the real executable bits, makes it a git repo, runs the named tests/shell suites with a time limit and prints one row per suite. A failing suite also prints its failed-case line and the log path.
  • --changed [--base REF] hands the paths this working tree changed (added, copied, modified, renamed; untracked included; deleted excluded) to tests/bin/run-tests.sh inside WSL, so it chooses the suites. Resolved before any WSL work: a bad ref or an empty change set costs nothing.
  • Why: native Windows spends about 64 to 100 ms per spawned process, so large suites do not finish locally (test-doctor.sh more than an hour, test-guards.sh more than 10 minutes). One pmctl dispatch run is about 320 process starts; removing them one batch at a time gives 3 to 10 percent per batch (CC-636, CC-637). WSL2 is already the project's release-sign-off platform.
  • Records CC-638, documents the helper in docs/platform-support.md (not a sandbox; does not replace native verification), CHANGELOG, registrations (suite registry, CI step, variable inventories).

Decision: no stand-in wsl.exe

An earlier round tested the sync, cleanup and --changed flow on Linux through a fake wsl.exe and a test-only environment variable. Removed on the maintainer's point that CI already runs the suites themselves on Linux: the fake could not catch the two Windows-only bugs found while building this (below), and it needed a test-only hook. The sync-and-run path therefore has no automated test; it was verified by hand against the real wsl.exe (see Evidence). The pure functions are tested on any platform.

Evidence (one machine, single measurements, noisy)

  • Suite time only; every call also spends about 6 s on the sync: test-lint-frontmatter 2 to 3 s in WSL vs 27 s natively, test-executor-router 4 vs 34 s, test-state-status 17 vs 62 s (23 passed in WSL; the native run has one pre-existing NTFS failure, so the work differs), test-guards 110 s in WSL (305 passed, 0 failed) vs more than 10 minutes natively. Small suites end up only about 3 times faster end to end.
  • By hand against the real wsl.exe on this host: a passing run (tree removed), a failing, a slow (--timeout 4) and a quiet suite (rows: exit 1 with failed-case line and log path; 124 "timed out"; 0 with a "prints no summary line" note; helper exit 1), --keep, --sync-only, and a real --changed run on this branch. That run found the paths against origin/main, and because this branch changes tests/lib/test-suite-runner.sh (widely used) run-tests.sh escalated to the whole suite and hit the 500 s limit: the helper reported "timed out ... (raise --timeout)", printed the log path and kept the tree. Documented.
  • New suite tests/shell/test-run-tests-in-wsl.sh: 10 passed, 1 skipped natively (the skip is the refusal off native Windows); 11 passed in WSL. Cases: help, usage errors, platform refusal, suite-name normalization, file and exec lists (including a file executable on disk but 100644 in the index), changed paths (modified, deleted, renamed, space in name, ignored, base HEAD, base HEAD~1, default origin/main when present, unknown ref refused with a hint), result parsing. Mutation checks, each killed by a case: every tracked file executable; first marker wins; no --diff-filter (deleted path appears).
  • In WSL: lint-jq-lf, lint-script-domain-inventory, lint-test-suite-registry, lint-test-docstrings, lint-doc-wikilinks, lint-portable-repo-paths, lint-surface-coverage, lint-readme-surface-lists, the whole-tree lint-shellcheck (301 files) and shellcheck on both new files in the CI form (no -x): clean.

Two bugs I introduced and fixed

  • Git Bash rewrote the /home/... argument to wsl.exe into C:/Program Files/Git/home/..., which created a junk directory named C: in the repo root (untracked, deleted by exact name). MSYS2_ARG_CONV_EXCL is now set for the one wsl.exe command only and never exported (git is a native program too).
  • The archive made every file executable, so git add -A in WSL recorded mode 100755 for everything and lint-jq-lf treated every library as an entry. Mode bits are now cleared first and only the real ones restored.

Gate and review

  • Full pr-gate: NOT run (not feasible on this host); no gate result artifact.
  • Two rounds of five reviewers each ran individually as subagents (same model family as the author). Round 1: security and architecture approve; critic, qa-tester and risk requested changes, all addressed (summary markers and a kept log instead of tail -n 3, NUL-separated exec list, in-WSL path re-check, non-empty $HOME, exec bit on the new files, timings including the sync, not-a-sandbox note, pure-function tests). Round 2 (before the stand-in was removed): security approve; critic and qa-tester changes-requested or advisory; risk warn; architecture advisory. Findings applied: --diff-filter=ACMR, git failures reported instead of silent exits, a bad base ref gets a hint, the change set is resolved before the sync, the sweep covers the whole scratch parent (not a glob built from the directory name), the path is re-checked inside WSL before writing the changed list, rtw_wsl_run prefix, the test-only variable removed, docs and the ticket rewritten.
  • Known gaps: the sync-and-run path and the mode-bit clearing (chmod 644) have no automated test (need native Windows plus WSL2); --changed can escalate to the whole suite and needs a larger --timeout; run-tests.sh without --base compares with HEAD only while the helper uses the merge base with origin/main (stated in the help); an early failure leaves its scratch tree until the next start sweeps it; test-gate-digest.sh and test-jq-lf.sh are also committed without the exec bit (pre-existing, not touched).
  • Permanent test admissions: one new suite tests/shell/test-run-tests-in-wsl.sh (no case added to a pre-existing suite). Its cases pin the helper's contract on any platform: argument handling and platform refusal (observable by exit code and message), suite-name normalization (names become paths inside WSL, so a security-relevant boundary), the file and exec lists (a wrong mode set broke lint-jq-lf in WSL once), the changed-path set (a deleted path or a wrong base changes which suites run), and result parsing (a missing or malformed status must never look like a pass). Each is implementation-independent and cheap; mutation-checked where noted.

🤖 Generated with Claude Code

https://claude.ai/code/session_011c6rVDk6zyfgLPEfbVrZi9

screenleon and others added 5 commits October 6, 2026 14:48
…-638)

Native Windows spends tens of milliseconds per spawned process, so large
suites do not finish locally. The helper syncs the working tree into WSL,
fixes the mode bits NTFS does not carry, and runs named suites with timing.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011c6rVDk6zyfgLPEfbVrZi9
…NUL lists, safer paths, sourceable pure functions (CC-638)

Review: exec bit on the two new files, summary scraping lost the summary
line with 2+ skipped cases and deleted the log, newline/dash file names in
the exec list, scratch name collisions, no in-WSL re-check before rm -rf,
timing claims omitted the per-call sync, and no Linux-runnable coverage of
the pure functions.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011c6rVDk6zyfgLPEfbVrZi9
…-run-tests-in-wsl.sh

CI runs shellcheck without -x; matches the convention in test-state-status.sh.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011c6rVDk6zyfgLPEfbVrZi9
…ts against a stand-in wsl.exe (CC-638)

Concurrent runs no longer replace each other's tree; --changed hands the
changed paths to tests/bin/run-tests.sh inside WSL; the sync, run, cleanup
and --changed paths are covered on Linux by a stand-in wsl.exe.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011c6rVDk6zyfgLPEfbVrZi9
…hs as a pure function (CC-638)

CI already runs the suites on Linux; the stand-in could not catch the two
Windows-only bugs found. Keep pure-function tests (suite names, file/exec
lists, changed paths incl. deleted/renamed/space/origin-main/bad ref, result
parsing). --changed resolved before any WSL work with the ACMR filter; sweep
covers the whole scratch parent; in-WSL re-check before writing into it;
rtw_wsl_run; ticket text rewritten.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011c6rVDk6zyfgLPEfbVrZi9
@screenleon
screenleon merged commit e569ea9 into main Oct 6, 2026
82 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant