ci(handbook): make the handbook trigger fire on e2e changes + enforce spec-metadata parity - #1393
Closed
joshuakrueger-dfx wants to merge 4 commits into
Closed
Conversation
A new or changed spec under e2e/ did not match e2e/screenshots/**, so handbook-check.yaml never started. e2e/** covers the previous screenshot path and the specs themselves.
A top-level e2e/*.spec.ts that calls toHaveScreenshot must have a metadata.json key equal to its file name without .spec.ts. Orphan keys fail the same check. The new job is Node-only, shares the existing path filter, and refuses to pass when no spec files are found.
…ings metadata.json "docs" is documentation title overrides, the same reserved key build.js already skips. Four existing keys name screenshot prefixes rather than spec files; sell-complete lives in e2e/synpress/. New specs still need an exact key.
The spec already takes seven toHaveScreenshot baselines. Without this entry the new parity check reports a real missing key.
Contributor
Author
|
Zurückgestellt — Setup-Priorität liegt gerade auf Infrastruktur, nicht auf diesem Fix. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Symptom (verbatim)
TaprootFreak auf services#1262, wörtlich: "scripts/handbook/metadata.json holds 37 entries and
none of them is the partner dashboard; no spec under e2e/ mentions partner; there is no baseline
among the 154 committed screenshots. And handbook-check.yaml filters on e2e/screenshots/,
scripts/handbook/, ... — this PR touches 58 files and not one of them matches. The gate does
not skip, it never triggers."
Dasselbe Grundmuster (fehlende Baseline/Metadata, Gate feuert nicht) trat in 3 Tagen sechsmal auf:
#1262, #1274, #1275, #1267, #1271, #1162 — TaprootFreak zitiert bei jedem erneut die Vorgänger-PRs
als Präzedenz.
Scale
9 Instanzen in api + 6 in services desselben strukturellen Fehlertyps in einer 3-Tage-PR-Analyse
(nicht Teil dieses PRs, siehe verlinkte jk-tools-Änderung); hier: 6 Instanzen davon in services
allein, jedes Mal CHANGES_REQUESTED.
Smaller fix considered
Eine reine Pfadfilter-Verbreiterung (z.B.
src/**in diepaths:-Liste) wurde geprüft undverworfen: sie macht den bestehenden teuren Docker-Build-Job auf praktisch jedem Frontend-PR
scharf, löst aber das eigentliche Problem nicht — der Job baut nur, was im Manifest steht, prüft
nie, ob etwas fehlt. Stattdessen zwei gezielte Ergänzungen: Trigger um
e2e/**erweitert (deckte2e/screenshots/**ein) plus ein neuer, schneller Job ohne Docker, der Spec↔Metadata-Paritätprüft.
Was der PR tut
handbook-check.yaml: Trigger-Pfade2e/screenshots/**→e2e/**.spec-metadata-parity(kein Docker, ubuntu-latest, 5 Min Timeout): jede Top-Level-e2e/*.spec.ts, dietoHaveScreenshot(aufruft, braucht einen exakt gleichnamigen Key inscripts/handbook/metadata.json— und umgekehrt (Orphan-Erkennung)."docs"wie inbuild.jsals reservierter Nicht-Spec-Key behandelt; vier historische Alias-Namen (
bug-session,subpage,swap-btc-to-ln,swap-ln-to-btc) und der eine Spec untere2e/synpress/sindam Spec-Inhalt verifiziert, nicht geraten (siehe Kommentare in
check-parity.js).support-issue-receiver-iban— sieben
toHaveScreenshot-Baselines hatten nie einen Eintrag).Real gegen
developgelaufen: PASS, 70 Top-Level-Specs, 34 Screenshot-Specs, 41 Metadata-Keys,0 Missing, 0 Orphans. 9/9 Fixture-Tests grün (inkl. Gegenprobe und Mutationsprobe pro Ausnahme).
Bekannte Grenze
Eine komplett neue Route ohne jeden e2e-Bezug (Fall #1262) bleibt Review-Disziplin — dieser Job
prüft nur Spec↔Metadata-Parität, nicht ob eine neue Route überhaupt einen Spec braucht.
Nicht verifiziert
handbook-checkals required Check konfiguriert ist.