Skip to content

fix(e2e): gate tests on a served response and keep failure artifacts - #819

Merged
d-oit merged 1 commit into
mainfrom
fix/e2e-harness-readiness-and-artifacts
Sep 24, 2026
Merged

d-oit merged 1 commit into
mainfrom
fix/e2e-harness-readiness-and-artifacts

Conversation

@d-oit

@d-oit d-oit commented Sep 24, 2026 •

Copy link
Copy Markdown
Owner

Two defects that surfaced when PR #817's E2E job failed 140 of 149 tests, and left nothing behind to explain it.

1. Readiness was a bound port, not a served response

webServer.port: 3000 is satisfied the moment next dev binds, while it is still compiling — tests started 3 seconds after the server did. A server that cannot answer yet then fails every navigating test on its own 5-second readiness wait, which reads as 140 broken tests. The job now waits on url (a real HTTP response, 120 s budget), and pipes the dev server's own output into the log.

2. A failing E2E run kept no evidence

The job uploaded playwright-report/ on failure, but reporter: 'list' never creates that directory — the artifact list from that run holds only the coverage report. Meanwhile trace: 'on-first-retry' was writing traces into test-results/, which no step uploaded. The upload now covers both directories, and CI writes the HTML report so the report path exists at all.

      - name: Upload Playwright artifacts
        if: failure()
        with:
          name: playwright-artifacts
          path: |
            playwright-report/
            test-results/
          if-no-files-found: warn

Verification

src/lib/__tests__/e2e-harness.test.ts loads the real config (with CI stubbed, since the reporter depends on it) and the real workflow. Each assertion was mutation-checked:

Mutation Result
url → port: 3000 waits for a served response, not a bound port fails
drop test-results/ from the upload uploads the traces alongside the report fails
reporter → 'list' in CI writes the HTML report in CI fails

pnpm exec playwright test e2e/home.spec.ts e2e/graph-density.spec.ts --project=chromium — 10 passed. ./scripts/quality_gate.sh — all gates passed.

The cold-start path is verified by this PR's own CI run, which starts the server itself and must now see a served response before the first test begins.

plans/153 records what is not explained (the underlying stall) and what the next occurrence will now reveal.


📝 Summary by GitNexus

Summary

This appears to be a test-harness and CI workflow change with no graph-detected code dependents or execution-flow reach. Review attention should center on the workflow configuration, which carries the elevated file risk.

🟠 HIGH blast radius. A test and CI configuration change spanning playwright.config.ts, src/lib/__tests__/e2e-harness.test.ts, and .github/workflows/ci-and-labels.yml, with no detected dependents.

The change is concentrated in Playwright configuration, an e2e harness test, and the associated planning document at plans/153-e2e-harness-readiness-and-artifacts-2026-09-24.md. The title and paths indicate updates to test readiness gating and failure-artifact handling.

Review .github/workflows/ci-and-labels.yml first, as it is the HIGH-risk file. Then verify that playwright.config.ts and src/lib/__tests__/e2e-harness.test.ts remain aligned with the CI execution environment and expected artifact behavior.

Added by GitNexus for PR #819. Edit freely — this block is replaced on the next review, everything above it is left untouched.

@vercel

vercel Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
do-knowledge-studio Ready Ready Preview, v0 Sep 24, 2026 7:27pm UTC

@github-actions github-actions Bot added documentation Documentation improvements ci config tests Related to automated/manual tests labels Sep 24, 2026
@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Blocked merge diagnosis — blocked
⏳ Check run(s) still in progress: ["Codacy Static Code Analysis","Diagnose Blocked Merge State","Shell Script Security Analysis","Secret Detection","Detect Changes","Infrastructure as Code Security","Trivy Filesystem Security Scan","YAML Syntax Validation","GitHub Actions Workflow Validation","commitlint","Analyze (actions)","Analyze (javascript-typescript)"]

@nexuscheck

nexuscheck Bot commented Sep 24, 2026

Copy link
Copy Markdown
Contributor
Akon Labs

GitNexus Review · PR #819

GitNexus reviewed 4 changed files.

🤖 Agent context — 4 files

Summary

This appears to be a test-harness and CI workflow change with no graph-detected code dependents or execution-flow reach. Review attention should center on the workflow configuration, which carries the elevated file risk.

🟠 HIGH blast radius. A test and CI configuration change spanning playwright.config.ts, src/lib/__tests__/e2e-harness.test.ts, and .github/workflows/ci-and-labels.yml, with no detected dependents.

The change is concentrated in Playwright configuration, an e2e harness test, and the associated planning document at plans/153-e2e-harness-readiness-and-artifacts-2026-09-24.md. The title and paths indicate updates to test readiness gating and failure-artifact handling.

Review .github/workflows/ci-and-labels.yml first, as it is the HIGH-risk file. Then verify that playwright.config.ts and src/lib/__tests__/e2e-harness.test.ts remain aligned with the CI execution environment and expected artifact behavior.

🟠 HIGH blast radius — no downstream dependents were found in the code graph; review the dependent list before merging.

Blast Level Dependents Modules Files
🟠 HIGH 0 0 4

What changed

Changed Files (4)
File Status
.github/workflows/ci-and-labels.yml 🟡 modified
plans/153-e2e-harness-readiness-and-artifacts-2026-09-24.md 🟢 added
playwright.config.ts 🟡 modified
src/lib/__tests__/e2e-harness.test.ts 🟢 added

What to check

File Risk (2)
File Risk Category
.github/workflows/ci-and-labels.yml 🟠 HIGH CI/CD
plans/153-e2e-harness-readiness-and-artifacts-2026-09-24.md 🟢 LOW Documentation

Full detail lives in the GitNexus check run for this commit.

@codacy-production

Copy link
Copy Markdown
Contributor

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 0 complexity · 0 duplication

Metric Results
Complexity 0
Duplication 0

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@d-oit
d-oit force-pushed the fix/e2e-harness-readiness-and-artifacts branch from 71fc043 to 6f13a09 Compare September 24, 2026 19:25
@nexuscheck

nexuscheck Bot commented Sep 24, 2026

Copy link
Copy Markdown
Contributor
Akon Labs

GitNexus Review · PR #819

No issues found in 4 changed files.

🤖 Agent context — 4 files

Summary

This appears to be a test-harness and CI workflow change with no graph-detected code dependents or execution-flow reach. Review attention should center on the workflow configuration, which carries the elevated file risk.

🟠 HIGH blast radius. A test and CI configuration change spanning playwright.config.ts, src/lib/__tests__/e2e-harness.test.ts, and .github/workflows/ci-and-labels.yml, with no detected dependents.

The change is concentrated in Playwright configuration, an e2e harness test, and the associated planning document at plans/153-e2e-harness-readiness-and-artifacts-2026-09-24.md. The title and paths indicate updates to test readiness gating and failure-artifact handling.

Review .github/workflows/ci-and-labels.yml first, as it is the HIGH-risk file. Then verify that playwright.config.ts and src/lib/__tests__/e2e-harness.test.ts remain aligned with the CI execution environment and expected artifact behavior.

🟠 HIGH blast radius — no downstream dependents were found in the code graph; review the dependent list before merging.

Blast Level Dependents Modules Files
🟠 HIGH 0 0 4

What changed

Changed Files (4)
File Status
.github/workflows/ci-and-labels.yml 🟡 modified
plans/153-e2e-harness-readiness-and-artifacts-2026-09-24.md 🟢 added
playwright.config.ts 🟡 modified
src/lib/__tests__/e2e-harness.test.ts 🟢 added

What to check

File Risk (2)
File Risk Category
.github/workflows/ci-and-labels.yml 🟠 HIGH CI/CD
plans/153-e2e-harness-readiness-and-artifacts-2026-09-24.md 🟢 LOW Documentation

Full detail lives in the GitNexus check run for this commit.

@d-oit
d-oit merged commit 55033bf into main Sep 24, 2026
27 checks passed
@d-oit
d-oit deleted the fix/e2e-harness-readiness-and-artifacts branch September 24, 2026 19:32

This branch was successfully deployed

1 active deployment
Preview — 6f13a09a Deployed Sep 24, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci config documentation Documentation improvements tests Related to automated/manual tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants