Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 5 additions & 19 deletions .github/workflows/test-e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,39 +54,25 @@ jobs:
# we want to run the full build on all os: don't cancel running jobs even if one fails
fail-fast: false
matrix:
os: [macos-14, ubuntu-24.04, windows-2022]
os: [macos-15, ubuntu-24.04, windows-2022]
browser: [chromium, firefox, chrome]
include:
# only test WebKit on macOS
- os: macos-14
- os: macos-15
browser: webkit
# only test Edge on Windows
- os: windows-2022
browser: msedge
- os: macos-14
- os: macos-15
browser: chrome
steps:
- name: Checkout
uses: actions/checkout@v7
- name: Free disk space on Ubuntu runner
if: runner.os == 'Linux'
uses: ./.github/actions/free-runner-disk-space-ubuntu
# START temporary workaround
# Browser download failure with Node 24. See https://github.com/microsoft/playwright/issues/41185 and https://github.com/microsoft/playwright-cli/issues/419#issuecomment-4646958277 (that mentions that it should work with Node 22)
# - name: Build Setup
# uses: ./.github/actions/build-setup
- name: Setup node
uses: actions/setup-node@v6
with:
# node-version-file: '.nvmrc'
node-version: '22'
registry-url: ${{ inputs.registry-url }}
- name: Install dependencies
uses: bahmutov/npm-install@v1
# if: inputs.install-dependencies == 'true'
with:
install-command: npm ci --ignore-scripts --prefer-offline --audit false
# END OF temporary workaround
- name: Build Setup
uses: ./.github/actions/build-setup
- name: Install ${{matrix.browser}}
uses: ./.github/actions/install-playwright-browser
with:
Expand Down
18 changes: 2 additions & 16 deletions .github/workflows/test-npm-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,22 +47,8 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@v7
# START temporary workaround
# Browser download failure with Node 24. See https://github.com/microsoft/playwright/issues/41185 and https://github.com/microsoft/playwright-cli/issues/419#issuecomment-4646958277 (that mentions that it should work with Node 22)
# - name: Build Setup
# uses: ./.github/actions/build-setup
- name: Setup node
uses: actions/setup-node@v6
with:
# node-version-file: '.nvmrc'
node-version: '22'
registry-url: ${{ inputs.registry-url }}
- name: Install dependencies
uses: bahmutov/npm-install@v1
# if: inputs.install-dependencies == 'true'
with:
install-command: npm ci --ignore-scripts --prefer-offline --audit false
# END OF temporary workaround
- name: Build Setup
uses: ./.github/actions/build-setup
- name: Build npm package
run: npm pack
- name: List the size of the npm package bundles
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test-performance.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ jobs:
# triggerUncaughtException(err, true /* fromPromise */);
# ^
# cdpSession.detach: Browser closed.
os: [macos-14, ubuntu-24.04]
os: [macos-15, ubuntu-24.04]
env:
# Performance tests rely on chromium API
browser: chromium
Expand Down
27 changes: 27 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,33 @@ GraphConfigurator

**E2E Pattern:** Heavy use of image snapshots (in `__image_snapshots__/`) to verify visual rendering correctness across features.

### E2E Visual Regression Thresholds

E2E tests compare a freshly rendered screenshot against a reference image snapshot using `jest-image-snapshot` (SSIM comparison). A test fails when the measured difference exceeds its allowed `failureThreshold` (a percentage of differing pixels). Because rendering differs slightly by browser engine and OS (mostly font rendering, a few pixels, not visible to a human), thresholds are configured per browser family and per platform.

**How thresholds are resolved** (see `test/e2e/helpers/visu/image-snapshot-config.ts`):
- `MultiBrowserImageSnapshotThresholds` holds a default per browser family (`chromium`, `firefox`, `webkit`), passed to `super({ chromium, firefox, webkit })`.
- Each test file subclasses it and overrides `getChromiumThresholds()` / `getFirefoxThresholds()` / `getWebkitThresholds()` to return a `Map` of per-snapshot overrides.
- Each map entry is keyed by the snapshot identifier (the BPMN diagram name, sometimes with a suffix like `.ignored` / `.not-ignored`) and holds per-platform values: `{ linux?, macos?, windows? }`.
- At runtime, the browser family selects the map, `getSimplePlatformName()` selects the platform key, and the value found there wins. If there is no dedicated entry or no matching platform key, the browser-family default is used.
- On CI, macOS runs the `webkit` family, so webkit macOS thresholds come from the `macos` key inside `getWebkitThresholds()`.

**Threshold value convention:**
- Values are written as `X / 100` (a percentage expressed as a fraction), for example `macos: 0.53 / 100`.
- Add a trailing comment with the actual observed diff percentage from the report, for example `macos: 0.53 / 100, // 0.5228413635451014%`. The fit test file prefixes it with `max` because one entry covers several test variations sharing the snapshot key.
- Set the value by rounding the observed diff UP to 2 decimals (ceil), so the threshold sits just above the observed diff.
- Keep entries ordered by snapshot key to match the surrounding file.

**Updating thresholds from a CI test-results report** (recurring task):
1. The report bundle (downloaded artifact) contains `index-single-page.html` (jest-html-reporters) and a `__diff_output__/` folder with `<snapshot>-diff.png` per failure.
2. Parse `index-single-page.html`. Each failed block contains: the suite/title, `was <actual>% different from snapshot ... Failure threshold was set to <old>%`, and a `__diff_output__/<relative-path>-diff.png` link. The relative path (minus `-diff.png`) identifies the snapshot; note some names contain hyphens (`.not-ignored`) and the fit tests live in nested subfolders, so capture the full path up to `-diff.png`.
3. Group by snapshot key and keep the MAX actual diff (several test variations can share one threshold key, e.g. `with.outside.labels` in the fit tests).
4. For each failing snapshot, locate the matching entry in the relevant test file's `getWebkitThresholds()` (or the correct browser/platform) and set `macos: ceil2(actual) / 100, // <actual>%`. If no entry exists (the failure was against the browser-family default), add one in key order.
5. Files that currently carry webkit macOS thresholds: `bpmn.rendering.test.ts`, `bpmn.rendering.ignore.options.test.ts`, `bpmn.colors.test.ts`, `diagram.navigation.fit.test.ts`, `style.api.test.ts` (a test file may hold several threshold subclasses, one per configurator).
6. Reporting: flag any entry where the increase (`new threshold - old threshold`) exceeds 0.3 percentage points. A large jump usually signals a real rendering change or regression worth a human look, not just pixel noise.

Note (from the class JSDoc): prefer NOT adding a threshold for a new test until it actually fails on CI. Discrepancies mostly come from labels; if labels are not part of what the test verifies, remove labels from the BPMN diagram instead of raising a threshold.

## Key Architectural Patterns

### Converter Pattern
Expand Down
62 changes: 17 additions & 45 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@
"minimist": "~1.2.8",
"npm-run-all": "~4.1.5",
"pinst": "~3.0.0",
"playwright": "~1.58.2",
"playwright": "~1.61.1",
"postcss": "~8.5.14",
"postcss-cli": "~11.0.1",
"prettier": "~3.8.1",
Expand Down
4 changes: 2 additions & 2 deletions test/e2e/bpmn.colors.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ class ImageSnapshotThresholdsModelColors extends MultiBrowserImageSnapshotThresh
[
'elements.colors.02.labels',
{
macos: 0.41 / 100, // 0.40831372531949794%
macos: 1.04 / 100, // 1.033599854269418%
},
],
]);
Expand Down Expand Up @@ -140,7 +140,7 @@ class ImageSnapshotThresholdsIgnoreBpmnColors extends MultiBrowserImageSnapshotT
[
'elements.colors.02.labels',
{
macos: 0.49 / 100, // 0.483122009334358%
macos: 1.13 / 100, // 1.1249364408232543%
},
],
]);
Expand Down
8 changes: 4 additions & 4 deletions test/e2e/bpmn.rendering.ignore.options.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -70,13 +70,13 @@ class ImageSnapshotThresholdsActivityLabelBounds extends MultiBrowserImageSnapsh
[
'activities.with.wrongly.positioned.labels.not-ignored',
{
macos: 0.44 / 100, // 0.4382175377357411%
macos: 1.17 / 100, // 1.1642294732319813%
},
],
[
'activities.with.wrongly.positioned.labels.ignored',
{
macos: 1.5 / 100, // 1.4951298719464878%
macos: 2.01 / 100, // 2.0057344642194885%
},
],
]);
Expand Down Expand Up @@ -135,13 +135,13 @@ class ImageSnapshotThresholdsLabelStyles extends MultiBrowserImageSnapshotThresh
[
'labels.with.font.styles.not-ignored',
{
macos: 0.31 / 100, // 0.30621380597637415%
macos: 0.59 / 100, // 0.584031268762264%
},
],
[
'labels.with.font.styles.ignored',
{
macos: 0.38 / 100, // 0.37988509633168904%
macos: 0.84 / 100, // 0.8385178209979638%
},
],
]);
Expand Down
Loading
Loading