manifest.json index sha256 hashes are stale on main; no CI gate catches template manifest drift

[christian-byrne]

Summary

packages/core/src/comfyui_workflow_templates_core/manifest.json on main records stale sha256 values for index.json and all 12 locale variants. Running the documented sync step on a pristine checkout of main — with no edits of any kind — produces a 13-line diff.

Reproduction

git clone https://github.com/Comfy-Org/workflow_templates
cd workflow_templates          # currently at 5097de61
git status --porcelain         # clean
python scripts/sync/sync_bundles.py
git diff --stat
#  .../comfyui_workflow_templates_core/manifest.json | 26 +++++++++++-----------

The 13 entries whose hashes change:

index.json, index.ar.json, index.es.json, index.fa.json, index.fr.json, index.ja.json, index.ko.json, index.mcp.json, index.pt-BR.json, index.ru.json, index.tr.json, index.zh.json, index.zh-TW.json

Cause

#1088 ("Archive Getting Started category and Reve templates") moved templates from templates/ into archived/ and edited all 13 templates/index.*.json files. The manifest hash for each of those is computed over the pip-filtered index that filter_index_for_pip produces (scripts/sync/sync_bundles.py:126-141), so moving templates out changes every one of them. The manifest committed in that PR does not reflect the final state of those files.

filter_index_for_pip is a pure deterministic JSON transform with no environment dependence, so this is genuine drift and not a Python-version artifact — reproduced locally on 3.12.

Why CI did not catch it

Neither manifest-related workflow actually compares committed state against generated state for templates:

• .github/workflows/validate-manifests.yml runs sync_bundles.py first and then validates, so the manifest it checks is always the freshly generated one. It only detects orphaned templates/blueprints, never staleness.
• .github/workflows/version-check.yml also re-runs the sync, but its commit step bails out unless the root pyproject.toml version changed: "Root version unchanged; skipping commit (no version bump performed)". A PR that edits templates/index.*.json without triggering a version bump therefore leaves the manifest stale on main.

By contrast the blueprints path is gated — .github/workflows/validate-blueprints.yml:65-73 re-runs sync_blueprints.py and exit 1s on any diff. Templates have no equivalent check.

Note that README.md:170 states "This step must be run before committing; CI will fail if the manifest/bundles are out of sync", which is not what the workflows currently do for templates.

Impact

The published comfyui-workflow-templates-core package ships a manifest whose recorded sha256 for all 13 index files does not match the index files actually packaged, so any consumer that verifies checksums against the manifest fails on them.

Knock-on effect on unrelated PRs

Any PR that legitimately runs sync_bundles.py per the README picks up these 13 unrelated lines. #1092 carries them for exactly this reason — they are not caused by that change.

Suggested fix

1. Land a manifest-only refresh on main (python scripts/sync/sync_bundles.py, commit just the manifest).
2. Add a staleness gate for templates mirroring validate-blueprints.yml:65-73 — re-run sync_bundles.py and fail on a non-empty diff — so this cannot recur silently. Alternatively, drop the version-bump condition from version-check.yml's commit step so manifest changes are always committed back.
3. Either way, reconcile README.md:170 with the actual behaviour.

[coderabbitai]

🔗 Related PRs

#1067 - Fix krea2 issue and update translation [merged]
#1075 - Update template steps to 20 [closed]
#1076 - Fix cloud template visibility [closed]
#1081 - Bump template to 0.11.30 [merged]
#1085 - Fix Topaz template issue [merged]

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[christian-byrne]

Fixed rather than left as a ticket. Two stacked PRs, both green:

• Regenerate the core manifest and unbreak Build & Test, both left broken by PR 1088 #1094 — regenerates the manifest (13 lines, manifest-only).
• Fail CI on manifest drift instead of regenerating the manifest before validating it #1095 — makes CI actually fail on drift. Stacked on 1094; closes this issue.

The interesting part

scripts/validate/validate_manifests.py already does the right check. It hashes every asset the manifest references against templates/, and on stale main it exits 1 with 39 errors. It was neutered purely by ordering: validate-manifests.yml ran sync_bundles.py first, so the validator only ever saw a freshly regenerated manifest. build-test.yml had the same ordering ahead of its pytest step. The validator needed no change at all — only to be pointed at the committed state.

Removing the pre-sync turns out to require widening the sparse-checkout to all of templates/. The old checkout fetched only templates/*.json, which was survivable only because the sync regenerated a manifest with no media entries (the globs found no .webp). Validating the committed manifest needs the media present, or every asset reports "not found". Offset by narrowing packages/ to the two committed manifest files, ~389 MB of mirrored media the job never reads.

Red/green proof

A temporary commit zeroed the sha256 for templates/index.json, then was reverted. Both runs are on the PR branch:

• 🔴 https://github.com/Comfy-Org/workflow_templates/actions/runs/31135884834
• 🟢 https://github.com/Comfy-Org/workflow_templates/actions/runs/31136027933

❌ Template 'index': SHA256 mismatch for 'index.json'
   Expected: 0000000000000000000000000000000000000000000000000000000000000000
   Actual:   a9917f8192fcaebdd2c74b86837299c7dd7aae55dc2ef56b59419c2e17bf2234
##[error]Manifest is out of sync with templates/. Run 'python scripts/sync/sync_bundles.py' and commit the regenerated manifest.

Under the old workflow that commit would have passed.

Two corrections to this issue

The drift is confirmed and correctly attributed, with the bisect tightened: at 850a5fad (the parent of #1088) the manifest and templates/index.json both hashed to 8093af99…; at 5097de61 the file hashes to a9917f81… while the manifest records 9e3ec0bd…. Regenerating under Python 3.9, 3.11 and 3.12 produces byte-identical output (7e5d9273…), so it is not a serialization artifact.

PR #1088 also broke something this issue did not mention. packages/core/tests/test_http_routes.py hard-coded eight template ids and #1088 archived six of them, so main has been red on Build & Test since 5097de61 (https://github.com/Comfy-Org/workflow_templates/actions/runs/31102844125), green at 850a5fad. Repaired in 1094 by sampling from bundles.json instead, so the next archive cannot rot it.

Deliberately not changed

version-check.yml still gates its auto-commit on a version bump. That is a release-process decision and the existing comment says the gate exists to avoid churn on template-only PRs. With the new check, a PR that leaves the manifest stale fails with an actionable message instead of merging silently — which is the behaviour that was missing.

#1092

Needs no rebase. git merge-tree against 1094 is clean and the merged manifest is byte-identical to the one already on that branch, because both sides make the identical 13-line change. Its body has been updated to say so.