Skip to content

Patch Manifold v3.5.2: fix a heap overflow in CsgLeafNode::Compose - #227

Merged
revarbat merged 1 commit into
mainfrom
fix/manifold-compose-overflow
Oct 2, 2026
Merged

revarbat merged 1 commit into
mainfrom
fix/manifold-compose-overflow

Conversation

@revarbat

@revarbat revarbat commented Oct 2, 2026

Copy link
Copy Markdown
Member

Bug. Manifold v3.5.2 reads past the end of a heap array in CsgLeafNode::Compose:

  • Compose calls combined.RemoveDegenerates() and then sorts the mesh (SortGeometry → GatherFaces/ReindexFace).
  • On some inputs, the cleanup leaves the face data inconsistent, and the sort then reads 96 bytes past a 663 KB array.
  • MinkowskiDifference.ShrinksANonConvexBodyOnEverySide reaches it, a holed plate eroded by a sphere. It fails under AddressSanitizer on main. A normal build doesn't notice, which is why CI never has.

Isolated and bisected upstream:

  • A standalone program using only Manifold reproduces it: the evaluator's exact plate and sphere meshes, then plate.MinkowskiDifference(sphere).
  • Bisected across the 77 master commits since v3.5.2's branch point. It's fixed by Improved decimation elalish/manifold#1789 "Improved decimation" (969b1417), which deletes that same RemoveDegenerates() call. The rest of #1789 is a new decimator.
  • v3.5.2 with only that call removed runs the repro clean, with the same result as master.
  • #1789 isn't in any release: 3.5.3 and 3.5.4 change only the release pipeline and WASM. Moving to master isn't an option either. It removes CrossSection::FillRule, which we use in 8 places, and it has a separate regression: eroding a cube by a sphere returns an empty result.

Change. FetchContent now runs cmake/patch_manifold.cmake as Manifold's PATCH_COMMAND. The script deletes the call, re-runs as a no-op on already-patched source, and fails the configure if the code around the call has changed, so a version bump can't silently skip it. CLAUDE.md's Manifold entry records it and says to drop it once we move past #1789.

Tested:

  • Fresh AddressSanitizer build, where FetchContent downloads and patches Manifold: all 1354 tests pass, including the Minkowski one (on main it aborts with the overflow).
  • Patching paths:
    • re-running the script prints "already applied" and changes nothing;
    • reconfiguring an existing build directory patches its unpatched copy;
    • a changed source fails with a clear message.
  • Release build: ctest passes all 2013 tests.

🤖 Generated with Claude Code

CsgLeafNode::Compose calls RemoveDegenerates() on the combined mesh and
then sorts it; on some inputs the cleanup leaves the face data
inconsistent and GatherFaces reads past an array (heap-buffer-overflow
under ASan, reached by MinkowskiDifference.ShrinksANonConvexBodyOnEverySide).
Reproduced with Manifold alone and bisected upstream to #1789
(969b1417), which fixes it by deleting the same call; unreleased as of
3.5.4. Applied at fetch time by cmake/patch_manifold.cmake, which is
idempotent and fails the configure if the source has moved on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@revarbat
revarbat merged commit 78c8200 into main Oct 2, 2026
3 checks passed
@revarbat
revarbat deleted the fix/manifold-compose-overflow branch October 2, 2026 08:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant