Skip to content

blockmap's liveness bitmap races under abandon/adopt — deterministic abort at low core counts #8

Description

@Ttimmahlax

Summary

With --features blockmap, tests/stress_mt aborts in blockmap_abort()
(page.rs:177) deterministically on machines with few cores. The default
build is unaffected, and so is --features secure.

Pinned to 4 CPUs (taskset -c 0-3), 12 repeats each:

build result
default (no features) 12/12 pass
--features blockmap 12/12 abort
--features secure 8/8 pass

On 24 cores the same binary passes 15/15, which is why it does not show on a
development box. GitHub's runners have few cores and run --all-features, so
CI hits it every time.

Not a regression

v1.0.0 (18ff26b) fails identically — 9/10 aborts under the same pinning.
The bug is in the shipped v1.0.0 and in v1.0.1, unchanged.

Why it was invisible

The test job runs cargo fmt --all --check before cargo test. That check
had been failing since before v1.0.0, so the job exited 1 and never reached
the tests
. Fixing the formatting in the 1.0.1 release is what uncovered this.

Worth noting separately: the abort is silent (std::process::abort() with no
message, deliberately — see double_free_abort's doc). Any harness that
decides pass/fail by grepping output for FAILED/panicked will call this a
pass. Check the exit code.

Likely cause

blockmap_transition documents its own assumption:

Owner-only, therefore no atomics: the two callers are the allocation pop and
the local free.

Under abandon/adopt a page changes owner. Two threads can then perform the
non-atomic read-modify-write on the same bitmap byte
(byte.write(byte.read() ^ mask)), and a lost update leaves a live block
marked free — which the very next transition reports as a double free.

stress_mt is the abandon/adopt storm, so it is the workload that exercises
exactly this.

Suggested directions

  1. Make the bitmap byte atomic (AtomicU8 + fetch_xor, Relaxed is enough —
    the bit is diagnostic, not a synchronisation edge). Costs one instruction
    on a path that only exists in the opt-in feature, so the usual objection
    does not apply here.
  2. Or clear the map for a segment's pages at the adopt boundary, so no two
    owners can ever transition the same byte.

Option 1 is smaller and keeps the feature's guarantee intact across adoption;
option 2 is cheaper at runtime but weakens detection across an adopt.

Reproduce

cargo test -p rusty_alloc --test stress_mt --features blockmap --no-run
taskset -c 0-3 target/debug/deps/stress_mt-*   # exit 134, no message

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions