Skip to content

perf(thread): trim semaphore signal-side contention (min-wait fast path + signaler combining) - #1654

Draft
Coldwings wants to merge 2 commits into
alibaba:mainfrom
Coldwings:perf-semaphore-min-wait
Draft

Coldwings wants to merge 2 commits into
alibaba:mainfrom
Coldwings:perf-semaphore-min-wait

Conversation

@Coldwings

Copy link
Copy Markdown
Collaborator

Summary

Two independent optimizations to photon::semaphore that cut the signal-side
spinlock convoy seen when several vCPUs signal one semaphore -- the shape a
WorkPool produces. Based directly on main; does not depend on #1650.

Changes

  • Skip the resume path when the count cannot satisfy any waiter (m_min_wait):
    signal() keeps a lower bound of what any waiter needs (or -1 when nobody
    waits), adds its count and returns immediately while the total stays below that
    bound, without taking splock or walking the queue. wait_interruptible()
    gets a matching lock-free fast path. The two sides form a Dekker pair, both
    sequentially consistent, so at least one observes the other and no wake-up is
    lost:

    signal(): m_count.fetch_add(n)     then load(m_min_wait)
    wait():   m_min_wait.store(count)  then load(m_count)
    
  • Let one signaler at a time walk the resume path (combining): once
    m_min_wait keeps the hopeless signals out, the ones that remain all pile onto
    splock and the same queue, redundantly -- whoever is in the resume path
    re-reads m_count and covers the others' counts. A signaler now claims the
    path with a RESUMING bit and serves everyone; one that finds it taken sets
    RECHECK and leaves. The holder does not release until it has read m_count
    with RECHECK observed clear, and the release itself reports the bit back in
    case someone set it at the last moment. Every transition is a read-modify-write
    on m_resume_state, totally ordered with the fetch_add on m_count.

  • Adds thread/test/perf_semaphore.cpp, a benchmark (NO_REGISTER).

Motivation

With several vCPUs signalling one semaphore, signal() used to take splock and
walk the wait queue on every single call, even when the count in hand could
satisfy nobody, dragging in the waiters' thread structs -- hot lines of other
vCPUs. That spinlock is where the time goes.

Measured with thread/test/perf_semaphore.cpp, 8 signalling vCPUs x 50k ops,
4 waiters, 500ns of work between signals, aarch64, MinSizeRel, median of 3:

m_min_wait                wall        signal() avg    signal() max
  no waiter        53.5 -> 42.6ms     348 -> 171ns
  batch            98.1 -> 36.1ms    1807 -> 179ns    127us -> 0.7us
  stream          261.0 -> 61.6ms    5777 -> 636ns    473us -> 142us

combining (stream)        wall        signal() avg    signal() worst
  before                 58.3ms          597ns            230us
  after                  48.2ms          348ns             17us

With zero work between signals the aggregate throughput goes the other way:
serialising on the spinlock lets one core keep the counter in its own L1 and run
long uncontended bursts, while the fast path has all eight adding to the same
line fairly. That burst throughput comes with a much worse worst-case signal()
and a large spread across vCPUs, and it evaporates as soon as there is any work
between signals, which is the case this optimisation targets.

References

…sfy any waiter

signal() used to take splock and walk the wait queue on every single call, even
when the count in hand could satisfy nobody. With several vCPUs signalling one
semaphore -- the shape a WorkPool produces -- that spinlock is where the time
goes, and the walk drags in the waiters' thread structs, which are hot lines of
other vCPUs.

semaphore now keeps m_min_wait, a lower bound of what any waiter needs, or -1
when nobody waits. signal() adds its count and returns immediately as long as
the total stays below that bound. wait_interruptible() gets a lock-free fast
path as well, for the case where the count is already there.

The two sides form a Dekker pair, and both are sequentially consistent, so at
least one of them observes the other:

  signal(): m_count.fetch_add(n)          then load(m_min_wait)
  wait():   m_min_wait.store(count)       then load(m_count)

The invariant is that m_min_wait never exceeds the requirement of any sleeping
waiter. Being too low only costs a needless walk, being too high would lose a
wake-up, so it is only raised back to -1 while holding splock with the queue
observed empty, and every waiter re-publishes on each iteration of its loop --
it may have been resumed and dequeued, but not yet run, while another waiter
drained the queue and reset the bound. The publish-then-enqueue span is covered
by splock, since wait_interruptible() holds it until prepare_usleep() has
pushed the waiter onto the queue.

The wait-side fast path lets a newcomer take a count that queued waiters are
waiting for. That does not change the semantics: the queue was never strict
FIFO -- a resumed waiter has to win try_subtract() against everybody else
anyway, and the out-of-order mode reorders by design.

Measured with thread/test/perf_semaphore.cpp, 8 signalling vCPUs x 50k ops,
4 waiters, 500ns of work between signals, aarch64, MinSizeRel, median of 3:

                    wall        signal() avg    signal() max
  no waiter    53.5 -> 42.6ms    348 -> 171ns
  batch        98.1 -> 36.1ms   1807 -> 179ns   127us -> 0.7us
  stream      261.0 -> 61.6ms   5777 -> 636ns   473us -> 142us

With zero work between signals the aggregate throughput goes the other way,
2ns/op to 15ns/op: serialising on the spinlock lets one core keep the counter in
its own L1 and run long uncontended bursts, while the fast path has all eight
adding to the same line fairly. That burst throughput comes with a 1.4ms
worst-case signal() and a 10x spread across vCPUs, and it evaporates as soon as
there is any work between signals, which is the case this optimisation targets.
Once m_min_wait keeps the hopeless signals out, the ones that remain all pile
onto splock and then onto the same wait queue, each dragging in the waiters'
thread structs. The work is serialized anyway, and it is redundant: whoever is
in the resume path re-reads m_count, so he covers the counts of everybody else.

signal() now claims the path with a RESUMING bit and does the walk for all; a
signaler that finds it taken sets RECHECK instead and leaves. The holder does
not leave until he has read m_count with RECHECK observed clear, and since one
may still set it between that last look and the release, the release itself
reports the bit back and the holder takes the role again.

Every one of these transitions is a read-modify-write on m_resume_state, so
they are totally ordered with each other, and so is the fetch_add on m_count.
RECHECK is cleared before m_count is read, never after. A signaler that leaves
the work to somebody else therefore always has a resume behind it that observes
its count: either the current holder loops again, or -- if he has already gone
past his last look -- his release reports RECHECK and he comes back, or a third
signaler has claimed the path since, and that claim is ordered after our
RECHECK, hence after our count.

The trade is that the holder does the work of all comers, so a signaler can
stay in the loop as long as others keep feeding it. That is the point: it is
the same total work, done on one core without the convoy.

thread/test/perf_semaphore.cpp, the stream case (8 signaling vCPUs x 50k ops,
4 waiters, 500ns of work between signals), interleaved A/B, 10 rounds each,
aarch64, MinSizeRel:

                    wall (median)   signal() avg   signal() worst
  before                   58.3ms          597ns            230us
  after                    48.2ms          348ns             17us

The two wall-time ranges barely overlap (53.3-72.3ms against 43.6-57.1ms), and
the worst case of signal() went from 43-230us in every round down to 3-17us.
One 1.35ms sample did show up in an earlier three-round run, which is what a
long combining stretch looks like; ten rounds did not reproduce it.

test-throttle fails on this machine regardless of these changes -- its loss
ladder is dominated by thread_usleep() granularity, and a three-way run over
the ooo fix alone, m_min_wait, and this commit yields the same numbers within
noise (0.136/0.145/0.3532/0.6587/0.932 against 0.086/0.134/0.3728/0.6689/0.931).
@Coldwings
Coldwings requested a review from lihuiba September 15, 2026 08:50
@Coldwings
Coldwings marked this pull request as draft September 16, 2026 09:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant