From 269c8a20539e6bc83a0ce8aeeaf65abc377ab7f2 Mon Sep 17 00:00:00 2001 From: Scott Marchant <15382220+scottmarchant@users.noreply.github.com> Date: Tue, 18 Aug 2026 10:47:02 -0600 Subject: [PATCH 1/4] docs(wasi-threads): record toolchain, sysroot, and runtime findings Proven end to end on this machine: a 4-thread pthread program with atomics, semaphores, and accurate timed waits compiles with both the wasi-sdk 33 clang and the Swift 6.3.3 host clang against the swift-wasm-6.3-RELEASE threads artifactbundle, and passes under wasmtime v24 LTS with -S threads. Key findings: - _REENTRANT discriminates the threads triple at compile time. - Memory must be imported and shared via explicit link flags, or pthread_create fails with EAGAIN at runtime. - Current wasmtime (v47) has removed wasi-threads; v24 LTS is the one solid runtime today. The proposal is deprecated upstream, so threads mode should be an experimental knob, not the default. - The slice A seams need no change for threads mode: the poke-defer hooks compile to no-ops there, and no API signature moves. This answers the #5 pre-upstream blocker. - Full gate audit of every __wasi__ conditional with a KEEP / SPLIT / THREADS classification, and a threads-mode design sketch. Co-authored-by: Krzysztof Rodak Co-Authored-By: Claude Fable 5 --- tests/wasm/THREADS-RESEARCH.md | 160 +++++++++++++++++++++++++++++++++ 1 file changed, 160 insertions(+) create mode 100644 tests/wasm/THREADS-RESEARCH.md diff --git a/tests/wasm/THREADS-RESEARCH.md b/tests/wasm/THREADS-RESEARCH.md new file mode 100644 index 000000000..992bd9a76 --- /dev/null +++ b/tests/wasm/THREADS-RESEARCH.md @@ -0,0 +1,160 @@ +# Multi-threaded dispatch on wasm32-unknown-wasip1-threads + +Research notes for making libdispatch multi-threaded on WebAssembly. +Fork-only material. Findings below marked "proven" were reproduced on +this machine; the commands live at the end of each section. + +## Summary + +- The threads toolchain works end to end today: a 4-thread pthread + program with atomics, semaphores, and accurate timed waits passes + under wasmtime v24 LTS. Proven. +- `_REENTRANT` is the compile-time discriminator: the threads triple + defines it, the single-threaded triple does not. Proven. +- The slice A seams need no change for threads mode. The poke-defer + hooks compile to no-ops there, exactly as they do on Linux. No API + signature changes. This answers the #5 pre-upstream blocker. +- The runtime landscape is the main constraint, not the toolchain: + current wasmtime (v47) has removed wasi-threads support. Only + wasmtime v24 LTS (proven), WAMR, and toywasm (both unverified) run + it. The wasi-threads proposal is deprecated upstream in favor of + shared-everything-threads. Threads mode should therefore be an + experimental build knob; the cooperative backend stays the default. + +## Toolchain facts (proven) + +SDK: `swift-wasm-6.3-RELEASE-wasm32-unknown-wasip1-threads` +artifactbundle from swiftwasm/swift releases, installed via +`swift sdk install --checksum `. Layout: + + /6.3-RELEASE-wasm32-unknown-wasip1-threads/ + wasm32-unknown-wasip1-threads/WASI.sdk <- sysroot + wasm32-unknown-wasip1-threads/swift.xctoolchain + +Predefines added by `--target=wasm32-unknown-wasip1-threads -pthread`: +`_REENTRANT`, `__wasm_atomics__`, `__wasm_bulk_memory__`. The plain +wasip1 triple defines none of these. + +Two working compile+link combinations, both proven against the +pthread smoke test: + +1. wasi-sdk 33 clang with its own sysroot. +2. Swift 6.3.3 host clang + the artifactbundle sysroot + an explicit + compiler-rt builtins archive (the host toolchain does not ship + builtins for the threads triple). wasi-sdk 33 provides one at + `lib/clang/22/lib/wasm32-unknown-wasip1-threads/libclang_rt.builtins.a` + and it links fine from the Swift clang. This mirrors the + `DISPATCH_WASI_BUILTINS` pattern the WASI toolchain file already + uses. + +Required link flags, or the module gets a private non-shared memory +and every `pthread_create` fails with EAGAIN (we hit this): + + -Wl,--import-memory,--export-memory,--max-memory= + +`-pthread` alone adds `--shared-memory` but does not import/export the +memory, and wasi-threads hosts can only spawn threads against an +imported shared memory. + +## Sysroot surface (proven by header inspection) + +wasi-libc's posix THREAD_MODEL is full musl: `pthread_create`, `join`, +`detach`, mutexes (incl. recursive), condvars with `condattr_setclock`, +rwlocks, barriers, once, TLS keys, `pthread_getname_np`. `semaphore.h` +has `sem_init/wait/trywait/timedwait/post/getvalue`. Timed waits work +and are accurate: `sem_timedwait` 150 ms deadline observed at 160 ms, +`pthread_cond_timedwait` 100 ms observed at 100 ms (proven under +wasmtime v24). + +Still absent, unchanged from single-threaded WASI: signals to +processes (`kill`), `sigaction`, `fork`/`exec`, `pipe(2)` and socket +creation, `mmap`. The capability model is the same; only threads and +shared memory are new. + +## Runtime support for wasi-threads + +| Runtime | Status | +|---|---| +| wasmtime v24 LTS | Works: `wasmtime run -S threads app.wasm`. Proven. | +| wasmtime v47 (current) | Removed. `-S threads` rejected; `wasi::thread-spawn` import unfulfilled. Proven. | +| Node.js built-in WASI | No wasi-threads. A worker_threads-based shim is possible but nobody ships one. | +| browser (bjorn3 shim, uwasi) | No `thread-spawn`. Same shim caveat. | +| WasmKit | No wasi-threads. | +| WAMR, toywasm | Claim support. Not verified here. | + +Strategic context: the wasi-threads proposal is frozen/deprecated; +the successor is the shared-everything-threads proposal (component +model era). Compiled modules still target `wasi::thread-spawn` today. +Consequence: threads mode is a forward-looking experiment with one +solid LTS runtime, not a replacement for the cooperative backend. + +## Gate audit: every `__wasi__` conditional in the port + +Classification: KEEP (correct for both modes), SPLIT (single-thread +only; re-gate on `defined(__wasi__) && !defined(_REENTRANT)`), THREADS +(needs a threaded replacement). + +| Site | Class | Notes | +|---|---|---| +| `internal.h` header excludes (sys/mount, sysctl, syslog) | KEEP | libc surface identical | +| `shims.h`, `shims/getprogname.h`, `shims/time.h`, `shims/hw_config.h`, `transform.c` | KEEP | byte order, progname, clocks, cpu count unchanged (`sysconf(_SC_NPROCESSORS_ONLN)` path already exists) | +| `io.c` absolute-path + no-mkfifo gates | KEEP | capability model unchanged | +| `event/event_config.h` backend select | SPLIT+THREADS | threads mode needs an event strategy (below); cooperative backend is single-thread only | +| `shims/lock.h` tid/lock encoding arm | SPLIT | threads mode: derive `dispatch_tid` from `pthread_self()`; musl pthread pointers are >=4-aligned so the `tid<<2`-style low-bit space still works | +| `shims/lock.h` sema4 counter arm | SPLIT | threads mode: `USE_POSIX_SEM=1` (sem_t proven). Later option: real futex via `__builtin_wasm_memory_atomic_wait32/notify` for WAIT_ON_ADDRESS | +| `shims/lock.c` three wait/park arms | SPLIT | threads mode blocks for real; no pumping | +| `queue.c` poke -> eager-drain sites (~32 gates) | SPLIT | threads mode pokes wake workers, upstream shape | +| `queue.c` `DISPATCH_USE_PTHREAD_POOL` excludes | THREADS | re-enable the pthread root-queue pool, Linux shape | +| `queue.c` main-queue drain `#if DISPATCH_COCOA_COMPAT \|\| __wasi__` | KEEP | slice A hoist serves both modes; in threads mode `dispatch_main` can also park normally | +| `init.c` gates (4) | SPLIT | mix: priority/workqueue init returns | +| CMake `CMAKE_HAVE_LIBC_PTHREAD`, `HAVE_*` cache arm | SPLIT | threads mode: internal workqueue on, `USE_POSIX_SEM=1`, pthread pool on | + +Slice A files (`event_internal.h` hook macros, `queue.c` brackets, +`object.c`, `once.c`): NO CHANGE. In threads mode the hooks stay +`((void)0)` like every other threaded platform. The seams and API +signatures survive as designed. + +## Threads-mode design sketch + +- Worker model: `DISPATCH_USE_INTERNAL_WORKQUEUE=1` plus the pthread + root-queue pool, the same shape Linux uses. `pthread_create` is + proven under wasmtime v24. +- Locks: keep the dq_state encoding; `dispatch_tid` from + `pthread_self()`. +- Semaphores: POSIX sem_t (proven, incl. timed waits). +- Timers: a manager thread on `pthread_cond_timedwait` needs no fds. +- fd sources: the hard problem. `poll_oneoff` has no cross-thread + wakeup object in wasip1-threads (no self-pipe, no eventfd; atomics + cannot interrupt a poll). A poller thread must use bounded poll + slices (the WasmKit guard already established the pattern), trading + arm/cancel latency for correctness. +- `dispatch_main()`: park on a semaphore like Linux; the slice A + drain hoist still serves the main-queue drain path. + +## Consequences for the slice plan + +1. #5 (slice A) is unblocked: threads mode needs zero changes to the + seams or their signatures. Proceed with the upstream copy. +2. Threads mode fits as an additive experimental slice after B1 + (build knob + gate splits + worker pool), with fd sources deferred + until a runtime story firms up. +3. The test runner needs a wasmtime v24 pin (or WAMR) for any threads + CI lane; current wasmtime cannot run it. + +## Reproduction commands + + # smoke (4 threads, atomics + semaphore): PASS counter=10 threads=4 + wasi-sdk-33.0/bin/clang --target=wasm32-unknown-wasip1-threads \ + -pthread -O1 threads-smoke.c -o threads-smoke.wasm \ + -Wl,--import-memory,--export-memory,--max-memory=67108864 + wasmtime-v24.0.5/wasmtime run -S threads threads-smoke.wasm + + # same source through the Swift host clang: PASS + swift-6.3.3-RELEASE.xctoolchain/usr/bin/clang \ + --target=wasm32-unknown-wasip1-threads --sysroot= \ + -pthread -O1 -nodefaultlibs threads-smoke.c -o out.wasm -lc \ + wasi-sdk-33.0/lib/clang/22/lib/wasm32-unknown-wasip1-threads/libclang_rt.builtins.a \ + -Wl,--import-memory,--export-memory,--max-memory=67108864 + + # timed waits: sem_timedwait 150ms -> 160ms, cond_timedwait 100ms -> 100ms + # (same compile line, timed-waits.c) From ef888d45b9e121f4126c45694a2ea5fe2624d757 Mon Sep 17 00:00:00 2001 From: Scott Marchant <15382220+scottmarchant@users.noreply.github.com> Date: Tue, 18 Aug 2026 10:49:03 -0600 Subject: [PATCH 2/4] build(wasi-threads): add a DISPATCH_WASI_THREADS toolchain knob DISPATCH_WASI_THREADS=ON retargets the WASI toolchain file to wasm32-unknown-wasip1-threads: it switches the C, C++, and Swift target triples, adds -pthread, and links with --import-memory,--export-memory,--max-memory=DISPATCH_WASI_MAX_MEMORY (default 256 MiB, cache-overridable). wasi-threads hosts spawn every thread against one imported shared memory; a module that owns a private memory fails at the first pthread_create with EAGAIN, so the import flags are not optional. The knob defaults to OFF and the default configuration is unchanged: the existing single-threaded build reconfigures and builds identically. The threads bundle ships its builtins archive at the same relative path, so the DISPATCH_WASI_BUILTINS default holds for both SDK layouts (verified by linking and running a pthread smoke test with the bundle builtins under wasmtime v24). Co-authored-by: Krzysztof Rodak Co-Authored-By: Claude Fable 5 --- cmake/toolchains/WASI.cmake | 35 +++++++++++++++++++++++++++++------ 1 file changed, 29 insertions(+), 6 deletions(-) diff --git a/cmake/toolchains/WASI.cmake b/cmake/toolchains/WASI.cmake index b405cea5e..f64a82708 100644 --- a/cmake/toolchains/WASI.cmake +++ b/cmake/toolchains/WASI.cmake @@ -9,16 +9,22 @@ set(CMAKE_SYSTEM_PROCESSOR wasm32) set(SWIFT_WASI_TOOLCHAIN_PATH "${SWIFT_WASI_TOOLCHAIN_PATH}" CACHE PATH "Host Swift .xctoolchain used to build for WASI") set(SWIFT_WASI_SDK_PATH "${SWIFT_WASI_SDK_PATH}" CACHE PATH - "wasm32-unknown-wasip1 directory in a Swift WASI SDK") + "wasm32-unknown-wasip1 or wasm32-unknown-wasip1-threads directory in a Swift WASI SDK") set(SWIFT_WASI_STATIC_RESOURCES_OVERRIDE "" CACHE PATH "Optional override for the Swift static resource directory") set(DISPATCH_WASI_BUILTINS_OVERRIDE "" CACHE FILEPATH "Optional override for the WASI compiler-rt builtins archive") +option(DISPATCH_WASI_THREADS + "Experimental: target wasm32-unknown-wasip1-threads (wasi-threads hosts only)" OFF) +set(DISPATCH_WASI_MAX_MEMORY "268435456" CACHE STRING + "Shared-memory maximum in bytes for threads builds (wasm memories need a hard cap)") list(APPEND CMAKE_TRY_COMPILE_PLATFORM_VARIABLES SWIFT_WASI_TOOLCHAIN_PATH SWIFT_WASI_SDK_PATH SWIFT_WASI_STATIC_RESOURCES_OVERRIDE - DISPATCH_WASI_BUILTINS_OVERRIDE) + DISPATCH_WASI_BUILTINS_OVERRIDE + DISPATCH_WASI_THREADS + DISPATCH_WASI_MAX_MEMORY) if(NOT SWIFT_WASI_TOOLCHAIN_PATH) message(FATAL_ERROR "Set SWIFT_WASI_TOOLCHAIN_PATH to the host Swift .xctoolchain") @@ -85,25 +91,42 @@ if(ENABLE_SWIFT) endif() endif() +if(DISPATCH_WASI_THREADS) + set(_dispatch_wasi_triple wasm32-unknown-wasip1-threads) +else() + set(_dispatch_wasi_triple wasm32-unknown-wasip1) +endif() + set(CMAKE_C_COMPILER "${_dispatch_wasi_clang}") set(CMAKE_CXX_COMPILER "${_dispatch_wasi_clangxx}") set(CMAKE_AR "${_dispatch_wasi_ar}") set(CMAKE_RANLIB "${_dispatch_wasi_ranlib}") -set(CMAKE_C_COMPILER_TARGET wasm32-unknown-wasip1) -set(CMAKE_CXX_COMPILER_TARGET wasm32-unknown-wasip1) +set(CMAKE_C_COMPILER_TARGET ${_dispatch_wasi_triple}) +set(CMAKE_CXX_COMPILER_TARGET ${_dispatch_wasi_triple}) set(CMAKE_SYSROOT "${_dispatch_wasi_sysroot}") set(CMAKE_TRY_COMPILE_TARGET_TYPE STATIC_LIBRARY) set(CMAKE_EXECUTABLE_SUFFIX .wasm) +if(DISPATCH_WASI_THREADS) + # -pthread turns on atomics and shared memory. The memory must also be + # imported and exported with a hard maximum: wasi-threads hosts spawn + # each thread against the same imported shared memory, and a module + # that owns a private memory fails at the first pthread_create. + string(APPEND CMAKE_C_FLAGS_INIT " -pthread") + string(APPEND CMAKE_CXX_FLAGS_INIT " -pthread") + string(APPEND CMAKE_EXE_LINKER_FLAGS_INIT + " -Wl,--import-memory,--export-memory,--max-memory=${DISPATCH_WASI_MAX_MEMORY}") +endif() + if(ENABLE_SWIFT) if(NOT EXISTS "${_dispatch_wasi_swiftc}") message(FATAL_ERROR "Swift compiler does not exist: ${_dispatch_wasi_swiftc}") endif() set(CMAKE_Swift_COMPILER "${_dispatch_wasi_swiftc}") - set(CMAKE_Swift_COMPILER_TARGET wasm32-unknown-wasip1) + set(CMAKE_Swift_COMPILER_TARGET ${_dispatch_wasi_triple}) set(CMAKE_Swift_FLAGS "-sdk \"${CMAKE_SYSROOT}\" -resource-dir \"${SWIFT_WASI_STATIC_RESOURCES}\"") - set(dispatch_MODULE_TRIPLE wasm32-unknown-wasip1 CACHE STRING "Swift module triple") + set(dispatch_MODULE_TRIPLE ${_dispatch_wasi_triple} CACHE STRING "Swift module triple") set(dispatch_ARCH wasm32 CACHE STRING "Swift architecture") set(dispatch_PLATFORM wasi CACHE STRING "Swift platform") endif() From b5932211e0b0c9d59b21e74b19a5ff5f80fe65bc Mon Sep 17 00:00:00 2001 From: Scott Marchant <15382220+scottmarchant@users.noreply.github.com> Date: Tue, 18 Aug 2026 12:51:21 -0600 Subject: [PATCH 3/4] feat(wasi-threads): compile libdispatch for wasm32-unknown-wasip1-threads Split every single-thread assumption on a new DISPATCH_WASI_COOPERATIVE discriminator (defined(__wasi__) && !defined(_REENTRANT)). The threads triple now behaves like a generic POSIX platform: - queue.c: the cooperative poke/drain arms compile only in cooperative mode; the pthread worker pool, root-queue drain, and worker threads compile in threads mode, the Linux shape. _gettid() derives a unique nonzero per-thread id from pthread_self() (4-aligned, so it survives the tid << 2 lock-owner encoding). - shims/lock: threads mode gets a real futex via __builtin_wasm_memory_atomic_wait32/notify for _dispatch_wait_on_address(), sched_yield for _dispatch_thread_switch, and POSIX semaphores for sema4 (USE_POSIX_SEM=1 in the WASI CMake arm when DISPATCH_WASI_THREADS is on). - event: a new DISPATCH_EVENT_BACKEND_WASI_THREADS backend (event_wasi_threads.c). The manager thread parks on a CLOCK_MONOTONIC condition variable and merges due timers; pokes signal the condvar. wasip1 has no poll wakeup object (no eventfd, no pipe creation), so fd and signal sources crash with a named message until a bounded-slice poller thread lands. - dispatch_main() crashes with a named message in threads mode: wasi-libc has no pthread_exit (a wasi thread ends only by returning from its start function), so the generic park does not exist. - DISPATCH_HW_CONFIG_UP stays cooperative-only. The threads target is not a uniprocessor, and UP also shrinks continuations below sizeof(struct dispatch_apply_s) on wasm32. The library builds clean (-Werror) in both modes. The cooperative build is unchanged: 52 of 52 ctest cases still pass. Co-authored-by: Krzysztof Rodak Co-Authored-By: Claude Fable 5 --- CMakeLists.txt | 8 +- src/CMakeLists.txt | 10 +- src/event/event_config.h | 14 +- src/event/event_wasi_threads.c | 277 +++++++++++++++++++++++++++++++++ src/internal.h | 10 ++ src/queue.c | 65 +++++--- src/shims/lock.c | 30 +++- src/shims/lock.h | 5 +- 8 files changed, 386 insertions(+), 33 deletions(-) create mode 100644 src/event/event_wasi_threads.c diff --git a/CMakeLists.txt b/CMakeLists.txt index 35a3e03d7..4e8e883f8 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -207,7 +207,13 @@ if(CMAKE_SYSTEM_NAME STREQUAL "WASI") set(HAVE_STRLCPY 1 CACHE BOOL "WASI libc result") set(HAVE_SYSCONF 1 CACHE BOOL "WASI libc result") set(HAVE_ARC4RANDOM 1 CACHE BOOL "WASI libc result") - set(USE_POSIX_SEM 0 CACHE BOOL "WASI libc result") + if(DISPATCH_WASI_THREADS) + # wasip1-threads: sem_t is a real blocking semaphore (proven under + # wasmtime v24, including sem_timedwait) + set(USE_POSIX_SEM 1 CACHE BOOL "WASI threads libc result") + else() + set(USE_POSIX_SEM 0 CACHE BOOL "WASI libc result") + endif() endif() diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index 50338ea7d..6a57b27ab 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -39,6 +39,7 @@ add_library(dispatch event/event_internal.h event/event_kevent.c event/event_wasi.c + event/event_wasi_threads.c event/event_windows.c firehose/firehose_internal.h shims/android_stubs.h @@ -107,8 +108,13 @@ elseif(ANDROID) target_compile_options(dispatch PRIVATE -U_GNU_SOURCE) elseif(CMAKE_SYSTEM_NAME STREQUAL "WASI") - target_compile_definitions(dispatch PRIVATE - DISPATCH_HW_CONFIG_UP=1) + if(NOT DISPATCH_WASI_THREADS) + # single-threaded wasip1 is a uniprocessor by construction; the threads + # target is not (and DISPATCH_HW_CONFIG_UP also shrinks continuations + # below sizeof(struct dispatch_apply_s) there) + target_compile_definitions(dispatch PRIVATE + DISPATCH_HW_CONFIG_UP=1) + endif() target_compile_definitions(dispatch PUBLIC "$<$:_WASI_EMULATED_SIGNAL>" "$<$:_WASI_EMULATED_MMAN>" diff --git a/src/event/event_config.h b/src/event/event_config.h index 32d43707d..5066d9a4b 100644 --- a/src/event/event_config.h +++ b/src/event/event_config.h @@ -21,27 +21,39 @@ #ifndef __DISPATCH_EVENT_EVENT_CONFIG__ #define __DISPATCH_EVENT_EVENT_CONFIG__ -#if defined(__wasi__) +#if defined(__wasi__) && !defined(_REENTRANT) # define DISPATCH_EVENT_BACKEND_EPOLL 0 # define DISPATCH_EVENT_BACKEND_KEVENT 0 # define DISPATCH_EVENT_BACKEND_WASI 1 +# define DISPATCH_EVENT_BACKEND_WASI_THREADS 0 +# define DISPATCH_EVENT_BACKEND_WINDOWS 0 +#elif defined(__wasi__) +// wasm32-wasip1-threads: experimental threaded mode; a manager thread on a +// condition variable replaces the cooperative drain (event_wasi_threads.c) +# define DISPATCH_EVENT_BACKEND_EPOLL 0 +# define DISPATCH_EVENT_BACKEND_KEVENT 0 +# define DISPATCH_EVENT_BACKEND_WASI 0 +# define DISPATCH_EVENT_BACKEND_WASI_THREADS 1 # define DISPATCH_EVENT_BACKEND_WINDOWS 0 #elif defined(__linux__) # include # define DISPATCH_EVENT_BACKEND_EPOLL 1 # define DISPATCH_EVENT_BACKEND_KEVENT 0 # define DISPATCH_EVENT_BACKEND_WASI 0 +# define DISPATCH_EVENT_BACKEND_WASI_THREADS 0 # define DISPATCH_EVENT_BACKEND_WINDOWS 0 #elif __has_include() # include # define DISPATCH_EVENT_BACKEND_EPOLL 0 # define DISPATCH_EVENT_BACKEND_KEVENT 1 # define DISPATCH_EVENT_BACKEND_WASI 0 +# define DISPATCH_EVENT_BACKEND_WASI_THREADS 0 # define DISPATCH_EVENT_BACKEND_WINDOWS 0 #elif defined(_WIN32) # define DISPATCH_EVENT_BACKEND_EPOLL 0 # define DISPATCH_EVENT_BACKEND_KEVENT 0 # define DISPATCH_EVENT_BACKEND_WASI 0 +# define DISPATCH_EVENT_BACKEND_WASI_THREADS 0 # define DISPATCH_EVENT_BACKEND_WINDOWS 1 #else # error unsupported event loop diff --git a/src/event/event_wasi_threads.c b/src/event/event_wasi_threads.c new file mode 100644 index 000000000..377110782 --- /dev/null +++ b/src/event/event_wasi_threads.c @@ -0,0 +1,277 @@ +/* + * Copyright (c) 2026 Apple Inc. All rights reserved. + * + * @APPLE_APACHE_LICENSE_HEADER_START@ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * @APPLE_APACHE_LICENSE_HEADER_END@ + */ + +#include "internal.h" +#if DISPATCH_EVENT_BACKEND_WASI_THREADS + +// Experimental event backend for wasm32-unknown-wasip1-threads. +// +// Threaded WASI has real worker threads, so the queue machinery runs the +// upstream shape: pokes wake workers, blocking waits block. What it does not +// have is any poll wakeup object (no eventfd, no self-pipe: wasip1 cannot +// create pipes or sockets), so the manager thread parks on a pthread +// condition variable instead of a poller. That supports timers and manager +// work. File-descriptor and signal event sources need a bounded-slice +// poll_oneoff thread and are not implemented yet; registering one crashes +// with a named message. + +#include +#include +#include + +#if !DISPATCH_USE_MGR_THREAD +#error unsupported configuration +#endif + +#if !defined(_REENTRANT) +#error this backend requires the wasip1-threads target (-pthread) +#endif + +typedef struct dispatch_wasi_threads_timeout_s { + uint64_t dwt_deadline; // uptime-anchored, nanoseconds + bool dwt_armed; +} dispatch_wasi_threads_timeout_s; + +static dispatch_wasi_threads_timeout_s + _dispatch_wasi_threads_timeout[DISPATCH_CLOCK_COUNT]; + +static pthread_mutex_t _dispatch_wasi_threads_mgr_mutex = + PTHREAD_MUTEX_INITIALIZER; +static pthread_cond_t _dispatch_wasi_threads_mgr_cond; +static bool _dispatch_wasi_threads_mgr_poked; +static dispatch_once_t _dispatch_wasi_threads_init_pred; + +static void +_dispatch_wasi_threads_init(void *context DISPATCH_UNUSED) +{ + pthread_condattr_t attr; + (void)dispatch_assume_zero(pthread_condattr_init(&attr)); + (void)dispatch_assume_zero( + pthread_condattr_setclock(&attr, CLOCK_MONOTONIC)); + (void)dispatch_assume_zero( + pthread_cond_init(&_dispatch_wasi_threads_mgr_cond, &attr)); + (void)dispatch_assume_zero(pthread_condattr_destroy(&attr)); + + // hand the manager queue to its root queue so the manager thread spawns + _dispatch_trace_item_push(_dispatch_mgr_q.do_targetq, &_dispatch_mgr_q); + dx_push(_dispatch_mgr_q.do_targetq, &_dispatch_mgr_q, 0); +} + +#pragma mark unotes + +bool +_dispatch_unote_register_muxed(dispatch_unote_t du DISPATCH_UNUSED) +{ + DISPATCH_CLIENT_CRASH(0, "file-descriptor and signal dispatch sources " + "are not supported on threaded WASI yet"); +} + +void +_dispatch_unote_resume_muxed(dispatch_unote_t du DISPATCH_UNUSED) +{ + DISPATCH_CLIENT_CRASH(0, "file-descriptor and signal dispatch sources " + "are not supported on threaded WASI yet"); +} + +bool +_dispatch_unote_unregister_muxed(dispatch_unote_t du DISPATCH_UNUSED) +{ + DISPATCH_CLIENT_CRASH(0, "file-descriptor and signal dispatch sources " + "are not supported on threaded WASI yet"); +} + +#pragma mark timers + +static void +_dispatch_event_merge_timer(dispatch_clock_t clock) +{ + dispatch_timer_heap_t dth = _dispatch_timers_heap; + uint32_t tidx = DISPATCH_TIMER_INDEX(clock, 0); + + _dispatch_wasi_threads_timeout[clock].dwt_armed = false; + + _dispatch_timers_heap_dirty(dth, tidx); + dth[tidx].dth_needs_program = true; + dth[tidx].dth_armed = false; +} + +void +_dispatch_event_loop_timer_arm(dispatch_timer_heap_t dth DISPATCH_UNUSED, + uint32_t tidx, dispatch_timer_delay_s range, + dispatch_clock_now_cache_t nows) +{ + dispatch_clock_t clock = DISPATCH_TIMER_CLOCK(tidx); + + // all clocks advance in nanoseconds; anchoring every deadline on the + // uptime clock keeps a single sleepable domain (the documented + // wall-clock limitation of the cooperative backend applies here too) + uint64_t deadline = range.delay + + _dispatch_time_now_cached(DISPATCH_CLOCK_UPTIME, nows); + (void)dispatch_assume_zero( + pthread_mutex_lock(&_dispatch_wasi_threads_mgr_mutex)); + _dispatch_wasi_threads_timeout[clock].dwt_deadline = deadline; + _dispatch_wasi_threads_timeout[clock].dwt_armed = true; + (void)dispatch_assume_zero( + pthread_cond_signal(&_dispatch_wasi_threads_mgr_cond)); + (void)dispatch_assume_zero( + pthread_mutex_unlock(&_dispatch_wasi_threads_mgr_mutex)); +} + +void +_dispatch_event_loop_timer_delete(dispatch_timer_heap_t dth DISPATCH_UNUSED, + uint32_t tidx) +{ + dispatch_clock_t clock = DISPATCH_TIMER_CLOCK(tidx); + (void)dispatch_assume_zero( + pthread_mutex_lock(&_dispatch_wasi_threads_mgr_mutex)); + _dispatch_wasi_threads_timeout[clock].dwt_armed = false; + (void)dispatch_assume_zero( + pthread_mutex_unlock(&_dispatch_wasi_threads_mgr_mutex)); +} + +// caller must hold _dispatch_wasi_threads_mgr_mutex +static uint64_t +_dispatch_wasi_threads_next_deadline(void) +{ + uint64_t next = 0; + for (size_t i = 0; i < countof(_dispatch_wasi_threads_timeout); i++) { + if (!_dispatch_wasi_threads_timeout[i].dwt_armed) continue; + uint64_t deadline = _dispatch_wasi_threads_timeout[i].dwt_deadline; + if (!next || deadline < next) next = deadline; + } + return next; +} + +#pragma mark dispatch_loop + +void +_dispatch_event_loop_atfork_child(void) +{ +} + +void +_dispatch_event_loop_poke(dispatch_wlh_t wlh DISPATCH_UNUSED, + uint64_t dq_state DISPATCH_UNUSED, uint32_t flags DISPATCH_UNUSED) +{ + dispatch_once_f(&_dispatch_wasi_threads_init_pred, NULL, + _dispatch_wasi_threads_init); + (void)dispatch_assume_zero( + pthread_mutex_lock(&_dispatch_wasi_threads_mgr_mutex)); + _dispatch_wasi_threads_mgr_poked = true; + (void)dispatch_assume_zero( + pthread_cond_signal(&_dispatch_wasi_threads_mgr_cond)); + (void)dispatch_assume_zero( + pthread_mutex_unlock(&_dispatch_wasi_threads_mgr_mutex)); +} + +DISPATCH_NOINLINE +void +_dispatch_event_loop_drain(uint32_t flags) +{ + if (flags & KEVENT_FLAG_IMMEDIATE) { + // nothing to collect eagerly: there is no poller, and due timers + // are the manager thread's job (it wakes on its own timedwait) + return; + } + + (void)dispatch_assume_zero( + pthread_mutex_lock(&_dispatch_wasi_threads_mgr_mutex)); + for (;;) { + if (_dispatch_wasi_threads_mgr_poked) break; + uint64_t next = _dispatch_wasi_threads_next_deadline(); + uint64_t now = _dispatch_uptime(); + if (next && next <= now) break; + if (next) { + uint64_t delta = next - now; + struct timespec ts; + (void)dispatch_assume_zero(clock_gettime(CLOCK_MONOTONIC, &ts)); + ts.tv_sec += (time_t)(delta / NSEC_PER_SEC); + ts.tv_nsec += (long)(delta % NSEC_PER_SEC); + if (ts.tv_nsec >= (long)NSEC_PER_SEC) { + ts.tv_sec += 1; + ts.tv_nsec -= (long)NSEC_PER_SEC; + } + int rc = pthread_cond_timedwait(&_dispatch_wasi_threads_mgr_cond, + &_dispatch_wasi_threads_mgr_mutex, &ts); + if (rc != 0 && rc != ETIMEDOUT) { + DISPATCH_INTERNAL_CRASH(rc, "pthread_cond_timedwait"); + } + } else { + (void)dispatch_assume_zero( + pthread_cond_wait(&_dispatch_wasi_threads_mgr_cond, + &_dispatch_wasi_threads_mgr_mutex)); + } + } + _dispatch_wasi_threads_mgr_poked = false; + + uint64_t now = _dispatch_uptime(); + for (size_t i = 0; i < countof(_dispatch_wasi_threads_timeout); i++) { + if (_dispatch_wasi_threads_timeout[i].dwt_armed && + _dispatch_wasi_threads_timeout[i].dwt_deadline <= now) { + _dispatch_event_merge_timer((dispatch_clock_t)i); + } + } + (void)dispatch_assume_zero( + pthread_mutex_unlock(&_dispatch_wasi_threads_mgr_mutex)); +} + +void +_dispatch_event_loop_cancel_waiter(dispatch_sync_context_t dsc) +{ + (void)dsc; +} + +void +_dispatch_event_loop_wake_owner(dispatch_sync_context_t dsc, + dispatch_wlh_t wlh, uint64_t old_state, uint64_t new_state) +{ + (void)dsc; (void)wlh; (void)old_state; (void)new_state; +} + +void +_dispatch_event_loop_wait_for_ownership(dispatch_sync_context_t dsc) +{ + if (dsc->dsc_release_storage) { + _dispatch_queue_release_storage(dsc->dc_data); + } +} + +void +_dispatch_event_loop_end_ownership(dispatch_wlh_t wlh, uint64_t old_state, + uint64_t new_state, uint32_t flags) +{ + (void)wlh; (void)old_state; (void)new_state; (void)flags; +} + +#if DISPATCH_WLH_DEBUG +void +_dispatch_event_loop_assert_not_owned(dispatch_wlh_t wlh) +{ + (void)wlh; +} +#endif + +void +_dispatch_event_loop_leave_immediate(uint64_t dq_state) +{ + (void)dq_state; +} + +#endif // DISPATCH_EVENT_BACKEND_WASI_THREADS diff --git a/src/internal.h b/src/internal.h index 690c2579c..b669a0a6c 100644 --- a/src/internal.h +++ b/src/internal.h @@ -667,6 +667,16 @@ _dispatch_fork_becomes_unsafe(void) #define DISPATCH_PERF_MON 0 #endif +// wasm32-wasip1 is single-threaded and uses the cooperative event backend. +// wasm32-wasip1-threads (-pthread, which defines _REENTRANT) is an +// experimental threaded mode that behaves like a generic POSIX platform: +// real worker threads, blocking waits, no cooperative pumping. +#if defined(__wasi__) && !defined(_REENTRANT) +#define DISPATCH_WASI_COOPERATIVE 1 +#else +#define DISPATCH_WASI_COOPERATIVE 0 +#endif + /* #includes dependent on internal.h */ #include "shims.h" #include "event/event_internal.h" diff --git a/src/queue.c b/src/queue.c index 6b0c77af8..20e4156bd 100644 --- a/src/queue.c +++ b/src/queue.c @@ -5389,7 +5389,7 @@ _dispatch_queue_mgr_lock(struct dispatch_queue_static_s *dq) }); } -#if DISPATCH_USE_KEVENT_WORKQUEUE || defined(__wasi__) +#if DISPATCH_USE_KEVENT_WORKQUEUE || DISPATCH_WASI_COOPERATIVE DISPATCH_ALWAYS_INLINE static inline bool _dispatch_queue_mgr_unlock(struct dispatch_queue_static_s *dq) @@ -5402,7 +5402,7 @@ _dispatch_queue_mgr_unlock(struct dispatch_queue_static_s *dq) }); return _dq_state_is_dirty(old_state); } -#endif // DISPATCH_USE_KEVENT_WORKQUEUE || defined(__wasi__) +#endif // DISPATCH_USE_KEVENT_WORKQUEUE || DISPATCH_WASI_COOPERATIVE static void _dispatch_mgr_queue_drain(void) @@ -5432,7 +5432,7 @@ _dispatch_mgr_queue_drain(void) } } -#if defined(__wasi__) +#if DISPATCH_WASI_COOPERATIVE DISPATCH_NOINLINE void _dispatch_wasi_mgr_queue_drain(void) @@ -5452,7 +5452,7 @@ _dispatch_wasi_mgr_queue_drain(void) _dispatch_event_loop_poke(DISPATCH_WLH_MANAGER, 0, 0); } } -#endif // defined(__wasi__) +#endif // DISPATCH_WASI_COOPERATIVE void _dispatch_mgr_queue_push(dispatch_lane_t dq, dispatch_object_t dou, @@ -5736,12 +5736,12 @@ _dispatch_workloop_worker_thread(uint64_t *workloop_id, #pragma mark - #pragma mark dispatch_root_queue -#if DISPATCH_USE_PTHREAD_POOL && !defined(__wasi__) +#if DISPATCH_USE_PTHREAD_POOL && !DISPATCH_WASI_COOPERATIVE static void *_dispatch_worker_thread(void *context); #if defined(_WIN32) static unsigned WINAPI _dispatch_worker_thread_thunk(LPVOID lpParameter); #endif -#endif // DISPATCH_USE_PTHREAD_POOL && !defined(__wasi__) +#endif // DISPATCH_USE_PTHREAD_POOL && !DISPATCH_WASI_COOPERATIVE #if DISPATCH_DEBUG && DISPATCH_ROOT_QUEUE_DEBUG #define _dispatch_root_queue_debug(...) _dispatch_debug(__VA_ARGS__) @@ -5763,7 +5763,7 @@ DISPATCH_NOINLINE static void _dispatch_root_queue_poke_slow(dispatch_queue_global_t dq, int n, int floor) { -#if defined(__wasi__) +#if DISPATCH_WASI_COOPERATIVE // Single-threaded WASI: there is no thread to create. Record a single // pending "worker" in dgq_pending (consumed by the matching decrement in // _dispatch_wasi_root_queue_drain(), mirroring _dispatch_worker_thread2) @@ -5787,7 +5787,7 @@ _dispatch_root_queue_poke_slow(dispatch_queue_global_t dq, int n, int floor) return; } _dispatch_wasi_root_queue_poke(dq); -#else // defined(__wasi__) +#else // DISPATCH_WASI_COOPERATIVE int remaining = n; #if !defined(_WIN32) int r = ENOSYS; @@ -5895,7 +5895,7 @@ _dispatch_root_queue_poke_slow(dispatch_queue_global_t dq, int n, int floor) #else (void)floor; #endif // DISPATCH_USE_PTHREAD_POOL -#endif // defined(__wasi__) +#endif // DISPATCH_WASI_COOPERATIVE } DISPATCH_NOINLINE @@ -6195,7 +6195,7 @@ _dispatch_root_queue_drain_deferred_item(dispatch_deferred_items_t ddi } #endif -#if !defined(__wasi__) +#if !DISPATCH_WASI_COOPERATIVE DISPATCH_NOT_TAIL_CALLED // prevent tailcall (for Instrument DTrace probe) static void _dispatch_root_queue_drain(dispatch_queue_global_t dq, @@ -6242,9 +6242,9 @@ _dispatch_root_queue_drain(dispatch_queue_global_t dq, _dispatch_clear_basepri(); _dispatch_queue_set_current(NULL); } -#endif // !defined(__wasi__) +#endif // !DISPATCH_WASI_COOPERATIVE -#if defined(__wasi__) +#if DISPATCH_WASI_COOPERATIVE DISPATCH_NOINLINE void _dispatch_wasi_root_queue_drain(dispatch_queue_global_t dq) @@ -6281,7 +6281,7 @@ _dispatch_wasi_root_queue_drain(dispatch_queue_global_t dq) _dispatch_clear_basepri(); _dispatch_queue_set_current(old_dq); } -#endif // defined(__wasi__) +#endif // DISPATCH_WASI_COOPERATIVE #if !DISPATCH_USE_INTERNAL_WORKQUEUE static void @@ -6340,7 +6340,7 @@ _dispatch_root_queue_init_pthread_pool(dispatch_queue_global_t dq, _dispatch_sema4_create(sema, _DSEMA4_POLICY_LIFO); } -#if !defined(__wasi__) +#if !DISPATCH_WASI_COOPERATIVE // 6618342 Contact the team that owns the Instrument DTrace probe before // renaming this symbol static void * @@ -6482,7 +6482,7 @@ _dispatch_worker_thread_thunk(LPVOID lpParameter) return 0; } #endif // defined(_WIN32) -#endif // !defined(__wasi__) +#endif // !DISPATCH_WASI_COOPERATIVE #endif // DISPATCH_USE_PTHREAD_POOL DISPATCH_NOINLINE @@ -7003,7 +7003,7 @@ _dispatch_main_queue_update_priority_from_thread(void) } #endif // DISPATCH_COCOA_COMPAT -#if DISPATCH_COCOA_COMPAT || defined(__wasi__) +#if DISPATCH_COCOA_COMPAT || DISPATCH_WASI_COOPERATIVE // Shared between the CFRunLoop callback path (DISPATCH_COCOA_COMPAT) and the // cooperative WASI drain, which owns the thread-bound main queue's drain lock // for the lifetime of the program. The runloop-handle initialization and the @@ -7019,7 +7019,7 @@ _dispatch_main_queue_drain(dispatch_queue_main_t dq) } _dispatch_perfmon_start_notrace(); -#if defined(__wasi__) +#if DISPATCH_WASI_COOPERATIVE #define _DISPATCH_MAIN_QUEUE_DRAIN_CALLER "_dispatch_wasi_main_queue_drain" #else #define _DISPATCH_MAIN_QUEUE_DRAIN_CALLER "_dispatch_main_queue_callback_4CF" @@ -7078,7 +7078,7 @@ _dispatch_main_queue_drain(dispatch_queue_main_t dq) _dispatch_force_cache_cleanup(); _dispatch_perfmon_end_notrace(); } -#endif // DISPATCH_COCOA_COMPAT || defined(__wasi__) +#endif // DISPATCH_COCOA_COMPAT || DISPATCH_WASI_COOPERATIVE #if DISPATCH_COCOA_COMPAT static bool @@ -7249,7 +7249,7 @@ _dispatch_main_queue_push(dispatch_queue_main_t dq, dispatch_object_t dou, } } -#if defined(__wasi__) +#if DISPATCH_WASI_COOPERATIVE void _dispatch_wasi_main_queue_drain(void) { @@ -7259,7 +7259,7 @@ _dispatch_wasi_main_queue_drain(void) // sides of the QoS check are always 0) _dispatch_main_queue_drain(&_dispatch_main_q); } -#endif // defined(__wasi__) +#endif // DISPATCH_WASI_COOPERATIVE void _dispatch_main_queue_wakeup(dispatch_queue_main_t dq, dispatch_qos_t qos, @@ -7270,7 +7270,7 @@ _dispatch_main_queue_wakeup(dispatch_queue_main_t dq, dispatch_qos_t qos, return _dispatch_runloop_queue_wakeup(dq->_as_dl, qos, flags); } #endif -#if defined(__wasi__) +#if DISPATCH_WASI_COOPERATIVE if (_dispatch_queue_is_thread_bound(dq)) { // nothing else can run the thread-bound main queue on // single-threaded WASI: note it for the cooperative drain, after @@ -7315,7 +7315,7 @@ void dispatch_main(void) { _dispatch_root_queues_init(); -#if defined(__wasi__) +#if DISPATCH_WASI_COOPERATIVE // Cooperative single-threaded WASI: there is no way to park the main // thread while other threads do the work, so dispatch_main() itself // becomes the drain loop. Armed timers and event sources (fd readiness, @@ -7333,12 +7333,19 @@ dispatch_main(void) DISPATCH_CLIENT_CRASH(0, "dispatch_main(): no runnable work on single-threaded WASI"); } -#else // defined(__wasi__) +#elif defined(__wasi__) + // wasi-libc has no pthread_exit: a wasi thread ends only by returning + // from its start function, so the main thread cannot park the way the + // generic path below does. Support needs a dedicated design (unbind the + // main queue, then park on a semaphore); crash clearly until it lands. + DISPATCH_CLIENT_CRASH(0, + "dispatch_main() is not supported on threaded WASI yet"); +#else // DISPATCH_WASI_COOPERATIVE #if HAVE_PTHREAD_MAIN_NP if (pthread_main_np()) { #endif _dispatch_object_debug(&_dispatch_main_q, "%s", __func__); -#ifndef __linux__ +#if !defined(__linux__) && !defined(__wasi__) _dispatch_program_is_probably_callback_driven = true; #endif _dispatch_ktrace0(ARIADNE_ENTER_DISPATCH_MAIN_CODE); @@ -7364,7 +7371,7 @@ dispatch_main(void) } DISPATCH_CLIENT_CRASH(0, "dispatch_main() must be called on the main thread"); #endif -#endif // defined(__wasi__) +#endif // DISPATCH_WASI_COOPERATIVE } DISPATCH_NOINLINE @@ -7639,9 +7646,17 @@ DISPATCH_ALWAYS_INLINE static inline pid_t _gettid(void) { +#if DISPATCH_WASI_COOPERATIVE // WASI is single-threaded; any nonzero constant works as the sole // thread's id (the value seeds tsd->tid for lock-owner encoding). return 1; +#else + // threaded WASI: pthread_self() is a pointer to the (at least + // 4-aligned, never-null) musl thread structure. Shifting it right by 2 + // yields a unique nonzero id per thread that survives the tid << 2 + // lock-owner encoding in _dispatch_tid_self(). + return (pid_t)((uintptr_t)pthread_self() >> 2); +#endif } #else #error "SYS_gettid unavailable on this system" diff --git a/src/shims/lock.c b/src/shims/lock.c index 9618cc14f..43b4b6389 100644 --- a/src/shims/lock.c +++ b/src/shims/lock.c @@ -69,6 +69,7 @@ _dispatch_thread_switch(dispatch_lock value, dispatch_lock_options_t flags, #endif // HAVE_UL_UNFAIR_LOCK #elif defined(__wasi__) #if !HAVE_UL_UNFAIR_LOCK && !HAVE_FUTEX_PI +#if DISPATCH_WASI_COOPERATIVE DISPATCH_NOINLINE static void _dispatch_thread_switch(dispatch_lock value, dispatch_lock_options_t flags, @@ -85,6 +86,18 @@ _dispatch_thread_switch(dispatch_lock value, dispatch_lock_options_t flags, DISPATCH_CLIENT_CRASH(value, "single-threaded WASI deadlock: " "lock contended with no other thread to release it"); } +#else +DISPATCH_ALWAYS_INLINE +static inline void +_dispatch_thread_switch(dispatch_lock value, dispatch_lock_options_t flags, + uint32_t timeout) +{ + (void)value; + (void)flags; + (void)timeout; + sched_yield(); +} +#endif // DISPATCH_WASI_COOPERATIVE #endif #elif defined(__unix__) #if !HAVE_UL_UNFAIR_LOCK && !HAVE_FUTEX_PI @@ -356,7 +369,7 @@ _dispatch_sema4_timedwait(_dispatch_sema4_t *sema, dispatch_time_t timeout) _pop_timer_resolution(resolution); return wait_result == WAIT_TIMEOUT; } -#elif defined(__wasi__) +#elif DISPATCH_WASI_COOPERATIVE DISPATCH_ALWAYS_INLINE static inline bool _dispatch_sema4_try_consume(_dispatch_sema4_t *sema) @@ -686,7 +699,7 @@ _dispatch_wait_on_address(uint32_t volatile *_address, uint32_t value, return _umtx_op((void*)address, UMTX_OP_WAIT_UINT, value, (void*)(uintptr_t)sizeof(struct timespec), (void*)&ts); } return _umtx_op((void*)address, UMTX_OP_WAIT_UINT, value, 0, 0); -#elif defined(__wasi__) +#elif DISPATCH_WASI_COOPERATIVE (void)flags; while (os_atomic_load(address, relaxed) == value) { // re-check the deadline before draining so that a continuous stream @@ -710,6 +723,17 @@ _dispatch_wait_on_address(uint32_t volatile *_address, uint32_t value, } } return 0; +#elif defined(__wasi__) + // threaded WASI: memory.atomic.wait32 is a true futex. + // It returns 0 (woken), 1 (address != value), or 2 (timed out). + (void)flags; + int64_t timeout_ns = -1; + if (nsecs != DISPATCH_TIME_FOREVER) { + timeout_ns = nsecs > INT64_MAX ? INT64_MAX : (int64_t)nsecs; + } + int rc = __builtin_wasm_memory_atomic_wait32( + (int32_t *)address, (int32_t)value, timeout_ns); + return rc == 2 ? ETIMEDOUT : 0; #else #error _dispatch_wait_on_address unimplemented for this platform #endif @@ -726,6 +750,8 @@ _dispatch_wake_by_address(uint32_t volatile *address) WakeByAddressAll((uint32_t *)address); #elif defined(__FreeBSD__) _umtx_op((void*)address, UMTX_OP_WAKE, INT_MAX, 0, 0); +#elif defined(__wasi__) && !DISPATCH_WASI_COOPERATIVE + __builtin_wasm_memory_atomic_notify((int32_t *)address, UINT32_MAX); #else (void)address; #endif diff --git a/src/shims/lock.h b/src/shims/lock.h index 04d047b97..8e7fd0466 100644 --- a/src/shims/lock.h +++ b/src/shims/lock.h @@ -284,10 +284,11 @@ void _dispatch_sema4_init(_dispatch_sema4_t *sema, int policy); #define _dispatch_sema4_is_created(sema) ((void)sema, 1) #define _dispatch_sema4_create_slow(sema, policy) ((void)sema, (void)policy) -#elif defined(__wasi__) +#elif DISPATCH_WASI_COOPERATIVE // Single-threaded WASI: a plain counter; waiters make progress by // cooperatively draining pending dispatch work instead of blocking +// (threaded WASI builds use USE_POSIX_SEM instead) typedef uint32_t _dispatch_sema4_t; #define _DSEMA4_POLICY_FIFO 0 #define _DSEMA4_POLICY_LIFO 0 @@ -301,7 +302,7 @@ typedef uint32_t _dispatch_sema4_t; #error "port has to implement _dispatch_sema4_t" #endif -#if defined(__wasi__) +#if DISPATCH_WASI_COOPERATIVE // Cooperative drain support for single-threaded WASI, implemented in // src/event/event_wasi.c: blocking waits drain pending dispatch work and // timers instead of blocking the sole thread. From 65fb8717e6d4843329bc3c69d567887e7791bca9 Mon Sep 17 00:00:00 2001 From: Scott Marchant <15382220+scottmarchant@users.noreply.github.com> Date: Tue, 18 Aug 2026 12:53:00 -0600 Subject: [PATCH 4/4] test(wasi-threads): prove dispatch_async runs on worker threads Add tests/wasm/threads-dispatch-smoke.c: 8 dispatch_async blocks must run on worker pthreads (checked against pthread_self of main), dispatch_semaphore must block and wake across threads, dispatch_sync must funnel, and dispatch_after must fire through the manager thread. Under wasmtime v24 (-S threads) the test prints: PASS: async=8/8 on-worker-thread=8/8 sync=1 timer=1 Multi-threaded dispatch on wasm32-unknown-wasip1-threads is proven. The research notes record the build and run commands, the implementation results, and what stays unimplemented (fd/signal sources, dispatch_main, the Swift overlay). Co-authored-by: Krzysztof Rodak Co-Authored-By: Claude Fable 5 --- tests/wasm/THREADS-RESEARCH.md | 35 +++++++++++ tests/wasm/threads-dispatch-smoke.c | 90 +++++++++++++++++++++++++++++ 2 files changed, 125 insertions(+) create mode 100644 tests/wasm/threads-dispatch-smoke.c diff --git a/tests/wasm/THREADS-RESEARCH.md b/tests/wasm/THREADS-RESEARCH.md index 992bd9a76..e266d077d 100644 --- a/tests/wasm/THREADS-RESEARCH.md +++ b/tests/wasm/THREADS-RESEARCH.md @@ -141,6 +141,41 @@ signatures survive as designed. 3. The test runner needs a wasmtime v24 pin (or WAMR) for any threads CI lane; current wasmtime cannot run it. +## Implementation results (this branch) + +The sketch above is now implemented and proven on this branch: + +- `DISPATCH_WASI_COOPERATIVE` (`__wasi__ && !_REENTRANT`) splits every + cooperative gate; the threads triple takes the generic POSIX shape. +- `cmake ... -DDISPATCH_WASI_THREADS=ON` builds `libdispatch.a` clean + (-Werror) for `wasm32-unknown-wasip1-threads`. The cooperative build + is unchanged: 52 of 52 ctest cases still pass. +- New backend `event_wasi_threads.c`: the manager thread parks on a + CLOCK_MONOTONIC condvar, merges due timers, and wakes on pokes. + `_dispatch_wait_on_address` is a real futex via + `__builtin_wasm_memory_atomic_wait32/notify`. +- **Proof**: `tests/wasm/threads-dispatch-smoke.c` under wasmtime v24 + prints `PASS: async=8/8 on-worker-thread=8/8 sync=1 timer=1`. + Every dispatch_async block ran on a worker pthread, semaphores + blocked and woke across threads, dispatch_sync funneled, and + dispatch_after fired through the manager thread. + +Two more toolchain facts found during implementation: + +- wasi-libc deliberately does not declare `pthread_exit` (a wasi + thread ends only by returning from its start function), so + `dispatch_main()` cannot use the generic pthread_exit park. It + crashes with a named message until a dedicated design lands (unbind + the main queue, park on a semaphore). +- `DISPATCH_HW_CONFIG_UP` must stay cooperative-only: the threads + target is not a uniprocessor, and UP-sized continuations (32 bytes + on wasm32) are smaller than `struct dispatch_apply_s`. + +Not implemented yet in threads mode: fd/signal event sources (need a +bounded-slice poll_oneoff poller thread; registering one crashes with +a named message), `dispatch_main()`, and the Swift overlay (needs a +matching swiftwasm host toolchain rather than the 6.3.3 release). + ## Reproduction commands # smoke (4 threads, atomics + semaphore): PASS counter=10 threads=4 diff --git a/tests/wasm/threads-dispatch-smoke.c b/tests/wasm/threads-dispatch-smoke.c new file mode 100644 index 000000000..5a078b656 --- /dev/null +++ b/tests/wasm/threads-dispatch-smoke.c @@ -0,0 +1,90 @@ +/* + * Copyright (c) 2026 Apple Inc. All rights reserved. + * + * @APPLE_APACHE_LICENSE_HEADER_START@ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * @APPLE_APACHE_LICENSE_HEADER_END@ + */ + +// Proof of concept for multi-threaded dispatch on wasm32-wasip1-threads: +// dispatch_async blocks must run on worker threads (not the main thread), +// dispatch_semaphore must block and wake across threads, dispatch_sync must +// funnel across threads, and dispatch_after must fire through the manager +// thread. Run under a wasi-threads host (wasmtime v24: -S threads). + +#include +#include +#include +#include + +#define ASYNC_N 8 + +static pthread_t main_thread; +static _Atomic int async_ran; +static _Atomic int on_worker; +static _Atomic int timer_fired; +static _Atomic int sync_ran; + +int +main(void) +{ + main_thread = pthread_self(); + dispatch_semaphore_t done = dispatch_semaphore_create(0); + dispatch_queue_t gq = + dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0); + + for (int i = 0; i < ASYNC_N; i++) { + dispatch_async(gq, ^{ + if (!pthread_equal(pthread_self(), main_thread)) { + atomic_fetch_add(&on_worker, 1); + } + atomic_fetch_add(&async_ran, 1); + dispatch_semaphore_signal(done); + }); + } + for (int i = 0; i < ASYNC_N; i++) { + if (dispatch_semaphore_wait(done, + dispatch_time(DISPATCH_TIME_NOW, 10 * NSEC_PER_SEC))) { + printf("FAIL: async completion timeout (i=%d ran=%d)\n", i, + atomic_load(&async_ran)); + return 1; + } + } + + dispatch_queue_t sq = dispatch_queue_create("smoke.serial", NULL); + dispatch_sync(sq, ^{ + atomic_store(&sync_ran, 1); + }); + + dispatch_after(dispatch_time(DISPATCH_TIME_NOW, 100 * NSEC_PER_MSEC), gq, ^{ + atomic_store(&timer_fired, 1); + dispatch_semaphore_signal(done); + }); + if (dispatch_semaphore_wait(done, + dispatch_time(DISPATCH_TIME_NOW, 10 * NSEC_PER_SEC))) { + printf("FAIL: dispatch_after timeout\n"); + return 1; + } + + int ran = atomic_load(&async_ran); + int workers = atomic_load(&on_worker); + int timer = atomic_load(&timer_fired); + int sync = atomic_load(&sync_ran); + int pass = ran == ASYNC_N && workers == ASYNC_N && timer && sync; + printf("%s: async=%d/%d on-worker-thread=%d/%d sync=%d timer=%d\n", + pass ? "PASS" : "FAIL", ran, ASYNC_N, workers, ASYNC_N, sync, + timer); + return !pass; +}