Add Wayland/PipeWire desktop capture support - #261
Merged
Conversation
WebRTC's PipeWire/xdg-desktop-portal screen capture backend is never enabled: rtc_use_pipewire stays false and WEBRTC_USE_PIPEWIRE/WEBRTC_USE_GIO are never defined for the JNI wrapper, so DesktopCapturer::CreateScreenCapturer() returns null under Wayland. Since the recent null-check fix that fails gracefully with a log message instead of crashing, but capture still doesn't work. - CMakeLists.txt: WEBRTC_USE_PIPEWIRE/WEBRTC_USE_GIO defines, GIO/GBM/DRM linking, rtc_use_pipewire=true, and a fix for a PKG_CONFIG_SYSROOT_DIR/ PATH leak from the JNI configure step into gn/ninja that broke portal capturer header discovery. Also links Xext/Xdamage/Xtst, needed by the existing X11 capturer path but not previously linked. - VideoTrackDesktopSource.cpp / DesktopCapturer.cpp: set_allow_pipewire(true) + set_prefer_cursor_embedded(true), gated behind WEBRTC_USE_PIPEWIRE. - patches/linux/pipewire_force_shm.patch: forces SHM buffers over DMA-BUF in screencast_stream_utils.cc's BuildFullFormat(). Several GPU/driver combinations deliver DMA-BUF frames flagged SPA_META_HEADER_FLAG_CORRUPTED, which OnStreamProcess drops, producing an empty screen share. Verified building and linking cleanly on all three Linux architectures against branch-heads/7977.
B077AS
added a commit
to B077AS/webrtc-java
that referenced
this pull request
Aug 29, 2026
…tch) Windows screen/window capture has been completely broken since the branch-heads/7339 -> 7977 (m140 -> m152) bump: zero frames captured, and the "screens" enumeration returned application windows. Root cause is a DesktopCaptureOptions ABI mismatch between webrtc.lib and the JNI wrapper. gn builds the lib with its default rtc_enable_win_wgc=true (webrtc.gni: rtc_enable_win_wgc = is_win), so the lib-side class contains the RTC_ENABLE_WIN_WGC-guarded members; the wrapper compiled without the define sees a smaller class (64 vs 80 bytes on 7977) with different member offsets for everything after the WGC block. DesktopCaptureOptions::CreateDefault() and the copy constructor are defined out-of-line in the lib, so `auto options = CreateDefault()` in the wrapper writes an 80-byte object into a 64-byte stack slot: 16 bytes of adjacent stack are zero-smashed on every capturer construction. That deterministic smash is what zeroed jni::DesktopCapturer's spilled screenCapturer argument (every ScreenCapturer.initialize() logged screenCapturer=0 and constructed the window capturer instead - the "Screens tab lists applications" bug), and general field-offset skew after the WGC block broke frame delivery entirely. The mismatch already existed on 7339, but the guarded block was then just 5 bools at the tail-adjacent position: an 8-byte zero overflow and skewed reads of cosmetic flags (disable_effects_), which happened to be survivable - which is why 0.14.0 worked. 7977 grew the block (7 bools + LUID) and added std::optional<Environment> env_ after it, making the corruption fatal. Fix: define RTC_ENABLE_WIN_WGC for the wrapper so both sides agree on the layout. This is the same ABI bug class already fixed on Linux with WEBRTC_USE_X11 (devopvoid#236) and WEBRTC_USE_PIPEWIRE (devopvoid#261). All allow_wgc_* options default to false, so capturer selection behavior is unchanged (DirectX/GDI as before).
devopvoid
pushed a commit
that referenced
this pull request
Aug 29, 2026
* fix: define RTC_ENABLE_WIN_WGC for the JNI wrapper (Windows ABI mismatch) Windows screen/window capture has been completely broken since the branch-heads/7339 -> 7977 (m140 -> m152) bump: zero frames captured, and the "screens" enumeration returned application windows. Root cause is a DesktopCaptureOptions ABI mismatch between webrtc.lib and the JNI wrapper. gn builds the lib with its default rtc_enable_win_wgc=true (webrtc.gni: rtc_enable_win_wgc = is_win), so the lib-side class contains the RTC_ENABLE_WIN_WGC-guarded members; the wrapper compiled without the define sees a smaller class (64 vs 80 bytes on 7977) with different member offsets for everything after the WGC block. DesktopCaptureOptions::CreateDefault() and the copy constructor are defined out-of-line in the lib, so `auto options = CreateDefault()` in the wrapper writes an 80-byte object into a 64-byte stack slot: 16 bytes of adjacent stack are zero-smashed on every capturer construction. That deterministic smash is what zeroed jni::DesktopCapturer's spilled screenCapturer argument (every ScreenCapturer.initialize() logged screenCapturer=0 and constructed the window capturer instead - the "Screens tab lists applications" bug), and general field-offset skew after the WGC block broke frame delivery entirely. The mismatch already existed on 7339, but the guarded block was then just 5 bools at the tail-adjacent position: an 8-byte zero overflow and skewed reads of cosmetic flags (disable_effects_), which happened to be survivable - which is why 0.14.0 worked. 7977 grew the block (7 bools + LUID) and added std::optional<Environment> env_ after it, making the corruption fatal. Fix: define RTC_ENABLE_WIN_WGC for the wrapper so both sides agree on the layout. This is the same ABI bug class already fixed on Linux with WEBRTC_USE_X11 (#236) and WEBRTC_USE_PIPEWIRE (#261). All allow_wgc_* options default to false, so capturer selection behavior is unchanged (DirectX/GDI as before). * fix: pass full frame height as libyuv src_height (broken window capture) With the RTC_ENABLE_WIN_WGC ABI fix in place, screen capture works but window (application) capture still delivered no frames: neither thumbnails (DesktopCaptureCallback) nor the live share (VideoTrackDesktopSource). Both files feed libyuv::ConvertToI420 with the cropped output height (i420Buffer/buffer->height() == crop_h) in the src_height slot. libyuv bounds-checks crop_y + crop_height <= src_height, so the conversion fails with -1 whenever crop_y > 0. The Windows border-crop branch sets crop_y = -top_left().y() for frames that start above the screen origin, which is the case for every MAXIMIZED window (its frame sits at -SM_CXPADDEDBORDER). Screen frames never enter that branch at all (exact stride, `fullscreen == true`), which is why only window capture appeared broken. Confirmed at runtime on a diagnostic build (Komm, Windows 11): size=1928x1043 crop_y=7 crop_w=1920 crop_h=1032 Failed to convert desktop frame to I420, libyuv result=-1 Also propagate non-SUCCESS capture results to the Java callback (with a null frame) instead of silently returning, so callers don't have to burn a timeout to notice a failed capture.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
WebRTC's PipeWire/xdg-desktop-portal screen capture backend is never enabled:
rtc_use_pipewirestaysfalsein the gn build, andWEBRTC_USE_PIPEWIRE/WEBRTC_USE_GIOare never defined for the JNI wrapper. As a resultDesktopCapturer::CreateScreenCapturer()returns null under Wayland — since #236 that now fails gracefully with a log message instead of crashing, but screen capture still doesn't work on Wayland sessions.This PR enables the PipeWire backend end-to-end:
CMakeLists.txt: addsWEBRTC_USE_PIPEWIRE/WEBRTC_USE_GIOcompile definitions and GIO/GBM/DRM linking, kept in sync withrtc_use_pipewire=truein the gn args —DesktopCaptureOptionshas members gated byWEBRTC_USE_PIPEWIRE, so these must match the gn build the same wayWEBRTC_USE_X11already has to, or the JNI wrapper andlibwebrtc.adisagree on struct layout.PKG_CONFIG_SYSROOT_DIR/PKG_CONFIG_PATHleak from the JNI configure step into gn/ninja, which double-prefixes sysroot include paths and breaks portal-capturer header discovery.VideoTrackDesktopSource.cpp/DesktopCapturer.cpp:set_allow_pipewire(true)+set_prefer_cursor_embedded(true), gated behindWEBRTC_USE_PIPEWIREso non-Linux builds are unaffected.patches/linux/pipewire_force_shm.patch: forces SHM buffers over DMA-BUF inscreencast_stream_utils.cc'sBuildFullFormat(). On several GPU/driver combinations, the Wayland compositor delivers every DMA-BUF frame flaggedSPA_META_HEADER_FLAG_CORRUPTED, whichOnStreamProcessdrops — producing an entirely empty screen share. Clearing the modifier list before it's offered to the compositor keeps negotiation on the SHM path, which always works.XextandXdamage, which the existing X11 capturer's damage-based capture (XDamageQueryExtension/XDamageCreate/XDamageSubtract) calls but weren't previously linked.Xtstis linked as well.Verification
Builds and links cleanly on all three Linux architectures (x86-64, arm, arm64) against
branch-heads/7977, confirmed via this fork's CI:gio-2.0/gio-unix-2.0resolve via pkg-config from the existing Debian-bullseye sysroot.libgio-2.0.so,libgbm.so,libdrm.sofrom the sysroot with no missing symbols.