Add macOS (Apple Silicon) host support - #1424
Open
dywongcloud wants to merge 27 commits into
Open
dywongcloud wants to merge 27 commits into
dywongcloud wants to merge 27 commits into
Conversation
This crate was removed on the alternate branch's main; drop it here too ahead of syncing the rest of litebox core and its platform/runner crates to that branch's tip, which no longer has it as a member.
…te main Bring litebox core and every already-present platform, runner, and shim crate up to the alternate branch's tip: the Platform trait and several downstream APIs changed shape there (shared-futex-backed VmArea, dirfd- relative sys_openat replacing sys_open, aarch64 syscall-rewriter support, 9P/proc/flock/framebuffer filesystem backends, a new litebox::broker module, etc.), so every implementor needs the same update to keep building. litebox_packager additionally gains the musl x18-register patching support macOS-hosted guests need (see docs/roadmap.md's "XNU destroys a live guest x18"); its desktop-image-specific manifest and build script are left out, matching the exclusion of the XFCE/HVF-desktop example recipes elsewhere in this series.
litebox_runner_linux_userland now unconditionally depends on litebox_broker_transport_linux_userland (for its multi-process broker socket support), whose own dependency chain and dev-dependencies pull in the rest of the broker crates: litebox_broker_core, _host, _local, _protocol, and _transport, plus litebox_broker_userland (the standalone broker daemon binary, needed for the runner's own broker integration tests). This is the full closure required for the already-synced runner crates to build and test; it does not add the broker daemon as a user-facing feature on its own.
litebox_platform_macos_userland: a LiteBox platform for running on userland macOS on Apple Silicon, backed by Hypervisor.framework -- HVF vCPU/memory management, guest-thread state, the W^X toggle for JIT regions, and the guest-entry monitor trampoline. litebox_runner_linux_on_macos_userland: the runner that hosts a Linux guest on top of that platform, including its NAT-style network proxy. litebox_rfb: an RFB (VNC) server used to expose the macOS runner's guest framebuffer. litebox_platform_multiplex: dispatches to whichever of the platforms above (or the existing ones) is active, needed by litebox_runner_lvbs. Desktop/VNC-testing demo image-build infrastructure (XFCE and HVF-desktop example recipes) is intentionally left out.
… the new crates - Add a macos-14 build_and_test_macos job, an AArch64 shim build step and binutils install to the existing build/test jobs, and a miri job scoped to litebox's mm:: tests. - Update confirm_no_std's allow-list: litebox_platform_macos_userland, litebox_runner_linux_on_macos_userland, litebox_rfb, litebox_broker_transport_linux_userland, and litebox_broker_userland are userland implementations needing std, matching the existing litebox_platform_linux_userland/litebox_platform_windows_userland entries; litebox_platform_multiplex needs it transitively through the macOS platform it multiplexes over. - Update dev_tests' copyright-header and ratchet (transmute/global/ MaybeUninit usage count) checks for the crates this series adds, removes, or otherwise changes, and skip the one file each check cannot apply to for a structural reason (an Xcode entitlements property list, requiring `<?xml ...?>` as its literal first bytes). - Extend the tun-access nextest retry allowance to test_handle_futex_death_wakes_waiter_and_sets_owner_died: under enough concurrent CPU contention from the rest of the suite, its real-threads fixed-sleep synchronization can race handle_futex_death's one-shot wake against a waiter that has not registered yet.
initialize_socket stored SockType as fd-scoped metadata, which descriptor_table().duplicate() deliberately does not copy. A dup'd INET socket therefore had no SockType, so get_socket_type (and everything routed through it: connect, accept, getsockopt(SO_TYPE), sendto) failed NoSuchMetadata -> ENOTSOCK once any path consulted it, regardless of whether the original fd was still open. SO_TYPE is a property of the socket, shared by every descriptor referring to it, so store it as entry metadata alongside SocketOptions and SocketOFlags. Un-ignores test_tun_tcp_connection_refused, which exercised exactly this.
The test kept the futex word on the host test-thread stack. handle_futex_death wakes with shared (non-PRIVATE) semantics, whose key lookup resolves the word through the VMM, and a host-stack page is only known to the VMM if this thread's stack happened to be mapped when the process-wide test platform took its one-time host-mapping snapshot. Whenever it was not, the wake failed EFAULT (swallowed, as on Linux), the waiter timed out, and the test failed -- about half the time under a full-suite run, deterministically by test order rather than timing, which is why widening the fixed sleep did not help. Map the word through the task instead, as a real guest's would be. Drops the nextest retry allowance added for this test: the fix removes the flake rather than tolerating it.
A MAP_SHARED file mapping's writes land in the file, so Linux requires the backing fd to be open for writing: mmap(MAP_SHARED | PROT_WRITE) on an O_RDONLY fd fails with EACCES, and such a mapping never gets VM_MAYWRITE, so a later mprotect(PROT_WRITE) fails with EACCES too. The shim enforced neither. A guest could open a file read-only, map it MAP_SHARED | PROT_READ, mprotect it writable, and dirty the file on munmap through the copy-back path without ever holding a writable fd. Record whether the fd was open for writing (from the open-time FdOpenFlags entry metadata) on each SharedFileMapping, carry it across splits and remaps, and check it in both do_mmap and sys_mprotect. Fds with no open-flags record keep the previous behaviour. test_map_shared_readonly_file, which asserted the mprotect half of this and was #[ignore]d for it, is re-enabled and also covers the mmap half.
CI installs the current stable toolchain (1.98), which rejects a few things 1.94 accepted, and two of its jobs build for aarch64, which compiles code the x86_64 host never sees: - litebox_common_linux: `UserPtRegs::write_into` casts `u64` to `usize`; the module is aarch64-only, so the widths match. Expect the lint there instead of adding a fallible conversion that cannot fail. - litebox_shim_linux: SCM_RIGHTS parsing now uses `as_chunks` for the constant-size fd chunks (chunks_exact_to_as_chunks), and a test's one-byte array is a byte string (byte_char_slices). - litebox_shim_linux: on aarch64 `ThreadInitState::NewThread` carried a 528-byte FPSIMD register file inline (large_enum_variant); box it. - litebox: the `Network` struct's doc comment had `accept_ready` inserted between it and the struct, so the docs (and their `Self::` links) were attached to the function. Move the helper above the docs. - litebox: three doc links pointed at private items; make them plain code spans so `cargo doc` with `-Dwarnings` passes without `--document-private-items`. - litebox_shim_linux: two doc links did not resolve. `InotifyEvent` lives in litebox_common_linux, and `elf_patch_cache` is a field of `Process`, not of the struct whose doc called it `Self::`.
DiodServer::start picks a port by binding port 0 and releasing it, then
starts diod on it, and the tests run in parallel. Two tests can get the
same port. The second diod then fails to bind ("Address already in
use") and exits, but the readiness probe was a plain TCP connect, which
can reach the first test's diod before the second one exits. The test
then attaches with its own export path to a server that does not export
it, and FileSystem::new fails with Io.
This showed up as roughly one failing run in sixteen of the nine_p
suite, always in a different test. Every failure with diod's bind error
also showed the listener was not owned by the test's diod, and ownership
recorded at the moment the old probe declared ready was false.
Judge readiness by the spawned diod owning a LISTEN socket on the port:
look the port up in /proc/net/tcp and match the socket inode against the
child's /proc/<pid>/fd links. The module is already Linux-only. A diod
that lost the race still exits and is retried on a new port as before.
With the check in place the suite passed 80 runs out of 80.
The transactional rewrite of WindowsUserland::allocate_pages refused every fixed allocation that touched a committed page, for Replace as well as NoReplace, because decommitting cannot be rolled back. But Vmem asks the platform for Replace exactly when its own mappings cover the whole range: that is MAP_FIXED over an existing mapping, which the shim does while setting up every process. Loading any program on Windows therefore failed with LoadError(Map(ENOMEM)). Plan committed pages under Replace as RecommitCommitted: decommit, then commit afresh with the new protection, which also gives the zeroed pages MAP_FIXED promises (the pre-rewrite code did the same). It runs after every reversible step. A failure before any page is discarded rolls back as before; after that, returning would leave Vmem believing the old mapping is intact, so it aborts, like an incomplete rollback. Hint and NoReplace are unchanged.
run_test_thread is a ThreadProvider method, not an inherent one, so `WindowsUserland::run_test_thread` did not resolve under rustdoc.
The dynamic-binary test built its rootfs with the Windows `tar`, which has no execute bit to record and archives the hooked program, libc and ld.so as 0644. The shim now checks execute permission on the program and its interpreter the way Linux execve does, so the runner failed with EACCES opening the program. Write the archive with the tar crate instead (already in Cargo.lock via litebox_packager), giving every entry mode 0755 as a Linux rootfs ships these files, and filling in the uid, gid and mtime fields that litebox's tar reader parses. Checked on Linux with the same shim: the test's own binaries, rewritten and archived by this helper, run through litebox_runner_linux_userland and print "hello world."; the same files archived 0644 fail with EACCES, the error the Windows job reported.
litebox_platform_macos_userland's Hypervisor.framework shim refuses to build against anything older than the macOS 26 SDK. The macos-14 image ships Xcode 15.4 with the 14.5 SDK, so its build script failed before clippy could run.
The vectored exception handler returned EXCEPTION_CONTINUE_SEARCH as soon as the faulting thread had no TlsState, before it consulted the exception table. So a guarded guest-memory access (memcpy_fallible and friends) that faulted on a host thread that had never entered the guest crashed the process instead of returning a failure. Guest threads always have TLS state, so the runner never hit this. The shim's unit tests do: they call syscall handlers directly on the test thread, and test_setgroups_getgroups_round_trip_boundaries_and_copy_on_write passes a null list to setgroups expecting EFAULT, which aborted with 0xc0000005 on Windows. (test_fallible_read is cfg'd out on Windows for the same reason.) A thread without TLS state cannot be in the guest, so treat its faults like any other fault outside the guest: look up the exception table and resume at the recovery point, else continue the search as before.
These only build for aarch64, so the x86_64 doc job never saw them; the macOS job's `cargo doc -Dwarnings` does. - litebox_common_linux: the `ptrace` module's own doc linked NT_PRSTATUS and NT_ARM_TLS, which resolve from the parent scope only as `ptrace::...`. - litebox_shim_linux: ptrace.rs linked `crate::wait::Task`, a path that does not exist; `prepare_to_run_guest` is a method of `crate::Task` (implemented in wait.rs).
The HVF backend landed in bulk without passing the macOS CI job (which stopped at the SDK build step), so `cargo clippy -Dwarnings` now reports 274 lints and 2 dead methods in this crate, and `cargo doc -Dwarnings` broken or private links. Fixed without blanket allows: - result_large_err (143): the lane and backend errors carried a whole HvfVcpuExit / register-state snapshot inline. Box those payloads (HvfVcpuLaneError's exit/state/request fields, HvfBackendError's Lane variant and UnexpectedExit exit, HvfVcpuDiagnosticError's Lane), and box the error of the few functions that hand values back in their error (LaneMaintenanceFailure, admit's rejected Command, the two claim helpers, behind named aliases). - large_enum_variant: box the register file in Command::Execute. - Machine-applicable fixes from `cargo clippy --fix` (map_or, let-else, is_multiple_of, contains, collapsible if, etc.), except the one whose method-path rewrite does not type-check; that site keeps its closure with an expect. - Dead code: HvfVcpu::reject and HvfAddressSpace::executable_generation had no callers. - `cancel_unbound_vcpu_creation` never failed; return () and drop the unreachable cleanup branches in its two callers. - Smaller fixes: finish_non_exhaustive for Debug impls that omit internal fields, a used `_monitor_mapping` field renamed, typed truncations via TruncateExt, `Box::new_zeroed` for the 80 KiB probe buffer instead of a stack array, merged duplicate match arms, an item hoisted above statements. - Per-site #[expect(..., reason)] where the lint flags a deliberate shape, as elsewhere in the repo: diagnostic report and state structs whose bools are independent properties, functions taking separately locked manager state, VM-scoped failure-injection hooks, and ratio math in a diagnostic report. - Doc links from public items to private ones become code spans; a field link uses Self::. Checked locally with the build script's xcrun step skipped: `cargo clippy --all-targets --all-features --workspace` and `cargo doc --document-private-items` for aarch64-apple-darwin pass with -Dwarnings.
- Box the TLS stream in `Upstream`, which otherwise sized every upstream connection by the whole rustls state. - as_chunks for the ICMP checksum, let-else for the recvfrom length, a char-array pattern for the authority end, and no redundant continue. - Expect similar_names on the DNS header parser (qdcount/ancount/ nscount/arcount are the protocol's field names) and the final cast of the already-folded checksum.
Bring in the two upstream commits on top of 4964033: - df162a1 Relocate page management into dedicated modules (microsoft#1415) - fcad93e Use bottom-up ELF placement policy (microsoft#1395) Resolution: - litebox::mm::linux is now litebox::mm::vmem; every path on our side (including the macOS platform and the shims) follows the rename. - The Linux and Windows userland platforms move prot_flags and their PageManagementProvider impl, with helpers and page tests, into page_mgmt.rs. Our Windows additions (the virtual-memory mutation lock, RecommitCommitted, allocation rollback) move with them unchanged. - The mm test backend gains upstream's Windows TASK_ADDR bounds next to our Apple ones. - Bottom-up ELF placement is ported onto our loader: the main image is claimed from default_low_addr() (which honours the platform's TASK_ADDR_MIN) with MAP_FIXED_NOREPLACE and retried past racing mappings, the interpreter stays top-down, and do_mmap is pub(crate).
Vmem picks a move destination from its own bookkeeping, and the default remap_pages claimed exactly that range with NoReplace. A hosted platform can already hold part of it for the host -- on Windows even the unusable tail of a host allocation's 64 KiB granule, which VirtualQuery reports as free but MEM_RESERVE cannot claim -- so the claim failed and a guest mremap(MREMAP_MAYMOVE) came back EFAULT. That is the intermittent test_mremap failure on the Windows CI job (EFAULT at the MAYMOVE call, passing on a re-run of the same commit). When the NoReplace claim reports the range in use, retry with Hint and let the platform place the pages, as Linux's own mremap without MREMAP_FIXED does. Vmem::move_mappings already records the address remap_pages returns; the permission restore after the copy now targets where the pages landed too. The trait docs say the returned address need not be new_range.start. Platforms that override remap_pages are unchanged. Tests cover both paths against a real-memory backend: a free target is claimed as before, and a host-owned one falls back, carrying the data and the final permissions to the placed range.
Bring the Windows guest crates over from main so master can run Windows PE programs on the Linux and Windows userland hosts: - litebox_common_windows: NT syscall numbers, PE loader types, NTSTATUS. - litebox_shim_windows: the NT shim (x86_64 only). - litebox_runner_windows_on_linux_userland and litebox_runner_windows_userland. - litebox_syscall_rewriter gains PE rewriting (ntdll syscall stubs and GS->FS TEB accesses) and the --host flag. Adapted to master: - litebox::mm::linux paths follow the rename to litebox::mm::vmem. - The event object implements IOPollable::unregister_observer by forwarding to its Pollee. - read_utf16_string uses as_chunks, which clippy now asks for over chunks_exact with a constant size. The workspace, lockfile, ratchet (the shim's test-only PLATFORM static) and CI (the Windows job now builds, tests and documents the Windows runner and the Windows shim; the no_std check skips the two std runners but still covers the shim) are updated to match.
reserve_alignment_outputs_match_host_ntdll frees a host probe range and then asks the guest to reserve inside it, retrying at a fresh address on a conflict. It built a new test task on every attempt, between the host free and the guest reserve. Building a task maps its CSR shared section through a fresh page manager, which does not know the sections earlier tasks left behind; its preferred spot is taken, so the Windows platform lets the host place the section, and right after the free the host's lowest free range is the probe itself. Every attempt therefore conflicted (STATUS_CONFLICTING_ADDRESSES against the host's STATUS_SUCCESS) and the retries ran out -- the Windows CI failure on this test. Build the task once, before the warm-up and the probe loop, which is what the test's own comment already requires: no shim-side allocation between the probe's free and the guest's reserve.
litebox_shim_darwin runs x86-64 Mach-O executables that need no dynamic linker. dyld and Apple's system libraries cannot be redistributed, so an image that links a dylib, or uses chained fixups, is refused at load time with a precise error rather than half-run. - Loading: the image is mapped at its preferred address (so the LC_DYLD_INFO rebase opcodes never need to run), each segment gets its initprot, and `main` is entered the way dyld enters it -- main(argc, argv, envp, apple) on an XNU-shaped stack, returning into a stub that calls exit(2) with its result. - System calls: the `syscall` instructions in the image's instruction sections (not the strings and constants that share __TEXT) are redirected at load time with the runtime patcher the Linux shim uses, into a trampoline placed right above the image. A site the patcher cannot redirect becomes `icebp; hlt`, so it faults instead of reaching the host kernel. - ABI: BSD-class calls (rax = 0x2000000 | n), results with the carry flag clear, failures with it set and Darwin's errno in rax. Implemented: exit, read, write, open, close, lseek, mmap (anonymous and private file mappings), munmap, mprotect, getpid, getppid, get[e]uid, get[e]gid, issetugid, getentropy and the _nocancel variants; the rest is ENOSYS. Runners: litebox_runner_darwin_on_linux_userland and litebox_runner_darwin_on_windows_userland, taking the program from a tar archive like the other runners. Tests: seven small Mach-O programs (assembly sources, a build script using clang and ld64.lld, and the committed binaries under litebox_shim_darwin/test-bins) run end to end under both runners -- stdout, exit status, the carry-flag errno convention, ENOSYS, reading a file from the archive, argv, and an anonymous mmap round trip -- plus parser unit tests against the same binaries. The Windows CI job builds, tests and documents the Windows-host runner; the Linux job picks up the rest as default members; the no_std check covers the shim and skips the two std runners. The copyright-header check learns the binary .macho extension.
WindowsUserland::new printed "System information." and the user address bounds to stdout in debug builds, ahead of anything the guest writes. The Darwin runner's end-to-end tests on the Windows CI job compare the guest's stdout and failed on exactly that prefix; the guests themselves ran correctly. Stdout belongs to the guest, so the diagnostic moves to stderr.
7a1eeb3 removed the HEKI/HVCI service crate because the alternate branch's main never had it, and the sync that followed replaced the LVBS and OP-TEE crates with that branch's copies, which predate upstream's split of HEKI out of the LVBS platform (microsoft#1093) and its removal of the platform singletons (microsoft#1127). Bring the crate back, and with it the upstream design it is written against. The crate itself is restored byte-for-byte as it was before 7a1eeb3 (identical to upstream's). The crates it depends on or serves are three-way merged against 6a03ec8, where the two histories diverged, so the branch's own changes to them are kept: - litebox_common_lvbs: the Vtl0Gate / Vtl0PrivilegedWrite / ReservationStatus interfaces heki needs. - litebox_platform_lvbs: HEKI and memory integrity move out to the service (mshv/heki.rs goes away, mem_integrity.rs moves to the crate); the platform provides the VTL0 gates instead. - litebox_runner_lvbs: owns the HEKI service, the session registry and the boot platform instead of reading them from singletons. - litebox_shim_optee, litebox_common_optee and litebox_runner_optee_on_linux_userland: upstream's singleton-free session management and ldelf segment-padding fix, with its test TA. Conflicts were import lists (resolved to what compiles) and release_memory's callback, which keeps this branch's Option<Range> signature. - litebox_common_linux: physical_pointers.rs, vmap.rs and loader.rs, and zerocopy's alloc feature, which the above build on. - litebox core: userspace_pointers.rs gains copy_from_raw / copy_to_raw (upstream microsoft#1257). litebox_platform_multiplex, which upstream deleted, is kept: nothing uses it any more, but it carries this branch's macOS entry. The workspace lists the crate again, the ratchet takes the new counts (each checked against the statics it counts), and the no_std check excludes it for the same crypto soft-float reason as the OP-TEE shim.
rust-cache's post-step cleanup walks target/ and treats any directory named `tests` as a build profile. rustdoc writes one for every documented module called `tests` (target/doc/litebox_runner_optee_on_linux_userland/ tests here), and for a profile named `tests` the cleanup probes nested `tests/target` and `tests/trybuild` build directories without awaiting the probe, so the ENOENT escapes its try block and lands as error annotations on an otherwise green job. The docs are rebuilt on every run, so each job that builds them now removes target/doc before the cache is saved.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
litebox_platform_macos_userland: a LiteBox platform for running on
userland macOS on Apple Silicon, backed by Hypervisor.framework -- HVF
vCPU/memory management, guest-thread state, the W^X toggle for JIT
regions, and the guest-entry monitor trampoline.
litebox_runner_linux_on_macos_userland: the runner that hosts a Linux
guest on top of that platform, including its NAT-style network proxy.
litebox_platform_multiplex: dispatches to whichever of the platforms
above (or the existing ones) is active, needed by litebox_runner_lvbs.