Skip to content

LATX, fix: Reclaim public-loader metadata and repeated search paths - #475

Open
LaurenIsACoder wants to merge 2 commits into
lat-opensource:masterfrom
LaurenIsACoder:latx-kzt-loader-lifetime
Open

LATX, fix: Reclaim public-loader metadata and repeated search paths#475
LaurenIsACoder wants to merge 2 commits into
lat-opensource:masterfrom
LaurenIsACoder:latx-kzt-loader-lifetime

Conversation

@LaurenIsACoder

@LaurenIsACoder LaurenIsACoder commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Scope and merge dependency

Based on upstream bfe17ae7c1. Merge #474 first: this PR queues unloaded-header retirement through RCU, so inherited queued retirements need the fork-accounting fix. The branch remains independently based on master to keep the review focused.

Two commits preserve the ownership boundaries:

  1. Reuse a normalized existing library-search path and move it to the front when prepending it again, preserving lookup precedence without accumulating repeated allocations.
  2. Retire stale public-loader ELF metadata after loader events. Perform the scan at a guest-execution safe point under the mapping lock, with a fresh loader snapshot; defer destruction through RCU so native frames that still borrow metadata can finish.

Keep the main executable, attached libraries, malloc-backed borrowers, live link-map entries and any ELF with a still-valid load segment. Clear/reuse vacant context slots without shifting other library indexes. This frees metadata, not guest mappings. A busy loader or exclusive-barrier timeout conservatively retains metadata until a later loader event; this is not unconditional immediate reclamation.

Author self-review and combined validation

The actual published diffs were reviewed individually and checked against the local candidates. Follow-up corrections were folded into their existing commits; author/sign-off metadata updates preserved the per-PR source trees. The series remains six commits across #472, #473, #474, #475 and #476.

All five branches merge without conflicts on bfe17ae7c1. The combined source tree 80aaaaf13688387640f528d1c0bccb6a800f8d6e builds and passes 33/33 lat-pr-fast tests. On that combined binary, the 500-cycle KZT unload and retained-handle checks, 520-cycle SMC/fork checks, and 512 successful TSYNC operations alongside thread creation also complete successfully. A tests-disabled configuration registers no tests.

The integration suite still reports 27 skips, including after selecting the available guest sysroot with LATX_X86_64_SYSROOT: 25 require clang and two require the x86 cross compiler. These are not passes. This author self-review does not substitute for the upstream CI matrix, maintainer review or real-application acceptance.

Validation

LoongArch64, GCC 15.3, x86_64-linux-user, LAT O1, KZT compiled in, release:

./configure --target-list=x86_64-linux-user --enable-latx --optimize-O1 \
  --extra-ldflags=-ldl --enable-kzt --enable-tests --disable-docs \
  --disable-werror --meson=meson
ninja -C build -j2 all meson-test-prereq
env -u LAT_LD_PREFIX meson test -C build --no-rebuild \
  --suite lat-pr-fast --print-errorlogs --num-processes 2
env -u LAT_LD_PREFIX meson test -C build --no-rebuild \
  --suite latx-integration --print-errorlogs --num-processes 2
  • Isolated branch: 26/26 lat-pr-fast passed. Author/sign-off-only rewriting preserved the exact validated source tree.
  • Maintained regressions cover normalized prepend order, production retirement/slot-reuse logic, busy/stale snapshots, pinned owners, surviving mappings and deferred-free behavior. ASan/UBSan production-body checks pass in debug and NDEBUG builds. Integer-overflow guards were checked in static review.
  • An isolated x86 Python workload performed 500 libbz2 dlopen/dlclose cycles with LATX_AOT=0, LATX_KZT=2, LATX_KZT_LIBS=core. GDB samples show my_context->elfsize remains 7, versus growth from 7 to 507 on the earlier unfixed baseline. With a retained handle and checked function calls it stays 8, and the guest exits successfully. A KZT-disabled control also exits successfully.
  • This proves the observed public-loader metadata lifecycle, not native libbz2 passthrough or full application memory stability. Private-dirty memory still grows about 4 MiB in the churn case while translation-code and TB allocation cursors advance; this PR does not bound those caches.
  • Integration suite: 27 skipped (exit 77) because the clang/guest-building toolchain is unavailable. Skips are not passes; broader loader/guest compatibility remains a gate.

No installed runtime changes or running-application restart.

Repeated prepends of the same normalized library-search path allocate new
entries indefinitely. Simply ignoring duplicates would change lookup
precedence when an existing path must become the first search location.

Reuse the normalized entry and move it to the front when prepending it
again. Preserve the relative order of the remaining paths.

Add path-collection regression coverage for normalized duplicates and
prepend order. This test passed in the loader-lifetime branch's 26-test
lat-pr-fast suite.

Signed-off-by: Hanlu Li <heuleehanlu@gmail.com>
Repeated public-loader dlopen/dlclose cycles leave stale ELF metadata in
the context even after its guest mappings disappear. Freeing it immediately
would invalidate metadata still borrowed by native frames or RCU readers.

At a guest-execution safe point, scan under the mapping lock using a fresh
loader snapshot and retire unowned metadata through RCU. Retain the main
executable, attached libraries, malloc-backed borrowers, live link-map
entries and ELFs with surviving load segments. Reuse vacant context slots
without shifting other library indexes. Busy or stale snapshots and an
exclusive-barrier timeout conservatively defer reclamation.

Add lifecycle regressions for retained owners, snapshots, slot reuse and
deferred destruction. The branch passed 26 lat-pr-fast tests and sanitizer
checks. A 500-cycle libbz2 loader workload kept elfsize at 7; retaining a
handle kept it at 8. This does not bound translation caches or establish
full application memory stability; unavailable integration tests were
skipped.

Requires the queued-RCU fork fix in PR lat-opensource#474 before merging, so a child
preserves inherited metadata retirements.

Signed-off-by: Hanlu Li <heuleehanlu@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant