Free a level's staging once its upload releases it, and report app memory

[athei]

Symptoms

In GTA IV the Metal HUD's app memory, the process footprint, creeps up during play. A run of v0.11.0-91-ge99fa026 with RUST_LOG=mtld3d::d3d9::mem_watch=debug (GTAIV-12703.log) shows the page boxes growing through the unattributed holder: other goes from 0 to 544 MiB in the first two minutes and then about 26 MiB a minute to 631 MiB, while the live textures (about 5000), their mip data (about 710 MiB), texture staging (about 299 MiB), the vertex/index backing and the parked pool (at its 128 MiB cap) stay flat. Free 32-bit address space falls about 80 MiB a minute late in the run, about three times the page-box growth, and nothing in the perf summary covers the process's footprint, the Metal device's allocations or d3d9.dll's heap, so the rest of the growth had no figure at all.

Cause

The encoder caches one bytesNoCopy MTLBuffer per texture staging level (MipStagingBuffer in unix/unix/src/encoder.rs) and keeps the native owner of the level's pages in the slot as the wrapper's keepalive. That owner is the Arc<PageBox> adopted from the upload lease that first delivered the pages, and a lease completes only when its last native owner drops, so while the wrapper is cached the PE side's GuestPageLease keeps its owner of the pages too. The slot was emptied only when the level's backing changed or the texture was destroyed.

A DEFAULT-pool static level releases its staging once its upload has been emitted: schedule_upload sets release_staging on the upload record, the encoder's answer reaches the API thread at the next Present, and release_emitted_staging drops the texture's copy. The cached wrapper kept the pages alive anyway, until the texture was destroyed, and since the texture no longer counted them the address-space watch charged them to other. The mechanism is read from the source and the existing unit test that pins it (cached_native_ownership_keeps_original_owner_alive in windows/core/src/guest_pages/tests.rs). A pin that lasts until texture destroy explains a plateau proportional to the released levels of the live textures; it did not obviously explain a steady creep of about 26 MiB a minute while the texture count stays flat. The GTA IV run with this change (Verification) settles the page-box part: other stays at 0 and the page boxes stay flat. The process footprint still grows, and the new figures place that growth outside the layer's allocations.

Changes

The fix: an emitted upload that carries release_staging takes the level's wrapper out of the cache and parks it on the resource retention queue behind the current submission, the same path a backing change already took (now park_staging_wrapper over take_staging_wrapper). Its destroy waits for both the upload and the draw counter to pass that submission, the keepalive drops after the destroy, and the lease then completes, so the pages are freed or parked in the staging lane once the GPU retires that upload.

The PE side applies the answer at the next Present, and only when no newer upload of the level overtook it (is_latest_upload), so a DEFAULT-pool level rewritten every frame (an UpdateTexture then a draw) carries release_staging on every upload yet keeps its staging. Retiring on every answer would then create and destroy a wrapper per level per frame over the same backing. The slot an answer empties therefore keeps the backing's address and length, and when the next upload wraps that same backing again (MipStagingBuffer::created), the new wrapper is marked kept_after_release and later answers leave it cached (take_released_staging_wrapper). The churn is one extra wrapper per backing. A kept wrapper still retires at a backing change or at texture destroy, so a level rewritten for a while and released afterwards holds its pages until then, as every level did before this change. The two new counters tex_wrapper_create_total and tex_wrapper_retire_total, with a churn row under the textures wrapped row, show the wrappers created and queued for destroy per window, so a bench run or a game log shows the churn directly; the bench reads them as noisy per-frame rates.

Nothing crosses the boundary that did not before: the encoder already reads release_staging from the upload record. The textures destroys row and tex_destroy_total now include these wrapper destroys; the key's meaning (texture-lifecycle destroys, staging wrappers included) is unchanged, but its count moves earlier, from texture release to upload retirement.

Three new perf-kv gauges follow existing paths. process_footprint_bytes is the process's phys_footprint, read with proc_pid_rusage (ri_phys_footprint, the same ledger value TASK_VM_INFO reports), which is what the HUD shows as app memory. metal_allocated_bytes is the MTLDevice's currentAllocatedSize, which until now was read only when a texture was refused. tex_staging_wrapped_bytes is the padded staging under the encoder's cached per-level wrappers.

The encoder reads the three once per perf window, only on the frame whose summary is about to emit, and carries them in CacheSizes as memory: Option<MemoryGauges>, beside the cache counts the summary already samples there. A PERF=1 build pays for them once every two seconds, a build without it never calls them. The grid gets footprint and metal alloc rows under the fault row and a wrapped row in Resources (textures), all through res_row. A window closed between the encoder's due check and the emit check gets no sample: the grid prints n/a and the line leaves the keys out, as it does with the fault keys.

In the address-space watch (windows/d3d9/src/device/mem_watch.rs) the page boxes gain an upload leases holder: texture staging that only the device's upload leases still keep. The sample walks the device's retained page leases (PacketRetirement::tally_page_leases) and counts an allocation only when every reference to it is a lease's (mtld3d_core::guest_pages::LeaseOnlyPages), so staging a texture still holds stays under texture staging and other is left with read-back pages of a call in progress and anything not yet named. Beside the page boxes the line logs d3d9.dll heap <n> MiB committed, from snmalloc_rs::SnMalloc::memory_stats(): snmalloc's own count of the chunks it has committed and handed to its allocators, two atomic loads, page boxes included. That is the existing pattern for a PE-side memory figure that every build reports, and it needs no new field in the per-frame payload; the walk runs only on the watch's debug samples and its threshold warnings.

perf_rule in windows/tests/tests/e2e/bench.rs records process_footprint_bytes and metal_allocated_bytes as info: they count Wine, the driver and what earlier benchmarks of a round left in the process, so they move with the machine and the round's order. tex_staging_wrapped_bytes keeps the bytes rule, since it counts the layer's own cache, which the frames' API calls decide. docs/ARCHITECTURE.md lists the three keys, the keys a window can leave out and the _bytes suffix's sampled form, describes the new holder and the heap figure in the address-space watch section with a reworked example line, and states the wrapper's retirement under "Retirement counters are exact".

A known limit of the bound: a level the game rewrote every frame for a while and then leaves alone keeps its one kept wrapper, and so that level's staging pages, until the backing changes or the texture is destroyed, as every level did before this change. That is at most one level's staging per such texture.

Alternatives considered

Retiring the cached wrapper only once the PE side has actually dropped the staging, by carrying the drop back to the encoder in a later frame. That retires exactly, the rewritten-then-released level included, but needs a new frame operation from the API thread. Remembering the retired backing in the slot needs no wire change and bounds the churn to one extra wrapper per backing; its residual is the level that is rewritten every frame for a while and released later, which holds its pages until texture destroy as before.

Not caching wrappers for droppable levels at all. A DEFAULT-pool level written by several UpdateTexture rects before its staging is fully written uploads more than once from the same pages, and those uploads would each create a wrapper; retiring at the release keeps the cache for them.

For the heap figure, carrying snmalloc's statistics in the per-frame payload to the perf summary, as the pe_pagebox_* keys are. That adds wire fields to every frame for a figure that changes slowly and that the address-space watch, which every build has, already sits next to.

For the upload-lease holder, a process-wide gauge charged when a lease is created, as held_pages does for surfaces and encoder leases. A read lease shares its owner with the texture until the texture lets go, so such a gauge would count every live texture's staging twice; the sample-time walk counts an allocation only once nothing but leases holds it.

Verification

make fmt, make check and make check PERF=1 are green. They cover rustfmt, clippy with nursery and pedantic, the audit (derive inventory, doc-block shape, test file placement, the COVERAGE index, the confined patterns) and the docs. make test-unit passes (1996 core tests, 758 unix and runner tests), and so does make test-unit PERF=1 (2065 and 760); make test-unit PERF=1 also passes on the first commit alone. make test ISOLATED=1 on 487f637 passes both end-to-end legs: i686 1085 tests, 1072 passed, 0 failed, 13 ignored, 6 processes; x86_64 1083 tests, 1070 passed, 0 failed, 13 ignored, 6 processes.

The fix is tested at helper level. encoder::tests::taking_a_staging_wrapper_empties_its_slot_and_keeps_the_others pins the retirement helper, which empties one slot and hands the pages' owner over with the wrapper. encoder::tests::a_release_retires_a_levels_wrapper_once_per_backing pins the churn bound: the first answer retires and the slot remembers the backing, a wrapper recreated over that backing survives later answers, a backing change still retires it, and a never-released slot, a live wrapper or another length start over. encoder::tests::the_wrapper_gauge_counts_populated_level_slots_only pins the wrapper gauge. guest_pages::tests::lease_only_pages_count_staging_the_texture_let_go_of_once pins the new holder: staging a texture holds is not counted, released staging counts once for two leases, a partial tally claims nothing, and a lease whose read was released still counts. The summary and perf-kv golden tests carry the new rows and keys; perf::tests::an_unsampled_window_prints_n_a_and_leaves_the_memory_keys_out pins a window without the memory sample (the three cells read n/a in their columns, the three keys are absent, nothing else moves); the holder tests in address_space carry the new clause.

No new end-to-end test, because the suite cannot observe the fix. The pages it frees are a DEFAULT-pool level's staging, which the application never sees mapped; the gauges that count them are statics of d3d9.dll that a test executable cannot read; and a freed staging box is often parked in the staging lane rather than decommitted, so not even the address space changes visibly. The suite already exercises every path that runs after a released level: re-uploads after the release through partial UpdateTexture, UpdateSurface, LockRect and GetDC (see the textures.rs row of windows/tests/COVERAGE.md). After this change each of them creates a fresh wrapper instead of replacing a stale one, and all of them pass.

make bench-ab BASE=5eb2ec52 BENCH_WAIT_IDLE=120 (default wow set, five round pairs, production i686 build): 9 benchmarks, 3125 metrics, 1 regressed, 3 improved, 0 exact changed, 35 added (the five new keys), 2 incomplete (the fault keys of streaming, absent from one base window), and both scene shapes unchanged. The one regression is perf.enc_op_binds_vbib_ms in wow112, 0.173 to 0.181 ms (+4.62 %; base rounds 0.173, 0.173, 0.174, 0.173, 0.178, candidate 0.181, 0.169, 0.181, 0.181, 0.178). That is the per-draw VB/IB bind phase, which this change does not touch, in a benchmark that uploads no textures (tex_uploads_pf 0 in both legs, wrapper creates and retires 0), so none of the new code runs there apart from the once-per-window gauge read; this reads as code placement, the kind of gap an A/A run of this benchmark has shown before and the same span as #1077, and wow112's perf.enc_ms (+0.71 %, sigma 0.83 %) and frame.p50 (-0.17 %) agree. The improvements are enc_op_resolve_ms and its children in api_call_cost (-11 %). frame.p50 per benchmark, base to candidate: buffers 0.217 to 0.218 ms, frame_shape 0.543 to 0.544, query_poll_spec 0.371 to 0.372, query_poll_wow 0.055 to 0.055, streaming 0.188 to 0.195 (+1.95 %, sigma 3.42 %), wow112 2.060 to 2.056. Submit and encode: submit_ms +3.16 % in wow112 (sigma 2.69 %), +0.55 % in frame_shape, +1.61 % in streaming (sigma 5.14 %); enc_work_ms +0.71 %, +3.41 % (sigma 3.33 %) and +3.12 % (sigma 2.57 %); none flagged. In streaming, the only benchmark that uploads, every texture upload row is unchanged (9.583 uploads a frame: 7.583 raw, 1.500 padded, 0.500 pass), tex_destroy_pf is 13.082 against 13.084, and the candidate creates and retires 9.583 wrappers a frame, one per upload, since that benchmark's textures live one frame and each upload wraps a new texture's staging; the destroy count does not move because those wrappers used to retire at texture release in the same frame.

GTA IV on 487f637 (GTAIV-3489.log) against e99fa02 (GTAIV-12703.log) at matched session times: page-box other stays at 0 MiB for the whole run (before 107 to 617 MiB), the page boxes stay flat at 425 to 445 MiB (before 523 to 1052 MiB), and at +4:34 the real free address space reads 1801 against 1238 MiB with the largest free block at 1758 against 1167 MiB. upload leases peaks at 6 MiB and wrapped at 7.8 MiB. Wrapper churn over the run is 56287 created and 56187 retired, a median of 1.35 a frame with spikes to 30 a frame while the game streams. The process footprint still grows by 237 MiB from 02:40:36 to the end of the run; of that, the Metal allocated size accounts for 3 MiB, d3d9.dll's heap for 8 and the page boxes for 1, and the rest is the game's and Wine's own 32-bit memory (about 133 MiB) and the 64-bit side (about 94 MiB), outside what this layer allocates. The second half of that run was submit-bound (submit 8.74 against 6.6 ms, 109 against 120 fps) with no correlation to the wrapper churn in the same windows; it is likely the scene, and it is not explained here.

Rules check

Statics: none added. The heap figure reads snmalloc's process-wide backend counters; LeaseOnlyPages is a value built per sample.

Config keys and environment variables: none.

Wire fields: none. The fix uses release_staging, a u32 already on TextureUploadRecord. The new counters are read on the side that reports them: the footprint, the Metal size and the wrapper bytes on the unix side, the heap and the upload leases on the PE side.

perf-kv keys: five added, process_footprint_bytes, metal_allocated_bytes, tex_staging_wrapped_bytes, tex_wrapper_create_total and tex_wrapper_retire_total, each with its row in the ARCHITECTURE table and its bench class. No existing key changes name, unit or aggregation.

Dependencies: none. proc_pid_rusage and rusage_info_v2 come from libc, memory_stats from the pinned snmalloc-rs.

Derives: LeaseOnlyPages derives Default only; no Clone or Copy was added, so the derive inventory is unchanged. MipStagingBuffer, which already derives Clone and Default, gains one bool field, its only one, so the booleans-pack rule does not apply.

Lint suppressions: none. Unsafe: two blocks in process_footprint, one operation each with its SAFETY comment.

Duplicates: park_staging_wrapper replaces the inline retirement at the backing change and at texture destroy, so all three retirement sites share it and its counter. process_footprint sits beside task_faults in handlers.rs and follows it.

Placement: the lease tally is pure and lives in mtld3d-core with its unit test; the wrapper helpers are free functions tested in unix/unix/src/encoder/tests.rs; every test is in its <stem>/tests.rs. The summary rows go through res_row and the golden layout and kv tests carry them.