feat(memtrack): collect RSS via rss_stat and folio-rmap reconstruction by not-matthias · Pull Request #453 · CodSpeedHQ/codspeed

not-matthias · 2026-07-13T13:04:00Z

Summary

Adds RSS (resident set size) collection to memtrack, in two layers:

Authoritative RSS from the kernel's kmem:rss_stat tracepoint — absolute per-mm resident bytes (anon/file/shmem/swap), latest-wins.
Reconstructed anonymous RSS from raw kernel folio-rmap events: fentry hooks on the anon folio-rmap add/remove functions emit signed page-count deltas, so anon RSS can be rebuilt over time as Σ(add − remove) × PAGE_SIZE.

The reconstruction is a total anon RSS delta-sum, not a per-vaddr resident map — the kernel remove hook (folio_remove_rmap_ptes) carries no address, so removals can't be attributed to a vaddr (the add hooks' faulting vaddr is emitted for observability only).

Commits

feat(memtrack): track RSS via kmem:rss_stat tracepoint — the baseline: EVENT_TYPE_RSS contract, MemtrackEventKind::Rss, parser arm, writer bench case, gated integration test.
feat(memtrack): reconstruct anon RSS from gated folio rmap fentry hooks — EVENT_TYPE_RMAP_ANON + RmapAnon event, five fentry programs (add_new / add_ptes / remove_ptes / remove_pmd / remove_pud), CO-RE folio helpers, and the load/attach gating.
test(memtrack): validate anon RSS reconstruction against rss_stat — ramps anon RSS via mmap/munmap and asserts the reconstructed estimate tracks the rss_stat MM_ANONPAGES peak within 25%.

What's on by default vs gated

rss_stat tracepoint: always on. Emitting Rss events is the intended new default behavior introduced by this change — the RSS tracepoint is not gated.
RmapAnon folio-rmap fentry programs: off by default, gated behind CODSPEED_MEMTRACK_TRACK_RMAP=1. When the flag is unset they are set_autoload(false) before load and never attached, so:
- the skeleton still loads on any kernel (a missing fentry BTF target would otherwise fail the whole load), and
- the folio-rmap reconstruction path stays out of production (--mode memory) and out of the existing test suites — no RmapAnon events are produced by default.

Verification

Run in a privileged, --pid=host container sharing the host kernel (7.0.12):

Reconstruction (flag on): ✅ real anon amplitude = 64 MiB, estimated peak = 64 MiB (ratio 1.00).
Regression (rss_tests, flag unset): ✅ passes — no RmapAnon events, folio-rmap programs stay unloaded.
Parser unit tests, cargo fmt, and clippy clean.
Kernel BTF signatures for all five folio_*_rmap* functions verified to match the BPF_PROG arg layouts.

Review notes (draft)

track_command ordering: the shared test helper spawns the child before enable()/track(root_pid). In practice the child's fork→execve→ld.so→libc-init far outlasts the two BPF-map updates, so tracking is armed before the workload allocates (both fixtures captured full event streams). Flagging in case we'd prefer a leading settle-usleep in the fixtures or an enable-before-spawn change in the helper.
PAGE_SIZE: hardcoded to 4096 (correct on x86_64). On a 16K/64K-page arm64 runner the estimate would need sysconf(_SC_PAGESIZE); rss_stat is already in bytes and unaffected. Happy to switch to sysconf if these tests run on arm64 CI.
The chart/inspection tooling used during development lives outside the tree (dev artifact) and is not part of this PR.

codspeed-hq · 2026-07-13T13:06:37Z

Merging this PR will not alter performance

⚠️

Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

✅ 17 untouched benchmarks

_{Comparing cod-3089-collect-rss-in-memtrack (b5a8e9e) with main (1dcc738)}

greptile-apps · 2026-07-13T13:07:01Z

Greptile Summary

This PR adds RSS reporting to memtrack. The main changes are:

Kernel rss_stat events for per-process resident counters.
Optional folio-rmap hooks for RSS reconstruction.
Fork, exec, and exit events for RSS lifecycle handling.
RSS fixtures, snapshots, and CI coverage.

Confidence Score: 4/5

This is close, but the startup failure path should be fixed before merging.

rss_stat is optional, but the related lifecycle tracepoints still abort tracker startup.
A host can lose all memory tracking even when only the RSS lifecycle path is unavailable.

Files Needing Attention: crates/memtrack/src/ebpf/memtrack/tracking.rs

Important Files Changed

Filename	Overview
crates/memtrack/src/ebpf/memtrack/tracking.rs	Attaches the RSS tracepoint, lifecycle tracepoints, and optional rmap fentry hooks.
crates/memtrack/src/ebpf/c/rss.bpf.h	Adds RSS and rmap event programs plus process lifecycle event emission.
crates/memtrack/src/ebpf/tracker.rs	Adds rmap gating and an RSS-only tracker construction path.

Prompt To Fix All With AI

### Issue 1
crates/memtrack/src/ebpf/memtrack/tracking.rs:12-14
**Lifecycle attaches abort**

These lifecycle tracepoints still make tracker startup fail before allocator tracking can run. `rss_stat` is handled as optional below, but `task_newtask`, `sched_process_exec`, and `sched_process_exit` still return attach errors through `attach_tracepoints()`. On a host where allocator probes can attach but one of these RSS reconciliation tracepoints cannot, `Tracker::new()` returns an error instead of starting memory tracking with that RSS path disabled.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

_{Reviews (28): Last reviewed commit: "ci(memtrack): run bpf tests on arm and a..." | Re-trigger Greptile}

not-matthias · 2026-07-20T16:52:22Z

@greptileai pause reviews for now until i enable them again

greptile-apps · 2026-07-20T16:52:41Z

Understood — pausing reviews. I'll hold off until you re-enable them.

Tip: You can customize Greptile's behavior for this repo with .greptile/rules.md and .greptile/config.json.

Set AllowShortFunctionsOnASingleLine: None in .clang-format and apply it, reformatting the allocator uprobe macros accordingly.

A forked child's inherited RSS is invisible to rss_stat: the fork-time counter copies fire outside the child's context, and anon COW faults are counter-neutral, so a child that only touches inherited memory never reports anything on its own. A fork event carrying the parent pid lets consumers seed the child from the parent's last absolutes; exec and exit mark where the address space is replaced or torn down.

Sample the kernel's per-mm resident counter through the kmem:rss_stat tracepoint, emitting absolute byte values per mm member. Adds the EVENT_TYPE_RSS contract, MemtrackEventKind::Rss, the parser arm, and a writer bench case. An rss_stat update from reclaim or another process's madvise fires in the actor's context; track (mm_id, member) -> owning pid so those updates reach the owner. External events may only lower a counter, so stale reads and mm_id collisions cannot invent peaks.

Attach fentry hooks on the folio-rmap add/remove functions, emitting signed page-count deltas per MM_* bucket so anon, file, and shmem RSS can be reconstructed over time. Gated behind CODSPEED_MEMTRACK_TRACK_RMAP; the programs stay autoload-off by default so the skeleton loads on any kernel, with the PUD pair (only present since v6.15) gated separately from the core set so rmap still works on older kernels. Adds the EVENT_TYPE_RMAP contract, MemtrackEventKind::Rmap, parser arm, and bench case. Recover the owning pid for rmap events run by another task (reclaim, process_madvise, khugepaged, KSM) from the mm_struct pointer, and maintain the ownership maps across exec and thread-group exit. The same ownership binding also validates external (curr==0) rss_stat updates, so a stale mm can no longer attribute a counter to the wrong pid.

Add the rss_tests integration suite: per-workload RSS/rmap reconstruction snapshots against /proc ground truth, fork-seeded child RSS, exec/exit resets, foreign-actor rmap attribution (reclaim, external madvise), and mm-ownership across CLONE_VM and exec. Extend tests/shared.rs with the tracker/fixture helpers these tests need and move compile_c_source into it for reuse. The suite needs two surfaces the production paths don't: a tracker mode that skips the allocator probes and exec-mapping watcher, and readers for the mm-ownership maps.

Add an aarch64 lane to the bpf-tests matrix and run the rss integration tests alongside the existing test binaries.

not-matthias · 2026-08-19T15:52:55Z

@greptileai review again, focus on the eBPF code (mostly RMAP tracking), be exhaustive

greptile-apps · 2026-08-19T15:56:32Z

+        self.attach_task_newtask()?;
+        self.attach_sched_process_exec()?;
+        self.attach_sched_process_exit()?;


Lifecycle attaches abort

These lifecycle tracepoints still make tracker startup fail before allocator tracking can run. rss_stat is handled as optional below, but task_newtask, sched_process_exec, and sched_process_exit still return attach errors through attach_tracepoints(). On a host where allocator probes can attach but one of these RSS reconciliation tracepoints cannot, Tracker::new() returns an error instead of starting memory tracking with that RSS path disabled.

Prompt To Fix With AI

This is a comment left during a code review. Path: crates/memtrack/src/ebpf/memtrack/tracking.rs Line: 12-14 Comment: **Lifecycle attaches abort** These lifecycle tracepoints still make tracker startup fail before allocator tracking can run. `rss_stat` is handled as optional below, but `task_newtask`, `sched_process_exec`, and `sched_process_exit` still return attach errors through `attach_tracepoints()`. On a host where allocator probes can attach but one of these RSS reconciliation tracepoints cannot, `Tracker::new()` returns an error instead of starting memory tracking with that RSS path disabled. --- For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

…ooks

GuillaumeLagrange

olgtm, obviously quite a big chunk to chew, but looks mostly fine to me. Main point of concern is the maintainability of "which function can be track for which kernel version" which IMO could be improved.

GuillaumeLagrange · 2026-08-21T09:35:38Z

+/* Kernels < 6.18 store folio->flags as a bare unsigned long instead of
+ * memdesc_flags_t; probe which layout the running kernel has. */
+struct folio___legacy {
+    unsigned long flags;
+} __attribute__((preserve_access_index));
+
+static __always_inline unsigned long folio_read_flags(struct folio* folio) {
+    if (bpf_core_field_exists(folio->flags.f)) {
+        return BPF_CORE_READ(folio, flags).f;
+    }
+    return BPF_CORE_READ((struct folio___legacy*)folio, flags);
+}
+
+/* Kernels < 6.6 store the large-folio order in a dedicated byte instead of
+ * the low byte of _flags_1. */
+struct folio___order_byte {
+    unsigned char _folio_order;
+} __attribute__((preserve_access_index));


this could benefit form being in a dedicated legacy or kernel support file

GuillaumeLagrange · 2026-08-21T09:35:39Z

+int BPF_PROG(tracepoint_task_newtask, struct task_struct* child, __u64 clone_flags) {
+    if (clone_flags & CLONE_THREAD) {
+        return 0;
+    }
+
+    __u32 parent_pid = current_tgid();
+    if (!is_tracked(parent_pid)) {
+        return 0;
+    }
+
+    /* Register the child here rather than on sched_process_fork: that
+     * tracepoint fires for CLONE_THREAD too and carries only raw task pids,
+     * which would fill the tracking maps with thread tids that no exit path
+     * removes (group death deletes only the tgid). task_newtask fires before
+     * wake_up_new_task, so registration precedes any event from the child.
+     * The BTF-typed variant is used because the child's pid must be read in
+     * the tracker's namespace, which the tracepoint's raw pid field cannot
+     * give. */
+    __u32 child_pid = task_ns_tgid(child);
+    if (!child_pid) {
+        return 0;
+    }
+    track_child(child_pid, parent_pid);
+
+    SUBMIT_EVENT_AS(child_pid, EVENT_TYPE_FORK, { e->data.fork.parent_pid = parent_pid; });
+}


Does not belong to the rss.bpf, it belongs to a specific fork/exec events watcher module

Same for the sched_process_exec probe, WDYT?

GuillaumeLagrange · 2026-08-21T09:35:39Z

-SEC("tp_btf/sched_process_fork")
-int BPF_PROG(tracepoint_sched_fork, struct task_struct* parent, struct task_struct* child) {
-    follow_fork(task_ns_tgid(parent), task_ns_tgid(child));
-    return 0;
-}


Related to the other comments, why was this removing from the process_tracking code and moved into the rss one?

…ction Rename the ownership maps to {object}_by_{key}.

…ooks Rewrite the rss_owner_by_counter doc comment as key/value/rule.

State that swap entries are out of scope.

…ooks Split rmap hooks and mm-ownership state out of rss.bpf.h.

…ooks Replace the two-bool RmapSupport with a three-level enum resolved from the kernel release, so the illegal core-off/pud-on state is unrepresentable and no vmlinux BTF load is needed. Every folio rmap target is an unconditional definition in mm/rmap.c (only the bodies are ifdef-ed), so the release is as precise a gate as a BTF symbol probe.

…ooks

greptile-apps Bot reviewed Jul 13, 2026

View reviewed changes

not-matthias force-pushed the cod-3089-collect-rss-in-memtrack branch 2 times, most recently from b670b0a to 2f41984 Compare July 13, 2026 17:32

not-matthias marked this pull request as ready for review July 14, 2026 15:38

greptile-apps Bot reviewed Jul 14, 2026

View reviewed changes

Comment thread crates/memtrack/src/ebpf/memtrack.rs Outdated

not-matthias force-pushed the cod-3089-collect-rss-in-memtrack branch from 41945a5 to 64688a9 Compare July 17, 2026 14:23

greptile-apps Bot reviewed Jul 17, 2026

View reviewed changes

Comment thread crates/memtrack/src/ebpf/memtrack.rs Outdated

not-matthias force-pushed the cod-3089-collect-rss-in-memtrack branch 2 times, most recently from b02f3cb to 411b713 Compare July 17, 2026 18:12

not-matthias changed the base branch from main to cod-1801-only-attach-to-used-libraries-in-memtrack July 20, 2026 09:07

Base automatically changed from cod-1801-only-attach-to-used-libraries-in-memtrack to main July 20, 2026 13:12

not-matthias force-pushed the cod-3089-collect-rss-in-memtrack branch 2 times, most recently from e00fb6b to a68424a Compare July 20, 2026 16:08

greptile-apps Bot reviewed Jul 20, 2026

View reviewed changes