PersistentFileMapped reads a truncated store when reopening an existing file, and grow_filled destroys it

[konard]

PersistentFileMapped cannot reopen a file written by an earlier process: it reports a fraction of the links the file holds, and there is no safe way to restore the real capacity.

Measured on a 64 MB store holding 307 records

how it is mapped	links reported
PersistentFileMapped::new(file)	91
PersistentFileMapped::new(file) then grow_filled(items, default)	0 — data destroyed
FileMapped::new(file) then unsafe { grow_assumed(items) }	524766 ✔ correct

let bytes = std::fs::metadata(&path)?.len() as usize;
let items = bytes / std::mem::size_of::<LinkPart<usize>>();
let file = OpenOptions::new().read(true).write(true).open(&path)?;
let m: PersistentFileMapped<LinkPart<usize>> = PersistentFileMapped::new(file)?;
unit::Store::<usize, _>::new(m)?.count();   // 91, not 524766

FileMapped starts with a logical capacity of zero however much the file already holds, so unit::Store::new reads a truncated store. Anything past the truncation is invisible, and a schema validation over the first N points fails at the first missing one.

grow_filled does not do what its documentation says

/// Fills only the genuinely uninitialised tail of the grown region,
/// keeping the bytes that were already persisted in the file.
fn grow_filled(&mut self, cap: usize, value: Self::Item) -> MemResult<&mut [Self::Item]>

It does not keep them. Reopening a 5-point store and calling grow_filled to restore capacity yields count() == 0 — the fill ran across the whole region, including the persisted part. That is the same wipe the type exists to prevent, on the one path a caller would reach for to restore capacity.

Request

A safe constructor that maps an existing file and adopts the capacity its bytes represent — the equivalent of grow_assumed, which is unsafe and therefore unusable in a crate with unsafe_code = "deny":

impl<T> PersistentFileMapped<T> {
    /// Maps `file` and adopts the capacity its existing contents represent.
    pub fn open_existing(file: std::fs::File) -> std::io::Result<Self>;
}

LinkPart<usize> is repr(C) over usize fields so every bit pattern is valid; the safety argument is the crate's to make once, rather than each caller's to make with its own unsafe block.

Either that, or grow_filled doing what it documents, would resolve it.

Impact

link-assistant/router v0.125.0 adopted PersistentFileMapped and cannot write to any store carried over from a previous release: reads answer from a second projection so nothing looks wrong, then every write fails schema validation. Reported as link-assistant/router#374, and it blocks upgrading a live deployment.