Skip to content

fix(lineage): preserve producers through identical-content copies - #302

Merged
TrevorBasinger merged 1 commit into
mainfrom
tb/preserve-checkpoint-copy-lineage
Sep 14, 2026
Merged

TrevorBasinger merged 1 commit into
mainfrom
tb/preserve-checkpoint-copy-lineage

Conversation

@TrevorBasinger

@TrevorBasinger TrevorBasinger commented Sep 11, 2026

Copy link
Copy Markdown
Member

When a packaging command copies a checkpoint or evaluation report and then reads back the copy, the input and output share one content identity. Publish traversal selected the packaging job as the latest producer, then stopped at its visited-artifact guard. The published graph consequently omitted the earlier training and evaluation jobs. A real private canary captured five jobs locally but published only three.

Resolve each input against the latest producer preceding its consuming job, and track visited artifacts together with that consumer boundary. Both database-backed publication and read-only step previews use the same ordering helper. Existing depth and visited-job guards remain; insertion order breaks equal timestamp ties.

Validation:

  • The initial six SQLite regression cases failed before the fix. Expanded coverage now exercises artifact publication, step publication, read-only previews, equal timestamps, self-reading outputs, and exclusion of later unrelated producers. All 36 focused publish/collection tests pass.
  • Reconstructed the real run's immutable, hash-verified stored package locally: baseline returns three jobs; this patch returns exactly five jobs, 71 artifacts, and 212 unique links, matching its recorded publication counts. No training or remote publication was repeated for this diagnostic.
  • ruff check ., ruff format --check . (910 files), mypy roar (407 source files), and git diff --check pass.
  • Broad local suite: initial run had 2,480 passes and 41 skips. After building the current native tracer and isolating host Git rewrites, the affected reruns passed 89 additional tests. Three runtime-fixture failures remain and reproduce with unchanged source: two copied uv-managed Python 3.10 executables cannot locate their standard library; one cross-version test discovers an already-populated parent Python 3.11 environment instead of exercising runtime installation. New tests and lineage reconstruction pass. Draft pending CI integration validation.

No API or schema changes. This prevents future ancestry loss; repairing a previously finalized incomplete publication remains a separate recovery step using its preserved package.

@TrevorBasinger
TrevorBasinger marked this pull request as ready for review September 14, 2026 14:07
@TrevorBasinger
TrevorBasinger merged commit 61e5e98 into main Sep 14, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant