themeparks-backfill: --since/--until, incremental reruns, final days only; changes() exposes opening

[cubehouse]

What this fixes

A customer-style run of themeparks-backfill 4.0.1 against a key that reaches the whole archive found four problems. Each was reproduced against the live API before it was fixed.

1. No date range. You always got the whole archive. New flags: --since YYYY-MM-DD and --until YYYY-MM-DD. Both days are inclusive, each must be a real calendar day (so 2025-02-30, 2025-1-1 and 20250101 are rejected), and --since after --until is refused before any request is made.
2. Reruns never updated. Once a park was marked complete, running it again printed already complete and exited 0 without fetching anything, so a nightly cron looked healthy but never updated. Now a finished file continues from the day after its last one. An interrupted run still resumes at its page boundary.
3. The last days were partial, and stayed partial. Runs ended on retrievableThrough, which is usually today. Today's row is the day so far, and the API docs say the archive records days 2 to 3 behind live data, so the last few days of every file were written while they could still change. Now:
   • a run ends at the new span().final_through (the earlier of recordedTo and retrievableThrough) and says when it held days back;
   • the next run adds those days once they are final;
   • every row in the file is final, so the file only ever gets appended to.
4. history.changes() threw away opening. That is the state in force at the start of the range. It is now exposed without breaking anything: iterating yields the same (entity id, row) pairs as before, and .opening is a dict of HistoryOpening keyed by entity id, costing no extra request. For async, the attribute is readable once you have iterated or after await changes.load().
5. README said waitTime is an int. The spec says it is a number and the models use float. The README now matches the models, and a test enforces it.

Also fixed along the way (each has a test and a committed mutant)

• A finished state file with a different column layout was appended to: the whole archive a second time, under a second header, exit 0. It is now refused, the same way an unfinished one already was.
• A rerun interrupted before its first page recorded no resume point. The next run would have started from the top of the archive. It now records the run's own start.
• If the data file had been deleted but its state file remained, the next run carried on part-way through. It now downloads the park again.
• A continued file never skips ahead to the key's first day. If the day a file continues from is older than the key may read (a cron that missed more days than the window, or a lapsed plan), the run used to carry on from the key's first day and leave a gap the state file did not record. It is now refused with exit 1, the file and state untouched, and a message naming --overwrite.
• A fixed --since older than the key's window keeps working. The file starts at the key's first day, and the same command line succeeds every later night. A test pins this.

Interrupted runs (from review)

• An interrupted run never appends a day twice. The state was written only at the end of a run or on an SDK error. Ctrl-C, SIGTERM or SIGKILL during a nightly extension therefore left complete: true with the old end, and the rerun appended the same days again. The state is now written atomically before the first request and after every page, together with the data file's size. A rerun first truncates the file to that size, so stopping at any point costs at most the page in flight. A file shorter than its checkpoint is refused.
• SIGTERM stops like Ctrl-C and exits 143.
• Two runs on the same park and --out are refused by an advisory lock (POSIX only).
• The state records the real first day. start is the first day actually written, after the key's window floor, and a new since field keeps the start that was asked for. The same --since keeps working. A different one earlier than the file's first day is refused.
• A partial first page loses no entity. Older states recorded only lastDay, the newest day any entity had reached. They now go back a whole page, 31 days, instead of resuming there.
• Finality is described honestly. Days through recordedTo are what the archive has recorded, and each is fetched once. The README explains how to fetch a range again if the archive re-records past days.
• Library additions: HistoryChanges.close(), AsyncHistoryChanges.aclose(), and an exported HistoryOpening, whose degraded field is now documented.

Compatibility

• State files move to version 2. end is now the newest final day, and the state gains since (the start asked for) and size (bytes of the file it vouches for).
• Version 1 files from this SDK are upgraded, not refused. Nobody recorded which of their newest days were final, so the upgrade removes rows dated within 7 days of the old end and fetches those days again. The other rows stay byte for byte, and a file whose newest row is older than that is not rewritten at all. A 4.0 file lying wholly inside that 7-day refetch window, as every anonymous 7-day file does, is simply downloaded again.
• --since on a rerun:
  • the same --since, or a later one inside the file, is accepted (a fixed or rolling cron line works);
  • one before the file's first day, or one that would leave a gap, is refused with a message saying what to do.
• backfill_park gains a keyword-only window argument.
• changes() now returns an iterator object instead of a generator.

Verification

• Live, without a key, on Magic Kingdom: a Ctrl-C raised 39 rows into an extension left the state at the previous checkpoint. The rerun discarded the 26,073 bytes written after it and finished with 314 rows, 0 duplicated (entityId, date) pairs and exit 0. A third run reported up to date. A different --since before the file's real first day was refused with exit 1.

• pytest tests/unit: 457 passed on 3.9, 3.12 and 3.13. ruff check, ruff format --check, mypy themeparks and mkdocs build --strict are clean.

• tests/mutation/run.py: 43/43 killed, including 25 new mutants for these defects.

• Live, without a key, against Magic Kingdom:

  • a 4.0.1 file was upgraded: the partial tail was replaced, it stopped at recordedTo, and no (entityId, date) pair was duplicated;
  • running it again reported "up to date";
  • --since/--until wrote CSV for exactly those days, and a rerun appended one new final day;
  • changes() returned 72 openings, 47 of them OPERATING at midnight.

Follow-up (not in this PR)

• Raw history export command.

🤖 Generated with Claude Code