Skip to content

Let a caller supply the storage backend to annual_cli.main() - #4584

Merged
springfall2008 merged 1 commit into
mainfrom
feat/annual-cli-storage-factory
Aug 19, 2026
Merged

Let a caller supply the storage backend to annual_cli.main()#4584
springfall2008 merged 1 commit into
mainfrom
feat/annual-cli-storage-factory

Conversation

@mgazza

@mgazza mgazza commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

What

annual_cli.main() gains an optional storage_factory, called as storage_factory(work_dir, log) — exactly how StorageLocalFiles is constructed today, so the default is that class itself and the command line behaves as it always has.

def main(argv=None, storage_factory=StorageLocalFiles):
    ...
    storage = storage_factory(args.work_dir, log)

Why

StorageBase is already an abstraction: annual_weather.py and annual_tariff.py only ever call self.storage.load() / save(), and AnnualPredictor uses whatever storage it is handed. But this entry point hard-coded the one implementation, so running the annual tool against a different backend meant forking the CLI — or reimplementing main(), and with it the --machine stdout/stderr contract. That last part is the fiddly bit worth not duplicating: the redirect_stdout guard that stops a stray print() from anything predictor.run() pulls in from corrupting the single JSON object a parent process parses off stdout.

The motivating case is embedding the annual tool in a long-lived service. There, a per-process work dir means every process re-downloads the same immutable ERA5 and Octopus data, none of it can be shared between them, and the duplicated requests land on the rate APIs. Pointing several processes at one shared backend (Redis, S3, a database) fixes both — and with this, that needs a factory rather than a fork.

Scope

Two lines of behaviour change, plus a docstring. No change to the CLI, its arguments, or its output. Nothing else in the file moves.

Testing

Added to tests/test_annual_cli.py, in the existing style:

  • the default still constructs a StorageLocalFiles — so a plain command-line run is unchanged
  • a supplied factory's storage reaches AnnualPredictor, and the factory receives --work-dir and the run's log callable

I checked the new assertions genuinely fail without the change (TypeError: main() got an unexpected keyword argument 'storage_factory') rather than passing vacuously.

Suites run and passing: annual_cli, annual_cli_machine, annual_cli_machine_end_to_end, annual_job, storage.

black 23.11.0 and ruff 0.11.4 — the versions pinned in .pre-commit-config.yaml — both report no changes. (The commit itself was made with --no-verify only because the pre-commit binary on this machine has a broken install; the hooks themselves were run by hand.)

…ain()

StorageBase is already an abstraction — annual_weather.py and annual_tariff.py
only ever call self.storage.load()/save(), and AnnualPredictor takes whatever
storage it is given — but annual_cli.main() hard-coded StorageLocalFiles. The
only way to run the annual tool against a different backend was therefore to
fork the CLI, or to reimplement main() and with it the --machine stdout/stderr
contract, which is the fiddly part: the stdout redirect that stops a stray
print() from corrupting the one-JSON-object stdout a parent process parses.

main() now takes storage_factory, called as storage_factory(work_dir, log) —
exactly how StorageLocalFiles is constructed, so the default is that class
itself and the command line behaves as it always has.

The motivating case is embedding the annual tool in a long-lived service. There
a per-process work dir means every process re-downloads the same immutable ERA5
and Octopus data and none of it can be shared, which also multiplies requests
against the rate APIs. A shared backend (Redis, S3, a database) fixes both, and
now needs a factory rather than a fork.

Tests cover both paths: the default still constructs StorageLocalFiles, and a
supplied factory reaches AnnualPredictor and receives --work-dir and the run's
log callable. The new assertions were verified to fail without the change
(TypeError: unexpected keyword argument 'storage_factory'). The annual_cli,
annual_cli_machine, annual_cli_machine_end_to_end, annual_job and storage
suites all pass, and black 23.11.0 and ruff 0.11.4 (the pinned pre-commit
versions) report no changes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@springfall2008
springfall2008 merged commit 3ae6825 into main Aug 19, 2026
2 checks passed
@springfall2008
springfall2008 deleted the feat/annual-cli-storage-factory branch August 19, 2026 07:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants