Skip to content

Add public RT/load contracts, complete iteration, and wheel typing - #3

Open
kvkenyon wants to merge 1 commit into
feat/public-retrievalfrom
feat/public-rt-load
Open

Add public RT/load contracts, complete iteration, and wheel typing#3
kvkenyon wants to merge 1 commit into
feat/public-retrievalfrom
feat/public-rt-load

Conversation

@kvkenyon

@kvkenyon kvkenyon commented Sep 6, 2026

Copy link
Copy Markdown
Owner

TinyERCOT's opt-in public client was limited to two DAM endpoints and sampling. This PR adds a reusable verified-contract registry, RT settlement prices and weather-zone actual load, complete caller-selected API/annual-file iteration, and installed-wheel PEP 561 proof. Legacy imports, signatures, defaults, exceptions, DataFrames, and runtime files remain unchanged.

Stacked on #2 (feat/public-retrieval, 01d8b69c4378040ca14d9683e9693e65cc3f93a8), above #1. Do not merge these PRs.

Head: 12d60654632ce5c47543ef5f3410ec55972c8471. GitHub CI passed on both Python 3.11 and 3.14.

Exact coverage

Generated typed coverage is 4 operations in 4 products out of 243 observed public data paths across 98 namespaces. This PR adds NP6-905-CD RT settlement prices and NP6-345-CD weather-zone load to NP4-190-CD DAM settlement and NP4-188-CD DAM capacity prices. It does not claim broad family coverage.

  • Offline discovery verifies primary OpenAPI hashes and small response/receipt pairs. Generation rejects missing/unknown schemas, unsupported types/formats, collisions, and pin drift. TypedDict filters and endpoint identities share one retrieval implementation.
  • ReportsClient streams all pages/rows for an explicit selection when callers remove total budgets. Per-response bytes, retry attempts and pacing remain bounded. Changing totals, preceding-page repetition, count mismatch, and budget exhaustion raise explicit opt-in errors.
  • Public MIS report 13060 retains listing/download/cache and sampling. iter_dam_archive adds every selected worksheet/row with source member, sheet, row and hash. Synthetic tests exhaust 2,010 rows across two sheets; no complete real annual extraction occurred.
  • PEP 561 wheel metadata and companion legacy stubs pass isolated positive assertions for all 510 legacy methods and 1,334 row fields, public row/filter/iterator contracts, and four rejected invalid uses. CI uses an installed wheel outside the checkout.

Evidence and boundaries

Evidence class Exact scope
New typed installed current Two RT HB_HOUSTON rows and two weather-zone load rows for 2026-09-04.
New typed installed historical One oldest-first row per new endpoint, both dated 2023-12-11; no retention/completeness guarantee.
Complete bounded installed query Two RT rows over two one-row pages for one hub/date/hour and two intervals; source totals match.
Prior typed installed evidence Both DAM endpoints current/oldest-first; four cached annual rows each from 2010/Dec_1 and 2026/Aug; rolling website ESR snapshot.
Real source retrieval only DAM product metadata; no generic product-metadata API claim.
Fixture-only Long API and annual iteration, retry/auth failure cases, malformed schemas/files, DST and publication/cache boundaries.
Deferred Other 239 data paths, API archive/bundle routes, other MIS/live formats, current async clients, schema epochs and as-of/revision reconstruction.
Restricted Secure, Certified, EWS, private telemetry/bids/COP/awards, participant settlements and customer data.
Unavailable Audited retired AS/SASM sources and hourly-load 2001.

All 39 audited source families remain enumerated in docs/public-retrieval.md and the bundled coverage inventory. Four API operations and two narrow web subsets are covered; no whole broad family is marked complete. Unknown schemas remain unsupported, with no guessed model or generic raw bypass.

Authorized Public API authentication occurred, following persisted inbox 001's explicit expansion of the initial unauthenticated audit scope. Six bounded public data requests returned eight total rows in this iteration. No restricted request, credential/token disclosure or copying, paid access, historical bulk extraction, forecaster work, no-mistakes, or merge occurred. No annual file was downloaded by this iteration's new probe. Normal tests are offline and never load .env.

Primary sources: ERCOT Public Reports specification, Public API authentication, annual DAM product, website ESR feed. Exact new public receipt hashes, times and counts are in docs/evidence/installed-reports-receipts.json; fixture transformations and prior receipts remain alongside it.

Validation

uv sync --frozen --group dev --extra files
uv run --extra files pytest -q
  976 passed, 1 skipped on Python 3.14.0
UV_PROJECT_ENVIRONMENT=/tmp/tinyercot-py311 uv sync --python 3.11 --frozen --group dev --extra files
/tmp/tinyercot-py311/bin/python -m pytest -q
  976 passed, 1 skipped on Python 3.11.14
uv run python tools/generate_client.py --check
uv run python tools/generate_public.py --check
  Both pinned generators passed
uv run ruff check tools tinyercot/catalog.py tinyercot/public tests
uv run ruff format --check tools tinyercot/catalog.py tinyercot/public tests
  Passed; 36 files formatted
uv build
  Wheel and sdist built
uv export --frozen --no-emit-project --output-file /tmp/tinyercot-iteration-requirements.txt
uv venv --python 3.11 /tmp/tinyercot-locked-wheel
uv pip sync --python /tmp/tinyercot-locked-wheel/bin/python /tmp/tinyercot-iteration-requirements.txt
uv pip install --python /tmp/tinyercot-locked-wheel/bin/python --no-deps dist/tinyercot-0.2.2-py3-none-any.whl
/tmp/tinyercot-locked-wheel/bin/python -I tests/wheel_smoke.py
uv run python tools/check_typing.py --python /tmp/tinyercot-locked-wheel/bin/python
  Installed import/DataFrame/mocked RT and complete static-contract proof passed
/tmp/tinyercot-retrieval-wheel/bin/python -I tools/probe_public.py --live --reports-only --credentials-file /Users/kevin/.treehouse/tinyercot-1bad13/1/tinyercot/.env --output /Users/kevin/.treehouse/tinyercot-1bad13/1/tinyercot/.local-evidence/installed-reports
  Six bounded public data requests passed
git diff d1daad25df42d3fff41f88b907d99ef325b970e0 -- tinyercot/__init__.py tinyercot/_client.py tinyercot/_generated.py
  Empty; frozen runtime preserved
git diff --check
  Passed; status, full change scope and remote PR base inspected

Additional isolated wheel smoke and PEP 561 checks passed with Python 3.11/NumPy 1.24.0/pandas 2.0.0/Pydantic 2.10.0 and Python 3.14/NumPy 2.5.2/pandas 3.0.5/Pydantic 2.13.5. The yanked NumPy 2.4.0 pin predates this task; only that existing runtime lock version changed, to 2.4.1. Runtime dependency constraints remain unchanged. The six new locked packages are development typing tools/dependencies. Exact install commands and documentation review notes: docs/current-iteration-task-note.md.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant