Skip to content

Add 52 public API operations, archive retrieval, and four live adapters - #4

Open
kvkenyon wants to merge 2 commits into
feat/public-rt-loadfrom
feat/public-metadata-expansion
Open

Add 52 public API operations, archive retrieval, and four live adapters#4
kvkenyon wants to merge 2 commits into
feat/public-rt-loadfrom
feat/public-metadata-expansion

Conversation

@kvkenyon

@kvkenyon kvkenyon commented Sep 6, 2026

Copy link
Copy Markdown
Owner

This adds 52 source-backed Public Reports operations through reusable product/artifact discovery and generation, bringing typed installed-wheel retrieval to 56 unique paths of 243. It also adds four public rolling dashboard adapters, RT annual-file iteration, public archive/bundle metadata, and a seven-column Resource DME CSV decoder. Existing legacy imports, signatures, defaults, models, exceptions and DataFrame behavior are unchanged; the three legacy runtime files are byte-identical to audit base d1daad25df42d3fff41f88b907d99ef325b970e0.

Stacked on PR #3 (feat/public-rt-load, 12d60654632ce5c47543ef5f3410ec55972c8471). Do not merge as part of this task.

Exact coverage and evidence

  • docs/evidence/installed-registry-index.json: 56 generated/installed unique paths; 57 current requests across 56 paths and 56 oldest-first requests across 56 paths. The new 52-path batch contributes 104 one-row requests. Every entry contains public receipt hashes/URLs/times and installed-module evidence.
  • Pagination remains separate: two paths have live multiple-page evidence, and one bounded RT selection completed two pages/two rows. New paths have no live pagination or full-history completeness claim. Offline tests cover complete iteration and safeguards.
  • Installed anonymous snapshots cover fuel mix, grid conditions/PRC, generation outages, and DC-tie flows. RT annual listing plus cached 2010/2026 source rows and current/oldest DME documents decode through the installed wheel.
  • Public root metadata lists 116 products and 242 artifacts. Archive metadata verifies DME documents and available retired total-AS-offer/SASM history. First-page retired listings are 1,000/4,236 and 1,000/2,602 documents. A 103-entry DAM monthly bundle listing passed; one bounded oldest bundle transfer failed and no successful transfer is claimed.
  • The remaining 187 observed data paths and other public document/history families remain active subsequent work, not authorized deferral. BINARY content cannot generate row models; public document retrieval remains in scope. Retired publication and available history are separate. Unknown/missing/ambiguous row schemas stay unsupported.
  • Secure, Certified, participant EWS, private telemetry/bids/COP/awards, participant settlements, and customer data remain restricted. Hourly-load 2001 remains unavailable. Published delayed disclosures explicitly marked Public are a separate public category.

Captain-authorized Public API authentication occurred. No restricted request, paid access, secret copying, forecaster computation, historical bulk extraction, no-mistakes run, or merge occurred. Offline import/generation/tests need no credentials. The ignored mode-0600 .env is not committed.

Primary sources: ERCOT Public Reports OpenAPI, public product root, and the per-source ERCOT URLs/receipts checked into docs/evidence/. Generation reads pinned local projections only and never refreshes the legacy generated file from a moving URL.

Validation

Passed in the working tree:

uv run python tools/generate_client.py --check
uv run python tools/generate_public.py --check
uv run --extra files pytest -q
uv build
uv pip install --python /tmp/tinyercot-typing-proof.fbtIin/venv/bin/python --no-deps --reinstall dist/tinyercot-0.2.2-py3-none-any.whl
uv run python tools/check_typing.py --python /tmp/tinyercot-typing-proof.fbtIin/venv/bin/python
git diff --cached --check
git diff d1daad25df42d3fff41f88b907d99ef325b970e0 -- tinyercot/__init__.py tinyercot/_client.py tinyercot/_generated.py
git check-ignore .env

Full suite: 1259 passed, 1 skipped. Real installed-wheel PEP 561 proof: 510 legacy methods, 1,334 legacy row fields, 56 current endpoint/product facades, 498 current row fields, 847 filter fields, live/archive/metadata APIs, and 64 invalid uses rejected. Legacy diff empty; credential file ignored.

Concurrent follow-up work stays uncommitted. Final PR-specific validation used the staged snapshot below, after installing openpyxl==3.1.5 in both separate test environments:

git checkout-index --all --prefix=/tmp/tinyercot-pr4-check.fjDuNo/
cd /tmp/tinyercot-pr4-check.fjDuNo
/tmp/tinyercot-locked-wheel/bin/python -m pytest -q
/tmp/tinyercot-typing-proof.fbtIin/venv/bin/python -m pytest -q
/tmp/tinyercot-typing-proof.fbtIin/venv/bin/python -m ruff check tools tinyercot/catalog.py tinyercot/public tests
/tmp/tinyercot-typing-proof.fbtIin/venv/bin/python -m ruff format --check tools tinyercot/catalog.py tinyercot/public tests
/tmp/tinyercot-typing-proof.fbtIin/venv/bin/python tools/generate_client.py --check
/tmp/tinyercot-typing-proof.fbtIin/venv/bin/python tools/generate_public.py --check

Python 3.11.14 and 3.14.0: 1259 passed, 1 skipped each. Lint passed; 62 files formatted; both generators reproduced pinned output. Source/fixture inventories were checked independently. Public Python documentation was reviewed for Google-style argument, result, lifecycle and failure contracts.

docs/public-metadata-task-note.md records exact installed probe commands, source limits, and next active batches. docs/api-archive-installed-task-note.md records the bounded bundle failure. No live requests were repeated merely to assemble the receipt index. Next stacked work expands the already captured renewable/load/RUC batch and continuing public disclosure, archive, and live coverage.

CI follow-up: the first run caught stale fixed counts in standalone tests/wheel_smoke.py after generation/tests/lint/build passed. It now reconciles installed coverage with the installed registry. uv run ruff check tests/wheel_smoke.py, /tmp/tinyercot-typing-proof.fbtIin/venv/bin/python -I tests/wheel_smoke.py, and /tmp/tinyercot-locked-wheel/bin/python -I tests/wheel_smoke.py passed. Only validation/docs changed in this follow-up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant