Implement Stage 9 v2 metadata catalog and preview routes

[anth-volk]

Fixes #3807

Summary

• Add the versioned API v2 metadata catalog for the US and UK using PolicyEngine.py 5.2.0 as the sole canonical package-version source.
• Add transactional, serialized, idempotent PostgreSQL bulk publication after the v2 Alembic migration.
• Add typed, read-only Cloud Run resource routes under /v2 for tax-benefit models, model versions, variables, parameters, parameter-tree children, canonical parameter values, logical input datasets, regions, and compact economy-selection options.
• Default every resource read to the PolicyEngine.py version installed in the API artifact while allowing an exact policyengine_version override.
• Preserve all unprefixed metadata routes, API v1 persistence, simulations, reports, user-created records, and production read behavior.

Migration boundaries

• Covered routes and workflows: the metadata route group through 18 dormant /v2 collection, detail, hierarchy, model-selection, and economy-option GET routes; the v2 schema migration and catalog-initialization workflow; model, variable, parameter, logical input dataset, and region reference data.
• Remaining on Flask/API v1: GET /us/metadata, GET /uk/metadata, all existing callers, all creation operations, Cloud SQL reads and writes, simulations, reports, and output datasets.
• Newly prepared: version-scoped SQLModel metadata, the generated v2 Alembic revision, the explicit deployment initializer, Supabase/PostgreSQL query services, Cloud Run resource routing, typed response schemas, and OpenAPI components at GET /v2/openapi.json.
• Intentional API contract changes: the previously dormant combined /v2/{country_id}/metadata preview is replaced before production use by independently paginated resources. No existing unprefixed response or route changes.

Resource read behavior

• Collection routes use required country_id, optional exact policyengine_version, offset defaulting to 0, and limit defaulting to 100 with a maximum of 500.
• Collection results contain items, offset, limit, and has_more; queries read one additional row and do not run a complete-result count.
• Parameter collection and detail responses do not load parameter values. Canonical values are served from /v2/parameter-values, while /v2/parameters/children reads one hierarchy level at a time.
• Every success result identifies the selected PolicyEngine.py version once. Typed error responses cover invalid requests, absent resources or versions, unavailable catalogs, and internal failures.
• The resource routes are GET-only and remain dormant: production callers continue to use the unprefixed v1 metadata routes.

Catalog and deployment behavior

• Expected PolicyEngine Core, US, and UK versions are derived from the installed PolicyEngine.py bundle and checked against installed distributions; only the PolicyEngine.py version is persisted as catalog identity.
• US national metadata uses populace_us_2024; subnational regions use PolicyEngine.py regional alternatives where available and otherwise use the documented national fallback. User-facing dataset labels use “Microcosm.” UK regions use enhanced_frs_2024_25.
• The initializer creates only reusable logical input datasets, never package-derived DatasetVersion rows or simulation/report output datasets.
• Publication uses PostgreSQL COPY, set-based reconciliation, one transaction, and a transaction-scoped advisory lock.
• Initialization runs after alembic upgrade head and before candidate deployment, never during imports, application startup, process restart, or request handling.
• App Engine remains Flask/API v1-only and receives no v2 database URL or credential.

Verification

• OpenSpec implementation and strict validation: 67/67 tasks complete.
• Focused unit, contract, deployment, and migration coverage: 367 tests passed.
• Focused disposable-PostgreSQL resource-route coverage: 9 tests passed, including all 18 routes and repeated read-only requests.
• Branch coverage across Stage 9 catalog, route, and migration modules: 91%; resource-route subset: 93%.
• Added-line coverage for Python changes on this branch: 94.66% (1,276/1,348 executable lines).
• Production-scale publication previously verified 1,172,130 parameter values from PolicyEngine.py 5.2.0, with no __pycache__ parameter nodes.
• Changed Python formatting and lint checks passed.
• Migration-contract generation completed with no generated drift.
• scripts/run_quality_guards.py and strict OpenSpec validation passed.

[codecov]

Codecov Report

❌ Patch coverage is 91.26001% with 120 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.59%. Comparing base (c72cf22) to head (4954ea7).
⚠️ Report is 6 commits behind head on master.

Files with missing lines	Patch %	Lines
policyengine_api/data/v2/catalog/extraction.py	78.87%	27 Missing and 33 partials ⚠️
...olicyengine_api/data/v2/catalog/parameter_query.py	84.61%	3 Missing and 5 partials ⚠️
policyengine_api/data/v2/catalog/region_query.py	81.81%	3 Missing and 5 partials ⚠️
...yengine_api/data/v2/catalog/publication_staging.py	93.33%	1 Missing and 4 partials ⚠️
policyengine_api/data/v2/catalog/records.py	96.26%	3 Missing and 2 partials ⚠️
...icyengine_api/data/v2/catalog/catalog_selection.py	90.90%	3 Missing and 1 partial ⚠️
policyengine_api/data/v2/catalog/model_query.py	87.09%	2 Missing and 2 partials ⚠️
policyengine_api/data/v2/catalog/publication.py	93.65%	2 Missing and 2 partials ⚠️
policyengine_api/data/v2/settings.py	88.23%	4 Missing ⚠️
policyengine_api/data/v2/catalog/initialization.py	90.90%	2 Missing and 1 partial ⚠️
... and 10 more

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3808      +/-   ##
==========================================
+ Coverage   85.49%   86.59%   +1.10%     
==========================================
  Files          95      119      +24     
  Lines        5902     7261    +1359     
  Branches     1032     1283     +251     
==========================================
+ Hits         5046     6288    +1242     
- Misses        559      613      +54     
- Partials      297      360      +63     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.