Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 0 additions & 16 deletions .github/chronicle-agents.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,20 +19,6 @@ approved_agents:
required_judges:
- ledger-source-fidelity
- ledger-boundary
- id: ledger-target-profile-author
purpose: Add source-backed target profiles and model measurement contracts without target values.
allowed_paths:
- policyengine_chronicle/target_profiles/**
- chronicle/targets/**
- tests/test_policyengine_chronicle_target_profiles.py
- tests/test_us_poverty_target_coverage.py
required_deterministic_checks:
- no embedded target values
- sum-only operation
- selector references raw Chronicle facts
required_judges:
- ledger-target-profile
- ledger-boundary
- id: ledger-contract-maintainer
purpose: Change Chronicle schemas, identity, provenance, or consumer contracts.
allowed_paths:
Expand All @@ -54,8 +40,6 @@ approved_agents:
required_judges:
ledger-source-fidelity:
verdict: PASS if every new fact is directly traceable to publisher bytes/cells and no source values are invented.
ledger-target-profile:
verdict: PASS if profiles contain selectors and measurement contracts only, with no target values or active calibration decisions.
ledger-contract:
verdict: PASS if schema or identity changes preserve source provenance and do not move Microcosm responsibilities into Chronicle.
ledger-boundary:
Expand Down
4 changes: 1 addition & 3 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,15 @@

## Chronicle Governance

For any source package, target profile, consumer contract, schema, or source-data
For any source package, consumer contract, schema, or source-data
boundary change:

- Approved Chronicle agent role:
- `ledger-source-ingestor`
- `ledger-target-profile-author`
- `ledger-contract-maintainer`
- Deterministic checks run:
- LLM judge verdicts:
- `ledger-source-fidelity`:
- `ledger-target-profile`:
- `ledger-contract`:
- `ledger-boundary`:

Expand Down
1 change: 0 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,6 @@ jobs:
import policyengine_chronicle
import policyengine_chronicle.normalization
import policyengine_chronicle.sources
import policyengine_chronicle.target_profiles
import policyengine_chronicle.targets
from policyengine_chronicle.schema import (
CONSUMER_FACT_SCHEMA_SHA256,
Expand Down
13 changes: 7 additions & 6 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# Chronicle Agent Rules

Chronicle is a source-backed fact store. It may parse publisher artifacts, normalize
representation, preserve provenance, and declare target-profile contracts.
representation, and preserve provenance. Selection and measurement contracts live
in consumers such as Microcosm.

Every fact value must trace to a publisher. The boundary is who asserted the
value, not level versus projection: a publisher's own projection (CBO baseline,
Expand All @@ -16,13 +17,13 @@ Do not put Microcosm work in Chronicle:
- no imputation
- no support-aware target activation
- no solver-ready target construction
- no target values in target profiles
- no target profiles or model-measurement bindings
- no PolicyEngine-computed values stored as facts

Resolving profile targets at a period other than a fact's reference period
requires the consumer's explicit `PeriodAlignmentDeclaration`; Chronicle records
the declaration and returns the published level, never the aligned number.
Chronicle records every fact's publisher reference period. Consumers own and
enforce any declaration that aligns those facts to another period; Chronicle
returns the published level and never an aligned number.

Only approved Chronicle agent roles in `.github/chronicle-agents.yml` should add or
modify source packages, target profiles, or contract schemas. Source-data PRs
modify source packages or contract schemas. Source-data PRs
need deterministic validation plus the listed Chronicle judge reviews before merge.
10 changes: 5 additions & 5 deletions LANE_C2_REPORT.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ The cached PDF resolves three material ambiguities in the task shorthand:

- The Annex columns put most requested External series in publisher periods
2021–2023, not 2022–2024. Chronicle preserves the printed periods. A consumer
that needs another period must supply an explicit `PeriodAlignmentDeclaration`;
Chronicle returns the published level and never an aligned number.
that needs another period owns the contract for aligning it; Chronicle
returns the published level and never an aligned number.
- The `poa=2,273k` and `psu=79k` observations are in the SILC column. Their
External cells are `NaN`, so they are stored as `survey_aggregate` and are
validation-only.
Expand Down Expand Up @@ -439,8 +439,8 @@ acceptance, and both required Chronicle judge reviews pass.
`Add Belgium national accounts and JRC external facts`.
- No push and no R2 upload were performed. The two manifests merely declare the
required content-addressed R2 keys.
- A consumer using a fact outside its publisher period must supply an explicit
`PeriodAlignmentDeclaration`; Chronicle returns the published value and does
not compute an aligned value.
- A consumer using a fact outside its publisher period owns the alignment
contract; Chronicle returns the published value and does not compute an
aligned value.

LANE C2 DONE
4 changes: 2 additions & 2 deletions LANE_C5_REPORT.md
Original file line number Diff line number Diff line change
Expand Up @@ -355,7 +355,7 @@ a within-source pair ending in 2025 in this lane.
Any choice of proxy family, ratio calculation, aging, cross-source mapping,
period alignment, target activation, or solver construction belongs to the
consumer. When a target period differs from a fact reference period, the
consumer must provide its explicit `PeriodAlignmentDeclaration`; Chronicle
returns the published levels, never the aligned number.
consumer-owned contract must declare the alignment; Chronicle returns the
published levels, never the aligned number.

LANE C5 DONE
61 changes: 28 additions & 33 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,13 @@ values as structured, queryable facts.

Chronicle may normalize structure: parse files, type values, declare units and
scales, assign geography and period identifiers, preserve lineage back to
source artifacts, and publish target profiles that identify source-backed facts
and measurement contracts. Chronicle does not reconcile inconsistent sources,
impute missing data, store raw survey microdata, or execute simulator-specific
calibration.
source artifacts, and publish source-backed facts. Chronicle does not own
selection or measurement contracts, reconcile inconsistent sources, impute
missing data, store raw survey microdata, or execute simulator-specific calibration.

Microcosm consumes Chronicle facts and target profiles, selects the subset its
current support universe can target, applies minimal period alignment when
declared, and runs calibration. Thesis can consume the same facts and
measurement contracts as official observations.
Microcosm consumes Chronicle facts, owns the contracts that select and bind
them, applies declared period alignment, and runs calibration. Thesis can
consume the same facts as official observations.

## Purpose

Expand All @@ -35,11 +33,8 @@ This repository provides:
their `survey_instrument`.
- **Normalization**: Low-assumption representation changes such as unit/scale
conversion and source-published total/share arithmetic.
- **Target profiles**: Source-backed target contracts and model-measurement
bindings that Microcosm, Thesis, and future rule engines can consume.
- **Consumer artifacts**: Versioned, reproducible bundles of consumer-contract
fact rows plus profiles (`chronicle build-consumer-artifact`) with a resolution
API that enforces the period contract.
fact rows plus manifest hashes (`chronicle build-consumer-artifact`).
- **Jurisdiction loaders**: Source-specific ETL that emits the shared Chronicle
schema.

Expand All @@ -50,27 +45,24 @@ They are source-backed claims with provenance.

The load-bearing rule:

> Chronicle may re-express a published value and declare target contracts, but may
> not reconcile, impute, or transform published values in ways that change their
> meaning.
> Chronicle may re-express a published value, but may not select it for a
> consumer or transform it in ways that change its meaning.

The store is facts-only, and the line is who asserted the value. Everything a
publisher asserted — including the publisher's own projections — is a fact.
Everything PolicyEngine computes (aged, uprated, forecast, or reconciled
levels) is a downstream build artifact and never enters the store; Microcosm
owns aging as a named, versioned model over Chronicle growth-factor facts. A
fact's `period` is the period its value refers to, and resolving a target at
any other period hard-fails without an explicit consumer
`PeriodAlignmentDeclaration` — the guard against silent un-aged calibration
(see [`docs/adr-chronicle-facts-only.md`](docs/adr-chronicle-facts-only.md)).
fact's `period` is the period its value refers to. Consumers must enforce any
contract that aligns it to another period (see
[`docs/adr-chronicle-facts-only.md`](docs/adr-chronicle-facts-only.md)).

| Layer | Owns | Examples |
|-------|------|----------|
| Chronicle Sources | Source artifacts and provenance | URLs, checksums, source files, parsed tables/cells |
| Chronicle Facts | Structured source claims | SOI cells, ACS estimates, CPI values, CBO-published projections |
| Chronicle Normalization | Representation changes | Unit scales, typed values, geography/date identifiers |
| Chronicle Target Profiles | Source-backed calibration contracts | SOI EITC totals, CBO baselines, source-published growth factors, measurement bindings |
| Microcosm Targets | Build-ready active subset | Support-aware activation, solver inputs, diagnostics |
| Microcosm Target Contracts | Selection, measurement bindings, and active subset | Period alignment, support-aware activation, solver inputs, diagnostics |

The storage split is documented in
[`docs/storage-architecture.md`](docs/storage-architecture.md): `ledger-raw`
Expand Down Expand Up @@ -126,8 +118,8 @@ chronicle/
└── docs/ # Architecture and source documentation
```

New code should prefer `policyengine_chronicle` for source-backed fact and target
profile consumers. Existing in-repo implementation code may continue using
New code should prefer `policyengine_chronicle` for source-backed fact
consumers. Existing in-repo implementation code may continue using
legacy implementation modules while the namespace migration is completed.
Solver execution and calibrated dataset construction belong in Microcosm.

Expand Down Expand Up @@ -304,6 +296,11 @@ uv run chronicle build-bundle --suite uk --out /tmp/chronicle-uk --replace
uv run chronicle build-consumer-artifact --facts /tmp/chronicle-uk --out /tmp/chronicle-uk-artifact --replace
```

The command writes a `policyengine_ledger.consumer_artifact.v2` artifact containing
only `consumer_facts.jsonl` and `manifest.json`. Version 2 is incompatible with the
retired v1 profile-bearing contract: loaders reject v1 manifests so downstreams must
adopt the facts-only surface explicitly.

`--year` is inert for `--suite uk` because the UK packages are year-pinned.
The US off-year bundle behavior is unchanged and out of scope here.

Expand Down Expand Up @@ -493,10 +490,8 @@ LEDGER_EXPLORER_DATA_DIRS=/tmp/chronicle-build-a,/tmp/chronicle-build-b npm run

```python
from policyengine_chronicle.targets import DataSource, Target, TargetType, query_targets
from policyengine_chronicle.target_profiles import load_target_profile

target_rows = query_targets(jurisdiction="us", year=2024)
profile = load_target_profile("us_fiscal")
```

## Target Input Schema
Expand All @@ -508,8 +503,8 @@ Target inputs use a three-table schema:
- **stratum_constraints**: Rules defining each stratum.
- **targets**: Source-published aggregate values linked to strata.

These are inputs to Chronicle target profiles. Microcosm owns the active
support-aware subset and calibrated solver execution.
These are source-backed inputs. Microcosm owns the contracts that select them,
the active support-aware subset, and calibrated solver execution.

## Chronicle Facts And Microcosm Targets

Expand All @@ -521,8 +516,8 @@ publishes the total/share relationship.

Inflation, cross-source reconciliation, and support-aware activation belong in
Microcosm unless the source itself publishes the adjusted or projected series.
Target profiles in Chronicle may declare the source-backed rows and measurement
bindings Microcosm is allowed to activate.
Microcosm contracts declare which source-backed rows and measurement bindings a
build may activate.

```python
from policyengine_chronicle.facts import SourceFact
Expand Down Expand Up @@ -558,10 +553,10 @@ normalized_fact = convert_units(fact, 1000, "count")
## Boundaries

- **Chronicle** owns government-statistics release artifacts, provenance, source
facts, aggregate facts, target profiles, and measurement contracts.
- **Microcosm** owns support-aware target activation, minimal period alignment,
raw microdata access, simulation interfaces, entity modeling, weights,
diagnostics, and calibration execution.
facts, and aggregate facts.
- **Microcosm** owns selection and measurement contracts, support-aware target
activation, period alignment, raw microdata access, simulation interfaces,
entity modeling, weights, diagnostics, and calibration execution.
- **Jurisdiction source packages** such as `ledger-us` and `chronicle-uk` own
source-specific parsers and specs that emit shared Chronicle records.
- **Jurisdiction simulation packages** own simulation-specific variable
Expand Down
2 changes: 1 addition & 1 deletion chronicle/__init__.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
"""Chronicle source-data foundation.

Chronicle owns government-statistics releases: source artifacts, source-backed
facts, constraints, provenance, and target profiles. Raw microdata storage,
facts, constraints, and provenance. Selection contracts, raw microdata storage,
source reconciliation, aging, imputation, target activation, and calibration
belong in downstream systems such as Microcosm.
"""
Expand Down
15 changes: 0 additions & 15 deletions chronicle/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -82,21 +82,6 @@
}
ALLOWED_ASSERTIONS = {"observation", "source_projection"}
DEFAULT_ASSERTION = "observation"
# How profile resolution treats the assertion axis. observed_only is the safe
# default: projections are invisible and a projection-only family fails loudly.
# prefer_observed applies per series (one geography/entity/dimension tuple):
# a series with any observed fact resolves only from observations, and a
# series with none may fall back to projections — no single series ever
# mixes bases across periods, and a projection-only series is never starved
# by a neighbouring series' observation. allow_source_projection treats both
# equally (for forecast families such as the OBR EFO lines), with the
# observation winning an exact-period tie within a series.
ASSERTION_POLICIES = {
"observed_only",
"prefer_observed",
"allow_source_projection",
}
DEFAULT_ASSERTION_POLICY = "observed_only"
ALLOWED_PROVENANCE_CLASSES = {
"administrative",
"census",
Expand Down
21 changes: 2 additions & 19 deletions chronicle/harness.py
Original file line number Diff line number Diff line change
Expand Up @@ -754,30 +754,15 @@ def main(argv: list[str] | None = None) -> int:

consumer_artifact_parser = subparsers.add_parser(
"build-consumer-artifact",
help=(
"Build a versioned consumer artifact from consumer-contract facts "
"and Chronicle target profiles"
),
help="Build a versioned facts-only consumer artifact",
description="Build a versioned facts-only consumer artifact.",
)
consumer_artifact_parser.add_argument(
"--facts",
type=Path,
required=True,
help="Path to a consumer_facts.jsonl file or a bundle directory",
)
consumer_artifact_parser.add_argument(
"--profile",
action="append",
default=[],
help="Packaged target profile id (may be repeated)",
)
consumer_artifact_parser.add_argument(
"--profile-path",
action="append",
type=Path,
default=[],
help="Path to a target profile JSON file (may be repeated)",
)
consumer_artifact_parser.add_argument(
"--out",
type=Path,
Expand Down Expand Up @@ -1285,8 +1270,6 @@ def main(argv: list[str] | None = None) -> int:
artifact_report = build_consumer_artifact(
args.out,
facts_path=args.facts,
profile_ids=args.profile,
profile_paths=args.profile_path,
replace=args.replace,
)
print(json.dumps(artifact_report.to_dict(), indent=2, sort_keys=True))
Expand Down
29 changes: 15 additions & 14 deletions docs/adr-chronicle-facts-only.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,22 +20,19 @@ the value**, not level versus projection:
aging), implemented as named, versioned models that consume growth-factor
facts from Chronicle and emit their own lineage.

Instead of projection objects, Chronicle contributes three guarantees:
Instead of projection objects, Chronicle contributes two guarantees:

- **Reference-period semantics.** `PeriodDimension` identifies the period a
value refers to; `PeriodCoverage` records non-identity provenance (start
and end dates, basis, the publisher's period label, accounting basis) for
cases like BE-SILC incomes that reference the year before the survey
label.
- **Period-contract enforcement.** Resolving a profile target at a period
other than the fact's reference period raises `PeriodContractError`
unless the consumer passes an explicit `PeriodAlignmentDeclaration`
(model id, version, parameters — never values). Chronicle records the
declaration in resolved rows and returns the published level untouched.
- **Basis-aware diagnostics.** Resolved rows carry `basis` (`fact` or
`declared_alignment`), `fact_period`, `requested_period`, and the
declaration, so downstream diagnostics distinguish "missed a published
fact" from "missed an aged level."
- **Facts-only consumer artifacts.** Chronicle publishes schema-validated fact
rows with manifest hashes. Consumers own the selection, measurement,
period-alignment, and model-binding contracts that interpret those rows. The
facts-only artifact is `policyengine_ledger.consumer_artifact.v2`; the version
bump makes the removal of v1's embedded profiles and resolution surface an
explicit incompatible transition.

## Why not facts plus projections in one schema

Expand Down Expand Up @@ -67,9 +64,13 @@ Instead of projection objects, Chronicle contributes three guarantees:
values other than `observation` and `source_projection` fail validation
with an error explaining that PolicyEngine-computed values are not facts.
- Consumer-contract rows always carry `assertion` explicitly, and the
consumer artifact (`chronicle build-consumer-artifact`) embeds profiles,
fact rows, coverage diagnostics, and manifest hashes so Microcosm can
build a target registry without database access or copied values
(issue #61).
consumer artifact (`chronicle build-consumer-artifact`) contains only fact
rows and the manifest hashes needed to verify them. Microcosm packages its
own selection contracts and builds its target registry without Chronicle
profiles (issues #166 and #172).
- The retired `policyengine_ledger.target_profile.v1` and
`policyengine_ledger.resolved_target.v1` schema IDs have no v2 successor in
issue #143. The Chronicle-side Belgian profile plan in issue #70 is
superseded; Belgian contracts also live consumer-side.
- Geography vintage translation (microcosm#205) follows the same pattern: a
declared consumer-side transform over facts, never an edit to them.
Loading
Loading