Skip to content

Fix hours_worked units in participation dynamics - #1828

Merged
vahid-ahmadi merged 1 commit into
mainfrom
fix-participation-hours-units
Aug 28, 2026
Merged

Fix hours_worked units in participation dynamics#1828
vahid-ahmadi merged 1 commit into
mainfrom
fix-participation-hours-units

Conversation

@vahid-ahmadi

Copy link
Copy Markdown
Collaborator

Fixes #1826.

The bug

hours_worked is annual in policyengine-uk — variables/household/income/hours_worked.py labels it "Annual hours worked" with a YEAR definition period, and weekly_hours derives from it as hours_worked / WEEKS_IN_YEAR.

dynamics/participation.py::impute_wages_for_nonworkers divided by it a second time:

hourly_wages = np.where(working_mask, employment_income / (hours_worked * 52), 0)

On enhanced_frs_2024_25, among workers with positive earnings and hours:

value
hours_worked median 1,950 (2,080 = 40 × 52 at p75)
weekly_hours median 37.5
hourly wage, as annual median £15.98, mean £21.14
hourly wage, as coded median £0.31, mean £0.41

£16 is consistent with ONS median hourly earnings; £0.31 is a quarter of the National Living Wage.

Why it matters

Imputed annual earnings for a non-worker entering work at 18.8 hours a week came out at £194 against a realistic ~£21,600. Entering work appears to pay almost nothing, so the extensive margin collapses: in a childcare reform analysis built on this module the response was 21 entrants and £0.007 of exchequer revenue from them; with the units corrected the same code gives 10,031 entrants and £38m.

calculate_gain_to_work calls this helper with impute_nonworker_wages=True, so non-workers' in_work_income was affected too, and imputed_wages is written straight into employment_income via sim.set_input — nothing downstream compensates.

Changes

  • Extract the hourly wage calculation into a new hourly_wage(sim, year) helper with the correct units, so it can be unit-tested without a dataset.
  • apply_participation_responses had the same confusion in the other direction, assigning the weekly hours_for_new_entrants (18.8) into the annual hours_worked, recording a new entrant as working 18.8 hours per year. Now annualised.
  • Four regression tests pinning the units, using a small fake simulation. They fail on main and pass here.

Not fixed here

The donor pool for this imputation is selected by elasticity group, which is keyed off calculate_earnings_quintile — and that function ranks the whole population including children, leaving its bottom two quintiles with no employed donors at all. That is #1827 and is a separate, larger change; this PR is deliberately limited to the units so it can be reviewed and merged on its own.

Testing

policyengine_uk/tests/test_participation_hours_units.py ....                [100%]
4 passed

Existing test_labour_supply_response_formulas.py and test_behavioral_responses.py still pass (6 passed, 11 skipped without a dataset token). ruff format --check clean.

hours_worked is annual in policyengine-uk: its label is "Annual hours
worked", it has a YEAR definition period, and weekly_hours derives from it
as hours_worked / WEEKS_IN_YEAR.

impute_wages_for_nonworkers computed the hourly wage as
employment_income / (hours_worked * 52), dividing by a year a second time.
On enhanced_frs_2024_25 that gives an implied hourly wage of about £0.43
against a realistic £22, and imputes roughly £194 of annual earnings for a
non-worker entering work at 18.8 hours a week rather than about £21,600.
Entering work therefore appears to pay almost nothing, which collapses the
extensive-margin response to near-zero entrants and makes the exchequer
appear to recover nothing from anyone who moves into work.

The hourly wage calculation is extracted into a new hourly_wage helper so
the units can be unit-tested without a dataset.

apply_participation_responses had the same confusion in the other
direction, writing the weekly hours_for_new_entrants straight into the
annual hours_worked variable, recording a new entrant as working 18.8
hours per year. It is now annualised.

Fixes #1826

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011NKBpgpwwY5ZXHUwh1GJQT
@juaristi22

Copy link
Copy Markdown
Collaborator

Adversarial review — policyengine-uk PR 1828

Verdict: APPROVE WITH COMMENT.

The two unit fixes are correct. SHOULD ADDRESS — test_participation_hours_units.py:50-59: the test duplicates the annualization arithmetic but never executes the changed writeback at participation.py:560-569. Add an integration-style regression that forces entry and inspects annual hours_worked.

Validation: 10 relevant tests passed, 11 credential-dependent tests skipped, Ruff passed, and GitHub checks passed.

@vahid-ahmadi

Copy link
Copy Markdown
Collaborator Author

Thanks. Agreed on the gap: the tests pin the arithmetic in hourly_wage but never execute the hours_worked writeback in apply_participation_responses, so that half of the fix is asserted only by inspection.

A direct regression there needs a simulation that will actually produce an entrant — apply_participation_responses requires sim.baseline, a reform that moves the gain to work, and a stochastic draw to land — so it is a microsimulation-marked test needing dataset credentials rather than the fake-based unit tests in this PR. I would rather add it properly than approximate it with a fake that re-implements the writeback and so tests nothing.

Two options and I am happy either way:

  1. Add it here as a @pytest.mark.microsimulation test that runs a small reform, forces entry by setting the participation change directly, and asserts weekly_hours for new entrants is near 18.8 rather than 0.36.
  2. Merge this as the units fix and let Rank earnings quintiles over adults on potential earnings #1829 carry it, since that PR changes the same function's donor pool and will need microsimulation coverage anyway.

Given #1829 is stacked on this branch, my inclination is (1) so this PR stands on its own. Say which you prefer and I will push it.

@juaristi22

Copy link
Copy Markdown
Collaborator

Let #1829 carry it.

@vahid-ahmadi
vahid-ahmadi merged commit 0e8b9a2 into main Aug 28, 2026
8 checks passed
@vahid-ahmadi
vahid-ahmadi deleted the fix-participation-hours-units branch August 28, 2026 11:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

impute_wages_for_nonworkers divides by 52 twice: hours_worked is annual, not weekly

2 participants