Pro-rate the Tax-Free Childcare top-up, not only its cap - #1830
Conversation
Adversarial review — policyengine-uk PR 1830Verdict: REQUEST CHANGES.
Validation: 15 targeted YAML tests passed; Ruff passed; GitHub checks passed. The tests validate the implementation's assumptions, not the source mapping. |
|
Both points are well made. I have filed the first as #1832 rather than folding it in, and I think the second is right in a way that argues for keeping this PR narrow. 1. The 25%-of-gross question — real, filed as #1832You are right that there is a contract problem, and it is verifiable inside the model without settling the FRS question: I have not folded the fix into this PR for two reasons. It is orthogonal to what this PR fixes. The pro-rating bug is a multiplicative error on the eligible share; the gross/net question is a multiplicative error on the rate. Whether the top-up is 0.25 or 0.20 of the input, a family eligible for half the year should receive half of it. Neither fix depends on the other, and the tests here pin the eligible-share behaviour independently of the rate. I cannot settle the contract on the evidence I have. FRS variable-level documentation is behind UK Data Service registration; I could not confirm whether respondents are told to include money paid from a TFC account. You state they are — if you have the reference, adding it to #1832 would let someone change the formula with a citation rather than an inference. I did not want to change a rate that affects every TFC result on my inference alone. 2. Eligibility is not account activity — agreed, and this PR does not claim otherwiseThis is the sharper of the two points and it corrects something I did imply. This PR only fixes the pro-rating of the eligible fraction that the formula already computes; it does not introduce a duration assumption, and it changes no built dataset today because Where I overreached was the PR description, which used HMRC's monthly-to-annual ratio to motivate this fix as though eligibility duration could stand in for months containing provider payments. It cannot, for exactly the reason you give: a family can be eligible all year and pay through the account for part of it. The reviewer on PolicyEngine/policyengine-uk-data#472 made the same point and derived a cleaner figure from HMRC Table 2 — So the intensive margin belongs in a separate One caution for whoever does: a dataset built from I have left this PR as the mechanism only. Happy to close it instead if you would rather the pro-rating fix arrive alongside the resolved contract in #1832. |
juaristi22
left a comment
There was a problem hiding this comment.
Adversarial re-review — request changes
The code head is unchanged, and the response confirms rather than resolves the blocking input-contract problem now tracked in #1832.
BLOCKING — the tests still entrench a 25%-of-gross contribution (tax_free_childcare.py:52-54).
The formula computes childcare_expenses × 0.20 / 0.80, or 25% of childcare_expenses. That is correct only if the input is the parent's £8 net deposit. The repository/FRS use of childcare_expenses is gross provider spending, for which the government contribution is £2 per £10, or 20%. See the FRS question instructions and GOV.UK scheme explanation.
This is not orthogonal to the PR: the PR moves this exact expression and adds three below-cap expectations that lock in the disputed interpretation. If the input remains gross spend, the new £250/£250/£500 expectations should be £200/£200/£400. Otherwise, make the net-deposit input contract explicit and reconcile it with Universal Credit before relying on these tests.
I agree with keeping account activity separate from eligibility. However, the 59.3% average-monthly-used / annual-unique account statistic should not be applied directly as a spending multiplier. On the reported 1.12×-target baseline it would produce 1.12 × 0.593 ≈ 0.664× target. It supports adding an explicit account-payment/activity state, but that state should be calibrated to expenditure rather than set mechanically to 0.593.
The 25%-of-gross treatment remains a plausible reason TFC is high: it overstates uncapped awards by 25%, which can readily be attenuated to the reported 12% aggregate excess by caps and other offsets.
Validation: 15 targeted YAML cases passed; Ruff and formatting passed; GitHub shows eight passing checks. Those tests confirm the implementation matches its expectations, but the disputed input contract makes the expectations themselves the problem.
|
@juaristi22 — re-review request, because your second point is now implemented rather than argued about. On "eligibility is being used as a proxy for account activity". You were right, and #1833 does what you asked: it adds That PR is stacked on this one, so this PR supplies the pro-rating machinery and #1833 supplies the empirical fraction. Reviewing them together may be easier than this one alone, since in isolation this changes no built dataset at all ( On the 25%-of-gross question, I have filed it as #1832 rather than resolving it here, for two reasons. It is a multiplicative error on the rate, while this is a multiplicative error on the eligible share — neither depends on the other, and this fix is correct under either reading of the input. And I could not settle the input contract on the evidence available to me: FRS variable documentation is behind UK Data Service registration, so I could not confirm whether respondents are told to include money paid from a TFC account. If you have that reference, adding it to #1832 would let someone change the rate with a citation rather than an inference. What I could verify, and put in #1832, is that the contract is genuinely ambiguous inside the model: Happy to close this and fold it into #1833 if you would rather review one change than two. |
juaristi22
left a comment
There was a problem hiding this comment.
Adversarial review — PolicyEngine UK PR #1830
Recommendation: REQUEST_CHANGES
- PR: #1830 — Pro-rate the Tax-Free Childcare top-up, not only its cap
- Reviewed head:
0fd5b1a3b132f3b6faeb444df798991b10e0e43b - Base:
mainat19e67fe6a9797c18f46b8eae23f65cc827f2d93b - Mode: full fresh review of PR changes, read-only
- Diff: 1 commit, 3 files, +62/−4
- CI at review time: 8/8 GitHub checks successful
- Prior-review status: the head is unchanged from the existing changes-requested review; recent work is in stacked PRs #1834 and policyengine-uk-data #473, not this PR.
Critical issues
C1 — the changed contribution expression and new tests apply 25% to a gross-cost input
At tax_free_childcare.py:52-54, the PR computes:
eligible_childcare_expense * eligible_fraction * p.rate / (1 - p.rate)The 2025 parameter is p.rate = 0.20, documented as the government share of the combined household-and-government payment. Dividing by 0.80 therefore returns 25% of childcare_expenses.
That 25% is the right multiplier for a parent deposit, not for total childcare cost. The repository variable says childcare_expenses is “Total amount spent on childcare,” and the UK-data pipeline maps the FRS CHAmt childcare-cost field to it. FRS separately defines TFCCOutAmt, the payment out of a TFC account to a provider, as including the government contribution (FRS 2023–24 Question Instructions, file p.124), corroborating gross-outflow semantics. The Childcare Payments Act 2014 s.1(4), file p.7 says the top-up is 25% of the qualifying deposit; s.21, file p.18 converts that to R/(100+R), i.e. 20% of an outgoing gross account payment. HMRC TFC35300 gives the same example: £2,000 paid in + £500 top-up = £2,500 total.
The new below-cap expectations therefore encode the wrong contract:
| New case | PR expectation | Gross-cost contract |
|---|---|---|
| £2,000, two eligible periods | £250 | £200 |
| £4,000, one eligible period | £250 | £200 |
| £2,000, four eligible periods | £500 | £400 |
The denominator pre-existed the PR, but it remains in the changed expression and the PR adds regression tests that deliberately lock in its wrong outputs. This is therefore in scope. Fix by applying p.rate directly to gross routed cost, or consistently redefine the input and all of its consumers as parent net deposits.
Why this likely explains the remaining Tax-Free Childcare target miss
The companion data PR #473 reports £0.790bn after its activity adjustment against a £0.630bn target, or 1.25×. Removing this exact gross-up error gives:
£0.790bn / 1.25 = £0.632bn
That is essentially the £0.630bn target. This is much stronger evidence for the residual than another take-up correction: the residual multiplier exactly matches the statutory gross/net error.
PR #1830 itself is not the reason the current aggregate is off. In built data, tax_free_childcare_eligible_declaration_periods currently returns eligible * 4, so eligible_fraction is only 0 or 1. The PR's new partial-year path is latent unless callers override the count or upstream data later becomes non-binary.
C2 — the changelog fragment is not recognized
changelog.d/tfc-part-year-prorating.md does not follow the repository's <name>.<type>.md convention. Running uv run --frozen --extra dev towncrier build --draft returns No significant changes. Rename it with a configured type such as tfc-part-year-prorating.fixed.md.
Should address
A1 — prorating annual spending by eligible quarters is an unsupported exact allocation
The new comment says that because childcare_expenses is annual, “only this share of it is spent while eligible.” That is an approximation, not a rule established by the scheme.
The Act applies the top-up to actual qualifying deposits within three-month entitlement periods (ss.18–19, file pp.16–17). Annual provider expenditure plus the number of eligible periods does not identify those deposits. A family eligible for two periods could concentrate spending inside or outside those periods. Moreover, account balances can be spent after eligibility for new top-ups ends (Explanatory Notes para.102, file p.17).
If the model must use annual inputs, label eligible_periods / 4 as a uniform-timing assumption and validate it empirically. Prefer an explicit routed/qualifying spend measure when available.
This matters for the stack: PR #1834 multiplies by a routed-spend share, and data PR #473 derives 0.593 from average monthly account activity. Both that share and eligible_fraction contain duration-like exposure. Before multiplying them, define the routed share as conditional on eligible time, or make the scalars mutually exclusive; otherwise partial-year records can be duration-adjusted twice.
A2 — the changed policy logic has no source reference
The variable has no reference metadata, and the new formula/comment cites no authoritative source for either entitlement-period mechanics or the uniform-spend assumption. Add the Act/HMRC references and distinguish law from modeling approximation.
Suggestion
- Add an integration test through
tax_free_childcare_eligible_declaration_periodsrather than overriding it. That will document that the current production path yields only 0 or 4 and prevent reviewers from assuming this PR already changes aggregate spending.
Stacked-PR and calibration assessment
- #1834 is stacked directly on #1830 and retains
p.rate / (1 - p.rate). Its new routed-share scalar does not resolve C1. - policyengine-uk-data #473 uses
sum(monthly used children) / annual unique children = 0.593. “Used” is a binary monthly account-activity count, not a value-weighted share of annual spending. It is defensible as an exposure proxy, but it should not be described as direct measurement of routed spend. - policyengine-uk-data #472 reports materially different target ratios across builds: 1.87× for the published release and 1.12× for a 32-epoch smoke build. Calibration conclusions should be tied to the exact build artifact and training budget; the smoke result is not a substitute for release validation.
- The cleanest decomposition is: correct the legal/input rate contract first; then estimate the share of eligible gross childcare cost routed through TFC accounts; then calibrate weights. Avoid using a take-up/activity scalar to compensate for the 1.25 rate error.
Validation performed
| Validation | Result |
|---|---|
| Focused formula YAML | 15 passed |
| Full HMRC TFC tests | 373 passed |
| Ruff lint | passed |
| Ruff format check | passed |
| Whitespace check | passed |
| Towncrier draft | no fragment recognized |
| GitHub checks | 8/8 successful |
| Official PDFs | 100/100 pages rendered and visually reviewed at 300 DPI |
| Disputed pages | 2 pages re-rendered and visually confirmed at 600 DPI |
Passing tests do not resolve C1 because the expected values encode the same gross/net mistake.
PDF/source audit tally
- Confirmed mismatches: 1
- Rejected mismatches: 0
- Inconclusive mismatches: 0
- Pages visually reviewed: 102 render passes (100 at 300 DPI + 2 at 600 DPI)
- Primary conclusion: 25% applies to parent deposits; 20% applies to gross provider payments/account outflows.
Required changes before approval
- Correct the gross/net rate contract and the new below-cap expected values.
- Rename the Towncrier fragment so it is included.
- Document and validate the annual-spend timing assumption, including how it composes with #1834/#473.
- Add official source references to the changed policy logic.
No GitHub review was posted as part of this run.
0fd5b1a to
1e58b18
Compare
|
Merge-order note: this and #1834 are each mergeable against Suggested order is #1830 then #1834: #1830 is the statutory correctness fix and changes results on its own, #1834 adds an input that defaults to neutral and changes nothing until the companion data PR lands. I have verified the combined result locally. On
|
juaristi22
left a comment
There was a problem hiding this comment.
Program Review
Source Documents
- Law: Childcare Payments Act 2014, sections 1 and 21
- HMRC manual: TFC35300 worked contribution example
- Scope: Full merge-base review because the prior head is not an ancestor of the force-pushed head
- Reviewed head SHA:
1e58b18d6ad4894d2d9544f7376920a15a03eb31 - Mode: full
Critical (Must Fix)
None.
Should Address
- [A1 STILL OPEN] Annual spending is assumed uniform across eligible declaration periods. Multiplying annual
childcare_expensesbyeligible_periods / 4is a documented approximation, but statute operates on actual deposits and top-ups during each entitlement period. Quantify the bias or add an integration test against the data-side activity proxy before relying on both scalars together. —policyengine_uk/variables/gov/hmrc/tax_free_childcare/tax_free_childcare.py:43
Resolved Prior Findings
- [C1 RESOLVED] The contribution is now 20% of gross provider spend, matching the input contract and sections 1/21; below-cap tests now expect £200/£200/£400.
- [C2 RESOLVED] The
.fixed.mdfragment is consumed by Towncrier. - [A2 RESOLVED] The code and tests cite the Act and HMRC example and label the duration assumption.
Validation Summary
- Focused TFC tests: 15 passed.
- Ruff: passed on the changed formula.
- Towncrier: renders the fragment under Fixed.
- CI: 8/8 checks passed on the exact head.
- Integration note: exact heads #1830 and #1834 currently have content conflicts in both the formula and YAML tests.
Review Severity: COMMENT
Next Steps
Resolve the #1834 stack explicitly and test the combined eligibility-fraction and routed-share behavior.
|
A second adversarial pass found five things. Three are fixed; two are limitations I am recording rather than papering over. The pro-rating half of this PR is unreachable on any built dataset. The fix is still right and harmless, but the real defect is upstream: that variable is binary by construction and does not model partial-year eligibility at all. The changelog now says so instead of implying a behaviour change. "Overstated every award by a quarter" was wrong. Measured on the Enhanced FRS at 2024, the rate change moves spending £1.1781bn → £1.0075bn, a 14.5% fall rather than 20%, because the per-child cap binds for many recipients and capped awards do not move. Changelog corrected. One pre-existing test stopped testing the cap. "Eligible standard child - at cap" used £10,000, and 10,000 × 0.2 = 2,000 lands exactly on the cap, so it passed whether or not the cap was applied. Raised to £12,000 so it binds again. 78 tests still pass. The load-bearing assumption, stated plainly. This change is only correct if I could not settle this from this repository: the semantics come from the FRS mapping in And the aggregate cannot arbitrate it. The companion calibration reaching about 1.02× of target with a routed share of 0.593 is not evidence for 20% specifically — the same target is reachable under the old 25% reading with a routed share of about 0.474. The case for 0.2 rests on the statutory reading and on the input being gross, not on the calibration fit. I have made that explicit here so the number is not asked to carry weight it cannot. |
1e58b18 to
a345ecf
Compare
|
Correction to my earlier merge-order note: the conflict between this and #1834 is semantic, not textual. Resolving the overlapping hunks is not enough. #1834's routing tests encode the pre-#1830 rate — gross × 0.25 — including an explicit So whichever merges second has to re-derive those seven expectations, not just resolve the diff. Verified by merging both in a scratch copy; the merged aggregate is £0.6444bn at a routed share of 0.593, which is 1.02× the £0.632bn target. |
…gibility Two corrections to the same expression. The rate. gov.hmrc.tax_free_childcare.contribution.rate is documented in its own file as "the government contribution to childcare expenses (as a percentage of total household and government contributions)", and childcare_expenses is documented as "Total amount spent on childcare". The formula divided by (1 - rate), which converts a gross-side 20% into the deposit-side 25% — the right multiplier for a payment *into* an account, not for total spend. Every award was a quarter too large. Childcare Payments Act 2014 section 1(4) sets the top-up at 25% of the qualifying payment into the account; section 21 converts that to R/(100+R) of the topped-up total, i.e. 20% of the gross payment out. HMRC TFC35300 gives the worked example: £2,000 paid in, £500 top-up, £2,500 total. On the repository's input contract the base is the £2,500, so the rate applies directly. The eligible share. It was applied to the cap but not to the spend, so part-year eligibility bit only where the cap bound. A family eligible for two of four declaration periods with £2,000 of annual childcare spend received the full-year amount, because its pro-rated cap never binds on a contribution that small. Both existing part-year tests happened to be cap-bound, which is why this went unnoticed; they pass unchanged. Tests restated at the corrected values with the arithmetic shown, and four below-cap part-year cases added. Fixes #1832 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011NKBpgpwwY5ZXHUwh1GJQT
a345ecf to
25a588f
Compare
juaristi22
left a comment
There was a problem hiding this comment.
Program Review
Source Documents
- Law: Childcare Payments Act 2014, especially sections 1, 20 and 21.
- HMRC manual: TFC35300 worked contribution example.
- Reviewed head SHA:
25a588f70ce46b815c3ff1c2f07ab0b9bdf3f1a7 - Mode: full
- Scope: Full merge-base review. The previously reviewed head was not an ancestor of this force-pushed head.
Critical (Must Fix)
None.
Should Address
- [A1 STILL OPEN] The formula assumes annual childcare spending is uniform across eligible declaration periods. The law and HMRC example operate on actual qualifying payments/top-ups, while
childcare_expenses * eligible_periods / 4estimates the eligible-period amount without timing data. Section 20(5) also allows qualifying provider payments from an account after a valid declaration has lapsed, so provider-payment timing and top-up timing need not coincide. The expanded changelog appropriately discloses that this has no effect on the current built dataset because the declaration-period input is binary; keep the approximation explicit in variable documentation and add a combined data-side integration/sensitivity test before fractional periods are populated. —policyengine_uk/variables/gov/hmrc/tax_free_childcare/tax_free_childcare.py:33
Resolved Prior Findings
- [C1 RESOLVED] The formula applies the 20% gross-side rate directly, matching sections 1/21 and the
childcare_expensesinput contract. - [A2 RESOLVED] The standard-child cap test now uses £12,000 of spend and genuinely exercises the £2,000 cap.
- [C2 RESOLVED] The
.fixed.mdfragment is consumed by Towncrier and quantifies the aggregate change. - [C3 RESOLVED] #1834 is now stacked directly on this exact head; GitHub reports both PRs mergeable.
- [A3 RESOLVED] The latest commit adds a latest-data invariant that no award exceeds 20% of recorded childcare expenses.
Validation Summary
- Focused Tax-Free Childcare tests:
15 passed. - Latest-data statutory-rate test: present but skipped locally because no microsimulation dataset was configured.
- Towncrier: fragment renders under Fixed.
- Exact-head GitHub CI at final check: 7 passed, Test in progress; GitHub reports the PR mergeable.
- Cross-PR:
25a588f...is the exact base of #1834, eliminating the earlier formula/test conflict.
Review Severity: COMMENT
Next Steps
Mergeable as written, with the annual-spend timing approximation documented and monitored when fractional declaration-period data is introduced.
The bug
tax_free_childcarecomputes the top-up from the full annualchildcare_expensesand applieseligible_fractiononly to the cap:So part-year eligibility bites only when the cap binds. A family eligible for two of four declaration periods with £2,000 of annual childcare spend:
childcare_expensesis annual, so a family eligible for part of the year puts roughly that share of it through the account and receives roughly that share of the top-up.eligible_fractionnow applies to the spend as well as the cap.Why it was not caught
Both existing part-year tests are cap-bound:
Pro-rating the cap alone is enough to make both pass. They pass unchanged here. Four tests are added for the below-cap cases; two of them fail on
main.This change is currently latent
tax_free_childcare_eligible_declaration_periodsis:— a binary eligibility flag, so
eligible_fractionis always 1 or 0 and no built dataset changes today. This PR supplies the mechanism; representing partial years is a separate change, and I have deliberately not bundled it.Why representing partial years matters
HMRC publishes both an annual count of families with used Tax-Free Childcare accounts and a monthly one. For 2025-26 (March 2026 release, Table 1):
The spend route agrees: £599.8m ÷ 868,095 = £691 a year against about £92 per family-month, or 7.5 months. Monthly counts run 452k–601k across the year, so this is not seasonality.
The model treats every claiming family as active for twelve months. That is worth roughly 1.58× on Tax-Free Childcare spending, and the built dataset currently pays £1.24bn against HMRC's £599.8m while matching the caseload — 1.05× on families and 0.98× on children. The overshoot is entirely the average award, and part-year claiming is the largest identified component of it. See PolicyEngine/policyengine-uk-data#470.
A maintainer may reasonably want the part-year share as a calibrated parameter rather than a constant, since 7.6 months is an aggregate and the real distribution is a mix of full-year and short-spell claimants. That is why this PR stops at the mechanism.
Testing
The two new below-cap tests fail on
main:ruff formatclean.