Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/cgt-donor-sparsity.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- Reduce CGT band donors from 300 to 30 per HMRC size-of-gain band. The L0-regularised calibration pays a gate penalty per active record, so the first full build with 300 light donors per band (weight ~7 each in the top band) shrank total donor weight ~8x and left the £2m+ bands 94-97% under target, failing the built-dataset outcome tests before upload. Thirty donors per band carry the same band-exact initial totals with a tenth of the gate cost, making the per-band HMRC targets cheap for the optimiser to satisfy.
9 changes: 8 additions & 1 deletion policyengine_uk_data/datasets/imputations/capital_gains.py
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,14 @@ def impute_capital_gains(dataset: UKSingleYearDataset) -> UKSingleYearDataset:
# most likely to realise gains rather than on uniformly random households.

HMRC_SIZE_BANDS_FILE = "capital_gains_size_distribution_hmrc.csv"
DONORS_PER_BAND = 300
# Few, heavy donors rather than many light ones: the L0-regularised
# calibration pays a gate penalty per active record, so a band represented
# by 300 rows of weight ~7 is far more expensive to keep than 30 rows of
# weight ~67 carrying the same band total. The first full build with 300
# donors/band shrank donor weight ~8x and left the top bands 94-97% under
# target; fewer gates make satisfying the band targets cheap for the
# optimiser while donors still spread across income profiles.
DONORS_PER_BAND = 30
_DONOR_SEED = 1
# HMRC's rows below £12,300 mix AEA regimes and are definitionally
# incomplete (see the CSV header note); the spline body already covers
Expand Down