Build a reform simulation's baseline from the situation under the baseline system

[MaxGhenis]

Draft: CI and the independent GPT-6.1 Sol review are pending. The review is queued in Subfleet; the Codex lanes are on hold until 22:09Z.

Fixes #612.

Problem

Simulation(situation=..., reform=...) builds the situation under the reform's system and makes its baseline a branch of that simulation under the baseline system. A stored input is a whole array: SimulationBuilder gives the people a situation gives no value for a variable it gives others (or sets along axes) that variable's default from the system it builds under. The branch inherited those arrays, so the baseline read the reform's default for those people. It also had no input for a variable the reform neutralizes, because the reform's system ignores that input. Reproduced on master b909d5a:

situation, reform	baseline before	baseline now = situation built under the baseline system
salary 3,000 for a, none for b; salary default 0 → 1,000	[3000, 1000]	[3000, 0]
axis over a's salary (0, 5,000), none for b; same reform	[0, 1000, 5000, 1000]	[0, 0, 5000, 0]
income_tax 7 given; reform neutralizes income_tax	450 (calculated)	7

Change

• Simulation.__init__ now builds a reform simulation's baseline branch from the situation under the baseline system, through a new Simulation._build_from_situation. Before, it re-pointed the inherited holders with _bind_to_tax_benefit_system. The branch gets new populations from its own system, and the situation is built into them by the same builder code a simulation under that system alone uses. The inputs are stored under "default", as in a simulation built from the situation, and the branch reads them through that. The records that described the inherited populations go with them: _user_input_keys, _store_history, _fast_cache, _requested_variables and invalidated_caches. The branch is still bound as before when there is no situation (a dataset, or populations given) or no baseline system.

• SimulationBuilder.skip_rejected_inputs (off by default; only the baseline build turns it on). In this mode the builder leaves out, rather than raises for, the inputs its system cannot take, as _bind_to_tax_benefit_system drops holders built for the reform:

  • a variable the system lacks or defines for another entity;
  • a value it rejects (an enum member only the reform has);
  • an axis over a variable it lacks (the axis still shapes the population);
  • an array its set_input refuses with a ValueError. That covers a period the variable's definition does not take (a variable the reform makes eternal), and also a NaN or yearly and monthly inputs that do not add up.

  Entities, roles and the syntax of periods are still checked. The reform's strict build runs first on the same situation, so skip mode only acts where the two systems differ: a variable's definition (its entity, values, period or set_input handler), or a variable the reform neutralizes, whose inputs the reform's system ignores without checking. An input of the second kind that the baseline system rejects (a NaN, say) is left out of the baseline; master's baseline had no input for that variable either.

Why rebuild rather than mask: a stored input is all-or-nothing in Holder.set_input, _user_input_keys, StoreHistory, to_input_dataframe and subsample. set_input_divide_by_period and set_input_dispatch_by_period combine a larger period's array with the sub-period inputs already stored, so the default one person gets for one month also changes what the other months of a yearly input are divided into (test_a_default_given_for_one_month_changes_how_a_year_is_divided). A mask of given entries would have to replay that handler arithmetic for each system. Building the situation again under the baseline system gets it right by construction.

The reform simulation itself is unchanged. policyengine-us reaches this code through core's Simulation.__init__. policyengine-uk builds its baseline as a separate Simulation from the situation and is not affected.

Not changed here: apply_reform on a simulation built from a situation keeps the old system's defaults in its inputs. That is the same bug in the other direction, and so does Simulation(tax_benefit_system=..., reform=...), which builds the situation before applying the reform. Both are tracked in #614.

Overlap with #611, which also edits this part of __init__ and the builder: whichever merges second moves its baseline rebuild onto _build_from_situation (see the note on #611).

Invariants (property-tested)

tests/core/test_reform_baseline_situation_property.py, 200 Hypothesis examples per run.

The reforms are any mix of:

• changed defaults of input variables: person and household; float, int, enum, date and string; monthly variables whose yearly inputs are divided or dispatched, and an eternal one;
• neutralized inputs and formulas;
• a variable only the reform has;
• a changed tax rate.

The situations have one to three people in one or two households. They give any of those variables for any of several periods to some people and not others, with no axes, one dimension of parallel axes, or two perpendicular dimensions. Some situations have no entities and give each variable's values for everyone.

For every such reform and situation:

1. Differential: the baseline calculates what SimulationBuilder makes of the situation under the baseline system alone calculates, for 13 outputs at their periods. The reform-only variable is left out of the situation first.
2. It has the same population, records the same inputs (_user_input_keys, input_variables) and exports the same to_input_dict().
3. It is the reform simulation's "baseline" branch, under the baseline system.
4. The reform simulation calculates what the builder makes of the situation under another copy of the reform's system.

Tests and evidence

• tests/core/test_reform_baseline_situation.py has 27 examples. Against master, 22 of the 28 new tests fail, the property among them. The 6 that pass guard behaviour the rebuild must keep:

  • cases _bind_to_tax_benefit_system already handled: a variable only the reform has, an entity move, a period refusal and an axis over a reform-only variable;
  • entity checks in skip mode;
  • "default" input keys without entities.

• On generated examples alone (no @examples), the property fails on each of 5 seeds against the unfixed __init__, and again against the "baseline"-key slip described below.

• Mutation check, each part of the change removed in turn. Each of these fails at least one test:

  • the rebuild itself, and keeping the old populations;
  • each of the six skip sites in the builder;
  • storing under the branch's own name;
  • the resets of _user_input_keys, _fast_cache and has_axes.

  The resets of _store_history, _requested_variables and invalidated_caches survive. On the only call path the branch is a fresh clone, where these are empty or only widen drops. They stay because purge_cache_of_invalid_values deletes the arrays, inputs included, of stale invalidated_caches entries, and _requested_variables vouches for the old populations' prerequisites.

• Local checks:

  • ruff format --check and ruff check pass;
  • country-template YAML tests: 39 passed;
  • full pytest tests on 41848f9: 2,531 passed, 4 skipped, 3 xfailed;
  • on ac6236f, which changes only _build_from_situation's branch name, the comments and the tests: these tests and test_reform_baseline_system.py, 45 passed.

• Downstream, policyengine-us 1.822.5. The household had three people, earnings for one adult and an axis over the other's (five points), plus a CTC base-amount reform. It ran through master's core and this branch's.

  • The baseline and reform values (employment income, income tax, CTC, EITC, net income, SNAP) are identical, and the baseline equals a plain Simulation(situation=...). A parametric reform changes no input defaults.
  • The rebuild takes 84 ms on the US system (6,155 variables), against 5 ms for the branch it replaces.
  • Constructing Simulation(situation, reform) took 8.09 and 10.12 s (median of 5) on master and 8.11 and 9.14 s here, interleaved on a loaded host: no difference measurable above the noise.

• Review. An in-session Opus 5.5 adversarial review approved with three nits, each reproduced, all addressed in ac6236f:

  1. Without entities, the rebuilt baseline stored inputs under "baseline", not "default". Fixed, with an example; the property now draws situations without entities.
  2. The skip-mode comment overstated what is still checked. Corrected.
  3. Invalid inputs for a neutralized variable are left out silently. Documented above, and the changelog now says "valid inputs".

axiom: n/a: engine change in policyengine-core, no policy encoded.

🤖 Generated with Claude Code