Skip to content

fix(data): keep sample weights aligned with their training rows - #492

Open
ChrisW09 wants to merge 1 commit into
mainfrom
fix/sample-weight-alignment
Open

ChrisW09 wants to merge 1 commit into
mainfrom
fix/sample-weight-alignment

Conversation

@ChrisW09

Copy link
Copy Markdown
Collaborator

Fixes #491

Problem

When fit() creates the validation split automatically and random_state is None (the estimator default), explicit classifier sample_weights were attached to the wrong training rows. In the issue's example, 81 of 160 training rows carried another row's weight, so rows weighted 0 were sampled.

Root cause

TabularDataModule.preprocess_data split X/y (and embeddings) with one train_test_split call. _resolve_train_sample_weights then split the weight vector with a second, independent call, relying on "same random_state reproduces the partition". That only holds for an integer seed. With None, both calls draw different permutations from NumPy's global RNG.

Fix

The row positions np.arange(len(y)) are now split in the same train_test_split call as X/y/embeddings. _resolve_train_sample_weights then indexes the weights with the training positions instead of re-splitting. The weights are correct by construction, whatever the seed.

Adding an array to train_test_split doesn't change the partition, because the indices are computed once and applied to every array. Seeded splits, and so existing results, stay the same. The explicit-validation-set path and the length validation of sample_weight are unchanged.

Tests

tests/integration/test_class_imbalance.py::TestWeightedSampling::test_explicit_sample_weight_stays_with_its_row (with random_state=None and with a fixed seed) checks that every training row carries its own weight. It uses a row_id feature to identify rows after the split. The None case fails on main and passes with this change. The existing data-module and sample-weight contract tests still pass.

Docs

No user-facing docs change is needed. The _resolve_train_sample_weights docstring now describes the position-based alignment.

Checks

  • just test: 3238 passed, 46 skipped, 28 xfailed; coverage 91.29%
  • just docs: build succeeded (warnings treated as errors)
  • just check: all hooks passed (ruff, prettier, pyright)

TabularDataModule split X/y with one train_test_split call and the
sampling weights with a second one, relying on an equal random_state to
reproduce the partition. With random_state=None, the estimators'
default, both calls drew different permutations from NumPy's global RNG
and most training rows received another row's weight. Row positions are
now split together with the data and the weights are indexed with the
training positions, so they stay aligned for any seed.

Fixes #491

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] sample_weight is applied to the wrong training rows when random_state is None (the default)

1 participant