Repository navigation
Conversation
TabularDataModule split X/y with one train_test_split call and the sampling weights with a second one, relying on an equal random_state to reproduce the partition. With random_state=None, the estimators' default, both calls drew different permutations from NumPy's global RNG and most training rows received another row's weight. Row positions are now split together with the data and the weights are indexed with the training positions, so they stay aligned for any seed. Fixes #491
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #491
Problem
When
fit()creates the validation split automatically andrandom_stateisNone(the estimator default), explicit classifiersample_weights were attached to the wrong training rows. In the issue's example, 81 of 160 training rows carried another row's weight, so rows weighted 0 were sampled.Root cause
TabularDataModule.preprocess_datasplit X/y (and embeddings) with onetrain_test_splitcall._resolve_train_sample_weightsthen split the weight vector with a second, independent call, relying on "samerandom_statereproduces the partition". That only holds for an integer seed. WithNone, both calls draw different permutations from NumPy's global RNG.Fix
The row positions
np.arange(len(y))are now split in the sametrain_test_splitcall as X/y/embeddings._resolve_train_sample_weightsthen indexes the weights with the training positions instead of re-splitting. The weights are correct by construction, whatever the seed.Adding an array to
train_test_splitdoesn't change the partition, because the indices are computed once and applied to every array. Seeded splits, and so existing results, stay the same. The explicit-validation-set path and the length validation ofsample_weightare unchanged.Tests
tests/integration/test_class_imbalance.py::TestWeightedSampling::test_explicit_sample_weight_stays_with_its_row(withrandom_state=Noneand with a fixed seed) checks that every training row carries its own weight. It uses arow_idfeature to identify rows after the split. TheNonecase fails onmainand passes with this change. The existing data-module and sample-weight contract tests still pass.Docs
No user-facing docs change is needed. The
_resolve_train_sample_weightsdocstring now describes the position-based alignment.Checks
just test: 3238 passed, 46 skipped, 28 xfailed; coverage 91.29%just docs: build succeeded (warnings treated as errors)just check: all hooks passed (ruff, prettier, pyright)