[ChrisW09]

Fixes #85.

The transformers exported from pretab.transformers did not record feature_names_in_ when fitted on a pandas DataFrame. A frame with reordered or renamed columns at transform was processed by position, so a feature silently went through another column's fitted basis. Output names were x0_..., and PLE, numeric binning and the tensor-product spline accepted an input_features of the wrong length.

Root cause

The shared validator validate_2d_allow_nan (used by every BasePreTabTransformer), the PLE and binning validation, and the np.asarray conversion of the other exported transformers never called scikit-learn's feature-name check. Each get_feature_names_out built x{i} names itself, and only the BasePreTabTransformer version checked the length of input_features.

Fix

• validate_2d_allow_nan, PLETransformer, NumericBinningTransformer._check_array, ContinuousOrdinalTransformer, OneHotFromOrdinalTransformer, ToFloatTransformer, NoTransformer, MissingStateIndicator and LanguageEmbeddingTransformer call scikit-learn's _check_feature_names: at fit it records feature_names_in_ (or drops it for an array), and at transform renamed or reordered columns raise and input without names warns. This covers every class exported from pretab.transformers.
• The new resolve_input_features (pretab/core/validation.py) returns the names that output names are built from. Explicit input_features need one entry per input feature and, after a DataFrame fit, must equal feature_names_in_. None uses feature_names_in_, else x0, x1, .... BasePreTabTransformer, PLE, binning, the tensor-product spline and the two ordinal-based encoders use it. ToFloatTransformer, NoTransformer, MissingStateIndicator and LanguageEmbeddingTransformer use scikit-learn's _check_feature_names_in instead, so pretab.preprocessing and pretab.embedding keep importing nothing from pretab.core.
• OneHotFromOrdinalTransformer.get_feature_names_out checks that the transformer is fitted and accepts input_features=None, which raised TypeError before.
• The binning feature-count error uses scikit-learn's wording.

On main, other code relied on get_feature_names_out accepting any names, so the stricter check needs these changes as well:

• RepresentationSpecMixin.get_representation_spec resolves its input names with resolve_input_features instead of passing x0, x1, ..., which a DataFrame-fitted transformer now rejects.
• _representation_summary (pretab/compose/serialize.py) passes each block's column names to the spec and no longer swallows spec errors. With the strict names alone it would have silently dropped every representation fitted directly on its column.
• ToFloatTransformer and NoTransformer read the width of their input from np.shape. The one-hot encoder hands them a sparse matrix, which np.asarray turned into a 0-d object array, so one input feature was recorded instead of the encoder's width; the strict get_feature_names_out exposed this in every categorical_method="one-hot" preprocessor.
• Block-width rule: in _representation_summary and in the feature lineage (_resolve_block_representation in pretab/compose/inspection.py), a step fitted on more inputs than its block has columns names its own inputs. On main this is the representation behind the imputer's built-in missing indicator (add_missing_indicator=True), which receives the indicator as a second input. Without the rule, the lineage of such a block would newly raise for PLE and binning.
• build_feature_info probes a step fitted on the DataFrame column with a one-row frame that carries the step's feature_names_in_ (_probe_input), so get_feature_info() does not warn about missing feature names.

Behaviour changes

• Standalone transformers fitted on a DataFrame with string column names set feature_names_in_. Output names and set_output(transform="pandas") columns use the column names (age_rbf0, age__missing). Renamed or reordered columns at transform raise ValueError, an array at transform warns that X does not have valid feature names, and a refit on an array removes feature_names_in_.
• get_feature_names_out(input_features) raises for a wrong length (InvalidParamError where resolve_input_features is used, scikit-learn's ValueError in the four classes that use _check_feature_names_in) and raises ValueError for names that differ from feature_names_in_.
• The NumericBinningTransformer feature-count error (still PretabDataError) reads "X has N features, but NumericBinningTransformer is expecting M features as input."
• Preprocessor: output names and transform output are unchanged (the golden baselines pass unchanged). Entries in to_spec()["representations"] name their inputs and outputs after the block's column (input_features: ["age"], age_rbf0) instead of x0 / x0_rbf0, and a spec entry that fails to build raises from to_spec() instead of being dropped.
• get_feature_info() no longer warns that X does not have valid feature names for a step fitted on the DataFrame column (on main, a scikit-learn scaler fitted there warned).
• get_feature_lineage() with add_missing_indicator=True works for RBF, B-spline and the other BasePreTabTransformer representations; it raised on main.
• fingerprint_ changes for preprocessors with a PreTab step fitted directly on the DataFrame (no imputer and no scaler in front), which now stores feature_names_in_, and for categorical_method="one-hot" preprocessors, whose ToFloatTransformer now records the encoder's width. The default preprocessor's fingerprint is unchanged (compared with main).

Documentation

• docs/representations/spline_expansions.md ("The idea"): output names follow the DataFrame column (age_bs0; x0_bs0 for array input).
• docs/tutorials/custom_representation.md ("Subclass BaseRepresentation"): self._validate records and checks feature_names_in_, and the output names use them.
• Docstrings: BasePreTabTransformer (DataFrame contract); every changed get_feature_names_out (Parameters, Returns, Raises); feature_names_in_ in the Attributes of every class that documents n_features_in_ (n_features_in_ added for the two ordinal-based encoders); validate_2d_allow_nan (reset); RepresentationSpecMixin.get_representation_spec (input_features); _representation_summary; the new resolve_input_features and _probe_input.

Tests

• tests/transformers/test_feature_names_in.py (new), parametrized over all 24 exported transformers (LanguageEmbeddingTransformer with a stub model, so the optional dependency is not needed). 154 of its 170 cases fail on main.
  • test_dataframe_fit_records_feature_names_and_uses_them, test_reordered_columns_are_rejected, test_renamed_columns_are_rejected, test_array_after_a_dataframe_fit_warns_like_scikit_learn, test_input_features_are_validated: the contract from the issue. Fails on main, passes here.
  • test_scikit_learn_feature_name_checks_pass: scikit-learn's check_dataframe_column_names_consistency and check_transformer_get_feature_names_out_pandas, for all but PeriodicEncodingTransformer (the checks feed data outside its required [0, period] range; the other tests cover it). Fails on main, passes here.
  • test_sparse_input_records_its_width[ToFloatTransformer|NoTransformer]: a sparse input records its real width. Fails on main, passes here.
  • test_array_fit_keeps_generic_names_and_drops_stale_feature_names: after a refit on an array, the names are x0, ... and a wrong-length input_features raises. Fails on main for the 9 transformers without a length check; a guard for the 15 BasePreTabTransformer subclasses, which pass on both.
  • test_cases_cover_every_exported_transformer: the cases cover every exported class. Guard test: passes on both by design.
• tests/core/test_representation_spec.py::test_spec_after_a_dataframe_fit_follows_get_feature_names_out (20 exported representations): the default spec after a DataFrame fit uses feature_names_in_ and matches get_feature_names_out. Guard test: passes on both by design; fails with the transformer changes alone, without the get_representation_spec change.
• tests/core/test_representation_spec.py::test_cases_cover_every_exported_representation: the cases above cover every exported representation. Guard test: passes on both by design.
• tests/extension/test_extension_protocol.py::test_representation_spec_after_a_dataframe_fit_names_the_columns: a BaseRepresentation subclass's spec names the frame's columns. Fails on main, passes here.
• tests/integration/test_serialization.py::test_representation_summary_names_each_block_after_its_column[imputed|frame]: summary entries named after the block's column, for a leaf behind the imputer and for a leaf fitted on the frame. Fails on main, passes here.
• tests/integration/test_serialization.py::test_representation_summary_of_a_leaf_behind_the_imputer_indicator: with add_missing_indicator=True, the two-input leaf names its own inputs, and to_spec(), reproducibility_report() and fingerprint_ work. Fails on main, passes here.
• tests/integration/test_feature_lineage.py::test_lineage_of_a_representation_behind_the_imputer_indicator[rbf|bspline]: lineage with add_missing_indicator=True. Fails on main, passes here. The [ple] case passes on main and fails with the transformer changes alone.
• tests/compose/test_inspection.py::test_build_feature_info_probes_a_step_fitted_on_the_frame_without_warning[standardization]: build_feature_info raises no warning. Fails on main, passes here. The [rbf] case passes on main and fails with the transformer changes alone.

Reproduction

Output of the script from #85 on this branch:

RBFExpansionTransformer | feature_names_in_: ['age' 'income'] | names: ['age_rbf0', 'age_rbf1']
   reordered columns: The feature names should match those that were passed during fit.
PLETransformer | feature_names_in_: ['age' 'income'] | names: [np.str_('age_ple0'), np.str_('age_ple1')]
   reordered columns: The feature names should match those that were passed during fit.
SplineTransformer | feature_names_in_: ['age' 'income'] | names: ['age_sp_0', 'age_sp_1']
   reordered columns: The feature names should match those that were passed during fit.

(PLETransformer.fit still emits its LeakageWarning for fitting on (X, y) outside a pipeline.)

Checks

• just test: 1638 passed, 59 skipped, 7 xfailed; coverage 93.76% (gate 90%).
• Minimum dependencies (CI job: Python 3.10, numpy 1.24.4, pandas 2.0.3, scipy 1.10.1, scikit-learn 1.6.0): 1625 passed, 72 skipped, 7 xfailed; scripts/quickstart.py passes.
• just docs: strict build succeeds.
• just check: all pre-commit and pre-push hooks pass (ruff format, ruff lint, prettier, file hygiene, pyright with 0 errors).
• Import direction between subpackages is unchanged.

Merge notes

• Base: main.

• The PR for add_missing_indicator / impute_with_indicator feeds the indicator through the scaler and basis; lineage crashes #62 (fix(preprocessor): keep the missing indicator out of the representation #106) conflicts in pretab/compose/inspection.py, where it adds representation_leaf at the place this PR adds _probe_input (keep both), and in the docstring of _representation_summary (keep this PR's docstring and add add_missing_indicator / impute_with_indicator feeds the indicator through the scaler and basis; lineage crashes #62's sentence that the leaf of a missing-indicator / separate-state block is the last step of its representation branch). The loop body merges cleanly into leaf = representation_leaf(transformer) followed by this PR's block-width rule. With add_missing_indicator / impute_with_indicator feeds the indicator through the scaler and basis; lineage crashes #62 the representation no longer receives the indicator, so the rule no longer applies to Preprocessor blocks, and two tests here that pin main's structure must be updated: in test_lineage_of_a_representation_behind_the_imputer_indicator, the families become {family, "missing_state"}, and in test_representation_summary_of_a_leaf_behind_the_imputer_indicator, entries["a"]["input_features"] becomes ["a"]. With these changes, the full suite passes (1659 passed, 59 skipped, 7 xfailed).

• The PR for CrossFittedTransformer(Preprocessor) relearns column types and category codes per fold, unlike transform #79 (fix(cross-fitting): refit only target-aware Preprocessor blocks per fold #117) lets _resolve_block_representation take columns=None, and conflicts on the same lines as this PR's block-width rule. Combine both: fits_block = columns is not None and getattr(transformer, "n_features_in_", len(columns)) == len(columns), then input_features=list(columns) if fits_block else None. The PR for CrossFittedTransformer(Preprocessor(add_missing_indicator=True)) crashes when a fold's training rows contain no missing value #89 (fix(cross-fitting): keep missing indicators of a wrapped Preprocessor #124), stacked on CrossFittedTransformer(Preprocessor) relearns column types and category codes per fold, unlike transform #79, moves this code into _locate_block_representation, which returns index, (...); apply the same two lines there and keep both sides of the import conflict (import pandas as pd, from sklearn.base import clone, from sklearn.pipeline import FeatureUnion, Pipeline). Resolved this way, the full suite passes with CrossFittedTransformer(Preprocessor) relearns column types and category codes per fold, unlike transform #79 (1652 passed, 59 skipped, 7 xfailed) and with CrossFittedTransformer(Preprocessor(add_missing_indicator=True)) crashes when a fold's training rows contain no missing value #89 (1661 passed, 59 skipped, 8 xfailed).

• Conflicts in import lists and appended tests, described in the other PRs: Preprocessor.get_feature_names_out(input_features) fails after fitting on an ndarray, breaking Pipeline.get_feature_names_out() #65 (pretab/compose/inspection.py imports: keep import pandas as pd and from sklearn.pipeline import FeatureUnion, Pipeline; its _without_feature_names also strips the feature_names_in_ that PreTab steps now record), A 2-D y (multi-output target) crashes the B/M/I splines even when unsupervised, and target-aware methods with unrelated errors #88 (pretab/core/validation.py imports and __all__, and the core.validation import in ple.py), Preprocessor rejects polars DataFrames, so pipelines with set_output(transform="polars") crash #94 (pretab/core/validation.py imports, __all__ and the new functions) and Preprocessor crashes on an all-missing column at fit (numerical, one-hot) or silently drops it (int) #95 (tests/compose/test_inspection.py, both add a test at the same place). Keep both sides in each.

• Follow-up with CrossFittedTransformer converts DataFrames to ndarray; a wrapped Preprocessor treats every column as categorical #58 and Preprocessor.get_feature_names_out(input_features) fails after fitting on an ndarray, breaking Pipeline.get_feature_names_out() #65: once the PRs for CrossFittedTransformer converts DataFrames to ndarray; a wrapped Preprocessor treats every column as categorical #58 (fix(cross-fitting): pass DataFrames to the wrapped transformer unchanged #103) (cross-fitting passes DataFrames to the wrapped estimator), Preprocessor.get_feature_names_out(input_features) fails after fitting on an ndarray, breaking Pipeline.get_feature_names_out() #65 (Preprocessor records and validates feature_names_in_) and this one have all landed, CrossFittedTransformer(Preprocessor(...)).fit(X, y).get_representation_spec() on a DataFrame raises ValueError: input_features is not equal to feature_names_in_, because the fallback spec passes x0, x1, ... to the wrapped Preprocessor. No existing test catches this: with the three branches merged, the suite passes. On main the fix would be dead code, so it is not in this PR. Whichever of the three lands last should add, in CrossFittedTransformer.get_representation_spec (pretab/core/supervised.py), just before the fallback return super().get_representation_spec(input_features):

  if input_features is None:
      # Name the inputs as the wrapped estimator does, so a DataFrame fit
      # passes its column names (not x0, x1, ...) to get_feature_names_out.
      input_features = getattr(self.estimator_, "feature_names_in_", None)

  together with tests for a cross-fitted PLETransformer and a cross-fitted Preprocessor fitted on a DataFrame (both were in fix: resolve 10 high-priority bugs found in a library audit (#76-#85) #86). With this change on the three merged branches, the spec names the columns (('num', 'city')) and the full suite passes.

• The PRs for to_spec / fingerprint_ fail or write unloadable specs for object arrays of numpy scalars and builtin dtypes #66 (fix(serialization): encode object arrays and builtin dtypes in specs #132) and Default to_spec() files contain bare NaN tokens and are not valid JSON for strict parsers #80 (fix(serialization): write specs as strict JSON without NaN tokens #134) change other parts of pretab/compose/serialize.py; git merges both cleanly with this branch.

This change was previously part of #86, which is closed in favour of one pull request per issue.