Skip to content

sklearn: Add HDBSCAN, Tree node-array properties, and precision_recall_curve drop_intermediate/y_score - #404

Open
John Clow (Gamrix) wants to merge 1 commit into
microsoft:mainfrom
Gamrix:sklearn-hdbscan-tree-attrs
Open

John Clow (Gamrix) wants to merge 1 commit into
microsoft:mainfrom
Gamrix:sklearn-hdbscan-tree-attrs

Conversation

@Gamrix

Copy link
Copy Markdown

These are three gaps in stubs/sklearn that produce false type errors on current scikit-learn code. They affect Pylance users and also Pyrefly users, because Pyrefly bundles these stubs. I found them by running Pyrefly against a codebase on scikit-learn 1.9.0. Each signature below was checked against both 1.9.0 and the 1.6.1 runtime that CI's stubtest installs (scikit-learn <1.7.0).

1. sklearn.cluster.HDBSCAN is missing (added in 1.3)

from sklearn.cluster import HDBSCAN fails because the stub doesn't export it.

  • Adds cluster/_hdbscan/__init__.pyi and cluster/_hdbscan/hdbscan.pyi, written in the style of _dbscan.pyi and _optics.pyi: constructor parameters, fit, fit_predict, dbscan_clustering, and the fitted attributes labels_, probabilities_, centroids_, medoids_, n_features_in_ and feature_names_in_.
  • Exports HDBSCAN from cluster/__init__.pyi, including __all__.
  • Source: sklearn/cluster/_hdbscan/hdbscan.py#L423 and __init__ at L665.
  • copy is typed bool = False. In 1.9 the runtime default is a hidden "warn" sentinel (Hidden(StrOptions({"warn"}))) that behaves like False and raises a FutureWarning. In 1.6 the default is plain False, which stubtest checks.

2. sklearn.tree._tree.Tree lacks its public node-array properties

estimator.tree_.children_left, .feature, .threshold and the other node arrays are the documented way to inspect a fitted tree, but none of them are in the stub.

  • Adds read-only properties children_left, children_right, feature, threshold, impurity, n_node_samples, weighted_n_node_samples, missing_go_to_left (all ndarray) and n_leaves (int, which matches BaseDecisionTree.get_n_leaves() -> int). Source: sklearn/tree/_tree.pyx#L721-L761.
  • Fixes the typo max_dept: int to max_depth: int (cdef public intp_t max_depth, _tree.pxd#L50).
  • Changes n_classes from int to an ndarray property. At runtime it is a property that returns an array with one entry per output (_tree.pyx#L721-L723).

3. precision_recall_curve lacks drop_intermediate and y_score

  • drop_intermediate: bool = False was added in 1.3.
  • The second parameter was renamed probas_pred to y_score in 1.5, with probas_pred deprecated and removed in 1.7. The 1.9 signature is (y_true, y_score, *, pos_label=None, sample_weight=None, drop_intermediate=False) (sklearn/metrics/_ranking.py#L1059).
  • This follows the existing squared deprecation pattern in metrics/_regression.pyi. The main overload takes y_score. A second overload, marked @deprecated, accepts the keyword probas_pred= that 1.5 and 1.6 still allow. Positional calls precision_recall_curve(y, scores) are unaffected.

stubtest_allowlist.txt

  • Removes the entries these changes make unused: sklearn.cluster.HDBSCAN, sklearn.cluster.__all__, sklearn.cluster._hdbscan.hdbscan, the ten sklearn.tree._tree.Tree.* properties and max_depth, and both precision_recall_curve entries.
  • Adds entries for internals the new module doesn't stub, in the same way sibling estimators handle them: HDBSCAN.__sklearn_tags__ (both paths), hdbscan.FAST_METRICS and hdbscan.remap_single_linkage_tree.

Validation (local, Python 3.13, --group tests)

  • python tests/run_stubtest.py sklearn reports exactly the same set of errors before and after this change, so the PR adds no new stubtest errors and leaves no unused allowlist entries. Two caveats:
    • I had to pin numpy<2.3 locally. With numpy 2.5 the run stops with an unrelated build error: sklearn/utils/validation.pyi: Module "numpy.core.numeric" has no attribute "ComplexWarning".
    • Even pinned, the run reports 63 errors on main that this PR doesn't touch (_seq_dataset disjoint-base errors and others), so a CI run may still fail on those.
  • python -m mypy .: no issues.
  • pyright stubs/sklearn tests: 0 errors. A probe file checking HDBSCAN(...).fit(X).labels_, tree_.children_left, tree_.n_leaves and precision_recall_curve(..., drop_intermediate=True) also checks cleanly.
  • ruff check stubs/sklearn and ruff format --check stubs/sklearn pass. Repo-wide ruff check reports one existing RUF100 in stubs/matplotlib/pyplot.pyi, which this PR doesn't touch.

🤖 Generated with Claude Code

…l_curve drop_intermediate/y_score

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Gamrix

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree company="Safetykit"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant