Skip to content

Observing Alternative Feature Importance Techniques #374

Open
Open
Observing Alternative Feature Importance Techniques#374
Labels
enhancementNew feature or request

Description

@BradKML
BradKML
opened on Oct 18, 2022

TL;DR Original List with yet-to-be implemented FE algorithms in parrt/random-forest-importances#54

Seeing #364 and #218 I do notice that some of the feature importance algorithms are not on the list, particularly LOFO, Morris and "Unbiased" feature importance. Might wanna check those out?

Bonus: this visualization notebook exists https://github.com/shionhonda/feature-importance

Currently these are not in the ReadME:

Unsure if they have alternative name:

How are they different (missing data test vs prioritization, significant correlation)?

Activity

  1. paulbkoch
    added
    enhancementNew feature or request
    on Feb 16, 2023

  2. BradKML commented on Apr 30, 2023

    @BradKML
    BradKML
    on Apr 30, 2023
    Author

    For some descriptions:

    • LOFO: (a) Get baseline performance with all features withCV, (b) Remove singular feature, retrain the model and evaluate CV performance (c) measure new performance against baseline (note: LightGBM is default model and cv=4)
    • Permutation: shuffling the value of the column to see if the performance has changed (noted to be slow) https://christophm.github.io/interpretable-ml-book/feature-importance.html
    • UFI: Not sure about how they rely on decision path in a RandomForest model to make the decision.
    • Gini and Split: uses the LGBMRegressor internal function in LightGBM, simple enough

  3. paulbkoch commented on May 1, 2023

    @paulbkoch
    paulbkoch
    on May 1, 2023
    Collaborator

    Hi @BrandonKMLee -- Thanks for putting together this list and the descriptions. We'd be open to PRs that implement these alternative algorithms. Our core team is pretty focused on improving EBMs, so we don't have a lot of bandwidth to work on more tangential improvements.

  4. BradKML commented on Sep 13, 2024

    @BradKML
    BradKML
    on Sep 13, 2024
    Author

    @paulbkoch noted with thanks regarding the priority, and I also remember how booster-based feature selection was being heavily focused on by everyone https://github.com/scikit-learn-contrib/boruta_py https://github.com/Ekeany/Boruta-Shap https://github.com/chasedehan/BoostARoota
    Also some other small finds regarding MRMR (mutual information, not sure if it overlaps to other methods here) https://github.com/AutoViML/featurewiz https://github.com/smazzanti/mrmr https://github.com/danielhomola/mifs

    P.S. There are other super-repos for feature importance https://github.com/JingweiToo/Wrapper-Feature-Selection-Toolbox https://github.com/jundongl/scikit-feature

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions