Skip to content

Don't drop columns that are not specified - #1505

Merged
yaeldekel merged 10 commits into
dotnet:masterfrom
yaeldekel:dropcolumns
Nov 9, 2018
Merged

yaeldekel merged 10 commits into
dotnet:masterfrom
yaeldekel:dropcolumns

Conversation

@yaeldekel

Copy link
Copy Markdown

Fixes #1504 .

@sfilipi

sfilipi commented Nov 2, 2018 •

Copy link
Copy Markdown
Member
    /// <param name="keepHidden">If true will keep hidden columns and false will remove hidden columns.</param>

based on what the outcome of the #1504 discussion is, we should update this to be accurate. #Closed


Refers to: src/Microsoft.ML.Data/Transforms/SelectColumnsTransform.cs:59 in 05adabe. [](commit_id = 05adabe, deletion_comment = False)

@Zruty0

Zruty0 commented Nov 6, 2018

Copy link
Copy Markdown
Contributor
    public ColumnSelectingEstimator(IHostEnvironment env, string[] keepColumns,

Do we need this ctor?


Refers to: src/Microsoft.ML.Data/Transforms/SelectColumnsTransform.cs:64 in ea30ccb. [](commit_id = ea30ccb, deletion_comment = False)

@Zruty0

Zruty0 commented Nov 6, 2018

Copy link
Copy Markdown
Contributor
    public ColumnSelectingEstimator(IHostEnvironment env, string[] keepColumns,

as in, do we need it to be public?


In reply to: 436352086 [](ancestors = 436352086)


Refers to: src/Microsoft.ML.Data/Transforms/SelectColumnsTransform.cs:64 in ea30ccb. [](commit_id = ea30ccb, deletion_comment = False)

@Zruty0 Zruty0 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

{
IDataView savedData = TakeFilter.Create(_env, est.Fit(data.AsDynamic).Transform(data.AsDynamic), 4);
savedData = SelectColumnsTransform.CreateKeep(_env, savedData, "pca");
savedData = SelectColumnsTransform.CreateKeep(_env, savedData, new[] { "pca" });

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

new[] [](start = 79, length = 5)

is it unavoidable?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we want the "keepHidden" argument to be optional, then the string[] argument can't be a 'params' argument. I can bring back the other CreateKeep overload with the non-optional keepHidden argument if you think it's important.


In reply to: 231233064 [](ancestors = 231233064)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought that you can have optional and then params


In reply to: 231242980 [](ancestors = 231242980,231233064)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

..these days


In reply to: 231257225 [](ancestors = 231257225,231242980,231233064)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Turns out you're right, it is possible to have an optional followed by params. But to use it without specifying the optional it would have to be: keepColumns: new[] {"a", "b", "c"}, which is longer than just adding the "new[]"...


In reply to: 231257259 [](ancestors = 231257259,231257225,231242980,231233064)

@yaeldekel

Copy link
Copy Markdown
Author
    public ColumnSelectingEstimator(IHostEnvironment env, string[] keepColumns,

It is used in a few tests.


In reply to: 436352181 [](ancestors = 436352181,436352086)


Refers to: src/Microsoft.ML.Data/Transforms/SelectColumnsTransform.cs:64 in ea30ccb. [](commit_id = ea30ccb, deletion_comment = False)

@Zruty0

Zruty0 commented Nov 6, 2018

Copy link
Copy Markdown
Contributor
    public ColumnSelectingEstimator(IHostEnvironment env, string[] keepColumns,

Can we change the tests? :)


In reply to: 436366507 [](ancestors = 436366507,436352181,436352086)


Refers to: src/Microsoft.ML.Data/Transforms/SelectColumnsTransform.cs:64 in ea30ccb. [](commit_id = ea30ccb, deletion_comment = False)

@sfilipi

sfilipi commented Nov 9, 2018

Copy link
Copy Markdown
Member
    public ColumnSelectingEstimator(IHostEnvironment env, string[] keepColumns,

idk if you can annotate test classes with the new [BestFriend] and not need to change them.


In reply to: 436375208 [](ancestors = 436375208,436366507,436352181,436352086)


Refers to: src/Microsoft.ML.Data/Transforms/SelectColumnsTransform.cs:64 in ea30ccb. [](commit_id = ea30ccb, deletion_comment = False)

@sfilipi sfilipi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@yaeldekel
yaeldekel merged commit e803520 into dotnet:master Nov 9, 2018
@yaeldekel
yaeldekel deleted the dropcolumns branch November 9, 2018 18:03
@ghost ghost locked as resolved and limited conversation to collaborators Mar 27, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants