You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Why is ColumnAttribute's ordinal parameter a string? #132
Because the ordinal can be a range such as 1-10 or 1-* or 1,2,3. This is needed to support vector-valued columns. Perhaps this makes the "ordinal" to be a sub-optimal argument name
The type IrisData is not actually used for anything by the TextLoader. It just informs the types, and is used to describe what ranges in the TSV/CSV should map to what columns names. (This is one of several things I'd say make this "reflection based" schema a bad design, at least for the loader -- for programmatic access to pipelines, it's a bit more defensible.) So the result would be an IDataView, with a column named SepalLength, with vector length 3. Of course if you were to feed this type to a prediction engine, it would throw.
Regarding the loader: I'd encourage us to look at PR 38, since it at least introduces the entry point necessary to just declare the schema programmatically without this odd reflection-based field peeking scheme we somehow released.
That PR's entry point though introduces ranges by having people declare Range objects. I wonder if there's a better option. Obviously int is completely insufficient for the reasons @glebuk describes, but describing the ranges of fields from the TSV/CSV in this fashion is perhaps a bit verbose. Maybe we can imagine some ways either in that PR, or a followup PR, to clean it up a bit.
Ah... you were asking about Ordinal so I thought we were talking about the text loader. Note that the ordinal property on this attribute is used only in this attempt to derive the text loader settings from the class. That is, it was a hack thrown on top of this scheme to solve a problem that exists only because we decided to use the same attributes for the text loader as we had used for the prediction engine.
In the prediction engine, there is no concept of ordinal. All references to columns in IDataView from a user's perspective are by name, all transforms etc. refer to columns by name, so there it was not relevant.
So to answer your question: nothing, because it's not used for that.
This is, I hope you don't mind my observing, another reason why I consider deriving the text loader settings from a class as a mistake.
You could make ordinal take int as an overload, for cases when it isn't a vector.
I agree that it feels very weird declaring a class just to pass metadata along. There are a few alternatives, which you could support one or more of:
Passing in a JSON config (great for more dynamic scenarios)
Having a builder class that you call methods on to define the schema (eg, builder.Column("SepalLength", typeof(float), 1).Column("Color", typeof(string), 2))
Activity
glebuk commentedon May 11, 2018
Because the ordinal can be a range such as 1-10 or 1-* or 1,2,3. This is needed to support vector-valued columns. Perhaps this makes the "ordinal" to be a sub-optimal argument name
KrzysztofCwalina commentedon May 11, 2018
What happens when a range is mapped to/from a single property? i.e.
TomFinley commentedon May 11, 2018
The type
IrisDatais not actually used for anything by theTextLoader. It just informs the types, and is used to describe what ranges in the TSV/CSV should map to what columns names. (This is one of several things I'd say make this "reflection based" schema a bad design, at least for the loader -- for programmatic access to pipelines, it's a bit more defensible.) So the result would be anIDataView, with a column namedSepalLength, with vector length 3. Of course if you were to feed this type to a prediction engine, it would throw.Regarding the loader: I'd encourage us to look at PR 38, since it at least introduces the entry point necessary to just declare the schema programmatically without this odd reflection-based field peeking scheme we somehow released.
TomFinley commentedon May 11, 2018
That PR's entry point though introduces ranges by having people declare
Rangeobjects. I wonder if there's a better option. Obviouslyintis completely insufficient for the reasons @glebuk describes, but describing the ranges of fields from the TSV/CSV in this fashion is perhaps a bit verbose. Maybe we can imagine some ways either in that PR, or a followup PR, to clean it up a bit.KrzysztofCwalina commentedon May 11, 2018
I was thinking more about how the ColumnAttributes is used to serialize data passed to Predict and deserialized when returned from Predict.
TomFinley commentedon May 11, 2018
Ah... you were asking about
Ordinalso I thought we were talking about the text loader. Note that the ordinal property on this attribute is used only in this attempt to derive the text loader settings from the class. That is, it was a hack thrown on top of this scheme to solve a problem that exists only because we decided to use the same attributes for the text loader as we had used for the prediction engine.In the prediction engine, there is no concept of ordinal. All references to columns in
IDataViewfrom a user's perspective are by name, all transforms etc. refer to columns by name, so there it was not relevant.So to answer your question: nothing, because it's not used for that.
This is, I hope you don't mind my observing, another reason why I consider deriving the text loader settings from a class as a mistake.
MgSam commentedon May 11, 2018
You could make ordinal take int as an overload, for cases when it isn't a vector.
I agree that it feels very weird declaring a class just to pass metadata along. There are a few alternatives, which you could support one or more of:
builder.Column("SepalLength", typeof(float), 1).Column("Color", typeof(string), 2))sfilipi commentedon Nov 27, 2018
@KrzysztofCwalina can this issue be closed, since the API has changed and this doesn't apply anymore?
Ivanidzo4ka commentedon Feb 6, 2019
https://github.com/dotnet/machinelearning/blob/master/src/Microsoft.ML.Data/DataLoadSave/Text/LoadColumnAttribute.cs
It's no longer accepts string, and work with integers, so I'm closing this issue
validate AutoFit 'Features' column must be of type R4 (dotnet#132)
code migration from machinelearning-automl to machinelearning repo fe…
validate AutoFit 'Features' column must be of type R4 (dotnet#132)
Added onnx export functionality for LpNormNormalizingTransformer (#4161)
Added Onnx export functionality to PCATransformer (#4188)