Add FieldAwareFactorizationMachine to AutoML #3985
Description
FieldAwareFactorizationMachine is good for large dataset like the Criteo 1TB dataset.
Currently FieldAwareFactorizationMachine is not swept over in AutoML.
Task:
- Add trainer to default list of binary learners to try
- Add sweep range
- Add to CLI's C# CodeGen
Should be easy to just replicate an existing trainer like SDCA:
Activity
zHaytam commentedon Aug 25, 2019
Hi, I would like to take this issue if possible.
justinormont commentedon Aug 25, 2019
Sounds great.
This will be a change to the master branch as the AutoML API code has been merged into master. The CLI's CodeGen (which creates the C# project) still lives in the AutoML feature branch.
Perhaps @XiaroanZhang can create the CodeGen part, or walk you though what's needed. @daholste may also be able to offer advice. I'll be out for the next week, but feel free to submit a PR and have folks approve/merge in.
Thanks so much.
gokart23 commentedon Oct 6, 2019
Hi @justinormont, looks like this issue is still open - do you mind if I work on it?
mstfbl commentedon Jan 9, 2020
Hi @gokart23 and @zHaytam , if you guys are still interested, go for it!
Sandy4321 commentedon Jan 26, 2020
if somebody tried to run locally ?
like
https://github.com/rambler-digital-solutions/criteo-1tb-benchmark
justinormont commentedon May 10, 2021
Related: #590
The related issue will allow
FieldAwareFactorizationMachineto also be added to the multi-class AutoML task. CurrentlyFieldAwareFactorizationMachineis only for binary classification due to not implementing the needed interfaces for OVA (one-vs-all / one-vs-rest).@Sandy4321 : No one has run that specific set of benchmarks, which sweep over the training dataset size and records {AUC, log-loss, training time, max memory, cpu load}. I would be very interested in seeing the results if someone would like to run it against various ML․NET trainers.
I have run the Criteo 1TB dataset extensively within ML․NET. Beyond the single trainers, it does run successfully in the AutoML code as all steps in the process are resilient to datasets well beyond memory size. Streamable trainers will succeed on small machines, and other memory bound trainers should also succeed on very large memory machines.
Disabling caching and non-streamable trainers will help small machines succeed for the full Criteo 1TB dataset. Subsampling to reduce the training dataset size, as the benchmark shows, is the other main route to reduce memory usage.