LightGBM.Fit() exits with 139 on osx-arm64

System Information:

OS & Version: MacOS (M3) Sonoma 14.2.1
ML.NET Version: ML.NET 3.0.1
.NET Version: 7.0
ML.LightGbm 3.0.1
LightGBM 4.2.0

Describe the bug
ML.LightGBM by default does not include a binary for osx-arm64. To circumvent this we manually built a lib_lightgbm.dylib binary for arm and copied it to /bin/Debug/net7.0/runtimes/osx-arm64/native. This worked for ML.NET 2.0.0 and ML.LightGBM 2.0.0. However, after upgrading to the above versions and manually adding the lib_lightgbm.dylib of LightGBM 4.2.0, calling BinaryClassificationModel.Trainers.LightGbm.Fit() exits with exit code 139 without any error message.

To Reproduce
Steps to reproduce the behavior:

Add lib_lightgbm.dylib of LightGBM 4.2.0 to /bin/Debug/net7.0/runtimes/osx-arm64/native
Create a BinaryClassificationModel.Trainers.LightGbm
Call Fit() with some data

Expected behavior
I would expect the LightGBM model to simply train on the data.

@michaelgsharp

Perhaps this has to do with the version of LightGBM codebase you are building? @michaelgsharp updated our version and had to react to a breaking change in #6880.

If you were using an earlier version (that worked with 2.0.0 as you suggest) then that would be broken with 3.0.0. Could you update your codebase and see if that fixes things?

@ericstj

@ericstj as mentioned in my initial comment, it breaks when using the latest versions:

ML.NET Version: ML.NET 3.0.1
.NET Version: 7.0
ML.LightGbm 3.0.1
Manually adding the binary from the LightGBM 4.2.0 codebase

So I am not sure what more I can update in my codebase?

It's possible the latest version or LightGBM codebase will not work - it definitely wouldn't have worked with 2.0.0 due to the breaking changes we already had to adjust to - we haven't yet tested out the latest version of the LightGBM codebase and it may have more breaking changes. We're currently on 3.3.5

machinelearning/eng/Versions.props

Line 38 in 54fa44f

. If this method worked for you before, you might try building from their 3.3.5 tag and see if that works.

@ericstj

@ericstj I set the versions as follows:

<PackageReference Include="Microsoft.ML" Version="3.0.1"/>
<PackageReference Include="LightGBM" Version="3.3.5"/>

I built 3.3.5 lib_lightgbm.dylib from source and include it in bin.

Now LightGBM().Fit() gives "[LightGBM] [Fatal] Unknown importance type: only support split=0 and gain=1" and exits with 138.

Can you double check your reference to Microsoft.ML.LightGBM to ensure it's also 3.0.1? I see from previous issues this might be related to using an 3.x LightGBM package with older Microsoft.ML.LightGBM:
#5447

If it's possible to share your simplified repro - or even just the project.assets.json we might be able to spot any other version discrepencies.

@ericstj

Setting Microsoft.ML.LightGBM explicitly to 3.0.1 fixed the issue:

<PackageReference Include="Microsoft.ML" Version="3.0.1"/>
<PackageReference Include="LightGBM" Version="3.3.5"/>
<PackageReference Include="Microsoft.ML.LightGbm" Version="3.0.1" />

Thank you a lot for your help @ericstj.

Thank you for sticking with me to find the problem. I think the fact that 4.2.0 created problems might mean that we have some breaking changes to react to in LightGBM (assuming you were using the latest ML.LightGBM package in that case). We'll get to those when updating LightGBM next.

Navigation Menu

Repository navigation

LightGBM.Fit() exits with 139 on osx-arm64 #6966

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

LightGBM.Fit() exits with 139 on osx-arm64 #6966

Description

Activity

ericstj commented on Jan 22, 2024

gijs-koehorst commented on Jan 23, 2024

ericstj commented on Jan 24, 2024

gijs-koehorst commented on Jan 24, 2024

ericstj commented on Jan 24, 2024

gijs-koehorst commented on Jan 25, 2024

ericstj commented on Jan 25, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions