Skip to content

ImageLoadingTransformer hides exceptions in Mapper.MakeGetter #3154

New issue
New issue
Closed
Closed
ImageLoadingTransformer hides exceptions in Mapper.MakeGetter#3154
Labels
bugSomething isn't workingenhancementNew feature or request

Description

@mareklinka
mareklinka
opened on Apr 1, 2019

System information

  • OS version/distro: Windows 10 1809
  • .NET Version (eg., dotnet --info): 2.2.202

Issue

I recently tried to use ML.NET in a Xamarin-based UWP app. I targeted the earliest version of UWP that support .NET Standard 2.0 and everything installed correctly. My intention was to make use a separately-trained TensorFlow model to predict some data from an image.

While my code worked without issues in a .NET Core 2.2 console app, in the UWP it was failing. I was constantly getting an exception when attempting to load the target image - Image [whatever] was not found. No additional details, no inner exception.

Only after a fair amount of headscratching and trying various things I managed to find the culprit (which was obvious in hindsight): UWP does not support bitmaps. After I tried to replicate a little bit of the ImageLoader code, I got an unsupported platform exception.

This is all fair but then looking into the code some more, I found this:

machinelearning/src/Microsoft.ML.ImageAnalytics/ImageLoader.cs

Line 215 in b5a8d99

throw Host.Except($"Image {src.ToString()} was not found.");

The code is catching all exceptions and throwing a custom exception in their place, without providing the base exception. I think this could be improved as part of making the API surface friendlier to use - spending an hour on this, I started to think I was doing something insanely wrong. The inner exception would have told me the root cause in 10 seconds flat.

So there are basically two issues:

  1. Hiding the base exceptions
  2. Image analytics won't work in UWP at all because of bitmaps

I think a fix for 1. could be relatively simple. 2. will be much more difficult and might not be desirable but I wanted to throw it out there. Maybe using a platform-agnostic implementation of image handling could be useful.

Opinions? Thoughts?

Activity

  1. Ivanidzo4ka
    added
    bugSomething isn't working
    UWP
    on Apr 2, 2019

  2. Ivanidzo4ka commented on Apr 2, 2019

    @Ivanidzo4ka
    Ivanidzo4ka
    on Apr 2, 2019
    Contributor

    @eerhardt @danmosemsft Can you provide information regarding System.Drawing.Common and it's UWP support?
    If it doesn't support UWP what is best alternative for System.Drawing?

  3. eerhardt commented on Apr 2, 2019

    @eerhardt
    eerhardt
    on Apr 2, 2019
    Member

    @safern @JeremyKuhne @nattress - do you have any recommendations here on using System.Drawing on UWP?

  4. safern commented on Apr 2, 2019

    @safern
    safern
    on Apr 2, 2019
    Member

    @safern @JeremyKuhne @nattress - do you have any recommendations here on using System.Drawing on UWP?

    Not really, Drawing is just a platform not supported assembly in UWP, we basically just throw there. I don't know if there are any plans on supporting it in the near future (even if we can because of all the P/Invokes to GDI+).

  5. Ivanidzo4ka commented on Apr 2, 2019

    @Ivanidzo4ka
    Ivanidzo4ka
    on Apr 2, 2019
    Contributor

    So #1, yes it's a bug, and it need to be fixed
    #2 We will come up with plan how to enable image support in UWP. Our whole UWP story isn't great and I guess we would tackle this issue during work on UWP support.

  6. labChariot commented on Jan 10, 2020

    @labChariot
    labChariot
    on Jan 10, 2020

    I was working with the exact same setup - Xamarin-based UWP utilising a previously trained TensorFlow model - and hit the PlatformNotSupportedException for System.Drawing when calling into the PredictionEngine:

    System.PlatformNotSupportedException: 'System.Drawing is not supported on this platform.'

This exception was originally thrown at this call stack:
    System.Drawing.Bitmap.Bitmap(string)
    Microsoft.ML.Data.ImageLoadingTransformer.Mapper.MakeGetterImageDataViewType.AnonymousMethod__0(ref System.Drawing.Bitmap)
    Microsoft.ML.Transforms.Image.ImageResizingTransformer.Mapper.MakeGetter.AnonymousMethod__1(ref System.Drawing.Bitmap)
    Microsoft.ML.Transforms.Image.ImagePixelExtractingTransformer.Mapper.GetGetterCore.AnonymousMethod__1(ref Microsoft.ML.Data.VBuffer<TValue>)
    Microsoft.ML.Transforms.TensorFlowTransformer.TensorValueGetterVec<T>.GetTensor()
    Microsoft.ML.Transforms.TensorFlowTransformer.Mapper.UpdateCacheIfNeeded(long, Microsoft.ML.Transforms.TensorFlowTransformer.ITensorValueGetter[], string[], Microsoft.ML.Transforms.TensorFlowTransformer.Mapper.OutputCache)
    Microsoft.ML.Transforms.TensorFlowTransformer.Mapper.MakeGetter.AnonymousMethod__4(ref Microsoft.ML.Data.VBuffer<T>)
    Microsoft.ML.Data.TypedCursorable<TRow>.TypedRowBase.CreateDirectVBufferSetter.AnonymousMethod__0(TRow)
    Microsoft.ML.Data.TypedCursorable<TRow>.TypedRowBase.FillValues(TRow)
    Microsoft.ML.PredictionEngineBase<TSrc, TDst>.Predict(TSrc)
    ...
    [Call Stack Truncated]

    It's my first time doing anything with ML.NET. Has there been any plan formulated to bring this sort of image support to ML.NET on UWP?

  7. antoniovs1029
    added
    enhancementNew feature or request
    P2
    on Jan 10, 2020

  8. antoniovs1029 commented on Jan 10, 2020

    @antoniovs1029
    antoniovs1029
    on Jan 10, 2020
    Contributor

    Hi, @labChariot . As of now, ML.NET doesn't have any plan to provide image support on UWP.

  9. labChariot commented on Jan 12, 2020

    @labChariot
    labChariot
    on Jan 12, 2020

    Thanks @antoniovs1029

  10. brianfeucht commented on Jun 9, 2020

    @brianfeucht
    brianfeucht
    on Jun 9, 2020

    ImageSharp just released RC2. I would love an implementation of the ImagePixelExtractingTransformer which is based of ImageSharp instead of GDI

    I started to take a look at implementing something locally, but access modifiers on the base classes make that difficult. I haven't had time to look it is beyond that.

  11. XorZy commented on Aug 22, 2022

    @XorZy
    XorZy
    on Aug 22, 2022

    Is there any progress on migrating from System.Drawing to ImageSharp?
    I saw it was mentioned on the Roadmap but it's not really clear if it's still being worked on.
    It's a very desirable change since NET7 will not support System.Drawing at all (even with a configuration switch) on any platform other than Windows.

  12. dakersnar commented on Aug 25, 2022

    @dakersnar
    dakersnar
    on Aug 25, 2022
    Contributor

    cc @luisquintanilla

  13. mapo80 commented on Oct 2, 2022

    @mapo80
    mapo80
    on Oct 2, 2022

    Same issue here. Any update?

  14. luisquintanilla commented on Oct 11, 2022

    @luisquintanilla
    luisquintanilla
    on Oct 11, 2022
    Contributor

    There is a PR #6363 that should address this issue.

  15. rosebyte commented on Sep 7, 2026

    @rosebyte
    rosebyte
    on Sep 7, 2026
    Member

    #6363 replaced the System.Drawing implementation with MLImage, and ImageLoadingTransformer no longer hides exceptions through the catch-and-rethrow path reported here.

  16. rosebyte
    closed this as completedon Sep 7, 2026
  17. github-actions
    locked and limited conversation to collaborators on Oct 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingenhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions