Repository navigation
Resource contention in tests DownloadImageSet #7393
Description
Multiple tests can run concurrently and try to download this resource to the same folder.
Build Information
Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=954239
Build error leg or test failing: Microsoft.ML.AutoML.Test.AutoFeaturizerTests.AutoFeaturizer_image_test
Pull request: #7388
Error Message
Fill the error message using step by step known issues guidance.
{
"ErrorMessage": "flower_photos_tiny_set_for_unit_tests.zip' because it is being used by another process",
"ErrorPattern": "",
"BuildRetry": false,
"ExcludeConsoleLog": true
}Known issue validation
Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=954239
Error message validated: [flower_photos_tiny_set_for_unit_tests.zip' because it is being used by another process]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 2/20/2025 7:12:33 PM UTC
Report
| Build | Repository | Test | Pull Request |
|---|---|---|---|
| 1651045 | dotnet/machinelearning | Microsoft.ML.AutoML.Test.AutoFitTests.AutoFitMultiClassification_Image_TrainTest | #7863 |
| 1647380 | dotnet/machinelearning | Microsoft.ML.AutoML.Test.AutoFitTests.AutoFitMultiClassification_Image_TrainTest | #7857 |
| 1632367 | dotnet/machinelearning | Microsoft.ML.AutoML.Test.AutoFitTests.AutoFitMultiClassification_Image_TrainTest | #7762 |
Summary
| 24-Hour Hit Count | 7-Day Hit Count | 1-Month Count |
|---|---|---|
| 0 | 0 | 3 |
Activity
LittleLittleCloud commentedon Feb 20, 2025
Should the test dataset be downloaded to its own folder for every tests
michaelgsharp commentedon Mar 10, 2025
So this file is used for TensorFlow testing and AutoML Testing. TensorFlow runs sequentially, so this issue should only happen during the AutoML tests.
I don't think we should run the AutoML tests serially (unless we really need to). I would either have the process check if the file is already downloaded and if it is then ignore it, or it could go to its own folder, or depending on how you have structured the tests we could move all tests that need this file to a single test class and we could have just that one test class run sequentially. I think that third option is my preferred approach, but what are your thoughts @LittleLittleCloud?
rosebyte commentedon Oct 23, 2026
Triage: the test is still failing, let's fix it.
svick commentedon Nov 2, 2026
There appears to be another race involving the generated CSV, separate from the ZIP download.
AutoFitMultiClassification_Image_TrainTestandAutoFeaturizer_image_testboth callGetFlowersDataset(), which truncates and rewrites the sameflowersfile.CI logs showed these tests overlapping, with the first failing during column inference:
A concurrent rewrite could explain an empty read:
CreateFromHeaddivides by the sample buffer length without handling zero. The exact interleaving hasn't been reproduced locally, but fixing only ZIP contention would leave the shared CSV unprotected.Note
This comment was written by Copilot.