Repository navigation
MsQuic fails with QUIC_STATUS_OUT_OF_MEMORY on AzureLinux #123216
Copy link
Copy link
Closed
Labels
Known Build ErrorUse this to report build issues in the .NET Helix tabUse this to report build issues in the .NET Helix tabarea-System.Net.Quicblocking-clean-ci-optionalBlocking optional rolling runsBlocking optional rolling runsdisabled-testThe test is disabled in source code against the issueThe test is disabled in source code against the issuetracking-external-issueThe issue is caused by external problem (e.g. OS) - nothing we can do to fix it directlyThe issue is caused by external problem (e.g. OS) - nothing we can do to fix it directly
Milestone
Description
Build Information
Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=1254154
Build error leg or test failing: System.Net.Http.Functional.Tests.Http3ResponseStreamZeroByteReadTest.ZeroByteRead_BlocksUntilDataIsAvailable
Pull request: #123091
Error Message
Fill the error message using step by step known issues guidance.
{
"ErrorMessage": "System.Net.Quic.QuicException : An internal error has occurred. RegistrationOpen failed: QUIC_STATUS_OUT_OF_MEMORY",
"ErrorPattern": "",
"BuildRetry": false,
"ExcludeConsoleLog": false
} at System.Net.Quic.ThrowHelper.ThrowMsQuicException(Int32 status, String message) in /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/ThrowHelper.cs:line 144
at System.Net.Quic.MsQuicApi..ctor(QUIC_API_TABLE* apiTable) in /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/MsQuicApi.cs:line 50
at System.Net.Quic.MsQuicApi.AllocateMsQuicApi() in /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/MsQuicApi.cs:line 235
at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode) in /_/src/libraries/System.Private.CoreLib/src/System/Lazy.cs:line 321
--- End of stack trace from previous location ---
at System.Lazy`1.CreateValue() in /_/src/libraries/System.Private.CoreLib/src/System/Lazy.cs:line 429
at System.Net.Quic.MsQuicApi.get_Api() in /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/MsQuicApi.cs:line 57
at System.Net.Quic.QuicListener..ctor(QuicListenerOptions options) in /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicListener.cs:line 127
at System.Net.Quic.QuicListener.ListenAsync(QuicListenerOptions options, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicListener.cs:line 63
at System.Net.Test.Common.Http3LoopbackServer..ctor(Http3Options options) in /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackServer.cs:line 57
at System.Net.Test.Common.Http3LoopbackServerFactory.CreateServer(GenericLoopbackOptions options) in /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackServer.cs:line 109
at System.Net.Test.Common.Http3LoopbackServerFactory.CreateServerAsync(Func`3 funcAsync, Int32 millisecondsTimeout, GenericLoopbackOptions options) in /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackServer.cs:line 114
at System.Net.Http.Functional.Tests.ResponseStreamZeroByteReadTestBase.ZeroByteRead_BlocksUntilDataIsAvailable(Boolean async) in /_/src/libraries/System.Net.Http/tests/FunctionalTests/ResponseStreamZeroByteReadTests.cs:line 204
--- End of stack trace from previous location ---
Known issue validation
Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=1254154
Error message validated: [System.Net.Quic.QuicException : An internal error has occurred. RegistrationOpen failed: QUIC_STATUS_OUT_OF_MEMORY]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 1/15/2026 2:21:33 PM UTC
Report
Summary
| 24-Hour Hit Count | 7-Day Hit Count | 1-Month Count |
|---|---|---|
| 0 | 26 | 101 |
Metadata
Metadata
Assignees
Labels
Known Build ErrorUse this to report build issues in the .NET Helix tabUse this to report build issues in the .NET Helix tabarea-System.Net.Quicblocking-clean-ci-optionalBlocking optional rolling runsBlocking optional rolling runsdisabled-testThe test is disabled in source code against the issueThe test is disabled in source code against the issuetracking-external-issueThe issue is caused by external problem (e.g. OS) - nothing we can do to fix it directlyThe issue is caused by external problem (e.g. OS) - nothing we can do to fix it directly
Activity
dotnet-policy-service commentedon Jan 15, 2026
Tagging subscribers to this area: @karelz, @dotnet/ncl
See info in area-owners.md if you want to be subscribed.
ManickaP commentedon Jan 15, 2026
This is once again problem with AzL3. Downgrade of SymCrypt-OpenSSL from 1.9.4 to 1.9.3 will help. We need to update the helix docker images to pin that package to 1.9.3 for the time being.
I have internally contacted responsible people from AzL3, so it's in the process.
ManickaP commentedon Jan 20, 2026
This has stopped occurring since dotnet/dotnet-buildtools-prereqs-docker#1576.
We need to remove the downgrade in the docker images once it gets fixed in SymCrypt, so keeping this open.
KQL query:
jakobbotsch commentedon Jan 31, 2026
Happened in https://dev.azure.com/dnceng-public/public/_build/results?buildId=1274864&view=ms.vss-test-web.build-test-results-tab&runId=35575508&resultId=148567&paneView=debug (PR #123839)
15 remaining items
[QUIC] Disable platform tests as well (#125772)
[QUIC] Disable platform tests as well (#125772)
vcsjones commentedon Mar 23, 2026
Does this mean we can unpin the SCOSSL in the docker image now since the tests are disabled? SCOSSL has fixes in the latest version that are blocking #118807.
ManickaP commentedon Mar 23, 2026
I disabled this only when ran directly on AzL3 VM, not in docker, so that we don't completely lose the coverage.
vcsjones commentedon Mar 26, 2026
@ManickaP Do we have any progress or understanding of a resolution? Holding back SymCrypt-OpenSSL is preventing some work in System.Security from getting merged because SCOSSL 1.9.4+ have fixes we need for a clean CI run.
ManickaP commentedon Mar 27, 2026
It's not held back on AzureLinux3 VM, just in the docker images. If we disable our tests in docker as well, we're gonna lose majority of our test coverage, but if you think it's an acceptable trade-off, I can do it.
And just for an FYI, ETA for the SymCrypt (1.9.5) fix is next week AFAIK.
Disable HTTP/3 tests on AzureLinux 3 VMs due to QUIC OOM
Disable HTTP/3 tests on AzureLinux 3 VMs due to QUIC OOM (#126605)
Disable HTTP/3 tests on AzureLinux 3 VMs due to QUIC OOM (dotnet#126605)
xnox commentedon Apr 9, 2026
@ManickaP I see that https://github.com/microsoft/azurelinux/blob/3.0/SPECS/SymCrypt-OpenSSL/SymCrypt-OpenSSL.spec is now at 1.9.5 as merged two weeks ago microsoft/azurelinux#16139
Is it time to attempt to upgrade to 1.9.5 and see if it sticks?
[QUIC] Remove check for AzL3 for QUIC and H/3 tests (#126834)
[QUIC] Remove check for AzL3 for QUIC and H/3 tests (#126834)