.NET 11 RC1: process crash from unhandled ObjectDisposedException in SetHandleBlocking after ConnectAsync (Linux) #135681

Description

After moving a set of ASP.NET Core services to .NET 11 RC1 (11.0.0-rc.1.26425.128, Linux x64 containers), processes crash at random with an unhandled ObjectDisposedException on a thread-pool thread. No application code is on the stack, so the exception can't be caught and the process exits.

This looks like the same bug as #126183. That issue is tracked as a CI known-build-error with milestone 12.0.0. We're filing this one to show the bug crashes production workloads on RC1, and to ask for a fix in 11.0 GA.

The two stacks we see:

Unhandled exception. System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'System.Net.Sockets.SafeSocketHandle'.
   at Interop.Sys.Fcntl.SetIsNonBlocking(SafeHandle fd, Int32 isNonBlocking)
   at System.Net.Sockets.SocketAsyncContext.SetHandleBlocking()
   at System.Net.Sockets.SocketAsyncContext.ConnectOperation.InvokeCallback(Boolean allowPooling)
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
   at System.Threading.Thread.StartCallback(Thread* pThread)

Unhandled exception. System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'System.Net.Sockets.SafeSocketHandle'.
   at System.Net.Sockets.SocketAsyncContext.SetHandleBlocking()
   at System.Net.Sockets.SocketAsyncContext.WriteOperation.System.Threading.IThreadPoolWorkItem.Execute()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
   at System.Threading.Thread.StartCallback(Thread* pThread)

Impact

• It hits every kind of process we run (web APIs, web front ends, background workers), at several crashes per hour per environment.
• The same workloads ran with no crashes on .NET 10. The crashes started within minutes of deploying the RC1 build.
• Mostly idle processes crash about as often as busy ones, which points at background connection activity rather than request load.

Analysis

This code path is new in .NET 11, from #124200 ("Restore blocking mode after successful ConnectAsync on Unix"). The .NET 10 SocketAsyncContext.Unix.cs has no SetHandleBlocking.

In RC1, ConnectOperation.InvokeCallback calls SafeSocketHandle.SetBlocking():

if (!IsNonBlocking && !IsClosed)
{
    AsyncContext.SetHandleBlocking();   // -> Interop.Sys.Fcntl.SetIsNonBlocking(_socket, 0)
}

The !IsClosed check and the P/Invoke (whose marshaller calls DangerousAddRef on the handle) aren't atomic. If another thread disposes the Socket between the two, DangerousAddRef throws ObjectDisposedException. The throw happens inside a completion callback on a thread-pool thread, so it's unhandled and terminates the process. The ec != SocketError.Success branch (a connect that failed or was aborted) also calls SetBlocking(), so a socket disposed while its connect is in flight is the obvious trigger.

Disposing a socket while ConnectAsync is pending is ordinary client behavior. Any client library with a connect timeout or reconnect logic does it, and it was safe on .NET 10. Our crashes line up with a long-lived connection-pooling client (a TLS connection to a remote cache) whose reconnect logic can dispose a still-connecting socket from another thread.

As noted on #126183, SetHandleBlocking should be best-effort and never throw out of a completion callback. For example, it could use DangerousAddRef(ref success) and skip when the handle is already gone, or catch ObjectDisposedException there.

Workarounds

• There's no AppContext switch or environment variable to opt out.
• Setting Socket.Blocking = false before connecting skips the path, because IsNonBlocking is then true. It doesn't help when the socket is later wrapped in a NetworkStream (for example for TLS): the NetworkStream constructor throws "The operation is not allowed on a non-blocking Socket".
• The only workaround we've found is staying on the .NET 10 runtime.

Reproduction

We don't have a deterministic standalone repro yet. A tight loop that disposes sockets while ConnectAsync is pending (millions of iterations, both loopback and container-to-container) didn't hit the window. Loopback connects complete synchronously, and the gap between IsClosed and DangerousAddRef is only a few instructions. Real network latency hits it several times an hour. We're happy to run diagnostics or a private build.

Configuration

• Runtime: 11.0.0-rc.1.26425.128 (official aspnet:11.0.0-rc.1 container image)
• OS: Linux x64 containers

Regression?

Yes. It doesn't happen on .NET 10, and it was introduced by #124200.