Proposal: Zero allocation connectionless sockets

This proposal eliminates allocations for connectionless use of Socket. It augments the SocketAddress class to allow reuse across operations, becoming a high-perf alternative to EndPoint.

Rationale and Usage

APIs which need to translate between IPEndPoint and native sockaddr structures are performing a large amount of defensive copying and layering workarounds.

This affects UDP performance and contributes to excessive GC. A simple example is:

Socket socket = ...;
byte[] buffer = ...;
var remoteEndPoint = new IPEndPoint(IPAddress.Any, 0);

socket.ReceiveFrom(buffer, 0, buffer.Length, SocketFlags.None, ref remoteEndPoint);
socket.SendTo(buffer, 0, buffer.Length, SocketFlags.None, remoteEndPoint);

These two calls allocate 12 times:

3x IPEndPoint
3x IPAddress
3x SocketAddress
3x byte[] ...6x if IPv6

Proposed API

class Socket
{
	public int ReceiveFrom(Span<byte> buffer, SocketFlags socketFlags, SocketAddress socketAddress);
	public ValueTask<int> ReceiveFromAsync(Memory<byte> buffer, SocketFlags socketFlags, SocketAddress socketAddress, CancellationToken cancellationToken = default);

	public int SendTo(ReadOnlySpan<byte> buffer, SocketFlags socketFlags, SocketAddress socketAddress);
	public ValueTask<int> SendToAsync(ReadOnlyMemory<byte> buffer, SocketFlags socketFlags, SocketAddress socketAddress, CancellationToken cancellationToken = default);
}

class SocketAsyncEventArgs
{
	// Only one of RemoteEndPoint or RemoteAddress must be specified.
	public SocketAddress RemoteAddress { get; set; }
}

class SocketAddress
{
	// If we can merge System.Net.Primitives and System.Net.Sockets, these two methods are unnecessary. That would be ideal.
	public static void GetBuffer(SocketAddress address, out byte[] buffer);
	public static void SetSockaddrSize(SocketAddress address, int size);
}

class EndPoint
{
	public virtual void SerializeTo(SocketAddress socketAddress); // Already has "SocketAddress Serialize()"; default would be to call that and copy.
}

Details

It is intended that UDP servers will use SocketAddress as a dictionary key to lookup client state, to avoid first converting to EndPoint.
- It is assumed that users will only rarely care to actually get the IP/port/etc. from the SocketAddress. This duty continues to be delegated to EndPoint.
- If users ever want to deserialize a SocketAddress into an EndPoint, they can already use EndPoint.Create.
- Need to ensure only the actual sockaddr structure is compared/hashed, not the entire byte buffer.
SocketAddress is currently duplicated in System.Net.Primitives and System.Net.Sockets to avoid exposing its internal buffer. This change will allow avoiding duplication.
This relies on users not using a SocketAddress until the I/O is finished.
- This is a bit safer with EndPoint as we can take defensive copies before methods return.
We can currently do some optimizations to avoid all allocations for SendTo and SendToAsync IPv4/IPv6 with some special casing, so this API would primarily be to optimize ReceiveFrom variants as well as (less important) allowing non-IPv4/IPv6 protocols to benefit. Still, if we were to add an API for ReceiveFrom we would probably want an API on SendTo for symmetry.

Open Questions

We've put some effort into not doing something like this before. It would be great to understand why. Currently:
- We duplicate the SocketAddress class in multiple assemblies to avoid exposing its buffer, and have a step to marshal (byte-by-byte) between the two implementations.
- Tons of APIs take EndPoint, it's a nice abstraction that we wanted here despite performance implications.
It isn't immediately obvious from the API surface that socketAddress is written to by ReceiveFrom. Is there a better way we can indicate this?
The two new methods on SocketAddress exist purely because System.Net.Sockets needs access to internals in System.Net.Primitives. Any thoughts on how to avoid exposing these "pubternal" bits?
- One option is to merge System.Net.Primitives and System.Net.Sockets; I don't see harm in this but that is a much larger discussion :)
If we merge the Primitives and Sockets assemblies, we can get rid of some of the allocations for ReceiveFrom without making any API changes. It's not a perfect solution but might be good enough.

Related Issues

There are two additional issues to update our APIs with ValueTask/Span/Memory that this will need to be consistent with:

Update Socket class add Span overloads for Socket datagram functions #938
Update UdpClient class UdpClient - add Span support #864

If the core issue is repeated conversions from EndPoint to SocketAddress, why not cache it on the EndPoint instead?

And aren't some of the allocations you mention a side-effect of what used to be one implementation being split between multiple assemblies? If that split is resulting in a perf impact, we should consider moving types into the same assembly to avoid the overhead, rather than adding new APIs.

Also, we separately want ValueTask/Memory-based APIs for UDP. Are you proposing that these be them? Would we not also have EndPoint-based APIs?

This feels to me like it's breaking an abstraction that shouldn't be broken, and we should have exhausted every other avenue before doing so.

If the core issue is repeated conversions from EndPoint to SocketAddress, why not cache it on the EndPoint instead?

Yes, I considered this option. We've been careful to make defensive copies when using EndPoint, which indicates to me that this could be a subtly breaking behavior change if sendto was suddenly not doing this. Also, given that EndPoint has Serialize and Create to enable EndPoint / sockaddr translation, it looks like a decision was made to explicitly not bake things into a single class -- I'd love some background knowledge on this if you've got it.

And aren't some of the allocations you mention a side-effect of what used to be one implementation being split between multiple assemblies? If that split is resulting in a perf impact, we should consider moving types into the same assembly to avoid the overhead, rather than adding new APIs.

Agreed, this is one of my open questions. I don't really see a downside to merging at least System.Net.Primitives and System.Net.Sockets, but I don't have the background on why they were split in the first place. If you have any knowledge here it would be helpful. Doing so would help us cut out many of these allocations.

Also, we separately want ValueTask/Memory-based APIs for UDP. Are you proposing that these be them?

No, that is not part of this proposal. We should of course make both proposals consistent if it moves forward.

This feels to me like it's breaking an abstraction that shouldn't be broken, and we should have exhausted every other avenue before doing so.

Agreed, this is one of my open questions. Hoping to see some collaboration on those potential avenues.

span-based UDP using EndPoint add Span overloads for Socket datagram functions #938.
span-based UdpClient UdpClient - add Span support #864

User may not care about port but you alway have to specify one for sending.

socket.SendTo(buffer, 0, buffer.Length, SocketFlags.None, remoteAddress);

will not work unless you use Connect() before. And when you do, you do not need the remoteAddress/remoteEndPoint at all. Did I miss something?

@stephentoub

If you have any knowledge here it would be helpful.

I think there are two historical things going on here. @stephentoub may have more context.

(1) These APIs were originally designed a long time ago and at the time, minimizing allocations was not a major priority, unfortunately.
(2) During the assembly refactoring for .NET Core, some classes that were originally all in the same assembly got moved into separate assemblies. In some cases this has led to unfortunate code duplication and/or inefficiencies introduced to avoid private access to certain classes.

We can and should fix these issues. In other words, there's some bad legacy here we can improve on, I think.

It seems to me that we need to consider the send case and the receive case separately here.

On the send side, there's no reason we need to do any copies at all. All we need to do is produce the native sockaddr bytes when we do the native call. Currently, that means we need to call IPEndPoint.Serialize which will allocate a SocketAddress etc. But if we just had something like IPEndPoint.Serialize(Span sockAddrBuffer) then I think we could avoid allocation in this path entirely without changing the top-level API. I'm glossing over details here so please correct me if this is more complicated than I'm describing

On the receive side, we ultimately need to return the endpoint info somehow. In your proposal we're mutating an existing SocketAddress, so you are correct that this API does not allocate; but presumably every user of this API will convert the SocketAddress into an IPEndPoint, which will cause allocation at that point, right? I'm not sure what to do about this, but if the goal is truly zero allocation then we need to consider the common usage, not just the API itself.

@wfurt

User may not care about port but you alway have to specify one for sending.

@wfurt SocketAddress is an entire sockaddr_in/etc. structure, it includes a port.

@geoffkizer

presumably every user of this API will convert the SocketAddress into an IPEndPoint, which will cause allocation at that point, right?

@geoffkizer I don't think this needs to be a common use. Really you'd use the address to lookup a client's state in a dictionary -- you don't need an EndPoint to do this, but can instead use SocketAddress directly to avoid any conversions. Outside of this, the main reason I can think of to want ip/port is for display purposes... in which case, yes, you'd need to convert to IPEndPoint to get.

ok. that would make sense. I look at .Net API and I did not see port there. Would you envision adding Port property to it as well?

https://docs.microsoft.com/en-us/dotnet/api/system.net.socketaddress?view=netcore-2.2

Would you envision adding Port property to it as well?

Note it doesn't have an IP property either.

My vision is to keep SocketAddress opaque, and force users to use EndPoint.Serialize and EndPoint.Create to shuffle to/from EndPoint if they need to manipulate the IP/port. My expectation is that this would not be done once per op, but rather once per endpoint at the start of a "connection", so it's not needed.

@geoffkizer

if we just had something like IPEndPoint.Serialize(Span sockAddrBuffer)

@geoffkizer yea, I thought of that. I think keeping the APIs symmetrical is worthwhile, though. Plus, there's some (probably ultimately inconsequential, but...) benefit to avoiding a SerializeTo(span) overhead on every call to SendTo. Less important now if we end up taking another direction with QUIC, but that's a fair amount of redundant calls for such a packet-heavy thing.

A couple related questions that I don't quite understand:

(1) Why does SocketAddress even exist today? Are there any APIs that take it? Is it just so a user can get the bytes for the native sockaddr structure? When would they even need to do this?
(2) Why is the IPEndPoint on ReceiveFrom passed by ref, instead of out?

Really you'd use the address to lookup a client's state in a dictionary -- you don't need an EndPoint to do this, but can instead use SocketAddress directly to avoid any conversions.

Yeah, that's a good point.

I think keeping the APIs symmetrical is worthwhile, though.

I agree but I'm not quite sure the best way to do this.

(1) Why does SocketAddress even exist today? Are there any APIs that take it? Is it just so a user can get the bytes for the native sockaddr structure? When would they even need to do this?

I think it is to allow custom implementations of EndPoint to serialize random non-IP sockaddr without 1st party support.

Thats what I meant, using a native wrapper is of course not an ideal solution, needing to compile the native dll for each platform separately is very annoying. But for the performance it is worth it at the moment.

Well, I misunderstood you, and I apologize for that.

https://github.com/enclave-networks/Enclave.FastPacket
https://github.com/JohannesDeml/NetworkBenchmarkDotNet
https://github.com/enclave-networks/research.udp-perf

here is some repositories maybe little help

@antonfirsov

I have PoC for the

class Socket
{
	public int ReceiveFrom(Span<byte> buffer, SocketFlags socketFlags, SocketAddress socketAddress);
	public ValueTask<int> ReceiveFromAsync(Memory<byte> buffer, SocketFlags socketFlags, SocketAddress socketAddress, CancellationToken cancellationToken = default);

	public int SendTo(ReadOnlySpan<byte> buffer, SocketFlags socketFlags, SocketAddress socketAddress);
	public ValueTask<int> SendToAsync(ReadOnlyMemory<byte> buffer, SocketFlags socketFlags, SocketAddress socketAddress, CancellationToken cancellationToken = default);
}

The socketAddress is updated in place - assuming it was created for correct address family and has sufficient capacity. If not, the call would fail e.g. there is no attempt to resize the underlying buffer. And of course you cannot pass same instance to multiple calls.

From the conversation above: for the DNS server:
https://github.com/TurnerSoftware/DinoDNS/blob/main/src/TurnerSoftware.DinoDNS/Connection/Listeners/UdpQueryListener.cs
One can reduce allocations significantly by using the new API directly and allocating SocketAddress inside of the receive loop. It cannot be above the loop as the code does not await HandleRequestAsync. Probably best option would be to make it part of the transitData. Aside from that the SocketAddress is used as opaque object so it should just work.

The Mirroring is somewhat more complicated: https://github.com/MirrorNetworking/Mirror/blob/d226d577c21e557ae09086e0ddc1da63d2704e38/Assets/Mirror/Transports/KCP/kcp2k/highlevel/KcpServer.cs#L149

If the RawReceiveFrom is not called concurrently, one can simply create global SocketAddress for IPv6 - it is ok to pass bigger buffer - my assumption is that the receive call would set SocketAddress.Size to reflect the actually received amount e.g. v4 vs v6 or file length for UDS. The connectionID seems to be simple GetHashCode. That should work on SocketAddress as well - with the caveat that of course different endpoint can possibly produce same hash code. (but that problem already exists) #86872 would make it easier to do extra validation but it feels like it is not necessary strictly speaking.

At the moment I don't have the originally proposed change for SocketAsyncEventArgs. It is not too difficult but for now I did not do it to keep the scope as small as possible. The work is mostly around validation as the field is already used internally and we would need to define and verify behavior for cases when somebody would also specify RemoteEndPoint .

At least for projects discussed above this should be sufficient. If this is good enough for majority here I can drive it for API review and push it to 8.0 before the gates close.

I opened separate #86872 to provide some more convenient methods around SocketAddress but the goal goes beyond UDP. @antonfirsov opened #78993 and that should make it easier for anybody who want's to interact with (IP)EndPoint as some point. While both are somewhat related, we may or may not get them into 8.0. As the work is not excessive I'd be happy to tentatively push both to 8.0 and ditch them if we run out of time.

Lastly #86513 talks about cost of pinning. We had case when the GCHandles fragmented memory and caused OOM. Hopefully that should not be problem for the synchronous version but we talk with @stephentoub about possibility to use NativeMemory in some cases to remove work from GC.

@wfurt

@wfurt
The API you proposed looks really nice, changing socket address in place is a simple solution and solves the allocation problems.

@wfurt

@wfurt That's amazing. This will elevate C# to a whole new class of use cases.
Super excited about this, and it's great to see that C# devs care so much.

什么时候可以完成这项0GC的工作啊？？？期待

[EDIT]
Bing translation:
when will this 0GC job be done??? expect

~~it seems blocked~~

#87397
#86872

~~i think that means new api won't be release with .net 8.0~~

Note that the label is blocking, not "blocked". This means that the issue is high-priority (blocking the release) which pushes it to the top of the API review backlog.

Note that the label is , not "blocked". This means that the issue is high-priority (blocking the release) which pushes it to the top of the API review backlog.blocking

👍 great.

This was long journey but the end is near. #87397 added sync & async overloads with SocketAddress. It is possible now to pass it to SendTo & ReceiveFrom in and avoid allocations. Same object will be updated in place e.g. it cannot be shared by multiple threads. On receive, the Size is updated to reflect valid bytes e.g. one will need to slice the Buffer to get valid address. #89808 should help on Windows with avoiding GCHandle & pinning. #90086 also reduce allocation for legacy SendTo & Receive but there is still room for improvements in 9.0.

What missing is direct support for SocketAsyncEventArgs. There did not seems to be any compelling reasons besides completeness. And #86872 did not get the helper functions approved. If anybody needs API to extract useful data from the SocketAddress please comment here or on #86872.

All this should be available ion RC1 build -> just barely made it in for 8.0.
Since changes above should fix most use cases I'm going to close this issue. It would be great if people can pick up RC1 and experiment as everything got in pretty late.

Thank you, it's appreciated.

Should I use new SocketAddress(AddressFamily.InterNetworkV6); to ensure the internal buffer is large enough for a ReceiveFrom that could receive both V4 and V6?

yes, it should have at least what the given socket can receive. Or more generically

Socket socket = new Socket(SocketType.Dgram, ProtocolType.Udp)
SocketAddress socketAddress = new SocketAddress(socket.Address.Family);

it will default to DualMode when it can and fall-back to IPv4 if needed.
One more note that the the old ReceiveFrom would do work to map IPv4 to IPv6V4Mapped.
The new API leaves the buffer exactly as given by OS. e.g. if you do EndPoint.Create(socketAddress) you may end up with different address and if you care you need to do the mapping yourself.

Navigation Menu

Repository navigation

Proposal: Zero allocation connectionless sockets #30797

Rationale and Usage

Proposed API

Details

Open Questions

Related Issues

79 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Proposal: Zero allocation connectionless sockets #30797

Description

Rationale and Usage

Proposed API

Details

Open Questions

Related Issues

Activity

stephentoub commented on Sep 8, 2019

scalablecory commented on Sep 9, 2019

scalablecory commented on Sep 9, 2019

wfurt commented on Sep 10, 2019

geoffkizer commented on Sep 10, 2019

geoffkizer commented on Sep 10, 2019

scalablecory commented on Sep 10, 2019

scalablecory commented on Sep 10, 2019

wfurt commented on Sep 10, 2019

wfurt commented on Sep 10, 2019

scalablecory commented on Sep 10, 2019

scalablecory commented on Sep 10, 2019

geoffkizer commented on Sep 10, 2019

geoffkizer commented on Sep 10, 2019

scalablecory commented on Sep 10, 2019

79 remaining items

sgf commented on Mar 15, 2023

sgf commented on Mar 15, 2023

wfurt commented on May 30, 2023

FakeByte commented on May 30, 2023

miwarnec commented on May 30, 2023

qulingbianji commented on Jun 2, 2023

sgf commented on Jul 13, 2023

antonfirsov commented on Jul 13, 2023

sgf commented on Jul 14, 2023

wfurt commented on Aug 9, 2023

macaba commented on Aug 9, 2023

wfurt commented on Aug 9, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions