Repository navigation
Add global flag for enabling asynchronous DNS resolution #49394
Description
What is the problem this feature will solve?
Most Node libraries end up using dns.lookup which in turn uses blocking getaddrinfo from libc, this is discussed in detail here
This problem is particularly pronounced when making lots of outbound HTTP requests, it seems Node maxes out at around 20 requests per second being busy performing DNS lookups most of the time.
Patching applications to use asynchronous resolver requires in some cases extensive work so it would be good idea to add option to substitute dns.lookup with asynchronous one globally.
What is the feature you are proposing to solve the problem?
For example in Go this is achieved with environment variables:
export GODEBUG=netdns=go # force pure Go resolver
export GODEBUG=netdns=cgo # force native resolver (cgo, win32)
What alternatives have you considered?
No response
Activity
tniessen commentedon Aug 28, 2023
cc @nodejs/dns @bnoordhuis
bnoordhuis commentedon Aug 28, 2023
A flag is probably a bad idea. In the deep past node used c-ares (dns.resolve and friends) exclusively. dns.lookup was added explicitly because lots of things only work with the system resolver. Ex. mDNS.
At the moment dns.lookup goes through libuv's thread pool and that's a finite resource (look up UV_THREADPOOL_SIZE) that's shared with other operations but there's no reason uv_getaddrinfo and uv_getnameinfo couldn't use a bespoke thread pool that scales as needed.
bnoordhuis commentedon Aug 29, 2023
There's a misunderstanding in the linked article that hasn't been true since node 10:
Libuv's thread pool reserves a fraction of the thread pool for "fast" operations (file I/O, cryptography, etc.). Slow-running DNS lookups therefore never completely block other operations.
bnoordhuis commentedon Aug 31, 2023
I've thought about this for a bit and I think there is two things we can do:
Change the thread pool strategy in libuv. getaddrinfo(3) and getnameinfo(3) only need minimal stack space so it should be okay to spawn dozens or even hundreds of dns threads. Looks ugly in htop though. :-)
Cache DNS results very lightly in node. Strawman: cache for 1 second and refresh in the background so repeated lookups are fast without going stale. No knobs, no overrides, just a completely transparent implementation detail.
I don't plan on working on it myself but anyone who wants to exchange money for code is welcome to get in touch with me.
kshitjj commentedon Aug 31, 2023
Hey @bnoordhuis,
I would like to work on this issue, the libuv thread pool is single-threaded for networking I am looking for ways to change network I/O to multithreaded.
As for Caching the DNS, I have started working on that. And I will put out a PR tomorrow.
github-actions commentedon Feb 28, 2024
There has been no activity on this feature request for 5 months and it is unlikely to be implemented. It will be closed 6 months after the last non-automated comment.
For more information on how the project manages feature requests, please consult the feature request management document.
github-actions commentedon Mar 29, 2024
There has been no activity on this feature request and it is being closed. If you feel closing this issue is not the right thing to do, please leave a comment.
For more information on how the project manages feature requests, please consult the feature request management document.
codenoid commentedon Aug 12, 2024
Hi, so NodeJS have not support Multicast DNS resolution?