Error: read ECONNRESET at TLSWrap.onStreamRead (node:internal/stream_base_commons:217:20) Emitted 'error' event on ClientRequest instance...

Version

17.6.0

Platform

No response

Subsystem

No response

What steps will reproduce the bug?

Sending 1000 TPS request

node:events:505
      throw er; // Unhandled 'error' event
      ^

Error: read ECONNRESET
    at TLSWrap.onStreamRead (node:internal/stream_base_commons:217:20)
Emitted 'error' event on ClientRequest instance at:
    at TLSSocket.socketErrorListener (node:_http_client:442:9)
    at TLSSocket.emit (node:events:527:28)
    at emitErrorNT (node:internal/streams/destroy:164:8)
    at emitErrorCloseNT (node:internal/streams/destroy:129:3)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  errno: -104,
  code: 'ECONNRESET',
  syscall: 'read'
}

Node.js v17.6.0

How often does it reproduce? Is there a required condition?

When TPS increase then, its break and server get restart

What is the expected behavior?

Process getting terminated, in ideal case, it should not terminate the process to stop

What do you see instead?

due to the process break, kubernetes POD getting crash, resulting all that movement in progress requests are getting fail, nd most importly server is not available

Additional information

No response

Thanks for the report!

What is your code exactly using? https.request? If that's the case, stream.finished can be really useful for error handling.

If more data, such as a minimal reproduction case is possible, that will be really helpful.

Below code is use for the Accept the request for customer using HTTP and do process the request, fetch the date from DB and response back.

CODE

const httpServer 			= require('http');

/* Set connection keep alive */
httpServer.globalAgent.keepAlive = true;

httpServer.createServer((request, response) => {
	try{
		response.setHeader('Content-Type', 'text/plain');

	       response.setTimeout(parseInt(process.env.DEFAULT_API_REQUEST_TIME_OUT), function (_pConnectionTimeOut) {
			/* Send response */
			sendResponse(response, responseObj.HTTP_STATUS.REQUESTTIMEOUT, responseObj.ERROR_CODES.SERVICE_TIME_OUT, responseObj.ERROR.TYPE.REQUESTTIMEOUT, responseObj.MESSAGES.DEFAULT.REQUESTTIMEOUT);
		});

		let strEndPoint = urlObj.parse(request.url, true).pathname;

		let body = "";
		request.on('error', (err) => {
			console.log("Request error")
			console.log(err)
			request.destroy();
			response.end();
		}).on('socket',function (socket) {
			socket.on('timeout', function() {
				request.destroy();
				/* Send response */
				sendResponse(response, responseObj.HTTP_STATUS.REQUESTTIMEOUT, responseObj.ERROR_CODES.SERVICE_TIME_OUT, responseObj.ERROR.TYPE.REQUESTTIMEOUT, responseObj.MESSAGES.DEFAULT.REQUESTTIMEOUT);
			});
		}).on('data', (chunk) => {
			body += chunk.toString()
		}).on('end', () => {
			/* Validating and upstreating the requst */
                        console.log(Body);
                        /* Setting response headers */
  		response.statusCode = 200;
		/* writing the message into the writer buffer */
		await response.write(JSON.stringify({"responsebody":body}));
		/* Flusing the message closing the communation */
		response.end();
		});
	}catch(exception){
		console.log("===================HTTP OBJECT ERROR===============")
		console.log(exception);
		console.log("===================HTTP OBJECT ERROR===============")
	}
/* on client conention error */
}).on("clientError",function(err,socket){
	console.log("Client Connection Error ");
	console.log(err);
	console.log("Client Connection Error");
	/* On client connection error */
}).on("secureConnection",function(socket){
	console.log("secure Connection Error ");
	/* On client connection error */
}).listen(9091);

Same error here, while trying to retrieve Kubernetes pods logs as stream.

I am getting the same error trying to run webpack:

npm run build

> build
> npx webpack

node:events:505
      throw er; // Unhandled 'error' event
      ^

Error: read ECONNRESET
    at TLSWrap.onStreamRead (node:internal/stream_base_commons:217:20)
Emitted 'error' event on ClientRequest instance at:
    at TLSSocket.socketErrorListener (node:_http_client:442:9)
    at TLSSocket.emit (node:events:527:28)
    at emitErrorNT (node:internal/streams/destroy:164:8)
    at emitErrorCloseNT (node:internal/streams/destroy:129:3)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  errno: -54,
  code: 'ECONNRESET',
  syscall: 'read'
}

Node.js v17.7.0

It's quite random, sometimes I have to run npm run build 5 times until I don't get the error. Sometimes it works immediately. Very hard to reproduce.

Tried 16.14.0 as well, same result.

Connection reset is a normal consequence. It can't be avoided. It means the client left before it last sent packet can be acknowledged (how TCP works). I see them with iPhone clients with short url codes that simply send a 301 status. The client is gone the moment I sent headers, before I can finish the stream. Other times is related to a web app with a service worker that tells the page to stop fetching from the server and instead reload and use cache. HTTP

The problem I'm having is catching all instances of it. I'm getting application crashes with the same -104. On HTTP2 I'm listening on the sessions as well as the stream. I catch session and stream errors with ECONNRESET. But somewhere, somehow, it's slipping through the cracks.

console.warn('app starting:', new Date());

// Snip setup phase

// Error Handlers
this.http2Server.addListener('error', (err) => {
  console.error('HTTP/2 server error', err);
});
this.http2Server.addListener('session', (session) => {
  session.on('error', (err) => {
    if (err?.code === 'ECONNRESET') {
      console.warn('HTTP/2 session connection reset.');
    } else {
      console.error('HTTP/2 session error', err);
    }
  });
});
this.http2Server.addListener('stream', (stream, headers) => {
  stream.on('error', (err) => {
    if (err?.code === 'ECONNRESET') {
      console.warn('HTTP/2 stream connection reset.', headers[':path']);
    } else {
      console.error('HTTP/2 stream error', headers, err);
    }
  });
});

// Logic handlers
this.http2Server.addListener('stream', this.httpHandler.handleHttp2Stream);
this.http2Server.addListener('request', (req, res) => {
  if (req.httpVersionMajor >= 2) return;
  this.httpHandler.handleHttp1Request(req, res);
});

// Catch uncaughtException
process.on('uncaughtException', (error, origin) => {
  console.error('UNCAUGHT EXCEPTION');
  console.error(error);
  console.error(origin);
  process.exit(1);
});

Log output:

L01 |  app starting: 2022-03-19T17:02:47.326Z
L02 |  HTTP/2 stream connection reset. /api/events?channels=dispatch
L03 |  HTTP/2 stream connection reset. /api/getUserInfo
L04 |  HTTP/2 stream connection reset. /api/getCompanyInfo
L05 |  HTTP/2 session connection reset.
L06 |  Error: read ECONNRESET
L07 |      at TLSWrap.onStreamRead (node:internal/stream_base_commons:217:20) {
L08 |    errno: -104,
L09 |    code: 'ECONNRESET',
L10 |    syscall: 'read'
L11 |  }
L12 |  UNCAUGHT EXCEPTION
L13 |  Error: read ECONNRESET
L14 |      at TLSWrap.onStreamRead (node:internal/stream_base_commons:217:20) {
L15 |    errno: -104,
L16 |    code: 'ECONNRESET',
L17 |    syscall: 'read'
L18 |  }
L19 |  uncaughtException
L20 |  app starting: 2022-03-19T17:03:10.409Z

I've tried sessionError and it's mostly the same. I still get crashes. My guess is the the socket itself is getting a connection resetting somewhere after the session is closed, or before any session is negotiated.

I'm on v16.14.0 on Linux. Considering the crashes are so random and I get uncaughtException for origin in the 'uncaughtException' event, I'm ready to just ignore it there, despite it being so dangerous. I don't know where lines L06-L11 are coming from. It might be the pm2 module exporting that.

Edit: Same result with this.http2Server.addListener('connection', (socket) => socket.on('error', console.error)). This is happening with the same three URLs again and again. It seems to be at the tail end of the session. Maybe it's related to the SSE I use.

I made more progress with clientError handler, but it's still not stopping the crash:

Handlers:

this.httpsServer.removeListener('error', reject);

this.httpsServer.addListener('error', (err) => {
  console.error('HTTPS server error', err);
});
this.httpsServer.addListener('clientError', (err) => {
  console.error('HTTPS client error', err);
});
this.httpsServer.addListener('tlsClientError', (err) => {
  console.error('HTTPS TLS client error', err);
});
this.httpsServer.addListener('connection', (socket) => {
  socket.on('error', (err) => {
    console.error('HTTPS connection error', err);
  });
  socket.on('clientError', (err) => {
    console.error('HTTPS connection client error', err);
  });
  socket.on('tlsClientError', (err) => {
    console.error('HTTPS connection TLS client error', err);
  });
});

Output:

HTTP/2 session connection reset.
HTTP/2 client error Error: socket hang up
    at connResetException (node:internal/errors:691:14)
    at TLSSocket.onSocketClose (node:_tls_wrap:1080:23)
    at TLSSocket.emit (node:events:532:35)
    at TLSSocket.emit (node:domain:475:12)
    at node:net:687:12
    at Socket.done (node:_tls_wrap:580:7)
    at Object.onceWrapper (node:events:640:26)
    at Socket.emit (node:events:520:28)
    at Socket.emit (node:domain:475:12)
    at TCP.<anonymous> (node:net:687:12) {
  code: 'ECONNRESET'
}
HTTP/2 TLS client error Error: socket hang up
    at connResetException (node:internal/errors:691:14)
    at TLSSocket.onSocketClose (node:_tls_wrap:1080:23)
    at TLSSocket.emit (node:events:532:35)
    at TLSSocket.emit (node:domain:475:12)
    at node:net:687:12
    at Socket.done (node:_tls_wrap:580:7)
    at Object.onceWrapper (node:events:640:26)
    at Socket.emit (node:events:520:28)
    at Socket.emit (node:domain:475:12)
    at TCP.<anonymous> (node:net:687:12) {
  code: 'ECONNRESET'
}
HTTP/2 stream connection reset. /api/events?channels=dispatch
HTTP/2 stream connection reset. /api/getUserInfo
HTTP/2 stream connection reset. /api/getCompanyInfo
HTTP/2 session connection reset.
Error: read ECONNRESET
    at TLSWrap.onStreamRead (node:internal/stream_base_commons:217:20) {
  errno: -104,
  code: 'ECONNRESET',
  syscall: 'read'
}
UNCAUGHT EXCEPTION
Error: read ECONNRESET
    at TLSWrap.onStreamRead (node:internal/stream_base_commons:217:20) {
  errno: -104,
  code: 'ECONNRESET',
  syscall: 'read'
}
uncaughtException
app starting: 2022-03-19T18:59:18.219Z

I gave up. I've tried every error watcher I can think up and had to resort to:

process.on('uncaughtException', (error, origin) => {
  if (error?.code === 'ECONNRESET') return;
  console.error('UNCAUGHT EXCEPTION');
  console.error(error);
  console.error(origin);
  process.exit(1);
});

Unless I'm missing one, there's a leak somewhere.

Edit: I added 'sessionError' and upgraded to v16.14.2. Let's see how it goes...

Hi I was also having the same problem. found a few causes I get these responses in node V14, V16 and V17. I hope this can help anyone solve their problems in the future

Client Connection Error [Error: 140232514893760:error:1417A0C1:SSL routines:tls_post_process_client_hello:no shared cipher:../deps/openssl/openssl/ssl/statem/statem_srvr.c:2313: ] { library: 'SSL routines', function: 'tls_post_process_client_hello', reason: 'no shared cipher', code: 'ERR_SSL_NO_SHARED_CIPHER' } Client Connection Error Client Connection Error [Error: 140232514893760:error:142090FC:SSL routines:tls_early_post_process_client_hello:unknown protocol:../deps/openssl/openssl/ssl/statem/statem_srvr.c:1689: ] { library: 'SSL routines', function: 'tls_early_post_process_client_hello', reason: 'unknown protocol', code: 'ERR_SSL_UNKNOWN_PROTOCOL' } Client Connection Error Client Connection Error [Error: 140232514893760:error:14209102:SSL routines:tls_early_post_process_client_hello:unsupported protocol:../deps/openssl/openssl/ssl/statem/statem_srvr.c:1714:] { library: 'SSL routines', function: 'tls_early_post_process_client_hello', reason: 'unsupported protocol', code: 'ERR_SSL_UNSUPPORTED_PROTOCOL' } Client Connection Error Client Connection Error Error: socket hang up at connResetException (node:internal/errors:691:14) at TLSSocket.onSocketClose (node:_tls_wrap:1080:23) at TLSSocket.emit (node:events:538:35) at node:net:687:12 at Socket.done (node:_tls_wrap:580:7) at Object.onceWrapper (node:events:646:26) at Socket.emit (node:events:526:28) at TCP.<anonymous> (node:net:687:12) { code: 'ECONNRESET' }

I was able to solve all my leaks with the following:

Connection timeout: this.http2Server.setTimeout(120_000);
Client Error Handler: this.http2Server.addListener('clientError', (err, socket) => socket.destroy(err));
Session Error Handler: this.http2Server.addListener('sessionError', (err, session) => socket.destroy(err));
Session Timeout handler: this.http2Server.addListener('session', (session) => session.setTimeout(60_000, () => session.destroy(new Error('TIMEOUT')))
Stream Error handler: this.http2Server.addListener('stream', (stream) => stream.addListener('error', (err) => stream.destroy(err)));

(not exactly my code, but should handle all cases and leave no leaks)

Upgrading Node to 16.15 from 16.14 solved the issue for us.

Specifically it was happening on webpack start.

Hi team,
Need help on this issue. My application was deployed in GKE (Google Cloud Project) and seeing the below error

read ECONNRESET
at TLSWrap.onStreamRead (internal/stream_base_commons.js:209:20)

I am using the https call module and this happens only when there is connecting from external vendor api which is not internal network and I don't see this issue when there is internal applications with in the network.

Currently I am using the Node version in my docker file as14.15.0. So based on the above comments I see this is an bug in Node V14 in other forum. I just want to check is this fix on Node V16 LTS version

@meznaric

.

Upgrading Node to 16.15 from 16.14 solved the issue for us.

Specifically it was happening on webpack start.

@meznaric I am facing this issue while calling the https call from my GKE application to external vendor API. Yesterday I upgraded my node js version from 14 --> 16.15.1 but still see the error. could you please let me know which version of minor version of Node V16 resolved you..

@jaiswarvipin

Version

17.6.0

Platform

No response

Subsystem

No response

What steps will reproduce the bug?

Sending 1000 TPS request
node:events:505
      throw er; // Unhandled 'error' event
      ^

Error: read ECONNRESET
    at TLSWrap.onStreamRead (node:internal/stream_base_commons:217:20)
Emitted 'error' event on ClientRequest instance at:
    at TLSSocket.socketErrorListener (node:_http_client:442:9)
    at TLSSocket.emit (node:events:527:28)
    at emitErrorNT (node:internal/streams/destroy:164:8)
    at emitErrorCloseNT (node:internal/streams/destroy:129:3)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  errno: -104,
  code: 'ECONNRESET',
  syscall: 'read'
}

Node.js v17.6.0
How often does it reproduce? Is there a required condition?

When TPS increase then, its break and server get restart

What is the expected behavior?

Process getting terminated, in ideal case, it should not terminate the process to stop

What do you see instead?

due to the process break, kubernetes POD getting crash, resulting all that movement in progress requests are getting fail, nd most importly server is not available

Additional information

No response

@jaiswarvipin Is this issue resolved for you? If yes could you please let me know how it got fix and I am facing the same

I'm having the same issue 🗡️ The nodejs version doesn't matter, I used nvm and tried a bunch of them.

@coding-chris-kao

@coding-chris-kao You probably want to take a look at the Range header and how it works. With files, it's probably not too hard to implement, as long as you don't use Content-Encoding.

https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests

Again, this isn't a Node issue. If you don't add a catch for client errors, your app will crash whenever a client doesn't close a connection cleanly. I've been running with the logic in #42154 (comment) for over a month with public servers without anymore crashes or memory leaks. (Though I did add a session.ping() to each session to force timeouts to fire and prevent leaks.)

Best of luck to you guys implementing the error catchers. And even more luck to those of you that don't. 😆

The code is so lengthy. Why node emit a uncatched expecption? If it emit a normal error event on server object, it would be more safe and easy to handle.

Why node emit a uncatched expecption?

It doesn't. There's a missing 'error' event listener somewhere in the application.

I am also having similar issues when trying to make a GET Requests via https.

$ nodemon app.js
[nodemon] `2.0.20`
[nodemon] to restart at any time, enter `rs`
[nodemon] watching path(s): *.*
[nodemon] watching extensions: js,mjs,json  
[nodemon] starting `node app.js`
Server is up and running on port 5000.
node:events:491
      throw er; // Unhandled 'error' event
      ^

Error: read ECONNRESET
    at TLSWrap.onStreamRead (node:internal/stream_base_commons:217:20)
Emitted 'error' event on ClientRequest instance at:
    at TLSSocket.socketErrorListener (node:_http_client:481:9)
    at TLSSocket.emit (node:events:513:28)
    at emitErrorNT (node:internal/streams/destroy:157:8)
    at emitErrorCloseNT (node:internal/streams/destroy:122:3)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  errno: -4077,
  code: 'ECONNRESET',
  syscall: 'read'
}
[nodemon] app crashed - waiting for file changes before starting...

In our case we needed to use internal kubernets urls instead of the external link when making requests

fix nodejs 16 tls ECONNRESET bug. nodejs/node#42154

I'm going to close this out because (as has been mentioned several times in this thread), this is not a node bug.

ECONNRESET means the peer closed the connection (i.e., a fact of life you simply need to deal with) and should be handled by adding 'error' event listeners in the right places.

According to the HTTP specification, this error is related to a "data race" but no specific solution is provided.

A client, server, or proxy MAY close the transport connection at any time. For example, a client might have started to send a new request at the same time that the server has decided to close the "idle" connection. From the server's point of view, the connection is being closed while it was idle, but from the client's point of view, a request is in progress.

in our case, we decided to retry the request on reusedSocket when saw ECONNRESET error.

Same issue here.

I'm making https client from react running on localhost:9001 to my nodejs app running on localhost:8080 (http only).

When I make api call, the response frombackend is received but in case of graphql query, we are not getting any response though graphql-client is able to build and send response I cross-checked in debug-mode.

what could be the issue ?


 [1684140807677] ERROR (redbird/94311 on m-c02f5akymd6n): HTTPS Client Error
     Error: read ECONNRESET
         at TLSWrap.onStreamRead (node:internal/stream_base_commons:217:20)
 [1684140807679] ERROR (redbird/94311 on m-c02f5akymd6n): HTTPS Client Error
    Error: read ECONNRESET
         at TLSWrap.onStreamRead (node:internal/stream_base_commons:217:20)

ERROR (redbird/94311 on m-c02f5akymd6n): HTTPS Client Error
 Error: 140704594961216:error:14094416:SSL routines:ssl3_read_bytes:sslv3 alert certificate unknown:../deps/openssl/openssl/ssl/record/rec_layer_s3.c:1558:SSL alert number 46

What about a bit more helpful log messages?
At the moment there is a luck of information which can help find the root issue.
For example

Error: read ECONNRESET
  at TCP.onStreamRead (node:internal/stream_base_commons:217:20) {
    errno: -104,
    code: 'ECONNRESET',
    syscall: 'read'
 }

For my opinion it says almost nothing.

This is absolutely correct...

ECONNRESET means the peer closed the connection (i.e., a fact of life you simply need to deal with) and should be handled by adding 'error' event listeners in the right places.

...but the real issue here isn't the ECONNRESET. It's the fact that the error object does not contains enough information to identify the "right places" to add the needed listeners.

For example, this is the entire call stack for the one that restarted my service just now:

Error: read ECONNRESET
    at TLSWrap.onStreamRead (node:internal/stream_base_commons:217:20)
    at TLSWrap.callbackTrampoline (node:internal/async_hooks:130:17)

I suspect that those worthless call stacks are not a nodejs problem, but rather a Javascript runtime problem.

But even so, could nodejs do something to help? For example, provide default error listeners that just log information about which listener caught the error before terminating the process.

I'm too new to nodejs (and Javascript) to really have an opinion about how to mitigate this issue, but as someone who is stuck developing atop nodejs I would very much appreciate a workaround for those worthless call stacks.

I am studying nextjs from Vercel's courses: Learn Next.js .Everything is ok in first several days ,until this ECONNRESET error occurs . I didn't figure out why this happen now ,because I think I didn't change things on the project except the net env (cause I use VPN),but I found this link help me to keep going :Cannot seed database . Maybe try 'vercel dev' instead of 'npm run dev' will work .

With no stack trace in the exception, I have no idea where to catch this exception in my code.

Navigation Menu

Repository navigation

Error: read ECONNRESET at TLSWrap.onStreamRead (node:internal/stream_base_commons:217:20) Emitted 'error' event on ClientRequest instance... #42154

Version

Platform

Subsystem

What steps will reproduce the bug?

How often does it reproduce? Is there a required condition?

What is the expected behavior?

What do you see instead?

Additional information

Version

Platform

Subsystem

What steps will reproduce the bug?

How often does it reproduce? Is there a required condition?

What is the expected behavior?

What do you see instead?

Additional information

Version

Platform

Subsystem

What steps will reproduce the bug?

How often does it reproduce? Is there a required condition?

What is the expected behavior?

What do you see instead?

Additional information

7 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Error: read ECONNRESET at TLSWrap.onStreamRead (node:internal/stream_base_commons:217:20) Emitted 'error' event on ClientRequest instance... #42154

Description

Version

Platform

Subsystem

What steps will reproduce the bug?

How often does it reproduce? Is there a required condition?

What is the expected behavior?

What do you see instead?

Additional information

Activity

juanarbol commented on Mar 4, 2022

jaiswarvipin commented on Mar 5, 2022

ilteoood commented on Mar 7, 2022

Anatanokami commented on Mar 9, 2022

clshortfuse commented on Mar 19, 2022

clshortfuse commented on Mar 19, 2022

clshortfuse commented on Mar 19, 2022

novomo commented on Apr 22, 2022

clshortfuse commented on Apr 22, 2022

meznaric commented on May 20, 2022

gopinaresh commented on Jun 8, 2022

gopinaresh commented on Jun 10, 2022

gopinaresh commented on Jun 11, 2022

Version

Platform

Subsystem

What steps will reproduce the bug?

How often does it reproduce? Is there a required condition?

What is the expected behavior?

What do you see instead?

Additional information

Version

Platform

Subsystem

What steps will reproduce the bug?

How often does it reproduce? Is there a required condition?

What is the expected behavior?

What do you see instead?

Additional information

HoodyStiliyan commented on Jun 14, 2022

7 remaining items

clshortfuse commented on Sep 7, 2022

yunnysunny commented on Oct 11, 2022

bnoordhuis commented on Oct 11, 2022

David5183 commented on Oct 13, 2022

Chaingenhash commented on Jan 5, 2023

bnoordhuis commented on Mar 10, 2023

xr commented on Mar 10, 2023

bksun commented on May 15, 2023

alexey-sh commented on Dec 7, 2023

natewaddoups commented on Feb 23, 2024

Se7enGo commented on Apr 2, 2024

rrlapointe commented on Oct 6, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions