Skip to content

ERR_HTTP2_INVALID_STREAM #24470

Closed
Closed

Description

@StephenLynx

Node 10.13
CentOS 7
HTTP2

When trying to use writeHead on the compatibility layer, I get a ERR_HTTP2_INVALID_STREAM sometimes that crashes the worker process.

Error [ERR_HTTP2_INVALID_STREAM]: The stream has been destroyed
at Http2ServerResponse.writeHead (internal/http2/compat.js:569:13)

The header is in {key:[value,value],anotherkey:[value]} format.

Activity

  1. Trott commented on Nov 19, 2018

    @Trott
    Member

    @nodejs/http2 (Should this be transferred to the help repo?)

  2. StephenLynx commented on Nov 19, 2018

    @StephenLynx
    Author

    Trott, keep in mind that this isn't the first time someone has issues with this.
    #22135
    Now, it should have been fixed on 10.12, so it was either a regression or I am doing something very sketchy with the request object that is causing it to happen.

    Edit: the issue happened on ubuntu 16, actually. I am trying to reproduce on CentOS 7 now.
    Edit2: it just happened again on CentOS 7 too.

  3. StephenLynx commented on Nov 19, 2018

    @StephenLynx
    Author

    I would also like to point out that I am using the cluster module to spawn one worker process for each CPU core.

  4. StephenLynx commented on Nov 19, 2018

    @StephenLynx
    Author

    And after installing 10.12 and 10.13 again I can't seem to reproduce it anymore.
    Edit: nevermind, just happened again.

  5. antsmartian commented on Nov 22, 2018

    @antsmartian
    Contributor

    @StephenLynx Could you please share a reproducible code?

  6. StephenLynx commented on Nov 22, 2018

    @StephenLynx
    Author

    I can not, aside from my software gitgud.io/LynxChan/LynxChan

  7. Trott commented on Nov 25, 2018

    @Trott
    Member

    @nodejs/http2 Is it likely that the source of the problem can be determined with the information given?

  8. apapirovski commented on Nov 29, 2018

    @apapirovski
    Contributor

    If writeHead is crashing the process then in all likelihood your stream is getting closed in advance of you calling the function. Given that you're using compatibility mode, it's very much possible that some other code is causing an issue that leads to the stream prematurely closing. Note that this was also the case with the issue you linked above.

    I'm going to close this out as there's no easily usable reproduction and there's not much likelihood this is a Node core issue.

  9. StephenLynx commented on Nov 29, 2018

    @StephenLynx
    Author

    How could one close it? I can look around my code, I use the request and response exclusively as I have used them for http1 for years.

  10. StephenLynx commented on Nov 29, 2018

    @StephenLynx
    Author

    So while I am not discarding the possibility to be with my code, I am not so sure it isn't with node. I remember finding some similar intermittent crash around version 5 that I also couldn't consistently reproduce but ended up being a DoS on node itself.

  11. mcollina commented on Nov 29, 2018

    @mcollina
    SponsorMember

    I think this is a bug in our compat API.

    As you can see in

    node/lib/_http_outgoing.js

    Lines 240 to 247 in 9920dbc

    OutgoingMessage.prototype._writeRaw = _writeRaw;
    function _writeRaw(data, encoding, callback) {
    const conn = this.connection;
    if (conn && conn.destroyed) {
    // The socket was destroyed. If we're still trying to write to it,
    // then we haven't gotten the 'close' event yet.
    return false;
    }
    , http1 writeHead does not throw if the stream is destroyed, it just returns false.

    I think this is a proper bug.

  12. mcollina commented on Nov 29, 2018

    @mcollina
    SponsorMember

    @StephenLynx can you please upload a script to reproduce the crash?

  13. StephenLynx commented on Nov 29, 2018

    @StephenLynx
    Author

    I tried, but it wouldn't crash with a simple wait to hold concurrent requests and another script hammering it.
    The crashes seem to happen a bit at random with my software running. It works, then suddenly crash. It also seemed to be related with time. It would hardly crash right away, but once a worker crashed, it seems all the others became much more prone to crashing too.

  14. 2 remaining items

  15. StephenLynx commented on Nov 29, 2018

    @StephenLynx
    Author

    GOT IT
    @mcollina
    https://pastebin.com/vCbVS1LT
    It spent 30 minutes working fine, then crashed when outputting from gridfs, then crashed again 14 seconds later when outputting some error 500 page.

  16. StephenLynx commented on Nov 29, 2018

    @StephenLynx
    Author

    Also, notice how the error it threw originated from trying to output a response on the cache handler. But that one is inside a try catch, so it didn't crash there.
    I'd also like to point that I had about 3 tabs open, no sure if that influenced anything.

  17. StephenLynx commented on Nov 29, 2018

    @StephenLynx
    Author

    So, the issue here is that is expected for the browser to sometimes interrupt the connection before the response and is up to what handles the connection to deal with it?

  18. mcollina commented on Nov 29, 2018

    @mcollina
    SponsorMember

    The issue is just a mismatch between writeHead in http1 and http2. See #24723

  19. StephenLynx commented on Nov 29, 2018

    @StephenLynx
    Author

    Btw, did you check if the write and end also don't crash?

  20. mcollina commented on Nov 29, 2018

    @mcollina
    SponsorMember

    @StephenLynx please check the test in my PR.

  21. StephenLynx commented on Nov 29, 2018

    @StephenLynx
    Author

    Ah yeah, you do a write head, write and end on the test.
    The name of the pr made me think if you weren't looking just at writeHead.

  22. Trott commented on Dec 3, 2018

    @Trott
    Member

    Fixed in 32fed93

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions