Skip to content

test-http2-large-file is flaky #47409

Closed as not planned
Closed as not planned
test-http2-large-file is flaky#47409
Labels
flaky-testIssues and PRs involving tests that fail intermittently in CI.staleIssues and PRs marked stale due to inactivity and scheduled for automatic closure.

Description

@MoLow
MoLow
opened on Apr 4, 2023

Test

test-http2-large-file

Platform

AIX

Console output

22:30:07 not ok 3904 sequential/test-http2-large-file
22:30:07   ---
22:30:07   duration_ms: 120107.72300
22:30:07   severity: fail
22:30:07   exitcode: -15
22:30:07   stack: |-
22:30:07     timeout
22:30:07   ...

Build links

Additional information

No response

Activity

  1. MoLow
    added
    flaky-testIssues and PRs involving tests that fail intermittently in CI.
    on Apr 4, 2023
  2. github-actions
    added
    aixIssues and PRs related to the AIX platform.
    on Apr 4, 2023

  3. MoLow commented on Apr 4, 2023

    @MoLow
    MoLow
    on Apr 4, 2023
    MemberAuthor

    I took https://ci.nodejs.org/computer/test-osuosl-aix72-ppc64_be-2/ offline to investigate

  4. MoLow commented on Apr 4, 2023

    @MoLow
    MoLow
    on Apr 4, 2023
    MemberAuthor

    on AIX tools/test.py sequential/test-http2-large-file --timeout 30 --repeat 50 reproduces easily
    however, this change causes the test to be very stable and I have no idea why:

    diff --git a/test/sequential/test-http2-large-file.js b/test/sequential/test-http2-large-file.js
index 2f2cc2c80d..3afcec73f4 100644
--- a/test/sequential/test-http2-large-file.js
+++ b/test/sequential/test-http2-large-file.js
@@ -24,6 +24,7 @@ server.listen(0, common.mustCall(() => {
                                { settings: { initialWindowSize: 6553500 } });
   const request = client.request({ ':method': 'POST' });
   function writeChunk() {
+    console.log({ remaining });
     if (remaining > 0) {
       remaining -= chunkLength;
       request.write(chunk, writeChunk);

  5. Trott commented on Apr 4, 2023

    @Trott
    Trott
    on Apr 4, 2023
    Member

    on AIX tools/test.py sequential/test-http2-large-file --timeout 30 --repeat 50 reproduces easily however, this change causes the test to be very stable and I have no idea why:

    diff --git a/test/sequential/test-http2-large-file.js b/test/sequential/test-http2-large-file.js
index 2f2cc2c80d..3afcec73f4 100644
--- a/test/sequential/test-http2-large-file.js
+++ b/test/sequential/test-http2-large-file.js
@@ -24,6 +24,7 @@ server.listen(0, common.mustCall(() => {
                                { settings: { initialWindowSize: 6553500 } });
   const request = client.request({ ':method': 'POST' });
   function writeChunk() {
+    console.log({ remaining });
     if (remaining > 0) {
       remaining -= chunkLength;
       request.write(chunk, writeChunk);

    There's a long history of console.log() statements causing tests to succeed that otherwise fail because they are blocking or otherwise causes an I/O delay that "fixes" a race condition. I can probably find an issue from 2015 about this!

  6. MoLow commented on Apr 4, 2023

    @MoLow
    MoLow
    on Apr 4, 2023
    MemberAuthor

    it does not seem to be a race condition, this did not fix the issue:

    diff --git a/test/sequential/test-http2-large-file.js b/test/sequential/test-http2-large-file.js
index 2f2cc2c80d..300e0642ab 100644
--- a/test/sequential/test-http2-large-file.js
+++ b/test/sequential/test-http2-large-file.js
@@ -24,12 +24,14 @@ server.listen(0, common.mustCall(() => {
                                { settings: { initialWindowSize: 6553500 } });
   const request = client.request({ ':method': 'POST' });
   function writeChunk() {
-    if (remaining > 0) {
+   setTimeout(() => {
+   if (remaining > 0) {
       remaining -= chunkLength;
       request.write(chunk, writeChunk);
     } else {
       request.end();
     }
+   }, 10);

  7. Trott commented on Apr 4, 2023

    @Trott
    Trott
    on Apr 4, 2023
    Member

    I forget (or maybe never knew!) the details, but console.log() has weird side effects that cause I/O flushing or other things and the side effects vary from platform to platform. Maybe someone on @nodejs/console or @nodejs/libuv would know more. If I recall correctly, it blocks on some platforms but not on others.

  8. MoLow commented on Apr 4, 2023

    @MoLow
    MoLow
    on Apr 4, 2023
    MemberAuthor

    process.stdout.write(''); does not make the issue go await either so it is definitely very specific to console.log

  9. richardlau commented on Apr 4, 2023

    @richardlau
    richardlau
    on Apr 4, 2023
    Member

    @miladfarca has been investigating this as an upstream V8 issue: https://bugs.chromium.org/p/v8/issues/detail?id=13902

  10. MoLow commented on Apr 4, 2023

    @MoLow
    MoLow
    on Apr 4, 2023
    MemberAuthor

    https://ci.nodejs.org/computer/test-osuosl-aix72-ppc64_be-2/ is back online

  11. targos commented on Jan 11, 2024

    @targos
    targos
    on Jan 11, 2024
    Member

    Happened on Windows and Linux with https://ci.nodejs.org/job/node-test-pull-request/56726/

  12. targos commented on Jan 11, 2024

    @targos
    targos
    on Jan 11, 2024
    Member

    The V8 issue talks about disabling concurrent sparkplug to fix it temporarily but we already do it: 7071c1d

    Related:

  13. targos
    added a commit that references this issue on Jan 23, 2024

    test: mark test-http2-large-file as flaky

    0791ce0

  14. targos commented on Jan 23, 2024

    @targos
    targos
    on Jan 23, 2024
    Member

    This really happens a lot lately. I opened #51549

    From nodejs/reliability#772:

    Reason sequential/test-http2-large-file
    Type JS_TEST_FAILURE
    Failed PR 17 (https://github.com/nodejs/node/pull/45427/, https://github.com/nodejs/node/pull/51401/, https://github.com/nodejs/node/pull/51410/, https://github.com/nodejs/node/pull/50960/, https://github.com/nodejs/node/pull/51422/, https://github.com/nodejs/node/pull/51431/, https://github.com/nodejs/node/pull/51412/, https://github.com/nodejs/node/pull/51352/, https://github.com/nodejs/node/pull/51172/, https://github.com/nodejs/node/pull/51446/, https://github.com/nodejs/node/pull/51459/, https://github.com/nodejs/node/pull/51044/, https://github.com/nodejs/node/pull/51437/, https://github.com/nodejs/node/pull/51461/, https://github.com/nodejs/node/pull/51456/, https://github.com/nodejs/node/pull/51455/, https://github.com/nodejs/node/pull/51472/)
    Appeared test-rackspace-win2022_vs2022-x64-1, test-azure_msft-win11_vs2022-x64-3, test-rackspace-win2019_vs2019-x64-1, test-rackspace-win2019_vs2019-x64-2, test-azure_msft-win10_vs2019-x64-1, test-rackspace-win2022_vs2022-x64-4, test-azure_msft-win11_vs2022-x64-1, test-rackspace-win2019_vs2019-x64-3, test-rackspace-win2022_vs2022-x64-6, test-azure_msft-win11_vs2022-x64-2, test-digitalocean-fedora38-x64-1, test-digitalocean-ubuntu2204-x64-2, test-rackspace-win2019_vs2019-x64-4
    First CI https://ci.nodejs.org/job/node-test-pull-request/56712/
    Last CI https://ci.nodejs.org/job/node-test-pull-request/56791/
    Example 
    not ok 950 sequential/test-http2-large-file
  ---
  duration_ms: 120192.99600
  severity: fail
  exitcode: 1
  stack: |-
    timeout
  ...

  15. shnooshnoo commented on Jan 25, 2024

    @shnooshnoo
    shnooshnoo
    on Jan 25, 2024

    The first time it occurred in this build https://ci.nodejs.org/job/node-daily-master/3286/, so i guess something that landed around 7th of Jan might've caused this. From what I can see the difference is this commit 1a5acd0

  16. 1 remaining item

  17. rdw-msft
    added a commit that references this issue on Feb 9, 2024

    test: mark test-http2-large-file as flaky

    a218420
  18. targos
    added a commit that references this issue on Feb 15, 2024

    test: mark test-http2-large-file as flaky

    fc0c130

  19. mhdawson commented on Mar 25, 2024

    @mhdawson
    mhdawson
    on Mar 25, 2024
    Member

    Again on linux - https://ci.nodejs.org/job/node-test-commit-linux/nodes=fedora-latest-x64/56789/

  20. mhdawson
    removed
    aixIssues and PRs related to the AIX platform.
    on Mar 25, 2024

  21. mhdawson commented on Apr 25, 2024

    @mhdawson
    mhdawson
    on Apr 25, 2024
    Member

    again - https://ci.nodejs.org/job/node-test-commit-linux/nodes=fedora-latest-x64/57416/

  22. github-actions commented on Jun 6, 2026

    @github-actions
    github-actionsbot
    on Jun 6, 2026 – with GitHub Actions
    Contributor

    This issue has been marked as stale due to 210 days of inactivity.
    It will be automatically closed in 30 days if no further activity occurs. If this is still relevant, please leave a comment or update it to keep it open.

  23. github-actions
    added
    staleIssues and PRs marked stale due to inactivity and scheduled for automatic closure.
    on Jun 6, 2026

  24. github-actions commented on Jul 7, 2026

    @github-actions
    github-actionsbot
    on Jul 7, 2026 – with GitHub Actions
    Contributor

    This issue has been automatically closed after 30 days of inactivity following its stale status (no activity for a total of 240 days).
    If this is still relevant, feel free to reopen it or leave a comment with additional details so we can continue the discussion.

  25. github-actions
    closed this as not plannedon Jul 7, 2026
  26. nodejs-github-bot
    added a commit that references this issue on Sep 28, 2026

    test: unmark exit timeout tests as flaky

    c12d483
  27. HoonDongKang
    added a commit that references this issue on Sep 29, 2026

    test: unmark exit timeout tests as flaky

    8d6bedb
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    flaky-testIssues and PRs involving tests that fail intermittently in CI.staleIssues and PRs marked stale due to inactivity and scheduled for automatic closure.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions