Skip to content

investigate flaky test-vm-timeout-escape-promise-module-2 on freebsd #36627

Closed
Closed
investigate flaky test-vm-timeout-escape-promise-module-2 on freebsd#36627
Labels
flaky-testIssues and PRs involving tests that fail intermittently in CI.freebsdIssues and PRs related to the FreeBSD platform.

Description

@Trott
Trott
opened on Dec 25, 2020
  • Test: parallel/test-vm-timeout-escape-promise-module-2
  • Platform: freebsd
  • Console Output:
20:55:36 not ok 2591 parallel/test-vm-timeout-escape-promise-module-2
20:55:36   ---
20:55:36   duration_ms: 0.397
20:55:36   severity: fail
20:55:36   exitcode: 1
20:55:36   stack: |-
20:55:36     (node:79375) ExperimentalWarning: VM Modules is an experimental feature. This feature could change at any time
20:55:36     (Use `node --trace-warnings ...` to show where the warning was created)
20:55:36     /usr/home/iojs/build/workspace/node-test-commit-freebsd/nodes/freebsd11-x64/test/parallel/test-vm-timeout-escape-promise-module-2.js:22
20:55:36           throw new Error(
20:55:36                 ^
20:55:36     
20:55:36     Error: escaped timeout at 173 milliseconds!
20:55:36         at loop (/usr/home/iojs/build/workspace/node-test-commit-freebsd/nodes/freebsd11-x64/test/parallel/test-vm-timeout-escape-promise-module-2.js:22:13)
20:55:36         at vm:module(0):1:30
20:55:36         at SourceTextModule.evaluate (node:internal/vm/module:229:23)
20:55:36         at assert.rejects.code (/usr/home/iojs/build/workspace/node-test-commit-freebsd/nodes/freebsd11-x64/test/parallel/test-vm-timeout-escape-promise-module-2.js:38:16)
20:55:36         at processTicksAndRejections (node:internal/process/task_queues:93:5)
20:55:36   ...

Activity

  1. Trott
    added
    freebsdIssues and PRs related to the FreeBSD platform.
    flaky-testIssues and PRs involving tests that fail intermittently in CI.
    on Dec 25, 2020

  2. Trott commented on Mar 20, 2021

    @Trott
    Trott
    on Mar 20, 2021
    MemberAuthor

    https://ci.nodejs.org/job/node-test-commit-freebsd/38718/nodes=freebsd11-x64/console

    00:20:50 not ok 2672 parallel/test-vm-timeout-escape-promise-module-2
00:20:51   ---
00:20:51   duration_ms: 0.440
00:20:51   severity: fail
00:20:51   exitcode: 1
00:20:51   stack: |-
00:20:51     (node:46479) ExperimentalWarning: VM Modules is an experimental feature. This feature could change at any time
00:20:51     (Use `node --trace-warnings ...` to show where the warning was created)
00:20:51     /usr/home/iojs/build/workspace/node-test-commit-freebsd/nodes/freebsd11-x64/test/parallel/test-vm-timeout-escape-promise-module-2.js:22
00:20:51           throw new Error(
00:20:51                 ^
00:20:51     
00:20:51     Error: escaped timeout at 179 milliseconds!
00:20:51         at loop (/usr/home/iojs/build/workspace/node-test-commit-freebsd/nodes/freebsd11-x64/test/parallel/test-vm-timeout-escape-promise-module-2.js:22:13)
00:20:51         at vm:module(0):1:30
00:20:51         at SourceTextModule.evaluate (node:internal/vm/module:224:23)
00:20:51         at assert.rejects.code (/usr/home/iojs/build/workspace/node-test-commit-freebsd/nodes/freebsd11-x64/test/parallel/test-vm-timeout-escape-promise-module-2.js:38:16)
00:20:51         at processTicksAndRejections (node:internal/process/task_queues:94:5)
00:20:51   ...

  3. Trott commented on Mar 20, 2021

    @Trott
    Trott
    on Mar 20, 2021
    MemberAuthor

    My local results suggest that parallelism may help rather than hurt this test but I want to confirm that on CI, because that's counterintuitive, especially when FreeBSD in CI has a history of being our most parallel-sensitive platform (probably having to do with the way the CI host counts processors).

    Stress test with no parallelism: https://ci.nodejs.org/job/node-stress-single-test/231/

    Stress test with parallelism: https://ci.nodejs.org/job/node-stress-single-test/232/

  4. Trott commented on Mar 20, 2021

    @Trott
    Trott
    on Mar 20, 2021
    MemberAuthor

    Stress test with no parallelism: https://ci.nodejs.org/job/node-stress-single-test/231/

    Stress test with parallelism: https://ci.nodejs.org/job/node-stress-single-test/232/

    Results sort of show it's flaky with parallelism and reliable without it, but I imagine the p value is small. 0 failures in 1000 runs in the first case, and 2 failures in 1000 runs in the second place.

    Let's try them both again with 10000 runs to get a little more confidence:

    Not parallel: https://ci.nodejs.org/job/node-stress-single-test/233/
    Parallel: https://ci.nodejs.org/job/node-stress-single-test/235/

  5. Trott
    added a commit that references this issue on Mar 20, 2021

    test: fix flaky test-vm-timeout-escape-promise-module-2

    f6e6d87

  6. Trott commented on Mar 20, 2021

    @Trott
    Trott
    on Mar 20, 2021
    MemberAuthor

    Optimistically opened a PR moving it to sequential. #37842

  7. jasnell
    closed this as completedin 341812bon Mar 22, 2021
  8. ruyadorno
    added a commit that references this issue on Mar 24, 2021

    test: fix flaky test-vm-timeout-escape-promise-module-2

    d9ab1d5
  9. targos
    added a commit that references this issue on May 1, 2021

    test: fix flaky test-vm-timeout-escape-promise-module-2

    cd71199
  10. MayaLekova
    added a commit that references this issue on May 22, 2026

    test: fix flaky test-vm-timeout-escape-promise-module-2

    341812b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    flaky-testIssues and PRs involving tests that fail intermittently in CI.freebsdIssues and PRs related to the FreeBSD platform.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions