investigate flaky internet/test-dns-idna2008 test

@bnoordhuis

test/internet/test-dns-idna2008.js (added in four days ago when #25679 landed) failed in CI a few hours ago:

https://ci.nodejs.org/job/node-test-commit-custom-suites/855/default/console

Host: test-rackspace-ubuntu1604-x64-2

00:01:48 not ok 9 internet/test-dns-idna2008
00:01:48   ---
00:01:48   duration_ms: 1.14
00:01:48   severity: fail
00:01:48   exitcode: 1
00:01:48   stack: |-
00:01:48     (node:4568) ExperimentalWarning: The dns.promises API is experimental
00:01:48     assert.js:753
00:01:48         throw newErr;
00:01:48         ^
00:01:48     
00:01:48     AssertionError [ERR_ASSERTION]: ifError got unwanted exception: queryA ESERVFAIL straße.de
00:01:48         at QueryReqWrap.dns.resolve4.mustCall (/home/iojs/build/workspace/node-test-commit-custom-suites/default/test/internet/test-dns-idna2008.js:27:10)
00:01:48         at QueryReqWrap.callback (/home/iojs/build/workspace/node-test-commit-custom-suites/default/test/common/index.js:368:15)
00:01:48         at QueryReqWrap.onresolve [as oncomplete] (dns.js:199:10)
00:01:48         at QueryReqWrap.onresolve [as oncomplete] (dns.js:199:19)
00:01:48   ...

It had previously succeeded in CI (see https://ci.nodejs.org/job/node-test-commit-custom-suites/852/default/console).

I guess ESERVFAIL is a DNS problem likely unrelated to the test itself? Although other DNS tests in internet ran just fine, so maybe it is in fact specific to this test?

@bnoordhuis

It's been a one-time failure so far? ESERVFAIL is reported by the DNS server; could be a temporary service outage.

It's been a one-time failure so far? ESERVFAIL is reported by the DNS server; could be a temporary service outage.

I had the temporary-hiccup idea too, but other DNS tests in the internet suite didn't fail. Still, the implication I'm getting from your comment--that there's probably not much point in chasing it down unless we see it more than once--is no doubt correct. Maybe leave this open for a bit and close it after a week or so if we don't see it again?

Failed again today:

https://ci.nodejs.org/job/node-test-commit-custom-suites/856/default/console

test-rackspace-ubuntu1604-x64-1

00:16:53 not ok 9 internet/test-dns-idna2008
00:16:53   ---
00:16:53   duration_ms: 1.14
00:16:53   severity: fail
00:16:53   exitcode: 1
00:16:53   stack: |-
00:16:53     (node:22610) ExperimentalWarning: The dns.promises API is experimental
00:16:53     assert.js:753
00:16:53         throw newErr;
00:16:53         ^
00:16:54     
00:16:54     AssertionError [ERR_ASSERTION]: ifError got unwanted exception: queryA ESERVFAIL straße.de
00:16:54         at QueryReqWrap.dns.resolve4.mustCall (/home/iojs/build/workspace/node-test-commit-custom-suites/default/test/internet/test-dns-idna2008.js:27:10)
00:16:54         at QueryReqWrap.callback (/home/iojs/build/workspace/node-test-commit-custom-suites/default/test/common/index.js:367:15)
00:16:54         at QueryReqWrap.onresolve [as oncomplete] (dns.js:199:10)
00:16:54         at QueryReqWrap.onresolve [as oncomplete] (dns.js:199:19)
00:16:54   ...

I'll take a look but I wonder if it's an environmental issue (i.e., network issue) because it never fails for me locally. What about you?

I'll take a look but I wonder if it's an environmental issue (i.e., network issue) because it never fails for me locally. What about you?

I've not had it fail locally either. It passed in last night's run on test-rackspace-ubuntu1604-x64-2.

To get a baseline (or to see if it passes 7 times, and then the firewall kicks in and it fails the remaining 93 times or whatever), here's 100 runs in CI:

https://ci.nodejs.org/job/node-stress-single-test/2147/

To get a baseline (or to see if it passes 7 times, and then the firewall kicks in and it fails the remaining 93 times or whatever), here's 100 runs in CI:

https://ci.nodejs.org/job/node-stress-single-test/2147/

No failures in stress test on test-rackspace-ubuntu1604-x64-2.

Re-running, but with all internet tests, but only 10 consecutive runs:

https://ci.nodejs.org/job/node-stress-single-test/2148/

Re-running, but with all internet tests, but only 10 consecutive runs:

https://ci.nodejs.org/job/node-stress-single-test/2148/

Also successful on test-rackspace-ubuntu1604-x64-2.
¯\(ツ)/¯

Happened in CI again today.

https://ci.nodejs.org/job/node-test-commit-custom-suites/897/default/console

test-rackspace-ubuntu1604-x64-1

00:05:33 not ok 9 internet/test-dns-idna2008
00:05:33   ---
00:05:33   duration_ms: 25.277
00:05:33   severity: fail
00:05:33   exitcode: 1
00:05:33   stack: |-
00:05:33     (node:18514) ExperimentalWarning: The dns.promises API is experimental
00:05:33     assert.js:742
00:05:33         throw newErr;
00:05:33         ^
00:05:33     
00:05:33     AssertionError [ERR_ASSERTION]: ifError got unwanted exception: queryA ESERVFAIL straße.de
00:05:33         at QueryReqWrap.dns.resolve4.mustCall (/home/iojs/build/workspace/node-test-commit-custom-suites/default/test/internet/test-dns-idna2008.js:27:10)
00:05:33         at QueryReqWrap.callback (/home/iojs/build/workspace/node-test-commit-custom-suites/default/test/common/index.js:367:15)
00:05:33         at QueryReqWrap.onresolve [as oncomplete] (dns.js:199:10)
00:05:33         at QueryReqWrap.onresolve [as oncomplete] (dns.js:199:19)
00:05:33   ...

Again today:

https://ci.nodejs.org/job/node-test-commit-custom-suites/898/default/console

test-rackspace-ubuntu1604-x64-1

00:04:15 not ok 9 internet/test-dns-idna2008
00:04:15   ---
00:04:15   duration_ms: 35.304
00:04:15   severity: fail
00:04:16   exitcode: 1
00:04:16   stack: |-
00:04:16     (node:25878) ExperimentalWarning: The dns.promises API is experimental
00:04:16     /home/iojs/build/workspace/node-test-commit-custom-suites/default/test/common/index.js:675
00:04:16     const crashOnUnhandledRejection = (err) => { throw err; };
00:04:16                                                  ^
00:04:16     
00:04:16     Error: queryA ESERVFAIL straße.de
00:04:16         at QueryReqWrap.onresolve [as oncomplete] (internal/dns/promises.js:167:17)
00:04:16   ...

It failed again today. So it failed each of the last three runs. (The test is run once a day in CI.) Interesting that when it fails, it fails reliably. And when it doesn't fail, it doesn't fail at all.

https://ci.nodejs.org/job/node-test-commit-custom-suites/899/default/console

test-rackspace-ubuntu1604-x64-1

00:04:16 not ok 9 internet/test-dns-idna2008
00:04:16   ---
00:04:16   duration_ms: 30.292
00:04:16   severity: fail
00:04:16   exitcode: 1
00:04:16   stack: |-
00:04:16     (node:22481) ExperimentalWarning: The dns.promises API is experimental
00:04:16     /home/iojs/build/workspace/node-test-commit-custom-suites/default/test/common/index.js:675
00:04:16     const crashOnUnhandledRejection = (err) => { throw err; };
00:04:16                                                  ^
00:04:16     
00:04:16     Error: queryA ESERVFAIL straße.de
00:04:16         at QueryReqWrap.onresolve [as oncomplete] (internal/dns/promises.js:167:17)
00:04:16   ...

@nodejs/build Thoughts on this one?

Ooh, this time around things are different from when initially reported. The test is taking 30 seconds or so before failing. Probably some throttling going on somewhere, yeah? I'm logged on and will try running the lookups/resolves in series rather than in parallel and see if that does anything...

@Trott

@Trott I've (temporarily) unpinned this issue as we can only have three pinned and I think the current test breakage on master (#26528) should definitely be pinned. We can swap then back when master is unbroken.

So... node-daily-master is unbroke.
But... this test will still be flaky if the system's DNS server fail to resolve straße.de.
.
.
.
I just had an idea: pass the dns.lookup test if the error is Error: queryA ESERVFAIL straße.de since IIUC it mean node passed the correct query to the OS.

test: de-flake test-dns-idna2008.js

https://ci.nodejs.org/job/node-test-commit-custom-suites-freestyle/5173/testReport/junit/(root)/test/internet_test_dns_idna2008/, but but but, it passed CI!?

fix of fix in #26570

test: fix syntax error in test-dns-idna2008.js when failing

test: de-flake test-dns-idna2008.js

test: fix syntax error in test-dns-idna2008.js when failing

test: de-flake test-dns-idna2008.js

test: fix syntax error in test-dns-idna2008.js when failing

test: fix test-dns-idna2008.js

Navigation Menu

Repository navigation

investigate flaky internet/test-dns-idna2008 test #25870

6 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

investigate flaky internet/test-dns-idna2008 test #25870

Description

Activity

bnoordhuis commented on Feb 1, 2019

Trott commented on Feb 2, 2019

Trott commented on Feb 2, 2019

bnoordhuis commented on Feb 3, 2019

Trott commented on Feb 3, 2019

Trott commented on Feb 3, 2019

Trott commented on Feb 3, 2019

Trott commented on Mar 3, 2019

Trott commented on Mar 4, 2019

Trott commented on Mar 5, 2019

Trott commented on Mar 5, 2019

Trott commented on Mar 5, 2019

6 remaining items

richardlau commented on Mar 8, 2019

refack commented on Mar 8, 2019

refack commented on Mar 10, 2019

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions