Skip to content

URL parse fails #25879

Closed
Closed
URL parse fails#25879
Labels
i18n-apiIssues and PRs related to Node.js internationalization support.urlIssues and PRs related to the legacy built-in url module.

Description

@pt-rick
pt-rick
opened on Feb 1, 2019
  • Version: 11.6.0
  • Platform: FreeBSD 11.2-RELEASE-p4
  • Subsystem: url

Simple parse of a URL fails...

test.js
const url = require('url');

const test = new URL("https://www.test.com/test");

error
internal/url.js:243
throw error;
^

TypeError [ERR_INVALID_URL]: Invalid URL: https://www.test.com/test
at onParseError (internal/url.js:241:17)
at new URL (internal/url.js:319:5)
at Object. (/shared/rick/pushy/test.js:3:14)
at Module._compile (internal/modules/cjs/loader.js:721:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:732:10)
at Module.load (internal/modules/cjs/loader.js:620:32)
at tryModuleLoad (internal/modules/cjs/loader.js:560:12)
at Function.Module._load (internal/modules/cjs/loader.js:552:3)
at Function.Module.runMain (internal/modules/cjs/loader.js:774:12)
at executeUserCode (internal/bootstrap/node.js:342:17)

When using the deprecated url.parse, this is the result:

test.js
const url = require('url');

const test = url.parse("https://www.test.com/test");

error
url.js:385
this.hostname = toASCII(this.hostname, true);
^

Error: Cannot convert name to ASCII
at Url.parse (url.js:385:23)
at Object.urlParse [as parse] (url.js:149:13)
at Object. (/shared/rick/pushy/test.js:3:18)
at Module._compile (internal/modules/cjs/loader.js:721:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:732:10)
at Module.load (internal/modules/cjs/loader.js:620:32)
at tryModuleLoad (internal/modules/cjs/loader.js:560:12)
at Function.Module._load (internal/modules/cjs/loader.js:552:3)
at Function.Module.runMain (internal/modules/cjs/loader.js:774:12)
at executeUserCode (internal/bootstrap/node.js:342:17)

Activity

  1. mscdex commented on Feb 1, 2019

    @mscdex
    mscdex
    on Feb 1, 2019
    Contributor

    Both work for me on node master and node v10.15.0.

  2. devsnek commented on Feb 1, 2019

    @devsnek
    devsnek
    on Feb 1, 2019
    Member

    new URL(...) throws for me, as it should, since test.com/test is treated like a relative url, and URL objects are absolute.

    url.parse(...) returns an object with pathname = "test.com/test", which is expected.

  3. bnoordhuis
    added
    invalidIssues and PRs that are invalid.
    urlIssues and PRs related to the legacy built-in url module.
    on Feb 2, 2019

  4. bnoordhuis commented on Feb 2, 2019

    @bnoordhuis
    bnoordhuis
    on Feb 2, 2019
    Member

    I'll close this out, can reopen if necessary.

  5. bnoordhuis
    closed this as completedon Feb 2, 2019

  6. pt-rick commented on Feb 4, 2019

    @pt-rick
    pt-rick
    on Feb 4, 2019
    Author

    This is too simple to not work so was thinking it perhaps has something to do with my environment. But even when removing everything from node_modules, I get this error. So perhaps is there some configuration or environment setting or missing package that I should look for that might be causing this problem? Node on Free BSD perhaps?

    To @mscdex comment, if I make the url a relative url (without 'https://') it doesn't throw for me if I use the deprecated url.parse but does, as you say, throw on new URL. However, both throw for me on an absolute URL as shown in the code above.

  7. sam-github commented on Feb 4, 2019

    @sam-github
    sam-github
    on Feb 4, 2019
    Contributor
    % node -p 'require("url").parse("https://www.test.com/test")'; node --version
Url {
  protocol: 'https:',
  slashes: true,
  auth: null,
  host: 'www.test.com',
  port: null,
  hostname: 'www.test.com',
  hash: null,
  search: null,
  query: null,
  pathname: '/test',
  path: '/test',
  href: 'https://www.test.com/test' }
v11.6.0

    Works for me. Could be something about your system, but its hard to know. Try to strip back to something as simple as possible, like above. Maybe even build node from source if you are using a package. Its hard to know what else to suggest.

  8. pt-rick commented on Feb 4, 2019

    @pt-rick
    pt-rick
    on Feb 4, 2019
    Author

    Thanks @sam-github for some ideas. I'm actually using a library that is calling url.parse so I managed to strip this down to as far as I can - 2 lines and no node_modules. Doing a cut and paste of your command, I get this:

    node -p 'require("url").parse("https://www.test.com/test")'; node --version

    url.js:385
    this.hostname = toASCII(this.hostname, true);
    ^

    Error: Cannot convert name to ASCII
    at Url.parse (url.js:385:23)
    at Object.urlParse [as parse] (url.js:149:13)
    at [eval]:1:16
    at Script.runInThisContext (vm.js:123:20)
    at Object.runInThisContext (vm.js:312:38)
    at Object. ([eval]-wrapper:6:22)
    at Module._compile (internal/modules/cjs/loader.js:721:30)
    at evalScript (internal/bootstrap/node.js:720:27)
    at executeUserCode (internal/bootstrap/node.js:313:7)
    at startExecution (internal/bootstrap/node.js:276:5)
    v11.6.0

    In stepping through the debugger, I now can see that toASCII doesn't seem to be getting resolved correctly and this is causing it to fail. The definition on line 24 or url.js is as follows:

    const { toASCII } = internalBinding('config').hasIntl ? internalBinding('icu') : require('punycode');

    internalBinding('config').hasIntl is undefined so it is trying to destruct to punycode but that seems to be where the failure is occurring as 'toASCII' is defined as ƒ toASCII() { [native code] }. I see that punycode has been deprecated so not sure if that is the issue.

    Likewise when I change my code to new URL("https://www.test.com/test"), it fails on the call to 'parse' on line 319 of internal/url.js. 'parse' is defined as ƒ parse() { [native code] } and that throws the error there.

  9. ChALkeR
    added
    i18n-apiIssues and PRs related to Node.js internationalization support.
    on Feb 4, 2019

  10. ChALkeR commented on Feb 4, 2019

    @ChALkeR
    ChALkeR
    on Feb 4, 2019
    Member

    @bnoordhuis perhaps this needs reopening?

  11. sam-github commented on Feb 4, 2019

    @sam-github
    sam-github
    on Feb 4, 2019
    Contributor

    That's more interesting. Could you give your node -p process.versions output? I wonder if what you are running into is because of ICU? Perhaps with the right configure options it would be reproducable on other sytems, there are a number of ICU build/configure options.

    Also, where did you get node from? We don't distribute pre-built binaries for FreeBSD. If its a pre-built package, it might be linked against an ICU library that isn't compatible with node. I would suggest building node from source yourself and seeing if you can repro with your own node, that might allow you to isolate at what point this problem got introduced.

  12. sam-github
    reopened this on Feb 4, 2019

  13. sam-github commented on Feb 4, 2019

    @sam-github
    sam-github
    on Feb 4, 2019
    Contributor

    If this is a problem with how someone built node, it might not be a node issue, but I'll reopen while @pt-rick gathers a bit more information, since it looks like he can reproduce with just node and no external deps.

  14. pt-rick commented on Feb 4, 2019

    @pt-rick
    pt-rick
    on Feb 4, 2019
    Author

    @sam-github here's the answer to the process versions. I'll have to get back to you about the FreeBSD package and source later.
    { node: '11.6.0',
    v8: '7.0.276.38-node.13',
    uv: '1.24.1',
    zlib: '1.2.11',
    ares: '1.14.0',
    modules: '67',
    nghttp2: '1.35.1',
    napi: '3',
    llhttp: '1.0.1',
    http_parser: '2.8.0',
    openssl: '1.1.0j',
    icu: '63.1',
    unicode: '11.0',
    cldr: undefined,
    tz: undefined }

  15. pt-rick commented on Feb 4, 2019

    @pt-rick
    pt-rick
    on Feb 4, 2019
    Author

    @sam-github Node was installed using the FreeBSD 'pkg' command. It was the latest port available.

    Adding @bradleythughes as I believe he contributes to FreeBSD ports.

  16. bnoordhuis
    removed
    invalidIssues and PRs that are invalid.
    on Feb 4, 2019

  17. sam-github commented on Feb 4, 2019

    @sam-github
    sam-github
    on Feb 4, 2019
    Contributor

    @pt-rick have you tried doing your own build? Doing so will definitively show it to be a problem with the package. It should just be an untar, ./configure, make, maybe a bit slow to build, but easy.

  18. pt-rick commented on Feb 5, 2019

    @pt-rick
    pt-rick
    on Feb 5, 2019
    Author

    @sam-github we will give that a try and I'll post the results when available.

  19. pt-rick commented on Feb 5, 2019

    @pt-rick
    pt-rick
    on Feb 5, 2019
    Author

    @sam-github we're running FreeBSD 11.2 and here are the results...

    It’s not too promising that it doesn’t compile. For starters, it says it requires a version of OpenSSL that is newer than what’s on FreeBSD 11.2 (it’s the version from FreeBSD 12.0). But it has a “use bundled OpenSSL” so I enabled that and it took off compiling for a long, long time. Eventually, though, it got this:

    c++ -o /usr/ports/www/node/work/node-v11.8.0/out/Release/obj.target/node_lib/src/node_stat_watcher.o ../src/node_stat_watcher.cc '-DNODE_ARCH="x64"' '-DNODE_PLATFORM="freebsd"' '-DNODE_WANT_INTERNALS=1' '-DV8_DEPRECATION_WARNINGS=1' '-DNODE_OPENSSL_SYSTEM_CERT_PATH=""' '-DHAVE_INSPECTOR=1' '-DHAVE_DTRACE=1' '-DNODE_REPORT' '-D__POSIX__' '-DNODE_USE_V8_PLATFORM=1' '-DNODE_HAVE_I18N_SUPPORT=1' '-DHAVE_OPENSSL=1' '-DHTTP_PARSER_STRICT=0' -I../src -I/usr/ports/www/node/work/node-v11.8.0/out/Release/obj/gen -I/usr/ports/www/node/work/node-v11.8.0/out/Release/obj/gen/include -I/usr/ports/www/node/work/node-v11.8.0/out/Release/obj/gen/src -I../deps/v8/include -I../deps/http_parser -I../deps/llhttp/include -I../deps/brotli/c/include -I../deps/openssl/openssl/include -pthread -Wall -Wextra -Wno-unused-parameter -m64 -Wall -Wextra -Wno-unused-parameter -O3 -fno-omit-frame-pointer -fno-rtti -fno-exceptions -std=gnu++1y -MMD -MF /usr/ports/www/node/work/node-v11.8.0/out/Release/.deps//usr/ports/www/node/work/node-v11.8.0/out/Release/obj.target/node_lib/src/node_stat_watcher.o.d.raw -isystem /usr/local/include -O2 -pipe -fstack-protector -isystem /usr/local/include -fno-strict-aliasing -isystem /usr/local/include -c
    ../src/node_os.cc:107:3: error: unknown type name 'uv_utsname_t'
    uv_utsname_t info;
    ^
    1 error generated.

    We're going to try on FreeBSD 12.0 and see if we can get a build.

  20. sam-github commented on Feb 5, 2019

    @sam-github
    sam-github
    on Feb 5, 2019
    Contributor

    it says it requires a version of OpenSSL that is newer than what’s on FreeBSD 11.2 (it’s the version from FreeBSD 12.0). But it has a “use bundled OpenSSL” so I enabled that

    That makes me nervous. What are you building? I don't think its https://nodejs.org/dist/v11.9.0/node-v11.9.0.tar.gz Nodejs defaults to building against its own bundled OpenSSL, so if you are having to "enable" that option, something has gone wrong.

    unknown type name 'uv_utsname_t' -- node also builds against its own version of libuv by default, so this means you are not using default configure options, and are trying to build against a system libuv.

    Possibly you are rebuilding the FreeBSD package, NOT our vanilla tarball.

    Btw, we build and test green against freebsd 10 and 11 in CI. Every commit, every release.

  21. pt-rick commented on Feb 5, 2019

    @pt-rick
    pt-rick
    on Feb 5, 2019
    Author

    @sam-github We were building the FreeBSD port. Apparently the incompatibility must be there. If we do a port on FreeBSD 12, url.parse succeeds so the issue has to do with the 11.2 port. Thank you for your help!

  22. pt-rick
    closed this as completedon Feb 5, 2019

  23. bradleythughes commented on Feb 6, 2019

    @bradleythughes
    bradleythughes
    on Feb 6, 2019
    Member

    @pt-rick: Since this seems to be an issue with Node.js on 11.2-RELEASE, can you open a bug for it on the FreeBSD project's Bugzilla?

  24. bradleythughes commented on Feb 6, 2019

    @bradleythughes
    bradleythughes
    on Feb 6, 2019
    Member

    @pt-rick Also, for what it's worth, I cannot reproduce this problem on FreeBSD 11.2-RELEASE with Node.js 11.8.0 installed via pkg. As I mentioned, file a bug with the project, and we can discuss and debug your problem there. :)

  25. pt-rick commented on Feb 7, 2019

    @pt-rick
    pt-rick
    on Feb 7, 2019
    Author

    @bradleythughes prior to @sam-github suggestion, we have only used the pkg port so not sure why ours breaks. It does work as expected under FreeBSD 12. I submitted a bug report to Bugzilla.

  26. elcheco commented on Dec 31, 2020

    @elcheco
    elcheco
    on Dec 31, 2020

    I am trying to set up GitLab on FreeBSD and I ended up on here discussed issue. I am now trying to rebuild the Node to 15.4 and also the latest yarn versions because, on the current, it produces exactly the same above described error even without any modules.
    Let's see after the update, but it seems that the error could be still present. The port for node offers also built-in OpenSSL and not the system one. Could this be the thing?

    ❯ pkg version -vIL=
node-15.3.0                        <   needs updating (index has 15.4.0)
yarn-1.22.4_1                      <   needs updating (index has 1.22.10)

    Versions:

    ❯ freebsd-version
12.2-RELEASE

❯ node --version
v15.3.0

❯ npm -v
6.14.8

❯ node -p process.versions
{
  node: '15.3.0',
  v8: '8.6.395.17-node.22',
  uv: '1.40.0',
  zlib: '1.2.11',
  brotli: '1.0.9',
  ares: '1.16.1',
  modules: '88',
  nghttp2: '1.41.0',
  napi: '7',
  llhttp: '2.1.3',
  openssl: '1.1.1h-freebsd',
  cldr: '37.0',
  icu: '67.1',
  tz: '2019c',
  unicode: '13.0'
}

❯ node -p 'require("url").parse("https://www.test.com/test")';
node:url:390
      this.hostname = toASCII(this.hostname, true);
                      ^

TypeError: Cannot convert name to ASCII
    at Url.parse (node:url:390:23)
    at Object.urlParse [as parse] (node:url:155:13)
    at [eval]:1:16
    at Script.runInThisContext (node:vm:133:18)
    at Object.runInThisContext (node:vm:310:38)
    at node:internal/process/execution:77:19
    at [eval]-wrapper:6:22
    at evalScript (node:internal/process/execution:76:60)
    at node:internal/main/eval_string:23:3 {
  code: 'ERR_INVALID_ARG_VALUE'
}

  27. elcheco commented on Jan 1, 2021

    @elcheco
    elcheco
    on Jan 1, 2021

    After NodeJs upgrade to version 15.4.0, it works ;-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    i18n-apiIssues and PRs related to Node.js internationalization support.urlIssues and PRs related to the legacy built-in url module.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions