Hashbang not resolving --loader relative to entry

Version: 11.0
Platform: macOS 10.14 (18A389)
Subsystem: bootstrap, --experimental-modules (only)

When using shebangs to specify --loader specifically for --experimental-modules a file could want to use a relative path to it's own loader file.

In this specific case, the module file is it's own loader, as if things are not already complicated enough.

#!/usr/bin/env node --experimental-modules --loader ./index.mjs

export function resolve(... args) {
  // …
  if (resolved.url === import.meta.url) resolve.url += '#initialize';
  return resolved;
}

function bootstrap() {
  // … 
}

import.meta.url.endsWith('#initialize') && bootstrap();

The issue is not related to modules down the road, it is strictly related to the current way relative --loader paths are resolved.

Executing this file from other paths other than it's own directory looks for ./index.mjs in the cwd and not the file's own directory.

I believe that Loader being a variable aspect of a shebang is at least problematic, more so risky, and simply not practical (I could be wrong).

Can we fix (or justify) this bug (or feature) please?

this isn't a bug. most OSes only use the first argument of a shebang. we can't put flags in it.

also your example code won't really work because the file with the loader isn't used to resolve itself because it doesn't exist yet at that point.

It actually works for me fine, except if cwd is not the same.

In the folder, node works and the loader kicks in.

Outside the folder (anywhere else) calling node [relative or absolute]/index.mjs throws always indicating ./index.mjs not found, the path has no effect on the error message.

Edit: My use case is not most OSes.

I don't think we should try to encourage using a shebang like this because it won't work in most places, even in the same cwd. https://linux.die.net/man/2/execve

Can we consider this not less lowest-common-denominator shebangs, but more from the general case of a shebang that supports this (my case and maybe more):

Command line

[cwd]» node --experimental-modules --loader ./index.mjs index.mjs

Shebang

#!/usr/bin/env node --experimental-modules --loader ./index.mjs

Can we agree that in the cases where they work (and the do for me) between those two, ./index.mjs in shebang from the perspective of the creator of such file is intuitive to say the least to not implicitly resolve cwd. Where as the command line, it is obviously intentional because it is right then and there.

Contrast this --loader case with other paths passed to node, you may be inclined to say that ARGVs are more ambiguous (again assume cases where it works). Some might think their shebang #!/usr/bin/env node -- ./package.json (obviously fictitious just to elaborate thought) is intended to refer to the package.json for the cwd and few might still thing it resolves their own folder's package.json, this case is beyond the scope of this issue. I would hesitate to even go there because I don't know those guys anyways and one of them seems a little weird, but to each their own.

How can we rationalize --loader a little deeper, is there similar precedence with --require, I really don't know.

Any thoughts?

Note: there is only one index.mjs

@SMotaal

@SMotaal i'm not sure what you're trying to say, but I think that the current behaviour is how it should be.

Any other Collaborators want to chime in on this one? Seems like the flags-in-shebang issue/question/request could conceivably be especially relevant to @nodejs/tooling folks?

this isn't a bug, nor is it a "feature". hashbangs are an os level construct, not something node provides.

If this is a problem specific to shebangs, I don’t understand why. I do understand the lowest-common-denominator shebang behavior, but not how this relates to Node’s handling of relative paths.

If the desired behavior is limited to certain OSes and would break in others, it may be difficult to rally much support.

@SMotaal

my understanding of this issue was that @SMotaal wanted to use hashbangs for esm and something about it wasn't working. if the issue isn't specific to hashbangs can it be rephrased?

@boneskull

@boneskull I think it depends on how you look at it and I am finding it tricky to rephrase/reframe this as @devsnek pointed out but I will try.

One way to look at it is to look at it relative to the current implementation. Essentially, the fact that it is a hashbang does not factor into the behaviour. There is no distinction between running node --experimental-modules --loader ./index.mjs from the command line and having it execute by calling the actual file.

Another way to look at it is to consider the practical uses of the different aspects. A hashbang is a static header of the main entry point of the node process. A custom --loader is specified to be loaded before an entry module which requires the loader specifically in order for it to be loaded and executed. Between those two concepts, it is easy to appreciate the edge case where an entry point and a loader are lumped together in a folder or within a prescribed relative proximity so that the entry point will include a relative path so that only when the arguments are passed via a hasbang would this particular path be resolved relative to the entry point in which the hashbang resides.

I am not ignoring the technical considerations that this would require. At the same time I am considering the current limitations on the future usability of this flag.

@SMotaal

@SMotaal you're saying that paths in the hashbang should be resolved from the file location, not the cwd?

@devsnek

I am not sure I am prepared to make this generalization on all paths, but for loader specifically and cases where the path points to a resource that bootstraps the runtime needed for the entry point to actually function.

… paths in the hashbang … — @devsnek

This part is really where I see a divergence where I imagine that node will need a mechanism to distinguish when it is executing from a hashbang. If not offered by platforms, then a very raw idea to do this is to ensure that the stringified arguments (spaces notwithstanding) occurs at index 0 of the main entry before it even tries to locate the loader — obviously this needs work and just intended to inspire a better and cleaner solution.

For reference, --require works the same way:

. means "current working directory", so changing the meaning of . to "directory of the module to be required" (or loaded, etc) seems incorrect/inappropriate.

What could be done is add a new node flag, e.g., node --cwd /some/path or node --cwd .., which would switch directories before other flags were interpreted. I don't know if this is a good idea.

I don't know if this works for --loader, but a workaround would be to use NODE_PATH, e.g.:

$ NODE_PATH=..:${NODE_PATH} ../bar.js
foo

@SMotaal

@SMotaal ok thanks for rephrasing.

i would still say that the current behaviour is "working as intended":

1st, hashbangs are "above" node, we don't define their behavior or interpret them.

2nd, node exec flags (such as --loader) are purposely separated from the entry point. they effect the entire application and as such are tied to the environment, not the specific script.

@targos

@targos I think what is missing is input from those with experience to weigh in on my very raw suggestion first. While I normally didn't mind taking a few days or weeks to catch up when problem solving, I think that this one is one too deep in my 1.5 yr distraction which initial was just to be able to use modules back in 2017 😄.

Then, if we see that there is a possible solution that has due confidence, I am happy to do my part.

@boneskull

@boneskull I absolutely agree on the meaning of . for which a different meaning should be very carefully thought out.

I wanted to explore ideas like dropping the . or somehow wrapping the argument to denote it's special meaning, which can be a separate avenue from all together and avoids shebanging our heads on the more complicated shebang idea if it is not a favourable improvement or favourable.

@SMotaal

@SMotaal Sure. I didn't want to say that you must be the one who does the investigation, sorry if I wasn't clear on that point. I just want to make sure we answer the questions in the right order :)

Interpreter directives (Hashbangs / Shebangs) do not parse consistently across shells. I do not believe we should attempt to specify behavior within node itself due to this as node's behavior could differ from shell behavior.

I'm a bit confused by this, there seem to be a mix of issues discussed, but note that most scripting languages use posix option syntax, and have short-option versions of any long options, particularly any long options that are credible candidates for use in #! lines, so while

#!/usr/bin/node --experimental-modules --loader ./index.mjs

would not work, traditional scripting languages would allow something like

#!/usr/bin/node -EL/usr/lib/some-app/loader/index.js

which fits within the "only one optional arg" limitation of exec.

Node doesn't use standard option syntax, so this doesn't work for it. Perhaps it should.

This is something I've really been wanting, my current solution is to use the ":" //#; exec hack to pass arguments. I think a good alternative solution to changing how loader works would be to have an alternative flag --relative-loader that resolves the loader relative to the main module (rather than the cwd).

This doesn't tie to to shebangs in anyway as it would just work as a command e.g.:

# This resolves the path given relative to ./path/to/my/module.js rather than the cwd 
# (e.g. path.resolve("./path./to/my/module.js", "./my-loader.js")
node --relative-loader ./my-loader.js ./path/to/my/module.js

# This would resolve ts-node/esm relative to ./path/to/my/node_modules/ rather than the cwd
# (e.g. like import.meta.resolve("ts-node/esm") from within "./path/to/my/module.js")
node --relative-loader ts-node/esm ./path/to/my/module.js

This would make shebangs be as simple as:

#!/usr/bin/env -S node --relative-loader ts-node/esm

console.log("Hello");

I have an alternate proposal that could resolve this a little more properly: #40594

Worth noting that multiple arguments in a shebang is non-portable anyways: https://stackoverflow.com/questions/4303128/how-to-use-multiple-arguments-for-awk-with-a-shebang-i-e - that's one of the motivating factors of that feature request.

There has been no activity on this feature request for 5 months and it is unlikely to be implemented. It will be closed 6 months after the last non-automated comment.

For more information on how the project manages feature requests, please consult the feature request management document.

There has been no activity on this feature request and it is being closed. If you feel closing this issue is not the right thing to do, please leave a comment.

For more information on how the project manages feature requests, please consult the feature request management document.

Navigation Menu

Repository navigation

Hashbang not resolving `--loader` relative to entry #23868

11 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Hashbang not resolving --loader relative to entry #23868

Description

Activity

devsnek commented on Oct 25, 2018

SMotaal commented on Oct 26, 2018

devsnek commented on Oct 26, 2018

SMotaal commented on Oct 26, 2018

devsnek commented on Oct 26, 2018

Trott commented on Nov 19, 2018

devsnek commented on Nov 19, 2018

boneskull commented on Nov 19, 2018

devsnek commented on Nov 19, 2018

SMotaal commented on Nov 19, 2018

devsnek commented on Nov 19, 2018

SMotaal commented on Nov 19, 2018

boneskull commented on Nov 19, 2018

devsnek commented on Nov 19, 2018

11 remaining items

SMotaal commented on Nov 20, 2018

SMotaal commented on Nov 20, 2018

targos commented on Nov 20, 2018

bmeck commented on Feb 3, 2020

sam-github commented on Mar 17, 2020

Jamesernator commented on Dec 14, 2020

dead-claudia commented on Oct 25, 2021

github-actions commented on Apr 24, 2022

github-actions commented on May 25, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Hashbang not resolving `--loader` relative to entry #23868