This repository was archived by the owner on Sep 2, 2023. It is now read-only.
Repository navigation
This repository was archived by the owner on Sep 2, 2023. It is now read-only.
The type field and .node #474
Description
So far we've told people that type can be used to switch how .js is interpreted. In nodejs/node#31388 @bmeck pointed out that setting it to module also has side-effects on how .node is interpreted. To me that sounds really counter-intuitive and surprising. Are there any meeting notes for this? Right now I would consider it a bug that's worth fixing.
For reference, this is the current effective extension map delta:
const typeModule = {
'': 'module',
'.js': 'module',
'.node': 'error',
};
const typeCommonJS = {
'': 'commonjs',
'.js': 'commonjs',
'.node': 'commonjs',
};I'm omitting .json because with experimental JSON modules the difference goes away. But without the flag, it is also affected by type.
Activity
bmeck commentedon Jan 17, 2020
If memory serves it was pulled out due to compat concerns. Don't remember when, but seems we should discuss what type "module" is intended for rather than adding .node to it prematurely
ljharb commentedon Jan 17, 2020
I’ll again link to #283 as the imo preferred way to handle this ambiguity that will come up for an unknowable number of new module types in the future.
hybrist commentedon Jan 17, 2020
I think my memory was that we intended for
.nodeto always be an error when imported, not just when it happens to be atype:modulepackage. But it's also totally possible that I misunderstood the discussion when it happened.If node doesn't support a new extension, then it doesn't seem to be valuable for a package to declare it. If a loader hook adds the support for loading a format, it can also provide the format. Allowing packages to come up with their novelty extensions for existing formats doesn't seem like a super compelling use case to me..?
Our current direction for loader hooks (determined before loading user code, immutable once set) seem to preclude that the package's extension map can introduce new formats.
GeoffreyBooth commentedon Jan 17, 2020
We have two categories of files: those that have an explicit parse goal inherent in the file type, and ambiguous files that can be either Script or Module.
.mjsand.cjsare examples of the former,.jsis the latter. How would people categorize all of our other natively supported types:.node,.json,.wasm?My understanding, please correct me:
.mjs..js; but it’s always the same in either parse goal so the parse goal/"type"is irrelevant..node. . . ?Yes, this is my recollection as well. We document in https://nodejs.org/api/esm.html#esm_commonjs_json_and_native_modules that JSON and native modules need to be pulled into an ES module via
require(fromcreateRequire).ljharb commentedon Jan 17, 2020
@jkrems it's not about allowing novelty extensions (altho ofc it would allow that), it's about not having to decide the semantics of a field like "type" for all future extensions as well as the complex combo of current ones.
The extension map would be a subset of loader hooks; they don't conflict as far as I'm aware.
bmeck commentedon Jan 17, 2020
@GeoffreyBooth
I'd like to be wary of attempting to see if something "is a Module" (unclear if meaning an Abstract Module Record here or in "type":"module"). WASM can be loaded through import but also just via an API. Virtually anything can be transformed into an Abstract Module Record and many things can be exposed without using Module Records at all (JSON/WASM/etc.).
I don't fully understand the comment.
hybrist commentedon Jan 17, 2020
To me that comes down to the question: Do we believe that we'll add more ambiguous file extensions? As long as extension-less and
.jsare the only ambiguous file extensions, it feels perfectly fine to say "type determines how to interpret those". I would hope that we never add another, distinct, new kind of ambiguous file extension to node.hybrist commentedon Jan 17, 2020
Somewhat echoing @bmeck: No, it can be neither. "module" refers to ESM specifically, not any kind of entry in either "module map"/cache. JSON is always
application/json, independent of require vs. import.bmeck commentedon Jan 17, 2020
@jkrems at least in my last call with Dan they wanted WASI and WASM to be in the same file extension.
GeoffreyBooth commentedon Jan 17, 2020
The
"type"field is intended to define the parse goal of ambiguous files when they’re referenced viaimport. That’s why it applies to both.jsand extensionless files.We only need to worry about how
"type"affects other file types when we want to add support for them toimportand they’re supported inrequire. Assuming we never supportrequireof WASM, thenimportcan always import WASM as an ES module (e.g. using the Module parse goal) and that’s it.I’m referring to parse goal. I thought that all WASM files were in parse goal Module, are they not? If so,
.wasmis equivalent to.mjsin that regard. To be used in a CommonJS context, one would need to useimport().bmeck commentedon Jan 17, 2020
They are not. They are a completely different parser.
hybrist commentedon Jan 17, 2020
That feels a bit awkward for node. I guess it means that node wouldn't support "raw" WASM imports, only ones with WASI semantics (including capabilities..?). For the former there's always the direct APIs though and fortunately we still have a dedicated experimental flag for importing
.wasm.GeoffreyBooth commentedon Jan 17, 2020
How will browsers support loading WASM? Via
importstatements? Will they also support WASM via<script>tags, and if so, what would thetypeof such a tag be?I assume WASM has only one parse goal?
bmeck commentedon Jan 17, 2020
Yes, using script tags (type=module) and/or import. The type wouldn't determine the format/parser used, it determines the loader (classic vs module , though other kinds exist).
Yes, but it is starting to have 2 distinct semantic goals (WASI vs WASM).
GeoffreyBooth commentedon Jan 17, 2020
So it sounds like browsers are treated WASM the same way they treat ES module JavaScript. Is there an issue with Node treating
.wasmfiles the same way we treat.mjsfiles? In that they can only be used viaimport/import(), and they are always parsed the same way regardless of"type".bmeck commentedon Jan 17, 2020
No, they remain decoupled in terms of format. They use the same loader workflow though.
Yes, that would not allow WASM to be loaded. I don't think I understand this question fully.
This is not true today, per the
WebAssemblyglobal.This likely won't be possible if WASI/WASM fully diverge.
hybrist commentedon Jan 17, 2020
That's already true behind
--experimental-modules. But doesn't address extension-less files.I think one conceptual issue here is that the
typefield inpackage.jsonand thetypeattribute in the<script>tag are fundamentally different. The field inpackage.jsondetermines theContent-Typeof ambiguous files. It has no effect on the module system. The attribute on<script>tags determines the module system to use (or rather to enable a module system at all). It has no effect on theContent-Type.With this, the
typeattribute of a<script>tag that loads WASM is clearly "module" because it's loaded using the module system. But that doesn't tell us anything about thepackage.jsonfield because theContent-Typeheader is what determines the file format. And the file format is what is controlled by thepackage.jsonfield.bmeck commentedon Jan 17, 2020
This is not purely the case, it does have potential alterations to other things like which loader to use for the main entry point even if the file of the main entry point is not "ambiguous".
GeoffreyBooth commentedon Jan 17, 2020
@bmeck In all cases I’m just talking about the public API that end users see, with regard to importing/loading the file. It sounds like
.wasmis equivalent to.mjsas far as end users are concerned (both are imported via thescript type=module,import, etc.).bmeck commentedon Jan 17, 2020
I don't understand this. Not for/against any part of the comment; I simply am confused.
GeoffreyBooth commentedon Jan 17, 2020
End users don’t really need to know or care about the module system or what’s happening under the hood. They can just treat
.wasmfiles the same way they treat ES module JavaScript files, as far asscripttags andimportstatements are concerned. I assume the server needs to send a WASMContent-Type, which is equivalent to us requiring a.wasmextension.bmeck commentedon Aug 4, 2020
discussion seems to be focused around WASM after some minor back and forth; maybe we can migrate the last bit of discussion to #457 ; feel free to reopen if there is disagreement