Top-level await support in REPL

Hey there,

Great work on ts-node! I just looked through the code a bit, and you all have clearly put a lot of thought and time into it. Thank you.

One request: I love using the REPL for quick exploration/experimentation/learning. And that often involves async steps, e.g. DB queries or API requests.

Before TypeScript, we used to use Streamline.js, and I loved that its REPL supported making async calls and awaiting their responses:

$ _node
_node> fs.stat('/dev/null', _)
{ dev: 1011986504, ... }
_node> r.get('https://httpbin.org/ip', _).body
'{\n  "origin": "1.2.3.4"\n}\n'
_node> ...

I understand that TypeScript and ES6/7 as languages don't allow top-level await in modules, but would you be open to supporting that in the REPL?

The way the Streamline REPL does it is simply by wrapping the code in an async IIFE, and only calling the REPL eval callback once the async IIFE finishes:

https://github.com/Sage/streamlinejs/blob/v2.0.13/lib/repl.js#L23-L42

Thank you in advance for the consideration! Cheers.

Off then top of my head, I don't think it's possible to support without more hacking of the TypeScript compiler and/or into the future with ES6 modules. You need an async function for await to compile, but you can't import from inside a function (it must be at the top-level). Happy to keep the issue open though.

Thanks for the quick reply!

Could ts-node wrap in async IIFE only if the input had no imports? (And anything else problematic?)

It could still be really valuable to support top-level await for simple cases, even if it wasn't supported for more complex ones.

Maybe just .then() before printing?

> fetch('https://google.com').then(r => r.text())
... awaiting promise id: 2 (Ctrl-C to cancel)
'<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">\n<TITLE>302 Moved</TITLE></HEAD><BODY>\n<H1>302 Moved</H1>\nThe document has moved\n<A HREF="http://www.google.co.nz/?gfe_rd=cr&amp;ei=uK12WNbvMMTN8gfyir2ADw">here</A>.\n</BODY></HTML>'

Ideally do it more interactively, eg. show the prompt below the currently awaiting promises, print them as they arrive or something, but it's a start.

related topic:
https://gist.github.com/Rich-Harris/0b6f317657f5167663b493c722647221

Temporary workaround which blocks the REPL while resolving:

npm install deasync --save-dev
# or
yarn add deasync --dev

const awaiter = promise => { const nothing = Symbol(); let ret = nothing; promise.then(response => { ret = response }); while(ret === nothing) { require('deasync').runLoopOnce(); } return ret; }

const value = awaiter(somePromise)

There is top level await in the node 10+ repl and it could theoretically be utilized running node --experimental-repl-await --eval 'require("./node_modules/ts-node/dist/bin.js")'. However, this is stopped by the TypeScript compiler complaining

> const a: string = await Promise.resolve("a")
[eval].ts(1,19): error TS1308: 'await' expression is only allowed within an async function.

which is totally right, since top level await is not (yet?) valid JavaScript but a hack for repl convenience.

I don't see a way to get the await keyword untouched through the TypeScript compiler.

@jessedvrs

Updated version of @jessedvrs's workaround that works in a type safe fashion:

npm install deasync2 --save-dev
# or
yarn add deasync2 --dev

function wait<T>(promise: Promise<T>): T { return require('deasync2').await(promise); }

const value = wait(somePromise);

@webmaster128

@webmaster128 If I understand it, that flag would not do anything for ts-node since it would only apply to the default REPL instance. We'd have to go ahead and implement something like nodejs/node@eeab7bc ourselves here. It should be a bit easier to do for us, but we'd need to make sure we're handling all the same things.

@blakeembrey

@blakeembrey if you run node --experimental-repl-await --eval 'require("./node_modules/ts-node/dist/bin.js")', then you set process.binding('config').experimentalREPLAwait = true via the node command line (see nodejs/node#19604). This runs our bin.ts, which then uses lib/repl.js I guess via import { start, Recoverable } from 'repl'. Thus I expect it to work automatically. I think the only one that stops this from working is the TypeScript compiler, that says top level await is invalid.

@webmaster128

@webmaster128 I think there's a misunderstanding. All the code to handle top-level await is within defaultEval, which we don't use because we aren't evaling as JavaScript directly. Therefore we need to replicate all the logic to handle this into ts-node - the flag won't do anything since it only applies to the node.js REPL and you aren't opening the node.js REPL. The TypeScript compiler fix is the trivial part, we just ignore that diagnostic in the REPL.

I think there's a misunderstanding. All the code to handle top-level await is within defaultEval, which we don't use because we aren't evaling as JavaScript directly.

You are probably right. I was a bit too optimistic here and did not follow the core carefully enough.

Would it be possible to implement the await keyword in the ts-node REPL using the wait<T>(promise: Promise<T>): T function from above?

@blakeembrey

The TypeScript compiler fix is the trivial part, we just ignore that diagnostic in the REPL.

@blakeembrey could you elaborate on that part? I think I have a solution for running top level await in JavaScript utilizing the JavaScript AST (very similar to https://github.com/ef4/async-repl/blob/master/stubber.js and working in the ts-node REPL setup). However, I don't know how to teach typescript to pass the await untouched from TS code to JavaScript.

There’s a feature in the README that allows you to ignore TypeScript diagnostic codes, it’d just be reusing that from the REPL.

Top-level await seems to work at least to some extent with esm package after disabling both 1308 and 2304 diagnostics in source files. TS_NODE_IGNORE_DIAGNOSTICS=1308,2304 TS_NODE_COMPILER_OPTIONS='{"target":"es6"}' node -r esm -r ts-node/register topAwait.ts

Ignoring TS1308 is pretty simple and side-effect free since it seems to be related only to top-level await. Unfortunately TS2304 is required mainly due to error TS2304: Cannot find name 'await' and this suppresses all other 'cannot find name' errors.

It doesn't seem to work in command-line REPL though if started without a source file defined.

I don’t think we can rely on the target though, can we instead parse and wrap the file before giving to to TypeScript instead?

That would require this AST manipulation in TypeScript and not in JavaScript. I think the mayor challenge is how to expose local variables in TypeScript when rewriting:

let a = await 1;

a // 1

as

(async () {
  let a = await 1;
})();

a // not available

what happens in the JavaScript version is that local declarations are converted to global properties (which only works in non-strict mode), e.g.

let a = await 1;

a // 1

becomes

(async () {
  a = await 1;
})();

a // 1

@webmaster128

@webmaster128 That's a very good point, I didn't consider how the previous JS approach got around this issue either. Why couldn't you do AST manipulation with TypeScript though? I'm not sure myself how mature that API is, but transforms do exist as a feature.

Another question, but how would const variables be possible here? I guess we can rely on TypeScript type checking to workaround. It looks like in Chrome (and maybe node.js, haven't tested yet), it's rewritten const so it's no longer a const when using await.

Why couldn't you do AST manipulation with TypeScript though?

Didn't try it.

It looks like in Chrome (and maybe node.js, haven't tested yet), it's rewritten const so it's no longer a const when using await.

Oh, interesting. I thought they had a more sophisticated approach allowing const. The method I use does not support const either on the JS side. But TypeScript takes care of that. But I don't think that converting top level let/const to var semantics in a REPL is a big deal.

Do you have an idea for valid TypeScript that exposes variables declared in an async function body to the global scope?

Hello everyone, just stumbled on this one. Wondering if there are simple instructions on how to start a TS repl with ts-node and simply use async 100 without being hit with:

[eval].ts:1:1 - error TS1378: Top-level 'await' expressions are only allowed when the 'module' option is set to 'esnext' or 'system', and the 'target' option is set to 'es2017' or higher.

Would love to know if there has been any updates to this one. Is it possible to do?

@feliperyan

@feliperyan have you tried setting options they way the error message tells you to?

Solutions mentioned here (ignore TS error + --experimental-repl-await) work with scripts but not in REPL mode.

Does anyone know how node's REPL handles top-level await? Can we use their top-level await mechanism? Is it exposed as an API? Can we copy their source code?

Someone will need to propose how we can implement this.

In case it helps anyone else, I've had good results in REPL mode with node --require ts-node/register/transpile-only --experimental-repl-await (using node 14)

I was playing around with the repl today and managed to get it working. Looking at the issue about native ESM support with ts-node helped

Environment

Node.js Version: lts/fermium (v14.16.0)
Typescript Version: 4.2.2

Configuration

package.json (just the important bit to allow treating .js as ES6 modules):

{
  "type": "module",
  "main": "./src/index.js"
}

Typescript config (just the important bit):

{
  "compilerOptions": {
     "allowSyntheticDefaultImports": true,
      "target": "es2017",
      "module" "ESNext"
  },
  "include": ["src/**/*"]
}

Usage

If you are loading a typescript module that has a top level await:

node --experimental-repl-await --loader ts-node/esm ./src/clients.ts

node --experimental-repl-await --loader ts-node/esm

To import a module within the repl, use dynamic imports.

> const clients = await import('./src/clients.ts')

In case it helps anyone else, I've had good results in REPL mode with node --require ts-node/register/transpile-only --experimental-repl-await (using node 14)

It's worth noting this only helps for importing typescript files from the REPL, but you can't use any TS syntax in node REPL, ie:

const foo: string = 'bar';

const foo: string = 'bar';
      ^^^

Uncaught SyntaxError: Missing initializer in const declaration

Does anyone know how node's REPL handles top-level await? Can we use their top-level await mechanism? Is it exposed as an API? Can we copy their source code?

Someone will need to propose how we can implement this.

I've inspected the source and it seems it's handled here:

https://github.com/nodejs/node/blob/975bbbc443c47df777d460fadaada3c6d265b321/lib/repl.js#L424-L440

https://github.com/nodejs/node/blob/975bbbc443c47df777d460fadaada3c6d265b321/lib/internal/repl/await.js#L91-L176

Navigation Menu

Repository navigation

Top-level await support in REPL #245

2 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Top-level await support in REPL #245

Description

Activity

blakeembrey commented on Dec 2, 2016

aseemk commented on Dec 2, 2016

simonbuchan commented on Jan 11, 2017

unional commented on Mar 22, 2017

jessedvrs commented on Mar 28, 2018

webmaster128 commented on Jul 20, 2018

webmaster128 commented on Jul 20, 2018

blakeembrey commented on Jul 21, 2018

webmaster128 commented on Jul 21, 2018

blakeembrey commented on Jul 21, 2018

webmaster128 commented on Jul 21, 2018

webmaster128 commented on Aug 9, 2018

blakeembrey commented on Aug 10, 2018

2 remaining items

Pokute commented on Aug 23, 2018

webmaster128 commented on Aug 23, 2018

blakeembrey commented on Aug 26, 2018

webmaster128 commented on Aug 27, 2018

feliperyan commented on May 28, 2020

ilyakatz commented on Jul 1, 2020

cspotcode commented on Jul 1, 2020

guifel commented on Oct 5, 2020

cspotcode commented on Oct 5, 2020

dansebcar commented on Oct 6, 2020

vamsiampolu commented on Feb 25, 2021

ejose19 commented on Jul 3, 2021

ejose19 commented on Jul 3, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions