module: add --experimental-strip-types

[marco-ippolito]

It is possible to execute TypeScript files by setting the experimental flag --experimental-strip-types.
Node.js will transpile TypeScript source code into JavaScript source code.
During the transpilation process, no type checking is performed, and types are discarded.

Motivation

I believe enabling users to execute TypeScript files is crucial to move the ecosystem forward, it has been requested on all the surveys, and it simply cannot be ignored. We must acknowledge users want to run node foo.ts without installing external dependencies or loaders.

There is a TC39 proposal for type annotations

@legendecas is trying to organize a meeting with the TypeScript team, so that we could ask some questions about long term support, compiler options etc...

Why type stripping

Type stripping as the name suggest, means removing all the types, transform the input in a JavaScript module.

const foo: string = "foo";

Becomes:

const foo = "foo";

Other runtimes also perform transformation of some TypeScript only features into JavaScript, for example enums, which do not exists in JavaScript.
At least initially in this PR no trasformation is performed, meaning that using Enum will not be possible.
I believe as a project we don't want to vendor TypeScript directly and I would like to push the ecosystem to STANDARDIZE these features (example: https://github.com/tc39/proposal-decorators) so that they become part of the language.

The flag --experimental-require-module is currently not supported.

Why I chose @swc/wasm-typescript:

Because of simplicity.
I have considered other tools like swc/core and esbuild but they require either rust or go to be added to the toolchain.
@swc/wasm-typescript its a small package with a wasm and a js file to bind it.
swc is currently used by Deno for the same purpose, it's battle tested.
In the future I see this being implemented in native layer.
Currently why just npm pack @swc/wasm-typescript, perform some transformation and add it to our sources.
This is not the standard way we operate, we want to have the sources under versioning to be able to hotfix and monitor possible security issues.
This could be solved by vendoring swc and build the wasm ourselves.
I will do that in a immediate future iteration.
Massive shoutout to @kdy1 for releasing a swc version for us.

Why not TSC (typescript.js)

typescript.js is a (at the time of writing this) a ~8.5MB and it does not perform type-stripping, but only transformation.

Limitations

TypeScript only features

By design Node.js will not perform transformations on TypeScript only features.
This means Enum, experimentalDecorators, namespaces and other TS only features are not supported.

.tsx extension is not supported.

Importing modules without extension is not supported.

  import { foo } from './foo'; // it will not work
  import { foo } from './foo.ts'; // it will work

REPL

Executing source code in the REPL with TypeScript is not supported.
This also applies to --print, --check and inspect.

Source maps

Currently source maps are not supported, this could be trivial to add.
Thanks to whitespacing, (replacing removed syntax with white spaces) the original location is preserved in stacktraces.

No benchmarks

This PR does not have performance in mind, as its a first iteration so no benchmarks and no bundle size measurements are performed

Next Steps

Short term

• Add @swc/wasm-typescript sources and build the wasm ourselves
• Improve UX, performance, etc...

Long term

• Move to @swc/core as a native implementation.

Issue to track progress 👉 nodejs/loaders#208

[nodejs-github-bot]

Review requested:

• @nodejs/actions
• @nodejs/gyp
• @nodejs/loaders
• @nodejs/security-wg
• @nodejs/tsc

[legendecas]

🎉 thank you for the great work!

I think correct source locations in the error stacks are required in the short term as well.

One possible solution could be that avoid altering source locations in stripping so that source maps generation and mapping would not be required. Note that source maps come with a cost and it would not be trivial (per memory usage and performance). (idea from @acutmore)

[marco-ippolito]

🎉 thank you for the great work!

I think correct source locations in the error stacks are required in the short term as well.

One possible solution could be that avoid altering source locations in stripping so that source maps generation and mapping would not be required. Note that source maps come with a cost and it would not be trivial.

Sourcemaps are already supported by swc and we might just add a flag to enabled them.
We could ask swc an option to replace syntax that have been removed with whitespaces so that we don't alter original error stacks

[legendecas]

Support extensionless imports

Not supporting extension-less imports is by-design: https://nodejs.org/api/esm.html#mandatory-file-extensions

[panva]

@marco-ippolito, speaking as a module maintainer here, it is very common for TS module codebases to have its own file imports written with .js extensions in the import statements, despite the files being .ts.

If I'm to replace --import=tsx/esm with --experimental-strip-types, or skip a compile step before testing, this would need to be able to resolve to a .ts file even when .js was in the statement (but not in the filesystem), otherwise I'll have to change the project imports and then deal with more pain from the compiler (or tsc typecheck) before emitting the actual module code to publish.

[marco-ippolito]

@marco-ippolito, speaking as a module maintainer here, it is very common for TS module codebases to have its own file imports written with .js extensions in the import statements, despite the files being .ts.

If I'm to replace --import=tsx/esm with --experimental-strip-types, or skip a compile step before testing, this would need to be able to resolve to a .ts file even when .js was in the statement (but not in the filesystem), otherwise I'll have to change the project imports and then deal with more pain from the compiler before emitting the actual module code to publish.

This is something we definitely want look into, to be able to maintain compatibility with what you run and what you compile.
It's in the checklist for future iterations, it might be resolved with extensions guessing (?), it requires further discussion.

[panva]

This is something we definitely want look into

Glad it's on your mind then!

As a sidenote, would --strip-types be a no-op if https://github.com/tc39/proposal-type-annotations ever came to be?

[marco-ippolito]

This is something we definitely want look into

Glad it's on your mind then!

As a sidenote, would --strip-types be a no-op if https://github.com/tc39/proposal-type-annotations ever came to be?

It's still in Stage 1 so it will take at least a few years but I hope v8 will implement the api to make it happen 🔥

[GeoffreyBooth]

Resolution of import specifiers isn’t specific to module type, so we can’t have extension searching just for TypeScript files; and for many reasons we don’t support it in ESM. However it’s not necessary: users should write their import specifiers with .ts extensions, like import './file.ts', and there’s https://www.typescriptlang.org/tsconfig/#allowImportingTsExtensions allowImportingTsExtensions to support this via tsconfig.json. Such behavior also aligns us with Deno.

If the user wants to use the same source files both run directly via this new flag and run as transpiled JavaScript output by a build tool, it would be the responsibility of the build tool to convert file.ts into file.js; but this is something that build tools are more than capable of handling.

[joyeecheung]

There are two aspects of TypeScript support here:

1. Stripping the syntax (i.e. essentially polyfilling the type annotation as syntax proposal)
2. Modify the resolution rules so that .ts files are accepted, with 1 applied (and as @panva mentioned, allow e.g. import 'foo' to resolve to /path/to/foo/something.ts)

AFAICT this does both (although the second one does it in a limited way), so that brings up the question on whether we want to make this this unflagged eventually. If we want to unflag it eventually then 2 is going to be a breaking change for existing TypeScript loaders that also try to modify the resolution rules in their own way, then we should only strip the syntax for files with explicit formats but don't try to override the resolution rules. If we want to keep it flagged as --strip-syntax eventually, then doing 2 is fine, though we might want to warn that it would be in conflict with other TypeScript loaders that actually do more like supporting enums.

[marco-ippolito]

There are two aspects of TypeScript support here:

1. Stripping the syntax (i.e. essentially polyfilling the type annotation as syntax proposal)
2. Modify the resolution rules so that .ts files are accepted, with 1 applied (and as @panva mentioned, allow e.g. import 'foo' to resolve to /path/to/foo/something.ts)

AFAICT this does both (although the second one does it in a limited way), so that brings up the question on whether we want to make this this unflagged eventually. If we want to unflag it eventually then 2 is going to be a breaking change for existing TypeScript loaders that also try to modify the resolution rules in their own way, then we should only strip the syntax for files with explicit formats but don't try to override the resolution rules. If we want to keep it flagged as --strip-syntax eventually, then doing 2 is fine, though we might want to warn that it would be in conflict with other TypeScript loaders that actually do more like supporting enums.

import foo from "./foo" is not supported, file extension is needed.
With this pr there is no resolution override, the test 👇
https://github.com/nodejs/node/pull/53725/files#diff-2fcd423c995b0d2501236a6f427092c5ab1df06ba9241285a471eebc2f12970dR40
I added an item in the issue to track the work to discuss about this

[AugustinMauroy]

In test suite I think we should add some test to know how node will work with that:

const X = {} as const;

and

import type { Config } from 'myConfig.d.ts';

export const config = {} satisfies Config;

[benjamingr]

This is neat, I'd like to see extensionless imports work if possible, but I also think this can land as-is and those issues fixed before making it stable.

[enricopolanski]

I see few issues with the module approach:

1. Usage of .ts extensions in imports and the allowImportingTsExtensions flag in TypeScript land is virtually non existent. Imports are either extensionless or use .js one (see point 2).

2. To complicate things even further ESMs resolution in TypeScript is mostly achieved in codebases by adding .js extension to imports even if the .js file at that path does not exist.

https://www.typescriptlang.org/docs/handbook/modules/theory.html#module-resolution-for-libraries

This is especially important in library code (as many projects out of there simply opt-out of ESM entirely and either create bundles FE-wise, or they compile down to CJS for Node ones) but the number of real world products using this pattern increases consistently every day with CJS being phased out by an ever growing number of projects.

I wonder whether the flag could strip types and run @swc/wasm-typescript on files regardless of their extension. If it is a normal js file it will simply not need to strip anything (albeit it will still have to parse it).

[marco-ippolito]

This is neat, I'd like to see extensionless imports work if possible, but I also think this can land as-is and those issues fixed before making it stable.

@benjamingr this is currently not possible because of this mandatory-file-extensions, it really depends on the esm loader.

Usage of .ts extensions in imports and the allowImportingTsExtensions flag in TypeScript land is virtually non existent. Imports are either extensionless or use .js one (see point 2).

To complicate things even further ESMs resolution in TypeScript is mostly achieved in codebases by adding .js extension to imports even if the .js file at that path does not exist.

@enricopolanski That is something that might be resolved with guessing, as you can immagine it requires node to perform a few expensive operation on the file system and a heuristic attempt to guess the extension.

Example:

• import foo from './foo.js'
• stat ./foo.js => does not exists
• stat ./foo.ts => exists
  And
• import foo from './foo.ts'
• stat ./foo.ts => does not exists
• stat ./foo.js => exists

We will iterate on this and it's tracked in the issue nodejs/loaders#208

[marco-ippolito]

swc is stable so I will mark this PR as open for reviews @nodejs/loaders

[mcollina]

lgtm

[github-actions]

The notable-change PRs with changes that should be highlighted in changelogs. label has been added by @marco-ippolito.

Please suggest a text for the release notes if you'd like to include a more detailed summary, then proceed to update the PR description with the text or a link to the notable change suggested text comment. Otherwise, the commit will be placed in the Other Notable Changes section.

[magic-akari]

As ES decorators will become a part of JavaScript language (https://github.com/tc39/proposal-decorators),
the syntax, which is indistinguishable from TypeScript's legacy decorators, has led us to remove the associated errors.
In the upcoming release of swc, the presence of decorators will no longer trigger errors. See swc-project/swc#9178.

[GeoffreyBooth]

I thought the whole reason the legacy decorators were legacy was because the approved decorators proposal differed from the first decorators transform that TypeScript shipped? As in, experimentalDecorators is the old/non-standardized/legacy behavior, and the current unflagged syntax is what got standardized?

[magic-akari]

The current state of decorators is quite similar to the situation with class fields. Please refer to this: nodejs/loaders#208 (comment)

[marco-ippolito]

If swc is not going to trasform decorators and leave them untouched, the javascript will be invalid because our v8 version does not support decorators afaik 🤔

[GeoffreyBooth]

But V8 will support them eventually because they add all new syntaxes that become standardized. I think the relevance here is that we shouldn’t be testing for this, as some future V8 update will cause this test to start passing.

So if users write a decorator in their code, our SWC pass should leave it untouched, and then V8 will error on it (for now, until it starts working in a future version of V8). If there’s some way to test that our SWC output didn’t transform the decorator but left it as is, that would be what we’d want to test.

[marco-ippolito]

Moving back to draft since we need to figure out this #53725 (comment)
and maybe add cjs before landing

[marco-ippolito]

Added proper support and tests for commonjs syntax, .cts, .mts, --experimental-default-type, --experimental-detect-module, --eval, --input-type, package.json 🥵

[GeoffreyBooth]

In nodejs/loaders#208 (comment) @DanielRosenwasser of the TypeScript team asked to collaborate with us on this, so I’m blocking until the TypeScript team has had a chance to provide meaningful input.

I strongly support this PR and will happily lift this block once everyone has had a chance to discuss. I’ll also continue to review it and suggest changes in the meantime.

[mhdawson]

@marco-ippolito it would be good if the wasm could be externalized like other wasm components like this one

node/src/node_builtins.cc

Line 49 in b4e8f1b

#ifdef NODE_SHARED_BUILTIN_UNDICI_UNDICI_PATH

[marco-ippolito]

@marco-ippolito it would be good if the wasm could be externalized like other wasm components like this one

node/src/node_builtins.cc

Line 49 in b4e8f1b

#ifdef NODE_SHARED_BUILTIN_UNDICI_UNDICI_PATH

Sure, added to the todo list in the issue

[rauschma]

@GeoffreyBooth wrote:

Such behavior also aligns us with Deno.

Alignment with Deno seems an important consideration. Someone familiar with Deno should be able to provide useful feedback – given that that ecosystem has several years of experience with running TypeScript files directly.

[privatenumber]

I think this should specify that allowImportingTsExtensions needs to be enabled to type check with tsc.

And a major caveat worth noting is that allowImportingTsExtensions requires noEmit to be enabled which prevents tsc from compiling, so this setup is mutually exclusive with compiling with tsc unless TypeScript decides to support import rewriting in the future.

---

Here are some other configs that should be recommended for TS to behave consistently with the runtime:

• "module": "nodenext" or "preserve" (and moduleResolution) for Node.js features and resolution

• "allowJs": true for supporting JS files to be imported from TS files, and supporting TS resolution from JS files

• "resolveJsonModule": true for resolving .json files

• "verbatimModuleSyntax": true so type imports/exports can be stripped. This implicitly enables isolatedModules which introduces constraints in how you write TS

• "moduleDetection": "force" so each file is defaults to being treated as a module

[karlhorky]

Maybe it makes sense to have a Node.js-recommended tsconfig.json

[marco-ippolito]

This is something we want to discuss with the typescript team, but yeah 100% will be done

[GeoffreyBooth]

If this PR lands, I think ideally there would be two new options added to tsc:

• "allowTransforms": false which would cause tsc to error on syntax that results in JavaScript code generation. Having tsc optionally error on syntaxes like enums would provide hints in editors so that authors are aware of the invalid syntax as soon as they write it, not later when they try to run it.

• "rewriteTsExtensions": "js" or similar, as a complement to allowImportingTsExtensions, where tsc replaces .ts at the end of imports with .js for files that tsc is compiling. I know this might be a hard sell to the TypeScript team as they’ve taken a firm position over the years that tsc won’t do any import specifier rewriting, but the result of that stance is that other tools are doing it instead. Many people like using tsc as their only build tool, so it would be a shame if users can’t continue to use it alongside --strip-types. I don’t think it should be Node’s responsibility to handle something that a build tool could and arguably should, since the build tool is what’s generating the new files and rewriting their contents.

[RyanCavanaugh]

Knowing whether a module specifier actually refers to a .ts file, vs is a non-file specifier, is extremely fraught, and not an answerable question under the constraints that in-situ singe-file transpilers have.

Looking at a bare specifier foo/bar.ts, there are two possibilities:

• foo is a package with a package.json that says { exports: { "bar.ts": something } }, in which case rewriting this specifier will break it
• foo is a path mapping to a local folder with an actual bar.ts file. In this case, you "must" rewrite it

Notably, during single-file-transpile, you don't even know what the config is or what the relevant package.jsons are, so there's no way to resolve this question.

It's too fraught to break into jail to do this. Bundlers are the right level of abstraction for transforming import specifiers into host-appropriate semantics.

[privatenumber]

Amazing PR!

I'm not familiar with the Node release, but is there a way I can test this out in tsx before release? (e.g. can there be a test/pre release?)

I'm also curious about runtime performance impact on a large codebase, and whether TypeScript type-checking performance will be impacted when dependencies are uncompiled. Will there be an investigation before release?

[marco-ippolito]

Amazing PR!

I'm not familiar with the Node release, but is there a way I can test this out in tsx before release? (e.g. can there be a test/pre release?)

I'm also curious about runtime performance impact on a large codebase, and whether TypeScript type-checking performance will be impacted when dependencies are uncompiled. Will there be an investigation before release?

At the moment the only way you can test this, is to checkout on this branch and build the executable.
In this PR performance is not measured, when this feature is disable it does not add any overhead.
No type-checking is performed so no performance penalty.
But performance is in the checklist of next-steps

[privatenumber]

At the moment the only way you can test this, is to checkout on this branch and build the executable.

Will look into this.

In this PR performance is not measured, when this feature is disable it does not add any overhead.
No type-checking is performed so no performance penalty.

Let me clarify...

Particularly, I'm curious about the performance cost of loading a Node TS codebase with the flag enabled.

And thinking ahead: I imagine this will enable a trend of npm packages shipping .ts files. In a future scenario where dependencies ship uncompiled .ts files, I'm curious if TypeScript performance will be slower compared to type checking compiled .d.ts files in dependencies. (I'll guess no, but maybe given .ts files are larger than .d.ts files?)

But performance is in the checklist of next-steps

Great to hear!