Skip to content
This repository has been archived by the owner on Sep 2, 2023. It is now read-only.

Latest commit

 

History

History
134 lines (90 loc) · 10.3 KB

plan-for-new-modules-implementation.md

File metadata and controls

134 lines (90 loc) · 10.3 KB
Raw

This document has been archived.

This document summarized the work that went into the new ECMAScript Modules implementation that shipped in Node.js 12.0.0.

Plan for New Modules Implementation

This document outlines the plan for building a new implementation to support ECMAScript modules in Node.js. The effort is split up into phases:

  • Phase 0 branches off of current Node but removes much of the Node 8.5.0+ --experimental-modules implementation so that a new implementation could be built in its place.

  • Phase 1 adds the “minimal kernel,” features that the modules working group felt would likely appear in any potential new ES modules implementation.

  • Phase 2 fleshes out the implementation with enough functionality that it should be useful to average users as a minimum viable product.

    • At the completion of Phase 2, the old --experimental-modules implementation was replaced with this new one (still behind the --experimental-modules flag). It was released as part of Node 12 on 2019-04-23.

  • Phase 3 improves user experience and extends the MVP.

    • At the completion of Phase 3, the new implementation’s experimental flag was dropped. It was released as part of Node 13.2.0 on 2019-11-21.

  • Phase 4 are items that were under development in earlier phases but weren’t finished when the new implementation’s experimental flag was dropped; these items may continue development after unflagging and potentially ship in later versions of Node.js.

The effort is currently in Phase 4

At every phase, the following standards must be maintained:

  • Spec compliance (#132): We must always follow the ES spec.
  • Browser equivalence (#133): There’s room for debate in specific cases, but in general if Node is doing something that browsers also do, Node should do it in the same way. Alternatively, code that executes in both environments should produce identical results.
  • Don’t break CommonJS (#112): We cannot cause breaking changes with regards to CommonJS.

See also the features list in the README.

Phase 0: Start Fresh

From current shipping Node, the following changes were made to strip out most of the Node 8.5.0+ --experimental-modules implementation so that a new implementation could be built in its place:

  • Remove support in the import statement of formats other than ESM:

    • No CommonJS.
    • No JSON.
    • No native modules.

  • Remove dynamic path searching:

    • No extension adding.
    • No directory resolution, including no support for index.js or index.mjs.
    • No support for main field for ESM.

  • Remove current VM implementation

  • Remove current Loader implementation

These changes were implemented in nodejs/ecmascript-modules#6.

Phase 1: The Minimal Kernel

The “minimal kernel” consists of features that the @nodejs/modules group have agreed will be necessary for all potential iterations of our ESM implementation. Phase 1 does not include features that preclude other potential features or implementation approaches; and Phase 1 also does not include some features that should naturally be built in a later phase of development, for example because those features depend on features planned for Phase 1.

  • module.createRequireFromPath (nodejs/node#19360) is the only way to import CommonJS into an ES module, for now.

  • import statements will only support files with an .mjs extension, and will import only ES modules, for now.

    • No JSON or native modules; createRequireFromPath can be used to get these.

  • import.meta.url.

    • Already in the existing implementation.

  • Dynamic import().

    • Already in the existing implementation.

  • Support for built-in modules with named exports

    • Already in the existing implementation.

Phase 2: Minimum Viable Product: Required to Upstream

Phase 2 fleshes out the implementation with enough functionality that it should be useful to average users as a minimum viable product. At the completion of Phase 2, the old --experimental-modules implementation was replaced with this new one (still behind the --experimental-modules flag).

  • Define semantics for importing a package entry point, e.g. import _ from 'lodash'

  • Define semantics for determining when to load sources as CommonJS or ES module for both the top-level main (node x.js) and dependency loading.

  • Define semantics for enabling ESM treatment of source code loaded via --eval, STDIN, and extensionless files (both with and without shebang lines).

    • Proposal: “Entry Points Proposal” covers non-file forms of input as well as adding --type flag for controlling file-based input.
    • Landed in nodejs/ecmascript-modules#32.
    • Renamed to --entry-type as part of upstream PR to Node.js core.
    • Renamed to --intry-type and limited to --eval, --print and STDIN as part of follow-up PR to Node.js core.

  • File extension and directory index searching in ESM, behind its own flag, --es-module-specifier-resolution.

The work through the end of Phase 2 landed in Node.js master as part of nodejs/node#26745 and was released in Node 12.0.0.

Phase 3: Path to Stability: Removing --experimental-modules Flag

Phase 3 improves user experience and extends the MVP. Phase 3 is malleable based on how things proceed while working on this phase. At the end of this phase, the --experimental-modules flag is dropped.

  • Better mechanism for creating require function: createRequire.

  • "exports" field: for consumers of a package, map the paths of deep imports to provide encapsulation (an explicit public API); pretty specifiers (no file exensions or paths that include things like dist/) and flexibility for future package versions renaming or moving files without affecting the package’s public API. Applies to both ESM and CommonJS.

    • Proposal: Package Exports Proposal.
    • Landed in nodejs/node#28568 and shipped in 12.7.0 behind --experimental-exports. Further improvements are being made as additional PRs against core.
    • The separate --experimental-exports flag was dropped in nodejs/node#29867, merging the feature with overall --experimental-modules.

  • Define behavior for builtin globals between CommonJS and ESM. Does modifying a builtin in one module system carry over into the other? If it does, we may have major performance concerns.

    • Issue raised in: nodejs/node#29426.
    • Solution was to not sync bindings automatically, but provide an API to manually sync them when desired: module.syncBuiltinESMExports().
    • Landed in nodejs/node#29737 and shipped in 12.12.0.

  • Shortcut to resolve to package root.

  • Finalize behavior of import of CommonJS files and packages.

    • Overview: #264.
    • At time of unflagging: import only the CommonJS default export, so import _ from 'cjs-pkg' but not import { shuffle } from 'cjs-pkg'.
    • Conditional exports allows creating an ESM wrapper to provide named exports of an otherwise all-CommonJS package; see “Approach #1: Use an ES Module Wrapper.”
    • No further improvements are expected.

  • Dual CommonJS/ESM packages: Support packages with both CommonJS and ESM sources that can be used in either environment.

    • At time of unflagging: "main" (or "exports": { ".": "file.js" } overriding "main") points to exactly one file, and full filenames are required (by default), so there is no possibility of an import specifier pointing to different files in ESM versus CommonJS; unless --experimental-conditional-exports is used (see next bullet). Without that flag, dual packages must provide secondary entry point via a path, e.g. 'pkg/module' or 'pkg/commonjs'.
    • With --experimental-conditional-exports, paths within the package.json "exports" block can have separate entry points per environment.
    • Landed in nodejs/node#29978 and nodejs/node#30051 behind --experimental-conditional-exports flag.
    • Unflagged in nodejs/node#31001 and shipped in Node 13.7.0.

Phase 4: Further Improvements After Unflagging

Now that the implementation has shipped, further efforts are listed on the README.