From 76033cafa1165cfa556895c5866844b3b2e959f5 Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Thu, 7 Jan 2016 15:29:12 -0600 Subject: [PATCH 01/41] es6-modules --- 000-index.md | 1 + 001-es6-modules.md | 169 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 170 insertions(+) create mode 100644 001-es6-modules.md diff --git a/000-index.md b/000-index.md index 8769f47..c19e1cc 100644 --- a/000-index.md +++ b/000-index.md @@ -1,3 +1,4 @@ # Node.js Enhancement Proposals * [Public C++ Streams](001-public-stream-base.md) +* [ES6 Module Interoperability](./001-es6-modules.md) diff --git a/001-es6-modules.md b/001-es6-modules.md new file mode 100644 index 0000000..a032318 --- /dev/null +++ b/001-es6-modules.md @@ -0,0 +1,169 @@ +# ES6 Modules + +Status | Draft +------ | ---- +Author | Bradley Meck +Date | 7 Jan 2016 + +It is our intent to: + +* implement interoperability for ES6 modules and node's existing module system +* create a **Registry** compatible with the WhatWG Loader Registry + +## Purpose + +1. Allow a common module syntax for Browser and Server. +2. Allow a common registry for inspection by Browser and Server environments/tools. + * these will most likely be represented by metaproperties like `import.ctx` but spec is not in place fully yet. + +## Related + +--- + +### [ECMA262](https://tc39.github.io/ecma262/#sec-source-text-module-records) + +--- + +Discusses the syntax and semantics of related syntax, and introduces: + +#### Types + +* [ModuleRecord](https://tc39.github.io/ecma262/#sec-abstract-module-records) + - Defines the list of imports via [ImportEntry](https://tc39.github.io/ecma262/#table-39). + - Defines the list of exports via [ExportEntry](https://tc39.github.io/ecma262/#table-41). + +* [ModuleNamespace](https://tc39.github.io/ecma262/#sec-module-namespace-exotic-objects) + - Represents a read-only snapshot of a module's exports. + +### Operations + +* [ParseModule](https://tc39.github.io/ecma262/#sec-parsemodule) + - Creates a ModuleRecord from source code. + +* [HostResolveImportedModule](https://tc39.github.io/ecma262/#sec-hostresolveimportedmodule) + - A hook for when an import is exactly performed. + +* [CreateImportBinding](https://tc39.github.io/ecma262/#sec-createimportbinding) + - A means to create a shared binding (variable) from an export to an import. Required for the "live" binding of imports. + +--- + +### [WhatWG Loader](https://github.com/whatwg/loader) + +--- + +Discusses the design of module metadata in a [Registry](https://whatwg.github.io/loader/#registry). All actions regarding the Registry are synchronous. + +**NOTE** It is not Node's intent to implement the asynchronous pipeline in the Loader specification. There is discussion about including a synchronous pipeline in the specification as an addendum. + + +### [Summary Video on Youtube](youtube.com/watch?v=NdOKO-6Ty7k) + +## Semantics + +### Determining if source is an ES6 Module + +If a module can be parsed outside of the ES6 Module goal, it will be treated as a [Script](https://tc39.github.io/ecma262/#sec-parse-script). Otherwise it will be parsed as [ES6](https://tc39.github.io/ecma262/#sec-parsemodule). + +In pseudocode, it looks somewhat like: + +``` +try { + v8::Script::Compile +} +catch (e) { + v8::Script::CompileModule +} +``` + +V8 may or may not choose to use parser fallback to combine this into one step. + +### CommonJS consuming ES6 + +#### default exports + +ES6 modules only ever declare named exports. A default export just exports a property named `default`. + +Given + +```javascript +let foo = 'my-default'; +default export foo; +``` + +```javascript +require('./es6'); +// {default:'my-default'} +``` + +#### read only + +The objects create by an ES6 module are [ModuleNamespace Objects](https://tc39.github.io/ecma262/#sec-module-namespace-exotic-objects). + +These have `[[Set]]` be a no-op and are read only views of the exports of an ES6 module. + +### ES6 consuming CommonJS + +#### default imports + +`module.exports` shadows `module.exports.default`. + +This means that if there is a `.default` on your CommonJS exports it will be shadowed to the `module.exports` of your module. + +Given: + +```javascript +// cjs.js +module.exports = {default:'my-default'}; +``` + +You will grab `module.exports` when performing an ES6 import. + +```javascript +// es6.js +import foo from './cjs'; +// foo = {default:'my-default'}; + +import {default as bar} from './cjs'; +// bar = {default:'my-default'}; +``` + +### Known Gotchas + +All of these gotchas relate to opt-in semantics and the fact that CommonJS is a dynamic loader while ES6 is a static loader. + +No existing code will be affected. + +#### Circular Dep CJS => ES6 => CJS + +Given: + +```javascript +//cjs.js +module.exports = {x:0}; +require('./es6'); +``` + +```javascript +//es6.js +import * as ns from './cjs'; +// ns = undefined +import cjs from './cjs'; +// ns = undefined +``` + +The list of properties being exported is not populated until `cjs.js` finishes executing. + +The result is that there are no properties to import and you recieve an empty module. + +##### Option: Throw on this case + +Since this case is coming from ES6, we could throw whenever this occurs. This is similar to how you cannot change a generator while it is running. + +If taken this would change the ES6 module behavior to: + +```javascript +//es6.js +import * as ns from './cjs'; +// throw new EvalError('./cjs is not an ES6 module and has not finished evaluation'); +``` From 3993e09f4163aec22a73bfc5ec55b60a101267ab Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Thu, 7 Jan 2016 18:13:11 -0600 Subject: [PATCH 02/41] v8 api advisory --- 001-es6-modules.md | 71 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 71 insertions(+) diff --git a/001-es6-modules.md b/001-es6-modules.md index a032318..2e5ab79 100644 --- a/001-es6-modules.md +++ b/001-es6-modules.md @@ -167,3 +167,74 @@ If taken this would change the ES6 module behavior to: import * as ns from './cjs'; // throw new EvalError('./cjs is not an ES6 module and has not finished evaluation'); ``` + +## Advisory + +V8 currently does not expose the proper APIs for creating Loaders, it has done the brunt of the work by [exposing a parser in their issue tracker](https://bugs.chromium.org/p/v8/issues/detail?id=1569). + +It has been recommended that we list a potential API we could consume in order to create our loader. These extensions are listed below. + +### Double Parsing Problem + +Due to the fact that we are going to be performing detection of grammar goal using parse failure or the presense of `import`/`export`. It would be ideal if `Compile` and `CompileModule` where merged into a single call. Assuming such we will place our additions in the `CompileOptions` and `Script` types. If such change is not possible, we will be using the fallback method listed in this proposal above. + +### API Suggestion + +```c++ +namespace v8 + +CompileOptions { + // parse to Script or Module based upon invalid syntax + // default is Module + kDetectGoal + // tells the parser to change the detect goal default to Script + kDefaultGoalScript +}; + +class Module : Script { + // these are normally constructed via ScriptCompiler, however, + // + // in order for CommonJS modules to create fully formed + // ES6 Module compatibility we need to hook up a static + // View of an Object to set as our exports + // + // think of this as calling FillImports using the current + // properties of an object, enumerable or not. + // + // exports are never added or removed from the Module even + // though the exports object may do so, unlike `with()` + // + // construction via this will act as if it has already been + // run() and fill out the Namespace() + Module(Object exports); + + // get a list of imports we need to perform prior to evaluation + ImportEntry[] ImportEntries(); + + // get a list of what this exports + ExportEntry[] ExportEntries(); + + // cannot be called prior to Run() completing + // + // return a ModuleNamespace view of this Module's exports + ModuleNamespace Namespace(); + + // required prior to Run() + // + // this will add the bindings to the lexical environment of + // the Module + FillImports(ImportBinding[] bindings); +} +class ImportEntry { + String ModuleRequest(); + + // note: if ImportName() is "*", the module takes the Namespace() + // as required by ECMA262 + String ImportName(); + + String LocalName(); +} +class ImportBinding { + ImportBinding(String importName, Module delegate, String delegateExportName); +} +``` From ea692188d01874a2c11d83d4d8cf0239f77c0e44 Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Thu, 7 Jan 2016 19:17:32 -0600 Subject: [PATCH 03/41] notes about ModuleNamespaceCreate --- 000-index.md | 2 +- 001-es6-modules.md => 002-es6-modules.md | 6 ++++++ 2 files changed, 7 insertions(+), 1 deletion(-) rename 001-es6-modules.md => 002-es6-modules.md (94%) diff --git a/000-index.md b/000-index.md index c19e1cc..380593b 100644 --- a/000-index.md +++ b/000-index.md @@ -1,4 +1,4 @@ # Node.js Enhancement Proposals * [Public C++ Streams](001-public-stream-base.md) -* [ES6 Module Interoperability](./001-es6-modules.md) +* [ES6 Module Interoperability](./002-es6-modules.md) diff --git a/001-es6-modules.md b/002-es6-modules.md similarity index 94% rename from 001-es6-modules.md rename to 002-es6-modules.md index 2e5ab79..d24e56b 100644 --- a/001-es6-modules.md +++ b/002-es6-modules.md @@ -46,6 +46,9 @@ Discusses the syntax and semantics of related syntax, and introduces: * [CreateImportBinding](https://tc39.github.io/ecma262/#sec-createimportbinding) - A means to create a shared binding (variable) from an export to an import. Required for the "live" binding of imports. +* [ModuleNamespaceCreate](https://tc39.github.io/ecma262/#sec-modulenamespacecreate) + - Provides a means of creating a list of exports manually, used so that CommonJS `module.exports` can create ModuleRecords that are prepopulated. + --- ### [WhatWG Loader](https://github.com/whatwg/loader) @@ -206,6 +209,9 @@ class Module : Script { // // construction via this will act as if it has already been // run() and fill out the Namespace() + // this in a way mimics: + // 1. calling ModuleNamespaceCreate(this, exports) + // 2. populating the [[Namespace]] field of this Module Record Module(Object exports); // get a list of imports we need to perform prior to evaluation From b65e7caa07ebd356e994e9fd5d5735d5a996c1ed Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Fri, 8 Jan 2016 08:03:49 -0600 Subject: [PATCH 04/41] wording, split v8 SourceTextModule, DynamicModule --- 002-es6-modules.md | 57 +++++++++++++++++++++++++++------------------- 1 file changed, 33 insertions(+), 24 deletions(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index d24e56b..e2823b6 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -8,7 +8,7 @@ Date | 7 Jan 2016 It is our intent to: * implement interoperability for ES6 modules and node's existing module system -* create a **Registry** compatible with the WhatWG Loader Registry +* create a **Registry Object** (see WhatWG section below) compatible with the WhatWG Loader Registry ## Purpose @@ -38,7 +38,7 @@ Discusses the syntax and semantics of related syntax, and introduces: ### Operations * [ParseModule](https://tc39.github.io/ecma262/#sec-parsemodule) - - Creates a ModuleRecord from source code. + - Creates a SourceTextModuleRecord from source code. * [HostResolveImportedModule](https://tc39.github.io/ecma262/#sec-hostresolveimportedmodule) - A hook for when an import is exactly performed. @@ -62,6 +62,10 @@ Discusses the design of module metadata in a [Registry](https://whatwg.github.io ### [Summary Video on Youtube](youtube.com/watch?v=NdOKO-6Ty7k) +## Additional Structures Required + +### DynamicModuleRecord + ## Semantics ### Determining if source is an ES6 Module @@ -194,9 +198,29 @@ CompileOptions { kDefaultGoalScript }; -class Module : Script { - // these are normally constructed via ScriptCompiler, however, +class Module { + // return a ModuleNamespace view of this Module's exports + ModuleNamespace Namespace(); +} + +class SourceTextModule : Script, Module { + // get a list of imports we need to perform prior to evaluation + ImportEntry[] ImportEntries(); + + // get a list of what this exports + ExportEntry[] ExportEntries(); + + // cannot be called prior to Run() completing + ModuleNamespace Namespace(); + + // required prior to Run() // + // this will add the bindings to the lexical environment of + // the Module + FillImports(ImportBinding[] bindings); +} + +class DynamicModule : Module { // in order for CommonJS modules to create fully formed // ES6 Module compatibility we need to hook up a static // View of an Object to set as our exports @@ -212,35 +236,20 @@ class Module : Script { // this in a way mimics: // 1. calling ModuleNamespaceCreate(this, exports) // 2. populating the [[Namespace]] field of this Module Record - Module(Object exports); - - // get a list of imports we need to perform prior to evaluation - ImportEntry[] ImportEntries(); - - // get a list of what this exports - ExportEntry[] ExportEntries(); - - // cannot be called prior to Run() completing - // - // return a ModuleNamespace view of this Module's exports - ModuleNamespace Namespace(); - - // required prior to Run() - // - // this will add the bindings to the lexical environment of - // the Module - FillImports(ImportBinding[] bindings); + DynamicModule(Object exports); } + class ImportEntry { String ModuleRequest(); - // note: if ImportName() is "*", the module takes the Namespace() + // note: if ImportName() is "*", the Loader + // must take the Namespace() and not directly the module // as required by ECMA262 String ImportName(); String LocalName(); } class ImportBinding { - ImportBinding(String importName, Module delegate, String delegateExportName); + ImportBinding(String importLocalName, Module delegate, String delegateExportName); } ``` From 5798d482cd7950355a4dcb2f5d3359bb27223342 Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Fri, 8 Jan 2016 08:05:22 -0600 Subject: [PATCH 05/41] import.context --- 002-es6-modules.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index e2823b6..360af93 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -14,7 +14,7 @@ It is our intent to: 1. Allow a common module syntax for Browser and Server. 2. Allow a common registry for inspection by Browser and Server environments/tools. - * these will most likely be represented by metaproperties like `import.ctx` but spec is not in place fully yet. + * these will most likely be represented by metaproperties like `import.context` but spec is not in place fully yet. ## Related From 1f7d92dd10a269096c66ac407e62574753c5a276 Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Fri, 8 Jan 2016 08:39:40 -0600 Subject: [PATCH 06/41] write out specific algorithm --- 002-es6-modules.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/002-es6-modules.md b/002-es6-modules.md index 360af93..5e2a76c 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -66,6 +66,25 @@ Discusses the design of module metadata in a [Registry](https://whatwg.github.io ### DynamicModuleRecord +A Module Record that presents a view of an Object for its `[[Namespace]]` rather than coming from an environment record. + +The list of exports is frozen upon construction. No new properties may be added. No properties may be removed. + +## Algorithm + +When `require()`ing a file. + +1. Determine if file is ES6 or CommonJS (CJS). +2. If CJS + 1. Evaluate immediately + 2. Produce a DynamicModuleRecord from `module.exports` +3. If ES6 + 1. Parse for `import`/`export`s and keep record, in order to create bindings + 2. Gather all submodules by performing `require` recursively + * See circular dep semantics below + 3. Connect `import` bindings for all relevant submodules (see [ModuleDeclarationInstantiation](https://tc39.github.io/ecma262/#sec-moduledeclarationinstantiation)) + 4. Evaluate + ## Semantics ### Determining if source is an ES6 Module From 2632efe32db660fb8ca5f5fc4a0915459ac0bc68 Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Fri, 8 Jan 2016 12:27:37 -0600 Subject: [PATCH 07/41] rename FillImports to ImportDeclarationInstantiation to be closer to ES spec --- 002-es6-modules.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index 5e2a76c..5c0d592 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -236,7 +236,7 @@ class SourceTextModule : Script, Module { // // this will add the bindings to the lexical environment of // the Module - FillImports(ImportBinding[] bindings); + ImportDeclarationInstantiation(ImportBinding[] bindings); } class DynamicModule : Module { @@ -244,7 +244,7 @@ class DynamicModule : Module { // ES6 Module compatibility we need to hook up a static // View of an Object to set as our exports // - // think of this as calling FillImports using the current + // think of this as calling ImportDeclarationInstantiation using the current // properties of an object, enumerable or not. // // exports are never added or removed from the Module even From ffef35317943deb48cb36c8822b25be6d4539adf Mon Sep 17 00:00:00 2001 From: bradleymeck Date: Sun, 17 Jan 2016 12:59:37 -0600 Subject: [PATCH 08/41] [typo] --- 002-es6-modules.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index 5c0d592..c91dd3d 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -114,7 +114,7 @@ Given ```javascript let foo = 'my-default'; -default export foo; +export default foo; ``` ```javascript From 63d213c75d7f624d20cf811d88f144881d627d35 Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Tue, 19 Jan 2016 19:50:33 -0600 Subject: [PATCH 09/41] s/WhatWG/WHATWG/ --- 002-es6-modules.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index c91dd3d..2a742b3 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -8,7 +8,7 @@ Date | 7 Jan 2016 It is our intent to: * implement interoperability for ES6 modules and node's existing module system -* create a **Registry Object** (see WhatWG section below) compatible with the WhatWG Loader Registry +* create a **Registry Object** (see WHATWG section below) compatible with the WHATWG Loader Registry ## Purpose @@ -51,7 +51,7 @@ Discusses the syntax and semantics of related syntax, and introduces: --- -### [WhatWG Loader](https://github.com/whatwg/loader) +### [WHATWG Loader](https://github.com/whatwg/loader) --- From 6ca03faffc511e239b31bd3bb9fe19ed4f6676c1 Mon Sep 17 00:00:00 2001 From: bradleymeck Date: Thu, 21 Jan 2016 07:45:56 -0600 Subject: [PATCH 10/41] simplify detection, remove double-parse concern by using `.es` --- 002-es6-modules.md | 29 ++--------------------------- 1 file changed, 2 insertions(+), 27 deletions(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index 2a742b3..c08b60f 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -89,20 +89,7 @@ When `require()`ing a file. ### Determining if source is an ES6 Module -If a module can be parsed outside of the ES6 Module goal, it will be treated as a [Script](https://tc39.github.io/ecma262/#sec-parse-script). Otherwise it will be parsed as [ES6](https://tc39.github.io/ecma262/#sec-parsemodule). - -In pseudocode, it looks somewhat like: - -``` -try { - v8::Script::Compile -} -catch (e) { - v8::Script::CompileModule -} -``` - -V8 may or may not choose to use parser fallback to combine this into one step. +A new filetype will be recognised, `.es` as ES6 based modules. They will be treated as a different loading semantic but compatible with existing systems, just like `.node`, `.json`, or usage of `require.extension` (even though deprecated) are compatible. ### CommonJS consuming ES6 @@ -200,22 +187,10 @@ V8 currently does not expose the proper APIs for creating Loaders, it has done t It has been recommended that we list a potential API we could consume in order to create our loader. These extensions are listed below. -### Double Parsing Problem - -Due to the fact that we are going to be performing detection of grammar goal using parse failure or the presense of `import`/`export`. It would be ideal if `Compile` and `CompileModule` where merged into a single call. Assuming such we will place our additions in the `CompileOptions` and `Script` types. If such change is not possible, we will be using the fallback method listed in this proposal above. - ### API Suggestion ```c++ -namespace v8 - -CompileOptions { - // parse to Script or Module based upon invalid syntax - // default is Module - kDetectGoal - // tells the parser to change the detect goal default to Script - kDefaultGoalScript -}; +namespace v8; class Module { // return a ModuleNamespace view of this Module's exports From 5b9f72709081f5d46f152fb4476bbc62be35117a Mon Sep 17 00:00:00 2001 From: bradleymeck Date: Thu, 21 Jan 2016 08:50:24 -0600 Subject: [PATCH 11/41] initial example implementation --- 002-es6-modules.md | 83 +++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 82 insertions(+), 1 deletion(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index c08b60f..3d548fd 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -189,7 +189,7 @@ It has been recommended that we list a potential API we could consume in order t ### API Suggestion -```c++ +```cpp namespace v8; class Module { @@ -247,3 +247,84 @@ class ImportBinding { ImportBinding(String importLocalName, Module delegate, String delegateExportName); } ``` + +## Example Implementation + +These are written with the expectation that: + +* ModuleNamespaces can be created from existing Objects. +* WHATWG Loader spec Registry is available as ES6ModuleRegistry. +* ModuleStatus Objects can be created. + +The variable names should be hidden from user code using various techniques left out here. + +### CJS Modules + +#### Pre Evaluation + +```javascript +ES6ModuleRegistry.set(__filename, new ModuleStatus({ + 'ready': {'[[Result]]':undefined}; +})) +``` + +#### Trailer + +```javascript +;{ + // module_namespace, how an ES6 module would see a CJS module + let module_namespace = Object.create(null); + function gatherExports(obj, acc = new Set()) { + if (typeof obj !== 'object' && typeof obj !== 'function' || obj === null) { + return acc; + } + for (const key of Object.getOwnPropertyNames(obj)) { + const desc = Object.getOwnPropertyDescriptor(obj, key); + acc.add({key,desc}); + } + return gatherExports(Object.getPrototypeOf(obj), acc); + } + [...gatherExports(real_exports)].forEach(({key,desc}) => { + if (key === 'default') return; + Object.defineProperty(module_namespace, key, { + get: () => real_exports[key], + set() {throw new Error(`ModuleNamespace key ${key} is read only.`)}, + configurable: false, + enumerable: Boolean(desc.enumerable) + }); + }) + Object.defineProperty(module_namespace, 'default', { + value: real_exports, + writable: false, + configurable: false + }); + ES6ModuleRegistry.set(__filename, new ModuleStatus({ + 'ready': {'[[Result]]': module_namespace} + }); +} +``` + +### ES6 Modules + +#### Post Parsing + +``` +require.cache[__filename] = new Module(...); +require.cache[__filename].exports = undefined; +``` + +Parsing occurs prior to evaluation, and CJS may execute once we start to resolve `import`. + +#### Header + +```javascript +// assert(module.exports == undefined); +const exports = void 0; +``` + +# Immediately Post Evaluation + +```javascript +const module_namespace = ES6ModuleRegistry.get(__filename).GetStage('ready')['[[result]]']; +require.cache[__filename].exports = module_namespace; +``` \ No newline at end of file From aa7b2546fabcf97e2972a61fe6a42c8d6467846a Mon Sep 17 00:00:00 2001 From: bradleymeck Date: Thu, 21 Jan 2016 09:11:09 -0600 Subject: [PATCH 12/41] typo --- 002-es6-modules.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index 3d548fd..185081e 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -322,7 +322,7 @@ Parsing occurs prior to evaluation, and CJS may execute once we start to resolve const exports = void 0; ``` -# Immediately Post Evaluation +#### Immediately Post Evaluation ```javascript const module_namespace = ES6ModuleRegistry.get(__filename).GetStage('ready')['[[result]]']; From 0c06b51347e06d8e5d329c7d8b249b3ba2bd7935 Mon Sep 17 00:00:00 2001 From: bradleymeck Date: Thu, 21 Jan 2016 09:11:55 -0600 Subject: [PATCH 13/41] style --- 002-es6-modules.md | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index 185081e..31c21b1 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -275,14 +275,14 @@ ES6ModuleRegistry.set(__filename, new ModuleStatus({ // module_namespace, how an ES6 module would see a CJS module let module_namespace = Object.create(null); function gatherExports(obj, acc = new Set()) { - if (typeof obj !== 'object' && typeof obj !== 'function' || obj === null) { - return acc; - } - for (const key of Object.getOwnPropertyNames(obj)) { - const desc = Object.getOwnPropertyDescriptor(obj, key); - acc.add({key,desc}); - } - return gatherExports(Object.getPrototypeOf(obj), acc); + if (typeof obj !== 'object' && typeof obj !== 'function' || obj === null) { + return acc; + } + for (const key of Object.getOwnPropertyNames(obj)) { + const desc = Object.getOwnPropertyDescriptor(obj, key); + acc.add({key,desc}); + } + return gatherExports(Object.getPrototypeOf(obj), acc); } [...gatherExports(real_exports)].forEach(({key,desc}) => { if (key === 'default') return; @@ -294,9 +294,9 @@ ES6ModuleRegistry.set(__filename, new ModuleStatus({ }); }) Object.defineProperty(module_namespace, 'default', { - value: real_exports, - writable: false, - configurable: false + value: real_exports, + writable: false, + configurable: false }); ES6ModuleRegistry.set(__filename, new ModuleStatus({ 'ready': {'[[Result]]': module_namespace} From 02928fec729b36897ac176aac8272840e8b76ef9 Mon Sep 17 00:00:00 2001 From: bradleymeck Date: Thu, 21 Jan 2016 09:12:05 -0600 Subject: [PATCH 14/41] style --- 002-es6-modules.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index 31c21b1..f2c18fb 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -285,13 +285,13 @@ ES6ModuleRegistry.set(__filename, new ModuleStatus({ return gatherExports(Object.getPrototypeOf(obj), acc); } [...gatherExports(real_exports)].forEach(({key,desc}) => { - if (key === 'default') return; - Object.defineProperty(module_namespace, key, { - get: () => real_exports[key], - set() {throw new Error(`ModuleNamespace key ${key} is read only.`)}, - configurable: false, - enumerable: Boolean(desc.enumerable) - }); + if (key === 'default') return; + Object.defineProperty(module_namespace, key, { + get: () => real_exports[key], + set() {throw new Error(`ModuleNamespace key ${key} is read only.`)}, + configurable: false, + enumerable: Boolean(desc.enumerable) + }); }) Object.defineProperty(module_namespace, 'default', { value: real_exports, From 615e2d3d1545f1522afabab058c6606b192487e3 Mon Sep 17 00:00:00 2001 From: bradleymeck Date: Thu, 21 Jan 2016 09:23:17 -0600 Subject: [PATCH 15/41] [impl] CJS Trailer -> Immediately Post Evaluation --- 002-es6-modules.md | 59 +++++++++++++++++++++++----------------------- 1 file changed, 29 insertions(+), 30 deletions(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index f2c18fb..f494d06 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -268,40 +268,39 @@ ES6ModuleRegistry.set(__filename, new ModuleStatus({ })) ``` -#### Trailer +#### Immediately Post Evaluation ```javascript -;{ - // module_namespace, how an ES6 module would see a CJS module - let module_namespace = Object.create(null); - function gatherExports(obj, acc = new Set()) { - if (typeof obj !== 'object' && typeof obj !== 'function' || obj === null) { - return acc; - } - for (const key of Object.getOwnPropertyNames(obj)) { - const desc = Object.getOwnPropertyDescriptor(obj, key); - acc.add({key,desc}); - } - return gatherExports(Object.getPrototypeOf(obj), acc); +const real_exports = require.cache[__filename].exports; +// module_namespace, how an ES6 module would see a CJS module +let module_namespace = Object.create(null); +function gatherExports(obj, acc = new Set()) { + if (typeof obj !== 'object' && typeof obj !== 'function' || obj === null) { + return acc; } - [...gatherExports(real_exports)].forEach(({key,desc}) => { - if (key === 'default') return; - Object.defineProperty(module_namespace, key, { - get: () => real_exports[key], - set() {throw new Error(`ModuleNamespace key ${key} is read only.`)}, - configurable: false, - enumerable: Boolean(desc.enumerable) - }); - }) - Object.defineProperty(module_namespace, 'default', { - value: real_exports, - writable: false, - configurable: false - }); - ES6ModuleRegistry.set(__filename, new ModuleStatus({ - 'ready': {'[[Result]]': module_namespace} - }); + for (const key of Object.getOwnPropertyNames(obj)) { + const desc = Object.getOwnPropertyDescriptor(obj, key); + acc.add({key,desc}); + } + return gatherExports(Object.getPrototypeOf(obj), acc); } +[...gatherExports(real_exports)].forEach(({key,desc}) => { + if (key === 'default') return; + Object.defineProperty(module_namespace, key, { + get: () => real_exports[key], + set() {throw new Error(`ModuleNamespace key ${key} is read only.`)}, + configurable: false, + enumerable: Boolean(desc.enumerable) + }); +}) +Object.defineProperty(module_namespace, 'default', { + value: real_exports, + writable: false, + configurable: false +}); +ES6ModuleRegistry.set(__filename, new ModuleStatus({ + 'ready': {'[[Result]]': module_namespace} +}); ``` ### ES6 Modules From e97f303ce048b4d7685dd6d4c6a673f05074e34a Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Thu, 21 Jan 2016 18:10:13 -0600 Subject: [PATCH 16/41] .es -> .jsm file extension --- 002-es6-modules.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index f494d06..0a54bac 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -89,7 +89,7 @@ When `require()`ing a file. ### Determining if source is an ES6 Module -A new filetype will be recognised, `.es` as ES6 based modules. They will be treated as a different loading semantic but compatible with existing systems, just like `.node`, `.json`, or usage of `require.extension` (even though deprecated) are compatible. +A new filetype will be recognised, `.jsm` as ES6 based modules. They will be treated as a different loading semantic but compatible with existing systems, just like `.node`, `.json`, or usage of `require.extension` (even though deprecated) are compatible. ### CommonJS consuming ES6 @@ -326,4 +326,4 @@ const exports = void 0; ```javascript const module_namespace = ES6ModuleRegistry.get(__filename).GetStage('ready')['[[result]]']; require.cache[__filename].exports = module_namespace; -``` \ No newline at end of file +``` From 83f204c76f4328d5ce3c20d88740dc7522f9dd1c Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Mon, 25 Jan 2016 10:57:46 -0600 Subject: [PATCH 17/41] top level await fix, wrap ES6 modules in Promise --- 002-es6-modules.md | 57 +++++++++++++++++++++++++++++++++++----------- 1 file changed, 44 insertions(+), 13 deletions(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index 0a54bac..77bffdb 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -95,7 +95,7 @@ A new filetype will be recognised, `.jsm` as ES6 based modules. They will be tre #### default exports -ES6 modules only ever declare named exports. A default export just exports a property named `default`. +ES6 modules only ever declare named exports. A default export just exports a property named `default`. All ES6 modules are wrapped in a `Promise` in anticipation of top level `await`. Given @@ -105,8 +105,9 @@ export default foo; ``` ```javascript -require('./es6'); -// {default:'my-default'} +require('./es6').then((namespace) => { + console.log(namespace);// {default:'my-default'} +}) ``` #### read only @@ -263,13 +264,26 @@ The variable names should be hidden from user code using various techniques left #### Pre Evaluation ```javascript +const namespacePromise = new Promise((f,r) => { + fulfillNamespacePromise = f; + rejectNamespacePromise = r; +}); ES6ModuleRegistry.set(__filename, new ModuleStatus({ - 'ready': {'[[Result]]':undefined}; -})) + 'ready': {'[[Result]]':namespacePromise}; +})); ``` #### Immediately Post Evaluation +##### On Error + +```javascript +rejectNamespacePromise(error); +ES6ModuleRegistry.delete(__filename); +``` + +##### On Normal Completion + ```javascript const real_exports = require.cache[__filename].exports; // module_namespace, how an ES6 module would see a CJS module @@ -298,9 +312,7 @@ Object.defineProperty(module_namespace, 'default', { writable: false, configurable: false }); -ES6ModuleRegistry.set(__filename, new ModuleStatus({ - 'ready': {'[[Result]]': module_namespace} -}); +fulfillNamespacePromise(module_namespace); ``` ### ES6 Modules @@ -308,8 +320,19 @@ ES6ModuleRegistry.set(__filename, new ModuleStatus({ #### Post Parsing ``` -require.cache[__filename] = new Module(...); -require.cache[__filename].exports = undefined; +const module = ...; +const namespacePromise = new Promise((f,r) => { + fulfillNamespacePromise = f; + rejectNamespacePromise = r; +}); +Object.freeze(namespacePromise); +Object.defineProperty(module, 'exports', { + get() {return namespacePromise}; + set(v) {throw new Error(`${__filename} is an ES6 module and cannot assign to module.exports`)} + configurable: false, + enumerable: false +}); +require.cache[__filename] = namespacePromise; ``` Parsing occurs prior to evaluation, and CJS may execute once we start to resolve `import`. @@ -317,13 +340,21 @@ Parsing occurs prior to evaluation, and CJS may execute once we start to resolve #### Header ```javascript -// assert(module.exports == undefined); const exports = void 0; ``` #### Immediately Post Evaluation +##### On Error + +```javascript +rejectNamespacePromise(error); +delete require.cache[__filename]; +``` + + +##### On Normal Completion + ```javascript -const module_namespace = ES6ModuleRegistry.get(__filename).GetStage('ready')['[[result]]']; -require.cache[__filename].exports = module_namespace; +fulfillNamespacePromise(ES6ModuleRegistry.get(__filename).GetStage('ready')['[[result]]']); ``` From 4b1fb6e0858fe0a66e6cf08511dbc790e5f7a021 Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Mon, 25 Jan 2016 11:30:08 -0600 Subject: [PATCH 18/41] javascript highlighting --- 002-es6-modules.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index 77bffdb..10cded9 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -319,7 +319,7 @@ fulfillNamespacePromise(module_namespace); #### Post Parsing -``` +```javascript const module = ...; const namespacePromise = new Promise((f,r) => { fulfillNamespacePromise = f; From 3ca031be8b72f6225bf2a8719ea6d5b116a21f1e Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Mon, 25 Jan 2016 12:52:18 -0600 Subject: [PATCH 19/41] Reference MIME registration/IESG --- 002-es6-modules.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index 10cded9..0ee2410 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -89,7 +89,7 @@ When `require()`ing a file. ### Determining if source is an ES6 Module -A new filetype will be recognised, `.jsm` as ES6 based modules. They will be treated as a different loading semantic but compatible with existing systems, just like `.node`, `.json`, or usage of `require.extension` (even though deprecated) are compatible. +A new filetype will be recognised, `.jsm` as ES6 based modules. They will be treated as a different loading semantic but compatible with existing systems, just like `.node`, `.json`, or usage of `require.extension` (even though deprecated) are compatible. It would be ideal if we could register the filetype with IANA as an offical file type, see [TC39 issue](https://github.com/tc39/ecma262/issues/322). Though it seems this would need to go through the [IESG](https://www.ietf.org/iesg/) and it seems browsers are non-plussed on introducing a new MIME. ### CommonJS consuming ES6 From 0b2cd8314c0ddf1ecd716970a8209b4793d276d7 Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Mon, 25 Jan 2016 15:29:32 -0600 Subject: [PATCH 20/41] mime type/file extension rejected by standards --- 002-es6-modules.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index 0ee2410..974a05a 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -89,7 +89,9 @@ When `require()`ing a file. ### Determining if source is an ES6 Module -A new filetype will be recognised, `.jsm` as ES6 based modules. They will be treated as a different loading semantic but compatible with existing systems, just like `.node`, `.json`, or usage of `require.extension` (even though deprecated) are compatible. It would be ideal if we could register the filetype with IANA as an offical file type, see [TC39 issue](https://github.com/tc39/ecma262/issues/322). Though it seems this would need to go through the [IESG](https://www.ietf.org/iesg/) and it seems browsers are non-plussed on introducing a new MIME. +~~A new filetype will be recognised, `.jsm` as ES6 based modules. They will be treated as a different loading semantic but compatible with existing systems, just like `.node`, `.json`, or usage of `require.extension` (even though deprecated) are compatible. It would be ideal if we could register the filetype with IANA as an offical file type, see [TC39 issue](https://github.com/tc39/ecma262/issues/322). Though it seems this would need to go through the [IESG](https://www.ietf.org/iesg/) and it seems browsers are non-plussed on introducing a new MIME.~~ Both TC39 and WHATWG reject the notion of a new MIME, we would have to push this through without any support from the standards bodies. As such I am for the moment disregarding but keeping reference to this idea. + +We will be using a [Directive Prologue](https://tc39.github.io/ecma262/#directive-prologue) in order to detect if source code is ES6. The directive will be `"use module"`. ### CommonJS consuming ES6 From a39f9a613941b38df65742629f734a6f79279245 Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Tue, 2 Feb 2016 08:42:12 -0600 Subject: [PATCH 21/41] revive .jsm, remove search from es6 import statements --- 002-es6-modules.md | 68 +++++++++++++++++++++++++++++++++++++--------- 1 file changed, 55 insertions(+), 13 deletions(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index 974a05a..30a2b0e 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -89,9 +89,53 @@ When `require()`ing a file. ### Determining if source is an ES6 Module -~~A new filetype will be recognised, `.jsm` as ES6 based modules. They will be treated as a different loading semantic but compatible with existing systems, just like `.node`, `.json`, or usage of `require.extension` (even though deprecated) are compatible. It would be ideal if we could register the filetype with IANA as an offical file type, see [TC39 issue](https://github.com/tc39/ecma262/issues/322). Though it seems this would need to go through the [IESG](https://www.ietf.org/iesg/) and it seems browsers are non-plussed on introducing a new MIME.~~ Both TC39 and WHATWG reject the notion of a new MIME, we would have to push this through without any support from the standards bodies. As such I am for the moment disregarding but keeping reference to this idea. +A new filetype will be recognised, `.jsm` as ES6 based modules. They will be treated as a different loading semantic but compatible with existing systems, just like `.node`, `.json`, or usage of `require.extension` (even though deprecated) are compatible. It would be ideal if we could register the filetype with IANA as an offical file type, see [TC39 issue](https://github.com/tc39/ecma262/issues/322). Though it seems this would need to go through the [IESG](https://www.ietf.org/iesg/) and it seems browsers are non-plussed on introducing a new MIME. -We will be using a [Directive Prologue](https://tc39.github.io/ecma262/#directive-prologue) in order to detect if source code is ES6. The directive will be `"use module"`. +The `.jsm` file extension will have a higher loading priority than `.js`. + +### ES6 Import Resolution + +ES6 `import` statements will not perform non-exact searches on relative or absolute paths, unlike `require()`. This means that no file extensions, or index files will be searched for when using relative or absolute paths. `node_modules` based paths will continue to use searching for both compatibility and to not limit the ability to have `package.json` support both ES6 and CJS entry points in a single codebase. + +In summary: + +```javascript +// only looks at +// ./foo +// does not search: +// ./foo.js +// ./foo/index.js +// ./foo/package.json +// etc. +import './foo'; +``` + +```javascript +// only looks at +// /bar +// does not search: +// /bar.js +// /bar/index.js +// /bar/package.json +// etc. +import '/bar'; +``` + +```javascript +// continues to *search*: +// node_modules/baz.js +// node_modules/baz/package.json +// node_modules/baz/index.js +import 'baz'; +``` + +#### Vendored modules + +This will mean vendored modules are not included in the search path since `package.json` is not searched for outside of `node_modules`. Please use [bundledDependencies](https://docs.npmjs.com/files/package.json#bundleddependencies) to vendor your dependencies instead. + +#### Shipping both ES6 and CJS + +Since `node_modules` continues to use searching, when a `package.json` main is encountered we are still able to perform file extension searches. If we have 2 entry points `index.jsm` and `index.js` by setting `main:"./index"` we can let Node pick up either depending on what is supported, without us needing to manage multiple entry points separately. ### CommonJS consuming ES6 @@ -136,7 +180,7 @@ module.exports = {default:'my-default'}; You will grab `module.exports` when performing an ES6 import. ```javascript -// es6.js +// es6.jsm import foo from './cjs'; // foo = {default:'my-default'}; @@ -150,9 +194,9 @@ All of these gotchas relate to opt-in semantics and the fact that CommonJS is a No existing code will be affected. -#### Circular Dep CJS => ES6 => CJS +#### Circular Dep CJS => ES6 => CJS Causes Throw -Given: +Due to the following explanation we want to avoid a very specific problem. Given: ```javascript //cjs.js @@ -161,25 +205,23 @@ require('./es6'); ``` ```javascript -//es6.js +//es6.jsm import * as ns from './cjs'; // ns = undefined import cjs from './cjs'; // ns = undefined ``` -The list of properties being exported is not populated until `cjs.js` finishes executing. - -The result is that there are no properties to import and you recieve an empty module. +The list of properties being exported is not populated until `cjs.js` finishes executing. The result is that there are no properties to import and you recieve an empty module. -##### Option: Throw on this case +In order to prevent this sticky situation we will throw on this case. -Since this case is coming from ES6, we could throw whenever this occurs. This is similar to how you cannot change a generator while it is running. +Since this case is coming from ES6, we will not break any existing circular dependencies in CJS <-> CJS. It may be easier to think of this change as similar to how you cannot affect a generator while it is running. -If taken this would change the ES6 module behavior to: +This would change the ES6 module behavior to: ```javascript -//es6.js +//es6.jsm import * as ns from './cjs'; // throw new EvalError('./cjs is not an ES6 module and has not finished evaluation'); ``` From 8dfd028d2c2ab1db8076d880bc867b92797b9127 Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Tue, 2 Feb 2016 08:53:02 -0600 Subject: [PATCH 22/41] be more explicit that node_modules will remain recursive --- 002-es6-modules.md | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index 30a2b0e..f3299b2 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -95,7 +95,7 @@ The `.jsm` file extension will have a higher loading priority than `.js`. ### ES6 Import Resolution -ES6 `import` statements will not perform non-exact searches on relative or absolute paths, unlike `require()`. This means that no file extensions, or index files will be searched for when using relative or absolute paths. `node_modules` based paths will continue to use searching for both compatibility and to not limit the ability to have `package.json` support both ES6 and CJS entry points in a single codebase. +ES6 `import` statements will not perform non-exact searches on relative or absolute paths, unlike `require()`. This means that no file extensions, or index files will be searched for when using relative or absolute paths. `node_modules` based paths will continue to use searching for both compatibility and to not limit the ability to have `package.json` support both ES6 and CJS entry points in a single codebase. `node_modules` based behavior will continue to be unchanged and look to parent `node_modules` directories as well. In summary: @@ -123,9 +123,14 @@ import '/bar'; ```javascript // continues to *search*: -// node_modules/baz.js -// node_modules/baz/package.json -// node_modules/baz/index.js +// ./node_modules/baz.js +// ./node_modules/baz/package.json +// ./node_modules/baz/index.js +// and parent node_modules: +// ../node_modules/baz.js +// ../node_modules/baz/package.json +// ../node_modules/baz/index.js +// etc. import 'baz'; ``` From e19b225da20c660dce3283c20a199e70cca809bd Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Tue, 2 Feb 2016 08:54:20 -0600 Subject: [PATCH 23/41] use word recursive --- 002-es6-modules.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index f3299b2..e98d6f1 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -95,7 +95,7 @@ The `.jsm` file extension will have a higher loading priority than `.js`. ### ES6 Import Resolution -ES6 `import` statements will not perform non-exact searches on relative or absolute paths, unlike `require()`. This means that no file extensions, or index files will be searched for when using relative or absolute paths. `node_modules` based paths will continue to use searching for both compatibility and to not limit the ability to have `package.json` support both ES6 and CJS entry points in a single codebase. `node_modules` based behavior will continue to be unchanged and look to parent `node_modules` directories as well. +ES6 `import` statements will not perform non-exact searches on relative or absolute paths, unlike `require()`. This means that no file extensions, or index files will be searched for when using relative or absolute paths. `node_modules` based paths will continue to use searching for both compatibility and to not limit the ability to have `package.json` support both ES6 and CJS entry points in a single codebase. `node_modules` based behavior will continue to be unchanged and look to parent `node_modules` directories recursively as well. In summary: From c94362ae25311ddb77d1a4aa2275ed416ae16a45 Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Tue, 2 Feb 2016 09:11:00 -0600 Subject: [PATCH 24/41] remove non-local dependencies --- 002-es6-modules.md | 44 ++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 42 insertions(+), 2 deletions(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index e98d6f1..2498c13 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -95,9 +95,11 @@ The `.jsm` file extension will have a higher loading priority than `.js`. ### ES6 Import Resolution -ES6 `import` statements will not perform non-exact searches on relative or absolute paths, unlike `require()`. This means that no file extensions, or index files will be searched for when using relative or absolute paths. `node_modules` based paths will continue to use searching for both compatibility and to not limit the ability to have `package.json` support both ES6 and CJS entry points in a single codebase. `node_modules` based behavior will continue to be unchanged and look to parent `node_modules` directories recursively as well. +ES6 `import` statements will not perform non-exact searches on relative or absolute paths, unlike `require()`. This means that no file extensions, or index files will be searched for when using relative or absolute paths. -In summary: +`node_modules` based paths will continue to use searching for both compatibility and to not limit the ability to have `package.json` support both ES6 and CJS entry points in a single codebase. `node_modules` based behavior will continue to be unchanged and look to parent `node_modules` directories recursively as well. + +In summary so far: ```javascript // only looks at @@ -134,6 +136,44 @@ import '/bar'; import 'baz'; ``` +#### Removal of non-local dependencies + +All of the following: + +* `$NODE_PATH` +* `$HOME/.node_modules` +* `$HOME/.node_libraries` +* `$PREFIX/lib/node` + +will not be supported by the `import` statement. Use local dependencies, and symbolic links as needed. + +##### How to support non-local dependencies + +Although not recommended, and in fact discouraged, there is a way to support non-local dependencies. **USE THIS AT YOUR OWN DISCRETION*. + +Symlinks of `node_modules -> $HOME/.node_modules`, `node_modules/foo/ -> $HOME/.node_modules/foo/`, etc. will continue to be supported. + +Adding a parent directory with `node_modules` symlinked will be an effective strategy for recreating these functionalities. This will incur the known problems with non-local dependencies, but now leaves the problems in the hands of the user, allowing node to give more clear insight to your modules by reducing complexity. + +Given: + +```sh +/opt/local/myapp +``` + +Transform to: + +```sh +/opt/local/non-local-deps/myapp +/opt/local/non-local-deps/node_modules -> $PREFIX/lib/node (etc.) +``` + +And nest as many times as needed. + +#### Errors + +In the case that an `import` statement is unable to find a module, node should make a **best effort** to see if `require` would have found the module and print out where it was found, if `NODE_PATH` was used, if `HOME` was used, etc. + #### Vendored modules This will mean vendored modules are not included in the search path since `package.json` is not searched for outside of `node_modules`. Please use [bundledDependencies](https://docs.npmjs.com/files/package.json#bundleddependencies) to vendor your dependencies instead. From 2fdaa7db27c6d6ffed84979054620bce305b25cf Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Tue, 2 Feb 2016 09:11:39 -0600 Subject: [PATCH 25/41] typo --- 002-es6-modules.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index 2498c13..b204862 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -149,7 +149,7 @@ will not be supported by the `import` statement. Use local dependencies, and sym ##### How to support non-local dependencies -Although not recommended, and in fact discouraged, there is a way to support non-local dependencies. **USE THIS AT YOUR OWN DISCRETION*. +Although not recommended, and in fact discouraged, there is a way to support non-local dependencies. **USE THIS AT YOUR OWN DISCRETION**. Symlinks of `node_modules -> $HOME/.node_modules`, `node_modules/foo/ -> $HOME/.node_modules/foo/`, etc. will continue to be supported. From d435163a6b1bcc4f3fea61bb81100b984bfcbede Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Thu, 4 Feb 2016 10:26:17 -0600 Subject: [PATCH 26/41] remove Promise wrapping, mention System.loader.import and deadlock of top level await --- 002-es6-modules.md | 96 +++++++++++++++++++++------------------------- 1 file changed, 44 insertions(+), 52 deletions(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index b204862..06b5320 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -186,7 +186,7 @@ Since `node_modules` continues to use searching, when a `package.json` main is e #### default exports -ES6 modules only ever declare named exports. A default export just exports a property named `default`. All ES6 modules are wrapped in a `Promise` in anticipation of top level `await`. +ES6 modules only ever declare named exports. A default export just exports a property named `default`. `require` will not automatically wrap ES6 modules in a `Promise`. In the future if top level `await` becomes spec, you can use the `System.loader.import` function to wrap modules and wait on them (top level await can cause deadlock with circular dependencies, node should discourage its use). Given @@ -196,9 +196,7 @@ export default foo; ``` ```javascript -require('./es6').then((namespace) => { - console.log(namespace);// {default:'my-default'} -}) +const foo = require('./es6').foo; ``` #### read only @@ -294,7 +292,7 @@ class SourceTextModule : Script, Module { // get a list of what this exports ExportEntry[] ExportEntries(); - // cannot be called prior to Run() completing + // can be called prior to Run(), but all entries will have values of undefined ModuleNamespace Namespace(); // required prior to Run() @@ -320,6 +318,8 @@ class DynamicModule : Module { // this in a way mimics: // 1. calling ModuleNamespaceCreate(this, exports) // 2. populating the [[Namespace]] field of this Module Record + // + // see JS implementation below for approximate behavior DynamicModule(Object exports); } @@ -338,6 +338,33 @@ class ImportBinding { } ``` +```javascript +// JS implementation of DynamicModule +function DynamicModule(obj) { + let module_namespace = Object.create(null); + function gatherExports(obj, acc = new Set()) { + if (typeof obj !== 'object' && typeof obj !== 'function' || obj === null) { + return acc; + } + for (const key of Object.getOwnPropertyNames(obj)) { + const desc = Object.getOwnPropertyDescriptor(obj, key); + acc.add({key,desc}); + } + return gatherExports(Object.getPrototypeOf(obj), acc); + } + [...gatherExports(obj)].forEach(({key,desc}) => { + if (key === 'default') return; + Object.defineProperty(module_namespace, key, { + get: () => obj[key], + set() {throw new Error(`ModuleNamespace key ${key} is read only.`)}, + configurable: false, + enumerable: Boolean(desc.enumerable) + }); + }); + return module_namespace; +} +``` + ## Example Implementation These are written with the expectation that: @@ -353,12 +380,9 @@ The variable names should be hidden from user code using various techniques left #### Pre Evaluation ```javascript -const namespacePromise = new Promise((f,r) => { - fulfillNamespacePromise = f; - rejectNamespacePromise = r; -}); +// for posterity, will still throw on circular deps ES6ModuleRegistry.set(__filename, new ModuleStatus({ - 'ready': {'[[Result]]':namespacePromise}; + 'ready': {'[[Result]]':undefined}; })); ``` @@ -367,41 +391,21 @@ ES6ModuleRegistry.set(__filename, new ModuleStatus({ ##### On Error ```javascript -rejectNamespacePromise(error); ES6ModuleRegistry.delete(__filename); ``` ##### On Normal Completion ```javascript -const real_exports = require.cache[__filename].exports; -// module_namespace, how an ES6 module would see a CJS module -let module_namespace = Object.create(null); -function gatherExports(obj, acc = new Set()) { - if (typeof obj !== 'object' && typeof obj !== 'function' || obj === null) { - return acc; - } - for (const key of Object.getOwnPropertyNames(obj)) { - const desc = Object.getOwnPropertyDescriptor(obj, key); - acc.add({key,desc}); - } - return gatherExports(Object.getPrototypeOf(obj), acc); -} -[...gatherExports(real_exports)].forEach(({key,desc}) => { - if (key === 'default') return; - Object.defineProperty(module_namespace, key, { - get: () => real_exports[key], - set() {throw new Error(`ModuleNamespace key ${key} is read only.`)}, - configurable: false, - enumerable: Boolean(desc.enumerable) - }); -}) +let module_namespace = Object.create(module.exports); Object.defineProperty(module_namespace, 'default', { - value: real_exports, + value: module.exports, writable: false, configurable: false }); -fulfillNamespacePromise(module_namespace); +ES6ModuleRegistry.set(__filename, new ModuleStatus({ + 'ready': {'[[Result]]':v8.module.DynamicModule(module_namespace)}; +})); ``` ### ES6 Modules @@ -409,19 +413,12 @@ fulfillNamespacePromise(module_namespace); #### Post Parsing ```javascript -const module = ...; -const namespacePromise = new Promise((f,r) => { - fulfillNamespacePromise = f; - rejectNamespacePromise = r; -}); -Object.freeze(namespacePromise); Object.defineProperty(module, 'exports', { - get() {return namespacePromise}; + get() {return v8.Module.Namespace(module)}; set(v) {throw new Error(`${__filename} is an ES6 module and cannot assign to module.exports`)} configurable: false, enumerable: false }); -require.cache[__filename] = namespacePromise; ``` Parsing occurs prior to evaluation, and CJS may execute once we start to resolve `import`. @@ -429,7 +426,10 @@ Parsing occurs prior to evaluation, and CJS may execute once we start to resolve #### Header ```javascript -const exports = void 0; +// we will intercept this to inject the values +import {__filename,__dirname,require,module,exports} from 'CURRENT__FILENAME'; +// to prevent global problems, and false sense of writable exports object: +// exports = undefined ``` #### Immediately Post Evaluation @@ -437,13 +437,5 @@ const exports = void 0; ##### On Error ```javascript -rejectNamespacePromise(error); delete require.cache[__filename]; ``` - - -##### On Normal Completion - -```javascript -fulfillNamespacePromise(ES6ModuleRegistry.get(__filename).GetStage('ready')['[[result]]']); -``` From b0a439ca8211dd2066acaa984e8e0460fb3ce555 Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Fri, 5 Feb 2016 09:28:42 -0600 Subject: [PATCH 27/41] s/snapshot/live bindings/ --- 002-es6-modules.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index 06b5320..38da8dc 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -33,7 +33,7 @@ Discusses the syntax and semantics of related syntax, and introduces: - Defines the list of exports via [ExportEntry](https://tc39.github.io/ecma262/#table-41). * [ModuleNamespace](https://tc39.github.io/ecma262/#sec-module-namespace-exotic-objects) - - Represents a read-only snapshot of a module's exports. + - Represents a read-only static set of live bindings of a module's exports. ### Operations From 8b650a1daf94e193fe85d3f391385432d47ef9a9 Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Mon, 8 Feb 2016 14:10:00 -0600 Subject: [PATCH 28/41] incorrect example --- 002-es6-modules.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index 38da8dc..fa6088a 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -196,7 +196,7 @@ export default foo; ``` ```javascript -const foo = require('./es6').foo; +const foo = require('./es6').default; ``` #### read only From 6a91d02543c06f27d40e661caa4e4658c482c734 Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Wed, 24 Feb 2016 16:02:03 -0600 Subject: [PATCH 29/41] reword to match spec --- 002-es6-modules.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index fa6088a..cedb159 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -299,7 +299,7 @@ class SourceTextModule : Script, Module { // // this will add the bindings to the lexical environment of // the Module - ImportDeclarationInstantiation(ImportBinding[] bindings); + ModuleDeclarationInstantiation(ImportBinding[] bindings); } class DynamicModule : Module { From 9b0ea7f23f827dd5d0d89520630316acffacbfbb Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Wed, 9 Mar 2016 19:12:31 -0600 Subject: [PATCH 30/41] ES6 -> ES, some better clarity on read-only nature during import statements --- 002-es6-modules.md | 168 +++++++++++++++++++++++++++++---------------- 1 file changed, 110 insertions(+), 58 deletions(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index cedb159..016bd21 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -1,4 +1,4 @@ -# ES6 Modules +# ES Modules Status | Draft ------ | ---- @@ -7,7 +7,7 @@ Date | 7 Jan 2016 It is our intent to: -* implement interoperability for ES6 modules and node's existing module system +* implement interoperability for ES modules and node's existing module system * create a **Registry Object** (see WHATWG section below) compatible with the WHATWG Loader Registry ## Purpose @@ -68,36 +68,44 @@ Discusses the design of module metadata in a [Registry](https://whatwg.github.io A Module Record that presents a view of an Object for its `[[Namespace]]` rather than coming from an environment record. -The list of exports is frozen upon construction. No new properties may be added. No properties may be removed. +All exported values are declarative. That means that they are known after the file is parsed, but before it is evaluated. + +When creating a `DynamicModuleRecord`. The list of exports is frozen upon construction. No new exports may be added. No exports may be removed. ## Algorithm -When `require()`ing a file. +When loading a file. -1. Determine if file is ES6 or CommonJS (CJS). +1. Determine if file is ES or CommonJS (CJS). 2. If CJS 1. Evaluate immediately 2. Produce a DynamicModuleRecord from `module.exports` -3. If ES6 +3. If ES 1. Parse for `import`/`export`s and keep record, in order to create bindings - 2. Gather all submodules by performing `require` recursively + 2. Gather all submodules by performing loading dependencies recursively * See circular dep semantics below 3. Connect `import` bindings for all relevant submodules (see [ModuleDeclarationInstantiation](https://tc39.github.io/ecma262/#sec-moduledeclarationinstantiation)) 4. Evaluate + +This still guarantees: + +* that ES module dependencies are all executed prior to the module itself +* CJS modules have a full shape prior to being handed to ES modules +* allows CJS modules to imperatively start the loading of other modules, including ES modules ## Semantics -### Determining if source is an ES6 Module +### Determining if source is an ES Module -A new filetype will be recognised, `.jsm` as ES6 based modules. They will be treated as a different loading semantic but compatible with existing systems, just like `.node`, `.json`, or usage of `require.extension` (even though deprecated) are compatible. It would be ideal if we could register the filetype with IANA as an offical file type, see [TC39 issue](https://github.com/tc39/ecma262/issues/322). Though it seems this would need to go through the [IESG](https://www.ietf.org/iesg/) and it seems browsers are non-plussed on introducing a new MIME. +A new filetype will be recognised, `.jsm` as ES modules. They will be treated as a different loading semantic but compatible with existing systems, just like `.node`, `.json`, or usage of `require.extension` (even though deprecated) are compatible. It would be ideal if we could register the filetype with IANA as an offical file type, see [TC39 issue](https://github.com/tc39/ecma262/issues/322). Though it seems this would need to go through the [IESG](https://www.ietf.org/iesg/) and it seems browsers are non-plussed on introducing a new MIME. The `.jsm` file extension will have a higher loading priority than `.js`. -### ES6 Import Resolution +### ES Import Path Resolution -ES6 `import` statements will not perform non-exact searches on relative or absolute paths, unlike `require()`. This means that no file extensions, or index files will be searched for when using relative or absolute paths. +ES `import` statements will not perform non-exact searches on relative or absolute paths, unlike `require()`. This means that no file extensions, or index files will be searched for when using relative or absolute paths. -`node_modules` based paths will continue to use searching for both compatibility and to not limit the ability to have `package.json` support both ES6 and CJS entry points in a single codebase. `node_modules` based behavior will continue to be unchanged and look to parent `node_modules` directories recursively as well. +`node_modules` based paths will continue to use searching for both compatibility and to not limit the ability to have `package.json` support both ES and CJS entry points in a single codebase. `node_modules` based behavior will continue to be unchanged and look to parent `node_modules` directories recursively as well. In summary so far: @@ -178,95 +186,139 @@ In the case that an `import` statement is unable to find a module, node should m This will mean vendored modules are not included in the search path since `package.json` is not searched for outside of `node_modules`. Please use [bundledDependencies](https://docs.npmjs.com/files/package.json#bundleddependencies) to vendor your dependencies instead. -#### Shipping both ES6 and CJS +#### Shipping both ES and CJS Since `node_modules` continues to use searching, when a `package.json` main is encountered we are still able to perform file extension searches. If we have 2 entry points `index.jsm` and `index.js` by setting `main:"./index"` we can let Node pick up either depending on what is supported, without us needing to manage multiple entry points separately. -### CommonJS consuming ES6 +##### Excluding main + +Since `main` in `package.json` is entirely optional even inside of npm packages, some people may prefer to exclude main entirely in the case of using `./index` as that is still in the node module search algorithm. + +### ES consuming CommonJS + +#### default imports + +`module.exports` is a single value. As such it does not have the dictionary like properties of ES module exports. In order to facilitate named imports for ES modules, all properties of `module.exports` will be hoisted to named exports after evaluation of CJS modules with the exception of `default` which will point to `module.exports` directly. + +Given: + +```javascript +// cjs.js +module.exports = { + default:'my-default', + thing:'stuff' +}; +``` + +You will grab `module.exports` when performing an ES import. + +```javascript +// es.jsm +import foo from './cjs'; +// foo = {default:'my-default', thing:'stuff'}; + +import {default as bar} from './cjs'; +// bar = {default:'my-default', thing:'stuff'}; +``` + +### CommonJS consuming ES #### default exports -ES6 modules only ever declare named exports. A default export just exports a property named `default`. `require` will not automatically wrap ES6 modules in a `Promise`. In the future if top level `await` becomes spec, you can use the `System.loader.import` function to wrap modules and wait on them (top level await can cause deadlock with circular dependencies, node should discourage its use). +ES modules only ever declare named exports. A default export just exports a property named `default`. `require` will not automatically wrap ES modules in a `Promise`. In the future if top level `await` becomes spec, you can use the `System.loader.import` function to wrap modules and wait on them (top level await can cause deadlock with circular dependencies, node should discourage its use). Given ```javascript -let foo = 'my-default'; +// es.jsm +let foo = {bar:'my-default'}; export default foo; ``` ```javascript -const foo = require('./es6').default; +// cjs.js +const es_namespace = require('./es'); +// es_namespace = { +// default: { +// bar:'my-default' +// } +// } ``` -#### read only +### Known Gotchas -The objects create by an ES6 module are [ModuleNamespace Objects](https://tc39.github.io/ecma262/#sec-module-namespace-exotic-objects). +All of these gotchas relate to opt-in semantics and the fact that CommonJS is a dynamic loader while ES is a static loader. -These have `[[Set]]` be a no-op and are read only views of the exports of an ES6 module. +No existing code will be affected. -### ES6 consuming CommonJS +#### No reassigning `module.exports` after evaluation -#### default imports +Since we need a consistent time to snapshot the `module.exports` of a CJS module. We will execute it immediately after evaluation. Code such as: -`module.exports` shadows `module.exports.default`. +```javascript +// bad-cjs.js +module.exports = 123; +setTimeout(_ => module.exports = null); +``` -This means that if there is a `.default` on your CommonJS exports it will be shadowed to the `module.exports` of your module. +Will not see `module.exports` change to `null`. All ES module `import`s of the module will always see `123`. -Given: +#### ES exports are read only -```javascript -// cjs.js -module.exports = {default:'my-default'}; -``` +The objects create by an ES module are [ModuleNamespace Objects](https://tc39.github.io/ecma262/#sec-module-namespace-exotic-objects). -You will grab `module.exports` when performing an ES6 import. +These have `[[Set]]` be a no-op and are read only views of the exports of an ES module. Attempting to reassign any named export will not work, but assigning to the properties of the exports follows normal rules. -```javascript -// es6.jsm -import foo from './cjs'; -// foo = {default:'my-default'}; +### CJS exports allow mutation -import {default as bar} from './cjs'; -// bar = {default:'my-default'}; +Unlike ES modules, CJS modules have allowed mutation. When ES modules are integrating against CJS systems like Grunt, it may be necessary to mutate a `module.exports`. + +Remember that `module.exports` from CJS is directly available under `default` for `import`. This means that if you use: + +```javascript +import * as shallow from 'grunt'; ``` -### Known Gotchas +According to ES `*` creates a shallow copy and all properties will be read-only. -All of these gotchas relate to opt-in semantics and the fact that CommonJS is a dynamic loader while ES6 is a static loader. +However, doing: -No existing code will be affected. +```javascript +import grunt from 'grunt'; +``` -#### Circular Dep CJS => ES6 => CJS Causes Throw +Grabs the `default` which is exactly what `module.exports` is, and all the properties will be mutable. + +#### Circular Dep CJS => ES => CJS Causes Throw Due to the following explanation we want to avoid a very specific problem. Given: ```javascript -//cjs.js +// cjs.js module.exports = {x:0}; -require('./es6'); +require('./es'); ``` ```javascript -//es6.jsm +// es.jsm import * as ns from './cjs'; -// ns = undefined +// ns = ? import cjs from './cjs'; -// ns = undefined +// cjs = ? ``` -The list of properties being exported is not populated until `cjs.js` finishes executing. The result is that there are no properties to import and you recieve an empty module. +ES modules must know the list of exported bindings of all dependencies prior to evaluating. The value being exported for CJS modules is not stable to snapshot until `cjs.js` finishes executing. The result is that there are no properties to import and you recieve an empty module. In order to prevent this sticky situation we will throw on this case. -Since this case is coming from ES6, we will not break any existing circular dependencies in CJS <-> CJS. It may be easier to think of this change as similar to how you cannot affect a generator while it is running. +Since this case is coming from ES, we will not break any existing circular dependencies in CJS <-> CJS. It may be easier to think of this change as similar to how you cannot affect a generator while it is running. -This would change the ES6 module behavior to: +This would change the ES module behavior to: ```javascript -//es6.jsm +// es.jsm import * as ns from './cjs'; -// throw new EvalError('./cjs is not an ES6 module and has not finished evaluation'); +// throw new EvalError('./cjs is not an ES module and has not finished evaluation'); ``` ## Advisory @@ -304,7 +356,7 @@ class SourceTextModule : Script, Module { class DynamicModule : Module { // in order for CommonJS modules to create fully formed - // ES6 Module compatibility we need to hook up a static + // ES Module compatibility we need to hook up a static // View of an Object to set as our exports // // think of this as calling ImportDeclarationInstantiation using the current @@ -361,7 +413,7 @@ function DynamicModule(obj) { enumerable: Boolean(desc.enumerable) }); }); - return module_namespace; + return Object.freeze(module_namespace); } ``` @@ -370,7 +422,7 @@ function DynamicModule(obj) { These are written with the expectation that: * ModuleNamespaces can be created from existing Objects. -* WHATWG Loader spec Registry is available as ES6ModuleRegistry. +* WHATWG Loader spec Registry is available as a ModuleRegistry. * ModuleStatus Objects can be created. The variable names should be hidden from user code using various techniques left out here. @@ -381,7 +433,7 @@ The variable names should be hidden from user code using various techniques left ```javascript // for posterity, will still throw on circular deps -ES6ModuleRegistry.set(__filename, new ModuleStatus({ +ModuleRegistry.set(__filename, new ModuleStatus({ 'ready': {'[[Result]]':undefined}; })); ``` @@ -391,7 +443,7 @@ ES6ModuleRegistry.set(__filename, new ModuleStatus({ ##### On Error ```javascript -ES6ModuleRegistry.delete(__filename); +ModuleRegistry.delete(__filename); ``` ##### On Normal Completion @@ -403,19 +455,19 @@ Object.defineProperty(module_namespace, 'default', { writable: false, configurable: false }); -ES6ModuleRegistry.set(__filename, new ModuleStatus({ +ModuleRegistry.set(__filename, new ModuleStatus({ 'ready': {'[[Result]]':v8.module.DynamicModule(module_namespace)}; })); ``` -### ES6 Modules +### ES Modules #### Post Parsing ```javascript Object.defineProperty(module, 'exports', { get() {return v8.Module.Namespace(module)}; - set(v) {throw new Error(`${__filename} is an ES6 module and cannot assign to module.exports`)} + set(v) {throw new Error(`${__filename} is an ES module and cannot assign to module.exports`)} configurable: false, enumerable: false }); From 4034984a127043360aba6ecb70bc2c60b4de3ddf Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Wed, 9 Mar 2016 19:38:53 -0600 Subject: [PATCH 31/41] reorder / slightly expand gotchas --- 002-es6-modules.md | 55 +++++++++++++++++++++++++++++++++------------- 1 file changed, 40 insertions(+), 15 deletions(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index 016bd21..941f331 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -59,9 +59,6 @@ Discusses the design of module metadata in a [Registry](https://whatwg.github.io **NOTE** It is not Node's intent to implement the asynchronous pipeline in the Loader specification. There is discussion about including a synchronous pipeline in the specification as an addendum. - -### [Summary Video on Youtube](youtube.com/watch?v=NdOKO-6Ty7k) - ## Additional Structures Required ### DynamicModuleRecord @@ -251,18 +248,6 @@ All of these gotchas relate to opt-in semantics and the fact that CommonJS is a No existing code will be affected. -#### No reassigning `module.exports` after evaluation - -Since we need a consistent time to snapshot the `module.exports` of a CJS module. We will execute it immediately after evaluation. Code such as: - -```javascript -// bad-cjs.js -module.exports = 123; -setTimeout(_ => module.exports = null); -``` - -Will not see `module.exports` change to `null`. All ES module `import`s of the module will always see `123`. - #### ES exports are read only The objects create by an ES module are [ModuleNamespace Objects](https://tc39.github.io/ecma262/#sec-module-namespace-exotic-objects). @@ -289,6 +274,46 @@ import grunt from 'grunt'; Grabs the `default` which is exactly what `module.exports` is, and all the properties will be mutable. +#### ES will not honor reassigning `module.exports` after evaluation + +Since we need a consistent time to snapshot the `module.exports` of a CJS module. We will execute it immediately after evaluation. Code such as: + +```javascript +// bad-cjs.js +module.exports = 123; +setTimeout(_ => module.exports = null); +``` + +Will not see `module.exports` change to `null`. All ES module `import`s of the module will always see `123`. + +#### ES export list for CJS are snapshot immediately after execution. + +Since `module.exports` is snapshot immediately after execution, that is the point when hoisting of properties occurs, adding and removing properties later will not have an effect on the list of exports. + +```javascript +// bad-cjs.js +module.exports = { + yo: 'lo' +}; +setTimeout(_ => { + delete module.exports.yo; + module.exports.foo = 'bar'; + require('./es.js'); +}); +``` + +```javascript +// es.js +import * as namespace from './bad-cjs.js'; +console.log(Object.keys(namespace)); // ['yo'] +console.log(namespace.foo); // undefined + +// mutate to show 'yo' still exists as a binding +import cjs_exports from './bad-cjs.js'; +cjs_exports.yo = 'lo again'; +console.log(namespace.yo); // 'yolo again' +``` + #### Circular Dep CJS => ES => CJS Causes Throw Due to the following explanation we want to avoid a very specific problem. Given: From 182506cf6efdeb855da3487d2f09b9313df169cc Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Thu, 10 Mar 2016 10:29:18 -0600 Subject: [PATCH 32/41] add module.exports examples for CJS -> ES --- 002-es6-modules.md | 73 ++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 70 insertions(+), 3 deletions(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index 941f331..af4341d 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -197,6 +197,8 @@ Since `main` in `package.json` is entirely optional even inside of npm packages, `module.exports` is a single value. As such it does not have the dictionary like properties of ES module exports. In order to facilitate named imports for ES modules, all properties of `module.exports` will be hoisted to named exports after evaluation of CJS modules with the exception of `default` which will point to `module.exports` directly. +##### Examples + Given: ```javascript @@ -218,6 +220,71 @@ import {default as bar} from './cjs'; // bar = {default:'my-default', thing:'stuff'}; ``` +------ + +Given: + +```javascript +// cjs.js +module.exports = null; +``` + +You will grab `module.exports` when performing an ES import. + +```javascript +// es.jsm +import foo from './cjs'; +// foo = null; + +import * as bar from './cjs'; +// bar = {default:null}; +``` + +------ + +Given: + +```javascript +// cjs.js +module.exports = function two() { + return 2; +}; +``` + +You will grab `module.exports` when performing an ES import. + +```javascript +// es.jsm +import foo from './cjs'; +foo(); // 2 + +import * as bar from './cjs'; +bar.name; // 'two' +bar.default(); // 2 +bar(); // throws, bar is not a function +``` + +------ + +Given: + +```javascript +// cjs.js +module.exports = Promise.resolve(3); +``` + +You will grab `module.exports` when performing an ES import. + +```javascript +// es.jsm +import foo from './cjs'; +foo.then(console.log); // outputs 3 + +import * as bar from './cjs'; +bar.default.then(console.log); // outputs 3 +bar.then(console.log); // throws, bar is not a Promise +``` + ### CommonJS consuming ES #### default exports @@ -261,15 +328,15 @@ Unlike ES modules, CJS modules have allowed mutation. When ES modules are integr Remember that `module.exports` from CJS is directly available under `default` for `import`. This means that if you use: ```javascript -import * as shallow from 'grunt'; +import * as namespace from 'grunt'; ``` -According to ES `*` creates a shallow copy and all properties will be read-only. +According to ES `*` grabs the namespace directly whose properties will be read-only. However, doing: ```javascript -import grunt from 'grunt'; +import grunt_default from 'grunt'; ``` Grabs the `default` which is exactly what `module.exports` is, and all the properties will be mutable. From d241ebe2c40924f480ad6eef0d889b7978783ebc Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Mon, 14 Mar 2016 14:47:22 -0500 Subject: [PATCH 33/41] more examples, slight reordering --- 002-es6-modules.md | 47 +++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 44 insertions(+), 3 deletions(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index af4341d..713a1ee 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -213,9 +213,19 @@ You will grab `module.exports` when performing an ES import. ```javascript // es.jsm + +// grabs the namespace +import * as baz from './cjs'; +// baz = { +// get default() {return module.exports;}, +// get thing() {return this.default.thing}.bind(baz) +// } + +// grabs "default", aka module.exports directly import foo from './cjs'; // foo = {default:'my-default', thing:'stuff'}; +// grabs "default", aka module.exports directly import {default as bar} from './cjs'; // bar = {default:'my-default', thing:'stuff'}; ``` @@ -291,22 +301,53 @@ bar.then(console.log); // throws, bar is not a Promise ES modules only ever declare named exports. A default export just exports a property named `default`. `require` will not automatically wrap ES modules in a `Promise`. In the future if top level `await` becomes spec, you can use the `System.loader.import` function to wrap modules and wait on them (top level await can cause deadlock with circular dependencies, node should discourage its use). +##### Examples + Given ```javascript // es.jsm let foo = {bar:'my-default'}; +// note: +// this is a value +// it is not a binding like `export {foo}` export default foo; +foo = null; ``` ```javascript // cjs.js const es_namespace = require('./es'); -// es_namespace = { -// default: { -// bar:'my-default' +// es_namespace ~= { +// get default() { +// return result_from_evaluating_foo; // } // } +console.log(es_namespace.default); +// {bar:'my-default'} +``` + +------ + +Given + +```javascript +// es.jsm +export let foo = {bar:'my-default'}; +export {foo as bar}; +export function f() {}; +export class c {}; +``` + +```javascript +// cjs.js +const es_namespace = require('./es'); +// es_namespace ~= { +// get foo() {return foo;} +// get bar() {return foo;} +// get f() {return f;} +// get c() {return c;} +// } ``` ### Known Gotchas From b82646669bf5738f35dc0f29c73faba7dfa00000 Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Thu, 7 Apr 2016 09:05:45 -0500 Subject: [PATCH 34/41] fix missing file extensions, thanks to @onlywei for finding --- 002-es6-modules.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index 713a1ee..fc206fb 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -243,10 +243,10 @@ You will grab `module.exports` when performing an ES import. ```javascript // es.jsm -import foo from './cjs'; +import foo from './cjs.js'; // foo = null; -import * as bar from './cjs'; +import * as bar from './cjs.js'; // bar = {default:null}; ``` @@ -265,10 +265,10 @@ You will grab `module.exports` when performing an ES import. ```javascript // es.jsm -import foo from './cjs'; +import foo from './cjs.js'; foo(); // 2 -import * as bar from './cjs'; +import * as bar from './cjs.js'; bar.name; // 'two' bar.default(); // 2 bar(); // throws, bar is not a function @@ -287,10 +287,10 @@ You will grab `module.exports` when performing an ES import. ```javascript // es.jsm -import foo from './cjs'; +import foo from './cjs.js'; foo.then(console.log); // outputs 3 -import * as bar from './cjs'; +import * as bar from './cjs.js'; bar.default.then(console.log); // outputs 3 bar.then(console.log); // throws, bar is not a Promise ``` @@ -434,9 +434,9 @@ require('./es'); ```javascript // es.jsm -import * as ns from './cjs'; +import * as ns from './cjs.js'; // ns = ? -import cjs from './cjs'; +import cjs from './cjs.js'; // cjs = ? ``` From 87708e83cc1c59497f938fb0e41b2b43b6f0077f Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Thu, 7 Apr 2016 09:08:16 -0500 Subject: [PATCH 35/41] fix missing file extensions --- 002-es6-modules.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index fc206fb..51f406a 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -215,18 +215,18 @@ You will grab `module.exports` when performing an ES import. // es.jsm // grabs the namespace -import * as baz from './cjs'; +import * as baz from './cjs.js'; // baz = { // get default() {return module.exports;}, // get thing() {return this.default.thing}.bind(baz) // } // grabs "default", aka module.exports directly -import foo from './cjs'; +import foo from './cjs.js'; // foo = {default:'my-default', thing:'stuff'}; // grabs "default", aka module.exports directly -import {default as bar} from './cjs'; +import {default as bar} from './cjs.js'; // bar = {default:'my-default', thing:'stuff'}; ``` @@ -450,7 +450,7 @@ This would change the ES module behavior to: ```javascript // es.jsm -import * as ns from './cjs'; +import * as ns from './cjs.js'; // throw new EvalError('./cjs is not an ES module and has not finished evaluation'); ``` From 590d05c33bda3a5f8dab9b4258100a67770aed2a Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Fri, 15 Apr 2016 11:56:28 -0500 Subject: [PATCH 36/41] move from .jsm to .mjs due to firefox privileges --- 002-es6-modules.md | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index 51f406a..d9d9b6a 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -94,9 +94,9 @@ This still guarantees: ### Determining if source is an ES Module -A new filetype will be recognised, `.jsm` as ES modules. They will be treated as a different loading semantic but compatible with existing systems, just like `.node`, `.json`, or usage of `require.extension` (even though deprecated) are compatible. It would be ideal if we could register the filetype with IANA as an offical file type, see [TC39 issue](https://github.com/tc39/ecma262/issues/322). Though it seems this would need to go through the [IESG](https://www.ietf.org/iesg/) and it seems browsers are non-plussed on introducing a new MIME. +A new filetype will be recognised, `.mjs` as ES modules. They will be treated as a different loading semantic but compatible with existing systems, just like `.node`, `.json`, or usage of `require.extension` (even though deprecated) are compatible. It would be ideal if we could register the filetype with IANA as an offical file type, see [TC39 issue](https://github.com/tc39/ecma262/issues/322). Though it seems this would need to go through the [IESG](https://www.ietf.org/iesg/) and it seems browsers are non-plussed on introducing a new MIME. -The `.jsm` file extension will have a higher loading priority than `.js`. +The `.mjs` file extension will have a higher loading priority than `.js`. ### ES Import Path Resolution @@ -185,7 +185,7 @@ This will mean vendored modules are not included in the search path since `packa #### Shipping both ES and CJS -Since `node_modules` continues to use searching, when a `package.json` main is encountered we are still able to perform file extension searches. If we have 2 entry points `index.jsm` and `index.js` by setting `main:"./index"` we can let Node pick up either depending on what is supported, without us needing to manage multiple entry points separately. +Since `node_modules` continues to use searching, when a `package.json` main is encountered we are still able to perform file extension searches. If we have 2 entry points `index.mjs` and `index.js` by setting `main:"./index"` we can let Node pick up either depending on what is supported, without us needing to manage multiple entry points separately. ##### Excluding main @@ -212,7 +212,7 @@ module.exports = { You will grab `module.exports` when performing an ES import. ```javascript -// es.jsm +// es.mjs // grabs the namespace import * as baz from './cjs.js'; @@ -242,7 +242,7 @@ module.exports = null; You will grab `module.exports` when performing an ES import. ```javascript -// es.jsm +// es.mjs import foo from './cjs.js'; // foo = null; @@ -264,7 +264,7 @@ module.exports = function two() { You will grab `module.exports` when performing an ES import. ```javascript -// es.jsm +// es.mjs import foo from './cjs.js'; foo(); // 2 @@ -286,7 +286,7 @@ module.exports = Promise.resolve(3); You will grab `module.exports` when performing an ES import. ```javascript -// es.jsm +// es.mjs import foo from './cjs.js'; foo.then(console.log); // outputs 3 @@ -306,7 +306,7 @@ ES modules only ever declare named exports. A default export just exports a prop Given ```javascript -// es.jsm +// es.mjs let foo = {bar:'my-default'}; // note: // this is a value @@ -332,7 +332,7 @@ console.log(es_namespace.default); Given ```javascript -// es.jsm +// es.mjs export let foo = {bar:'my-default'}; export {foo as bar}; export function f() {}; @@ -433,7 +433,7 @@ require('./es'); ``` ```javascript -// es.jsm +// es.mjs import * as ns from './cjs.js'; // ns = ? import cjs from './cjs.js'; @@ -449,7 +449,7 @@ Since this case is coming from ES, we will not break any existing circular depen This would change the ES module behavior to: ```javascript -// es.jsm +// es.mjs import * as ns from './cjs.js'; // throw new EvalError('./cjs is not an ES module and has not finished evaluation'); ``` From 6a5c0074955338757201c343b48ee174af52f698 Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Sat, 16 Apr 2016 09:17:03 -0500 Subject: [PATCH 37/41] explain detection reasoning --- 002-es6-modules.md | 53 +++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 52 insertions(+), 1 deletion(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index d9d9b6a..818ade1 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -96,7 +96,58 @@ This still guarantees: A new filetype will be recognised, `.mjs` as ES modules. They will be treated as a different loading semantic but compatible with existing systems, just like `.node`, `.json`, or usage of `require.extension` (even though deprecated) are compatible. It would be ideal if we could register the filetype with IANA as an offical file type, see [TC39 issue](https://github.com/tc39/ecma262/issues/322). Though it seems this would need to go through the [IESG](https://www.ietf.org/iesg/) and it seems browsers are non-plussed on introducing a new MIME. -The `.mjs` file extension will have a higher loading priority than `.js`. +The `.mjs` file extension will have a higher loading priority than `.js` for `require`. This means that, once the node resolution algorithm reaches file expansion, the path for `path + '.mjs'` would be attempted prior to `path + '.js'` when performing `require(path)`. + +#### Reason for decision + +The choice of `.mjs` was made due to a number of factors. + +* `.jsm` + * conflict with Firefox, which includes escalated [privileges over the `file://` protocol](https://developer.mozilla.org/en-US/docs/Mozilla/JavaScript_code_modules/Using#The_URL_for_a_code_module) that can access [sensistive information](https://developer.mozilla.org/en-US/docs/Mozilla/JavaScript_code_modules/Services.jsm). This could affect things like [test runners providing browser test viewers](https://mochajs.org/#running-mocha-in-the-browser) + * [decent usage on npm](https://gist.github.com/ChALkeR/9e1fb15d0a0e18589e6dadc34b80875d) +* `.es` + * lacks conflicts with other major software + * removes the JS moniker/signifier in many projects such as Node.js, Cylon.js, Three.js, DynJS, JSX, ... + * removes JavaScript -> JS acronym association for learning + * is an existing TLD, could be place for squatting / confusion. + * [small usage on npm](https://gist.github.com/ChALkeR/261550d903ec9867bbab) +* `.m.js` + * potential conflict with existing software targetting wrong goal + * allows `*.js` style globbing to work still + * toolchain problems for asset pipelines/node/editors that only check after last `.` + * [small usage on npm](https://gist.github.com/ChALkeR/c10642f2531b1be36e5d) +* `.mjs` + * lacks conflicts with other major software, conflicts with [metascript](https://github.com/metascript/metascript) but that was last published in 2014 + +There is knowledge of breakage for code that *upgrades* inner package dependencies such as `require('foo/bar.js')`. As `bar.js` may move to `bar.mjs`. Since `bar.js` is not the listed entry point this was considered ok. If foo was providing this file explicitly like `lodash` has: this can be mitigated easily by using as proxy module should `foo` choose to provide one: + +```js +Object.defineProperty(module, 'exports', { + get() { + return require(__filename.replace(/\.js$/,'.mjs')) + }, + configurable:false +}); +Object.freeze(module); +``` + +Concerns of ecosystem damage when using a new file extension were considered as well. Firewall rules and server scripts using `*.js` as the detection mechanism for JavaScript would be affected by our change, largely just affecting browsers. However, new file extensions and mimes continue to be introduced and supported by such things. If a front-end is unable to make such a change, using a rewrite rule from `.mjs` to `.js` should suffieciently mitigate this. + +There were proposals of using `package.json` as an out of band configuration as well. + +* removes one-off file execution for quick scripting like using `.save` in the repl then running. +* causes editors/asset pipelines to have knowledge of this. most work solely on file extension and it would be prohibitive to change. + * Build asset pipelines in particular would be affected such as the Rails asset pipeline + * OS file associations could be complicated since they work after the last `.` + * HTTP/2 PUSH solutions would also need this and would be affected +* no direction for complete removal of CJS. A file extension leads to a world without `.js` and only `.mjs` files. This would be a permanent field in `package.json` +* per file mode requirements mean using globs or large lists + * discussion of allowing only one mode per package was a non-starter for migration path issues +* `package.json` is not required in `node_modules/` and raw files can live in `node_modules/` + * eg. `node_modules/foo/index.js` + * eg. `node_modules/foo.js` +* complex abstraction to handle symlinks used by tools like [link-local](https://github.com/timoxley/linklocal) + * eg. `node_modules/foo -> ../app/components/foo.js` ### ES Import Path Resolution From 18709ffb64b19e479a06473b1e8ad4c810218377 Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Sat, 16 Apr 2016 09:31:31 -0500 Subject: [PATCH 38/41] usage for .mjs --- 002-es6-modules.md | 1 + 1 file changed, 1 insertion(+) diff --git a/002-es6-modules.md b/002-es6-modules.md index 818ade1..59e678b 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -118,6 +118,7 @@ The choice of `.mjs` was made due to a number of factors. * [small usage on npm](https://gist.github.com/ChALkeR/c10642f2531b1be36e5d) * `.mjs` * lacks conflicts with other major software, conflicts with [metascript](https://github.com/metascript/metascript) but that was last published in 2014 + * [small usage on npm](https://gist.github.com/bmeck/07a5beb6541c884acbe908df7b28df3f) There is knowledge of breakage for code that *upgrades* inner package dependencies such as `require('foo/bar.js')`. As `bar.js` may move to `bar.mjs`. Since `bar.js` is not the listed entry point this was considered ok. If foo was providing this file explicitly like `lodash` has: this can be mitigated easily by using as proxy module should `foo` choose to provide one: From 033623ccf4dbedb7fae9952abd68dc6e0ba78509 Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Mon, 18 Apr 2016 09:01:24 -0500 Subject: [PATCH 39/41] remove comment about OS file association, no clear usecase for different bins --- 002-es6-modules.md | 1 - 1 file changed, 1 deletion(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index 59e678b..563e1c4 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -139,7 +139,6 @@ There were proposals of using `package.json` as an out of band configuration as * removes one-off file execution for quick scripting like using `.save` in the repl then running. * causes editors/asset pipelines to have knowledge of this. most work solely on file extension and it would be prohibitive to change. * Build asset pipelines in particular would be affected such as the Rails asset pipeline - * OS file associations could be complicated since they work after the last `.` * HTTP/2 PUSH solutions would also need this and would be affected * no direction for complete removal of CJS. A file extension leads to a world without `.js` and only `.mjs` files. This would be a permanent field in `package.json` * per file mode requirements mean using globs or large lists From 45cac133fd64646708ca8c5bd86552e726fef1c7 Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Wed, 20 Apr 2016 16:08:53 -0500 Subject: [PATCH 40/41] note on Draft status --- 002-es6-modules.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/002-es6-modules.md b/002-es6-modules.md index 563e1c4..0dfb5f0 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -5,6 +5,8 @@ Status | Draft Author | Bradley Meck Date | 7 Jan 2016 +**NOTE:** `Draft` status means that this has not been accepted for implementation into node core. This is the document of the standard node core would implement a feature should this draft be moved to `Accepted`. + It is our intent to: * implement interoperability for ES modules and node's existing module system From dd6c91850fd57c88cda767d80ead94a7a710da93 Mon Sep 17 00:00:00 2001 From: Bradley Farias Date: Wed, 20 Apr 2016 16:10:31 -0500 Subject: [PATCH 41/41] rephrase intent --- 002-es6-modules.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/002-es6-modules.md b/002-es6-modules.md index 0dfb5f0..919ff67 100644 --- a/002-es6-modules.md +++ b/002-es6-modules.md @@ -7,7 +7,9 @@ Date | 7 Jan 2016 **NOTE:** `Draft` status means that this has not been accepted for implementation into node core. This is the document of the standard node core would implement a feature should this draft be moved to `Accepted`. -It is our intent to: +--- + +The intent of this standard is to: * implement interoperability for ES modules and node's existing module system * create a **Registry Object** (see WHATWG section below) compatible with the WHATWG Loader Registry