diff --git a/proposals/html-module-spec-changes.md b/proposals/html-module-spec-changes.md index 0807cec1..bd119b78 100644 --- a/proposals/html-module-spec-changes.md +++ b/proposals/html-module-spec-changes.md @@ -1,45 +1,64 @@ -This is a list of spec areas that will need to be changed to implement our [HTML Modules proposal](https://github.com/w3c/webcomponents/blob/gh-pages/proposals/html-modules-proposal.md). Some of the proposed changes are very specific and some are more handwavy depending on how deep I've investigated into that particular area. +This is a list of spec areas that will need to be changed to implement our [HTML Modules proposal](https://github.com/w3c/webcomponents/blob/gh-pages/proposals/html-modules-proposal.md) Questions/corrections/feedback are welcome! I've left TODOs in several places where we still have open questions; any input regarding these is especially appreciated. -- [@dandclark](https://github.com/dandclark), with:\     [@bocupp](https://github.com/BoCupp) [@samsebree](https://github.com/samsebree) [@travisleithead](https://github.com/travisleithead) # ES6 Spec Changes ([full spec link](https://tc39.github.io/ecma262/)): -- Introduce a new subtype of [Abstract Module Record](https://tc39.github.io/ecma262/#sec-abstract-module-records) in addition to the existing [Source Text Module Record](https://tc39.github.io/ecma262/#sourctextmodule-record). Proposed name for the new subtype is HTML Module Record. - 1. HTML Module record has these fields in common with Source Text Module Record (perhaps they should move up to Abstract Module Record): [[Status]], [[EvaluationError]], [[DFSIndex]], [[DFSAncestorIndex]]. - 2. In addition HTML Module Records have this new field: - [[RequestedModules]]: A list of ScriptEntry records, appearing in document order per the position of the corresponding script elements in the HTML Document. ScriptEntry is defined as: + +These spec changes are built on top of the proposed refactoring here: https://github.com/tc39/ecma262/pull/1311 + +- Introduce a new subtype of Cyclic Module Record (TODO Add link to Cyclic MR once it's part of the official spec) in addition to the existing [Source Text Module Record](https://tc39.github.io/ecma262/#sourctextmodule-record), named HTML Module Record. + 1. HTML Module Records reuse the [[RequestedModules]] field of Cyclic Module Record, but instead of a list of strings it is a list of ScriptEntry records. See definition of ParseHTMLModule in HTML5 spec changes for a specification of how these are populated (although it is important to note that ES has no knowledge of this process which involves the HTML Parser, inline vs external script elements etc). + _[TODO: This reuse-name-with-different-type is pretty fishy. Should [[RequestedModules]] in Cyclic MR be generalized as a list that can hold objects of a type specified in each subclass? Or should we use an entirely new field in HTML MR? Or should we generate unique IDs for the inline script elements and place them in the Module Map, so that an HTML Module Record can just use strings in [[RequestedModules]]?]._ + ScriptEntry is defined as: | Field Name | Value Type | Meaning | | --- | --- | --- | - | [[IsInline]] |bool | Is this entry from an inline script? | - | [[ModuleRecord]] | Source Text Module Record \| null | The source text module record for the script element if IsInline == true | - | [[SourceName]] | String \| null | The name specified in the script’s src attribute if IsInline == false. Null otherwise. | - 3. Additionally, the [[HostDefined]] field in Abstract Module Record should be used to hold the document, or an HTML Module object on the HTML5 spec side that will hold the document. It’s basically meant as an abstract container for the HTML5 spec side of things to stash data that it defines -- for script modules this holds the [module script](https://html.spec.whatwg.org/multipage/webappapis.html#module-script) as defined in the HTML5 spec. -- HTML Module record should have a modified version of [InnerModuleInstantiation](https://tc39.github.io/ecma262/#sec-innermoduleinstantiation). In short, the existing definition of this function recursively calls InnerModuleInstantiation on each child module (calling [HostResolveImportedModule](https://tc39.github.io/ecma262/#sec-hostresolveimportedmodule) to resolve module names to Source Text Module Records), then calls [ModuleDeclarationEnvironmentSetup](https://tc39.github.io/ecma262/#sec-moduledeclarationenvironmentsetup) to set up the lexical environment and resolve imports/exports for the current module. In order to follow the structure of the newly defined ScriptEntry record as defined above, we will change the current definition of InnerModuleInstantiation’s step 9 (“For each string required that is an element of module.[[RequestedModules]]...) + | [[InlineModuleRecord]] | Module Record \| null | The Module Record for the module request if available at ScriptEntry creation time. Null otherwise. | + | [[ExternalScriptURL]] | String \| null | The URL for the module request if the Module Record was not available at ScriptEntry creation time. Null otherwise. | + 2. The [[HostDefined]] field in Abstract Module Record will be set to the HTML Module Script (see HTML5 Spec changes). This is analogous to script modules where this field holds the JavaScript [module script](https://html.spec.whatwg.org/multipage/webappapis.html#module-script) as defined in the HTML5 spec. +- HTML Module Record inherits the concrete Instantiate() method from Cyclic Module Record. +- HTML Module Record defines its own version of [InnerModuleInstantiation](https://tc39.github.io/ecma262/#sec-innermoduleinstantiation). Cyclic Module Record's definition recursively calls InnerModuleInstantiation on each child module (calling [HostResolveImportedModule](https://tc39.github.io/ecma262/#sec-hostresolveimportedmodule) to resolve module names to Module Records), then calls [ModuleDeclarationEnvironmentSetup](https://tc39.github.io/ecma262/#sec-moduledeclarationenvironmentsetup) to set up the lexical environment and resolve imports/exports for the current module. HTML Module Record's version will be similar, but will change the definition of step 9 (“For each string required that is an element of module.[[RequestedModules]]...) in order to follow the structure of the newly defined ScriptEntry record as defined above. - 9\. For each ScriptEntry *se* in *module*.[[RequestedModules]]) - a\. Let *requiredModule* be null. - - b\. If *se*.[[IsInline]]) == true - - i\. Let *requiredModule* be (*se*.[[ModuleRecord]]). + - b\. If *se*.[[InlineModuleRecord]]) != null + - i\. Let *requiredModule* be (*se*.[[InlineModuleRecord]]). - c\. Else - - i\. Let *requiredModule* be HostResolveImportedModule(*module*, *se*.[[SourceName]]) + - i\. Let *requiredModule* be HostResolveImportedModule(*module*, *se*.[[ExternalScriptURL]]) - d\. Set *index* to ? InnerModuleInstantiation(*requiredModule*, *stack*, *index*). - e\. Assert: *requiredModule*.[[Status]] is either "instantiating", "instantiated", or "evaluated". - f\. Assert: *requiredModule*.[[Status]] is "instantiating" if and only if *requiredModule* is in *stack*. - g\. If *requiredModule*.[[Status]] is "instantiating", then - - i\. Assert: *requiredModule* is a Source Text Module Record. - - ii\. Set *module*.[[DFSAncestorIndex]] to min(*module*.[[DFSAncestorIndex]], *requiredModule*.[[DFSAncestorIndex]]). -- HTML Module record should implement a modified version of [ModuleDeclarationEnvironmentSetup](https://tc39.github.io/ecma262/#sec-moduledeclarationenvironmentsetup). Called from InnerModuleInstantiation, this function sets up the lexical environment and resolves imports/exports for the current module. For HTML Modules, there is no distinct lexical environment so we may just redefine this to be a no-op, or further modify the HTML Module version of InnerModuleInstantiation to skip it altogether. Note that the ‘export * from all inline script elements’ stuff is performed in our redefined [ResolveExport](https://tc39.github.io/ecma262/#sec-resolveexport) below. + - i\. Set *module*.[[DFSAncestorIndex]] to min(*module*.[[DFSAncestorIndex]], *requiredModule*.[[DFSAncestorIndex]]). +- HTML Module Record provides a concrete implementation of InitializeEnvironment(), implementing the corresponding abstract method on Cyclic Module Record. This function is responsible for creating a mutable binding with the name "\*default\*" that will be used to set up the HTML Module's document as the module's default export. + 1. Let _module_ be this HTML Module Record. + 1. Let _realm_ be _module_.[[Realm]]. + 1. Assert: _realm_ is not *undefined*. + 1. Let _env_ be NewModuleEnvironment(_realm_.[[GlobalEnv]]). + 1. Set _module_.[[Environment]] to _env_. + 1. Let _envRec_ be _env_'s EnvironmentRecord. + 1. Perform ! _envRec_.CreateMutableBinding("\*default\*", *false*). + 1. Call _envRec_.InitializeBinding("\*default\*", *undefined*). + 1. Return NormalCompletion(empty). +- Declare an implementation-defined abstract operation HostGetDefaultExport(_module_) whose purpose is to return the value of *module*'s default export, or null if there isn't one. + - Note: See HTML5 spec changes below for the implementation. +- HTML Module Record provides a concrete implementation of ExecuteModule(), implementing the corresponding abstract method on Cyclic Module Record. For HTML modules there is no script to execute. This method just sets up the HTML Module's default export, obtained from the implementation-defined HostGetDefaultExport. + 1. _module_ be this HTML Module Record. + 1. Let _defaultExport_ be HostGetDefaultExport(_module_). + 1. Let _envRec_ be _module_.[[Environment]]'s EnvironmentRecord. + 1. Call _envRec_.SetMutableBinding("\*default\*", _defaultExport_, *false*). + 1. Return NormalCompletion(empty). - HTML Module Record should implement a modified version of [GetExportedNames](https://tc39.github.io/ecma262/#sec-getexportednames)(*exportStarSet*), as follows: - - 1\. Let *module* be this Source Text Module Record. + - 1\. Let *module* be this HTML Module Record. - 2\. If *exportStarSet* contains *module*, then - a\. Assert: We've reached the starting point of an import * circularity. - b\. Return a new empty List. - 3\. Append *module* to *exportStarSet*. - 4\. Let *exportedNames* be a new empty List. - 5\. For each ScriptEntry *se* in *module*.[[RequestedModules]]), do: - - a\. If *se*.[[IsInline]] == true: - - i\. Let *starNames* be *se*.[[ModuleRecord]].GetExportedNames(*exportStarSet*). + - a\. If *se*.[[InlineModuleRecord]] != nullInlineModuleRecord: + - i\. Let *starNames* be *se*.[[InlineModuleRecord]].GetExportedNames(*exportStarSet*). - ii\. For each element *n* of *starNames*, do - a\. If SameValue(*n*, "default") is false, then - i\. If *n* is not an element of *exportedNames*, then @@ -52,65 +71,90 @@ Questions/corrections/feedback are welcome! I've left TODOs in several places w - i\. Assert: This is a circular import request. - ii\. Return null. - 3\. Append the Record { [[Module]]: *module*, [[ExportName]]: *exportName* } to resolveSet. - - 4\. If SameValue(*exportName*, "default") is true, then - - a\. Return the HTML Document associated with this HTML Module record. TODO How do we actually return this as a resolved binding if the HTML record has no lexical scope? Maybe the record needs to have one? - - b\. NOTE: I assume here that we’re not trying to pass through default exports of the inline scripts. - - 5\. Let *starResolution* be null. - - 6\. For each ScriptEntry record *se* in *module*.[[RequestedModules]], do: - - a\. If *se*.[[IsInline]] == false, continue to next record. - - b\. Let *importedModule* be *se*.[[ModuleRecord]]). - - c\. Let *resolution* be ? importedModule.ResolveExport(*exportName*, *resolveSet*). - - d\. If *resolution* is "**ambiguous**", return "**ambiguous**". - - e\. If *resolution* is not null, then - - i\. Assert: *resolution* is a ResolvedBinding Record. - - ii\. If *starResolution* is null, set *starResolution* to *resolution*. + - 4\. Let *resolution* be null. + - 5\. For each ScriptEntry record *se* in *module*.[[RequestedModules]], do: + - a\. If *se*.[[ExternalScriptURL]] != null, continue to next record. + - b\. Let *importedModule* be *se*.[[InlineModuleRecord]]). + - c\. Let *singleResolution* be ? *importedModule*.ResolveExport(*exportName*, *resolveSet*). + - d\. If *singleResolution* is "**ambiguous**", return "**ambiguous**". + - e\. If *singleResolution* is not null, then + - i\. Assert: *singleResolution* is a ResolvedBinding Record. + - ii\. If *resolution* is null, set *resolution* to *singleResolution*. - iii\. Else, - a\. Assert: There is more than one inline script that exports the requested name. - b\. Return "**ambiguous**". - - 7\. Return *starResolution*. -- We need to redefine how the HTML Module is created and its fields are populated. Instead of populating the fields in [ParseModule](https://tc39.github.io/ecma262/#sec-parsemodule), we will create the module based on input from the result of parsing the HTML Document on the HTML5 side. See HTML5 spec proposed changes below. -- HTML Module Record should implement a modified version of [InnerModuleEvaluation](https://html.spec.whatwg.org/#fetch-the-descendants-of-a-module-script)(*module*, *stack*, *index*). This method calls InnerModuleEvaluation on each child module, then executes the current module. The HTML Module version will have the following changes: - - Change step 10 and step 10a to be the following, to account for the different structure of HTML Module Record vs Source Text Module Record. Steps 10b-10f remain the same: + - 6\. If *resolution* is null and SameValue(*exportName*, "default") is true, then + - a\. Let *resolution* be a ResolvedBinding Record { [[Module]]: *module*, [[BindingName]]: *\*default\** } + - b\. NOTE 1: *\*default\** was set up to reference the HTML Module's document during instantiation/execution + - c\. NOTE 2: I assume here that we’re not trying to pass through default exports of the inline scripts. + - 7\. Return *resolution*. +- HTML Module Record inherits the concrete Evaluate() method from Cyclic Module Record. +- HTML Module Record should implement a modified version of Cyclic Module Record's [InnerModuleEvaluation](https://html.spec.whatwg.org/#fetch-the-descendants-of-a-module-script)(*module*, *stack*, *index*). This method calls InnerModuleEvaluation on each child module, then executes the current module. The HTML Module version will have the following changes (TODO Make sure these step numbers are still correct after Cyclic Module Record is merged into the official spec): + - Change step 10 and step 10a to be the following, to account for the different structure of HTML Module Record vs Cyclic Module Record. Steps 10b-10f remain the same: - 10\. For each ScriptEntry *se* in *module*.[[RequestedModules]]), do: - - a\. If (*se*.[[IsInline]] == true), let *requiredModule* be *se*.[[ModuleRecord]], else let *requiredModule* be HostResolveImportedModule(*module*, *se*.[[SourceName]]) - - Omit step 11 (since the HTML module doesn’t have any JS code of its own to run; it only recurses to run the code of its requested modules per step 10). - + - a\. If (*se*.[[InlineModuleRecord]] != null), let *requiredModule* be *se*.[[InlineModuleRecord]], else let *requiredModule* be HostResolveImportedModule(*module*, *se*.[[ExternalScriptURL]]) +- Note that we don't define any operation in ES for creation of HTML Module Records. This is implemented entirely in HTML5 (see spec changes below). + # HTML5 spec changes ([full spec link](https://html.spec.whatwg.org/)): -- Broadly speaking, the language throughout most of the module fetching algos ([[1](https://html.spec.whatwg.org/#fetch-a-module-script-graph)], [[2](https://html.spec.whatwg.org/#internal-module-script-graph-fetching-procedure)], [[3](https://html.spec.whatwg.org/#fetch-a-single-module-script)], [[4](https://html.spec.whatwg.org/#fetch-the-descendants-of-a-module-script)], [[5](https://html.spec.whatwg.org/#fetch-the-descendants-of-and-instantiate-a-module-script)], [[6](https://html.spec.whatwg.org/#finding-the-first-parse-error)]) needs to be generalized from “module script” to “module”, e.g. s/fetch a single module script/fetch a single module/. +- Introduce a third type of [script](https://html.spec.whatwg.org/multipage/webappapis.html#concept-script) named HTML Module Script. It has the following item in addition to script: + - A `document`: The [Document](https://html.spec.whatwg.org/multipage/dom.html#document) for the HTML Module, or null. +- Rename the existing concept of [module script](https://html.spec.whatwg.org/multipage/webappapis.html#module-script) to JavaScript module script. +- Redefine "module script" as a union of JavaScript module script or HTML module script. + - Broadly speaking, the usage of "module script" in most of the module fetching algos ([[1](https://html.spec.whatwg.org/#fetch-a-module-script-graph)], [[2](https://html.spec.whatwg.org/#internal-module-script-graph-fetching-procedure)], [[3](https://html.spec.whatwg.org/#fetch-a-single-module-script)], [[4](https://html.spec.whatwg.org/#fetch-the-descendants-of-a-module-script)], [[5](https://html.spec.whatwg.org/#fetch-the-descendants-of-and-instantiate-a-module-script)], [[6](https://html.spec.whatwg.org/#finding-the-first-parse-error)]) will refer to this new definition of module script, as they will be generalized to both HTML and JavaScript modules. - In [prepare a script](https://html.spec.whatwg.org/#prepare-a-script), when defining a script’s type in step 7, always set it to “module” if we’re parsing an HTML Module. TODO How to determine that? New parser flag? -- Introduce a “create an HTML Module”, similar to [create a module script](https://html.spec.whatwg.org/#fetching-scripts:creating-a-module-script). The definition would look something like this: - - To create an HTML module, given a JavaScript string *source*, an environment settings object *settings*, a URL *baseURL*, and some script fetch options *options*: - - 1\. Let *htmlModule* be a new HTML Module that this algorithm will subsequently initialize. - - 2\. Set *htmlModule’s* settings object to *settings*. - - 3\. Set *htmlModule’s* *base URL* to *baseURL*. - - 4\. Set *htmlModule’s* fetch options to *options*. - - 5\. Set *htmlModule's* *parse error* and *error to rethrow* to null. - - 6\. Let *result* be ParseHTMLModule(*source*, *settings's* Realm, *htmlModule*). Note: Passing *htmlModule* as the last parameter here ensures *result*.[[HostDefined]] will be *htmlModule*. See below bullet for ParseHTMLModule definition. - - 7\. For each ScriptEntry *required* of *result*.[[RequestedModules]], such that *required*[[IsInline]] == false: - - a\. Let *url* be the result of resolving a module specifier given *htmlModule’s* *base URL* and *required*[[SourceName]]. +- Rename [create a module script](https://html.spec.whatwg.org/#fetching-scripts:creating-a-module-script) to "create a JavaScript module script". +- Introduce a new algorithm “create an HTML module script”, similar to [create a module script](https://html.spec.whatwg.org/#fetching-scripts:creating-a-module-script). The definition would look something like this: + - To create an HTML module script, given a JavaScript string *source*, an environment settings object *settings*, a URL *baseURL*, and some script fetch options *options*: + - 1\. Let *htmlModuleScript* be a new HTML module script that this algorithm will subsequently initialize. + - 2\. Set *htmlModuleScript*'s settings object to *settings*. + - 3\. Set *htmlModuleScript*'s *base URL* to *baseURL*. + - 4\. Set *htmlModuleScript*'s fetch options to *options*. + - 5\. Set *htmlModuleScript*'s *parse error* and *error to rethrow* to null. + - 6\. Let *result* be ParseHTMLModule(*source*, *settings's* Realm, *htmlModuleScript*). Note: Passing *htmlModuleScript* as the last parameter here ensures *result*.[[HostDefined]] will be *htmlModuleScript*. See below bullet for ParseHTMLModule definition. + - 7\. For each ScriptEntry *required* of *result*.[[RequestedModules]], such that *required*[[InlineModuleRecord]] == null: + - a\. Let *url* be the result of resolving a module specifier given *htmlModuleScript*'s *base URL* and *required*[[ExternalScriptURL]]. - b\. If *url* is failure, then: - i\. Let *error* be a new TypeError exception. - - ii\. Set *htmlModule’s* *parse error* to *error*. - - iii\. Return *htmlModule*. Note: This step is essentially validating all of the requested module specifiers. We treat a module with unresolvable module specifiers the same as one that cannot be parsed; in both cases, a syntactic issue makes it impossible to ever contemplate instantiating the module later. - - 8\. Set *htmlModule’s* *record* to *result*. - - 9\. Return *htmlModule*. -- Define ParseHTMLModule(*source*, *realm*, *htmlModule*) as roughly the following. TODO Given that we define HTML Module Record in ES6, should this function be defined over there? We’re using the HTML5 parser though... - - 1\. Let *record* be a new HTML Module record that this algorithm will subsequently initialize. - - 2\. Run the HTML5 parser on source to obtain the result *document*. - - 3\. Set *record*.[[HostDefined]] = *htmlModule*. - - 4\. For each HTMLScriptElement *script* in *document*: - - a\. Let *se* be a new ScriptEntry record (see definition in ES6 changes above). + - ii\. Set *htmlModuleScript*'s *parse error* to *error*. + - iii\. Return *htmlModuleScript*. Note: This step is essentially validating all of the requested module specifiers. We treat a module with unresolvable module specifiers the same as one that cannot be parsed; in both cases, a syntactic issue makes it impossible to ever contemplate instantiating the module later. + - 8\. Set *htmlModuleScript*'s *record* to *result*. + - 9\. Return *htmlModuleScript*. +- Introduce a new algorithm ParseHTMLModule(*source*, *realm*, *htmlModuleScript*) as the following. + - 1\. Run the HTML5 parser on *source* to obtain the result *document*. + - a\. TODO: This needs to be fleshed out more. Do we need to run the parser in a special mode to ensure that nothing is fetched and no script runs? Script execution should already be [disabled because the HTML Module document does not have a browsing context](https://html.spec.whatwg.org/#concept-n-noscript), but the case for fetching is less clear. We also need to specify the special handling for non-module `