Make AOT packages loadable as ES modules

[jasongin]

Currently an AOT module produced by node-api-dotnet must be loaded via CommonJS require(). And there is an example project that shows how to wrap an AOT module in an npm package, where that package also must be loaded via require().

Can we make the AOT module package loadable via ES import instead? Or a package that supports both? (This might require a small bootstrapper script.)

See also the comment from @ceztko that brought up this issue.

[ceztko]

It seems conditional exports are exactly what makes possible to support both conventions in a package. Together with the use of process.dlopen(module, exports) as described in my comment, you can make an ES Module compliant loader.

[ceztko]

We also noticed that AOT built packages are not truly standalone when operated from TypeScript: type definitions (in the usual example aot-npm-package.d.ts) wants to augment the node-api-dotnet module, which is in fact defined elsewhere by installing node-api-dotnet package. The latter carries runtime dependencies to run .NET CLR runtime, which shouldn't be required by definition, and has no useful type definitions for the AOT build packages. You can verify the issue by trying to import the AOT built aot-npm-package in a .ts compiled file, without having installed the node-api-dotnet package. We believe that the issue would be solved if the type definitions are generated with modern ES Module conventions (no manual module declaration, and static methods defined as such in a class, not inserted in a TS namespace). This is what we do in our own custom TS/Node.js interop solution, where for example this ES Module exports definition produces the attached SampleLibrary.d.ts.txt type definition. This definition can be safely used when compiling TS that consumes the AOT built aot-npm-package with either CommonJS or ES Module targets, so we believe making exactly this way would help both fixing this issue and removing the implicit dependency from node-api-dotnet package.

[jasongin]

@ceztko I'll take a look at that issue. Definitely it should not try to load the CLR when you are using an AOT module.

But the use of TS namespaces for the type definitions is pretty fundamental I think. When using .NET APIs, developers expect types from all loaded assemblies to be merged into a combined namespace hierarchy. It would not be a good experience if it was necessary to qualify .NET types by their assembly name (corresponding to the TS module that exported the type). It's often not obvious exactly which assembly contains every type, and that may even change across major versions.

[ceztko]

But the use of TS namespaces for the type definitions is pretty fundamental I think.

As long as that gets transpiled to valid ES Module you can use whatever you want. But in my experience namespace , module are really legacy construct and should not be used anymore and Microsoft documentation is very explicit about it:

It is also worth noting that, for Node.js applications, modules are the default and we recommended modules over namespaces in modern code.

I was trying to make some code that would safely compile both to CommonJS/ES Module, and I was walking in a valley of death and despair until I stopped trying to use namespace and module keywords all together, so take my words as a warning.

About the other recommendation for using conditional exports for having the AOT package loadable as ES Module: these typically works when the moduleResolution parameter compiling the code consuming the AOT built package is set to node16 or nodenext, which in most cases is desirable (some times it is not). In our TS/NodeJS solution we are using the following configuration (I'm continue to refer to names/locations of the aot example so it's more clear):

{
  ...
  // As a fallback, the "main" just point to the CommonJS entry, 
  "main": "./bin/aot-npm-package.js",
  "exports": {
    "import": "./bin/aot-npm-package.mjs",
    "require": "./bin/aot-npm-package.js"
  },
  ...
}

It works well both when compiling with the legacy node module resolution or auto detect everything with node16. With the latter, it picks the right the right directive even when using import but unconditionally compiling to CommonJS, as in this case import really gets transpiled to require().

It took us a lot of research to discover all this stuff: I hope it will be useful for you to improve node-api-dotnet, as we plan to both use your solution and our own as well (which is able to produce wrappers for other languages/platforms, not only TS/Node.js).

[jasongin]

I am aware of the documented guidance regarding TS namespaces. I considered several approaches for .NET type definitions. Namespaces was the only mechanism I could find that allows for a reasonably intuitive projection of .NET types into TypeScript. Any other construct that I tried required what I considered unacceptable API usability compromises. I agree that the combination of TS namespaces, modules, and declaration merging can be difficult to work with as a library author. But our intention is that this project should solve those challenges such that TS consumers of the .NET libraries don't have to worry about any of that complexity.

[jasongin]

I dug into this more, and realized this is related to a broader problem(#260): the generated .d.ts and .js files are not right for this kind of project. The namespacing and module augmentation are designed for dynamic invocation of ordinary .NET assemblies, but aren't supposed to be used with a module that was built with [JSExport].

After fixing that, it should be straightforward to also generate an ESM-compatible .js (or .mjs), to resolve this issue. An ES module requires explicitly declaring all of the top-level exports, which is not a challenge since the generator has that context anyway. Then there will be a slight variation in the JS code for the AOT case, which the AOT publish process will need to deal with somehow.