Require explicit require() registration? (And maybe local only?)

[aseemk]

(Pun intended in title.)

Node's local module system is great because it allows local dependencies, almost eliminating worries of version conflicts or shared global state.

But one big exception is require() extension registrations, including .coffee -- any code that has require('coffee-script') affects global state for the entire running app, because CoffeeScript automatically registers its extension upon execution:

https://github.com/jashkenas/coffee-script/blob/1.3.1/src/coffee-script.coffee#L15-L21

This can and has led to version conflicts: one module depends on one version of CoffeeScript and require()s it, while another module depends on a different version of CoffeeScript and require()s it.

Unfortunately, there's no way I can think of to fix this problem today for modules that are already published. That ship has sailed.

I'm proposing we fix it going forward. At the very least, I'd like to recommend that registering CoffeeScript's require() extension handler be an explicit operation. E.g.:

// strawman 1:
require('coffee-script').register();
// strawman 2:
require('coffee-script/require');

This would at least prevent code that uses CoffeeScript's compiling API directly (e.g. connect-assets/snockets compiler middleware) from unnecessarily setting or overwriting an extension handler.

And hopefully it, along with documentation on this matter, would also make it more explicit to module authors that this affects global state.

But ideally, we could go further and ensure our require() handler only had a local effect instead of a global one. This isn't trivial to do -- require.extensions is global, and Node's module system is locked -- but I think it might be possible with some engineering.

I'll share some thoughts on how in a comment, but I wanted to get this conversation started and an issue formalized. What do you guys think of the goals, and of explicit registration in particular?

[aseemk]

Cool, after some investigation, I think it might be easier to achieve local require() extension than I originally thought!

The key reason is that the module object has a parent property. When CoffeeScript's handler is registered, it can make a note of the module that's doing the registering. And then whenever the handler is invoked (by Node), it can only process modules that are ultimately coming from that original module.

Demonstrating this with a simple .json handler:

# app.coffee:
require './parent'
require './bar.json'

# parent.coffee:
require('./handler').register(module)
require './foo.json'

# handler.coffee:
exports.register = (caller) ->
    require.extensions['.json'] = (module, filename) ->
        if module.parent is caller
            console.log "handling: #{filename}"
        else
            console.log "ignoring: #{filename}"

$ coffee app.coffee
handling: /path/to/foo.json
ignoring: /path/to/bar.json

Pretty neat! This is a simplified example, but hopefully it demonstrates the technique. Additions that would need to be made:

• The caller could be anywhere up the parent chain, e.g. instead of module.parent, it might be module.parent.parent or so on, so the code should check the chain, not just the first parent.
• As you can see, this requires the caller to pass in their module. Hopefully that's fine. It might work to just take CoffeeScript's own module.parent, but I don't think that would support two sibling (or otherwise separate) modules sharing the same physical CoffeeScript dependency on disk, due to require.cache only executing CoffeeScript once. (Though I have an issue open for npm to potentially change this behavior.)
• To prevent version conflicts, register() could actually maintain a map/dictionary from caller to CoffeeScript instance. That way, a child of the caller always uses the CoffeeScript instance that the caller required().

What do you guys think?

[aseemk]

I'm happy to take a stab at this, but I'm hesitant to spend a bunch of time on that if the idea isn't welcome. Lemme know your guys' thoughts — much appreciated.

[vendethiel]

from #2969

@jashkenas said

Hang on a minute -- aren't we just removing require.extensions, since Node has deprecated it?

also, from IRC

[21:33] Vendethiel1: I'm going to implement this in CSR and release a beta 5
[21:33] great
[21:33] So require('coffee-script-redux') won't auto-register