Circular reference handling

[guybedford]

In a cycle like the following:

a.js

import { y } from './b.js';
await new Promise(resolve => setTimeout(resolve, 1000));
export let x = 42;

b.js

import { x } from './a.js';
await new Promise(resolve, setTimeout(resolve, 2000));
export let y = x + 1;

If I import a.js, b.js should execute first.

On the other hand, if I import b.js, a.js should execute first.

But what happens if I do both one after the other - import('a.js'), import('b.js').

1. Should the import of a.js "own" the execution, so that the import of a.js creates promises for the execution of a.js and b.js, and the import of b.js waits on that. This way a.js is not executed until b.js has finished executing ever.
2. Both graph executions run simultaneously: the load for a.js executes b.js, and the load for b.js executes a.js so they both execute immediately in parallel, and both resolve when both modules have finished executing.
3. What is specified in Variant B promise refactoring #33 is a third case: the load for a.js executes b.js, and then the load for b.js resolves as soon as b.js has finished executing, but before a.js has finished executing.

[bergus]

How about option 4: Throw an exception when detecting a cycle containing a module with TLA?

I can't think of any good use case where dependencies are expected to race with each other. Someone will eventually get bitten hard in either case. Disallowing it completely might be the best idea. If use cases come up, we can still add support for it later without breaking any working code.
Does anyone have a link ready about the motivation for allowing circular dependencies? The original discussion might help

[bergus]

I don't like option 3. import('b.js') should not fulfil until all of bs dependencies have been loaded and executed. (Or at least, unless the import('b.js') expression is inside one of the modules that form the cycle).

[bergus]

Some food for thought:

• How does import() handle this in case there is no TLA in the circularly dependent modules, or in case there is no circular dependency? Does the module execution race on whose code is fetched first, or is the order governed by the order of the import calls?
• In a dependency tree without circular dependencies, we can replace every import * as … from … with const … = await import(…); and no behaviour would change, right (maybe apart from the concurrency of resource fetching)? Would it be consistent to guarantee this equivalence also for a dependency graph with cycles?

[littledan]

I was expecting 3. , but if 4 is acceptable, I'd be happy to go with that option as well. I don't understand the intuition for 1 or 2.

Any other thoughts, on either this question or whether we could throw in other kinds of circularity cases? cc @GeorgNeis @MylesBorins @domenic

[zenparsing]

It would be surprising if the addition of a top-level-await somewhere changed import(something) from returning a promise for a completely evaluated namespace object, into a promise for a partially evaluated namespace object.

The current (synchronous) specification is carefully designed to support cycles, and I don't think that the addition of TLA should compromise on that design. None of the 4 options presented here seem to work.

Would the following work?

• Each module has an associated [[EvaluationPromise]].
• The core walking algorithm is basically the same, where all modules in a strongly-connected-component transition to "evaluated" at the same time, and all of the [[EvaluationPromise]]s for modules in a strongly-connected-component are resolved at the same time.

It's difficult for me to tell whether #33 moves in this direction or not.

[guybedford]

@zenparsing in this model is [[EvaluationPromise]] a promise for the module's own execution or a promise for the execution of the module and all its dependencies?

[zenparsing]

@guybedford Not exactly either; it's a promise for when the module transitions from "evaluating" to "evaluated". In the same way that all modules in an SCC transition to "evaluated" together, all modules in an SCC would have their [[EvaluationPromise]] resolved together.

[guybedford]

@zenparsing right, but the question then does one top-level evaluation job "own" that evaluation promise transition? Or is it just the first top-level evaluation job that does that transition?

[zenparsing]

I'm confused about the "own" terminology, so I'm not sure how to answer. I think the best way to talk about this is by using the Tarjan's algorithm/SCC terminology. The evaluation promise would be resolved by that algorithm (when the module is transitioned from "evaluating" to "evaluated"), not by any job.

[guybedford]

So right now the spec will transition the components together, but the complication with top-level wait is that this algorithm is now being changed from "sync" to "async", where new top-level jobs can be started during the async resolution. So we need to be handling the fact that there are potentially multiple runs of this algorithm happening simultaneously, where they may have overlapping modules.

[zenparsing]

Yes, part of the design validation should be proving that the "asyncified" version of the SCC algorithm still works in those overlapping cases.

[littledan]

I'm not sure if this algorithm is as "async-ified" as it may sound. In #33, I don't believe any module is left in the "evaluating" state when we get to the next turn. We are just left with some Promises for "evaluated" modules which might not be resolved yet (and might never). What do you see as remaining to be proven?

[zenparsing]

If the graph walk is not "asyncified" then I don't understand how this is supposed to guarantee that dependencies are fully evaluated prior to evaluation. Perhaps I just don't understand the proposal and need to see the algorithm in full (instead of a PR).

Also, I think we should be more conservative and try variant A first.

[littledan]

My understanding of the "B" algorithm is that the SCC component search and cycle breaking happens synchronously, while actually launching the module body will block on the dependencies' promises resolving.

Seems like we need to improve documentation here to explain the algorithms more clearly. I am looking into this. Also happy to discuss at a future TC39 meeting.

What do you mean "try first"? I think we have to make a decision on one or the other (or decide against this proposal). Anyway, I don't think A vs B affects the async-ness of the module evaluation graph walk.