Failure to resolve modules from yarn workspaces when module has rootDir/outDir

[jeantil]

Bug report

Describe the bug

In a yarn workspace based monorepo (derived from examples/with-yarn-workspace), with a typescript module which uses rootDir and outDir settings in tsconfig.json and path aliases in the shared tsconfig.json config.
Next fails to resolve the module even though the tsc build works fine.

To Reproduce

clone the reproduction repository
checkout 2c7f4bd
check that typescript builds correctly yarn workspace web-app tsc -b -v
run yarn dev
the build fails to resolve the baz module

error - ./pages/index.tsx
Module not found: Can't resolve 'baz' in '/home/jean/dev/jsdev/src/demo3/packages/web-app/pages'
ModuleNotFoundError: Module not found: Error: Can't resolve 'baz' in '/home/jean/dev/jsdev/src/demo3/packages/web-app/pages'
    at /home/jean/dev/jsdev/src/demo3/node_modules/webpack/lib/Compilation.js:925:10
    at /home/jean/dev/jsdev/src/demo3/node_modules/webpack/lib/NormalModuleFactory.js:401:22
    at /home/jean/dev/jsdev/src/demo3/node_modules/webpack/lib/NormalModuleFactory.js:130:21
    at /home/jean/dev/jsdev/src/demo3/node_modules/webpack/lib/NormalModuleFactory.js:224:22
    at /home/jean/dev/jsdev/src/demo3/node_modules/neo-async/async.js:2830:7
    at /home/jean/dev/jsdev/src/demo3/node_modules/neo-async/async.js:6877:13
    at /home/jean/dev/jsdev/src/demo3/node_modules/webpack/lib/NormalModuleFactory.js:214:25
    at /home/jean/dev/jsdev/src/demo3/node_modules/enhanced-resolve/lib/Resolver.js:213:14
    at /home/jean/dev/jsdev/src/demo3/node_modules/enhanced-resolve/lib/Resolver.js:285:5
    at eval (eval at create (/home/jean/dev/jsdev/src/demo3/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:13:1)
    at /home/jean/dev/jsdev/src/demo3/node_modules/enhanced-resolve/lib/UnsafeCachePlugin.js:44:7
    at /home/jean/dev/jsdev/src/demo3/node_modules/enhanced-resolve/lib/Resolver.js:285:5
    at eval (eval at create (/home/jean/dev/jsdev/src/demo3/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:13:1)
    at /home/jean/dev/jsdev/src/demo3/node_modules/enhanced-resolve/lib/Resolver.js:285:5
    at eval (eval at create (/home/jean/dev/jsdev/src/demo3/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:25:1)
    at /home/jean/dev/jsdev/src/demo3/node_modules/enhanced-resolve/lib/DescriptionFilePlugin.js:67:43

Expected behavior

the build completes and the index page displays all three strings foo bar baz

System information

• OS: Linux
• Browser (if applies) not relevant but the tests were done in firefox
• Version of Next.js: next@latest (resolved to 9.4.2 see yarn.lock)
• Version of Node.js: v12.16.3

Additional context

Trying to apply next-transpile-modules to baz didn't help either.
This does not use project references, only path aliases.
Removing the main entry in the baz module's package.json doesn't change the issue

[jeantil]

TLDR;

Use tsconfig-paths-webpack-plugin (see repro repository for the full config file )

    config.resolve.plugins = [
      ...config.resolve.plugins,
      new TsconfigPathsPlugin(),
    ];

The full rambling story

Since this was considered a good first issue I figured I might be able to figure it out

The only way I found to dump the actual effective webpack config and stat output was to plug console.log calls directly in packages/next/build/compiler.ts not sure if there was something more appropriate.

I was able to make the project compile by changing the webpack config but it opens almost more questions than it solves.
Assuming <rootDir> is the root of the monorepo (where the .git folder lives).
Webpack reports that it is running in <rootDir>/packages/web-app which makes sense.

Resolving the baz alias using a path relative to the current directory doesn't work so ../baz/src which would translate to <rootDir>/packages/web-app/../baz/src doesn't resolve the module.

However baz/src does resolve to <rootDir>/node_modules/baz/src probably thanks to
webpack.resolve.modules which contains [node_modules]. (nevermind the fact that <rootDir>/packages/web-app/node_modules is empty. Webpack somehow manages to find the <rootDir>/node_module, I haven't been able to figure out how it does that.

So the imports of baz now resolve to <rootDir>/node_modules/baz/src/index.ts which gets transpiled to javascript without any additional configuration. just like <rootDir>/node_modules/foo/index.ts.
Since duplicating all the mappings with all the right relative paths is fairly tedious, there is a webpack plugin for this which worked right away.

The reproduction repository master has been updated accordingly at jeantil/next-9-ts-aliases-workspaces@ad0b342

Oh my tsx

Interestingly, the bar directory which contains index.tsx won't compile without enabling the next-transpile-modules even thought the .tsx files in <rootDir>/packages/web-app/pages compile just fine and the .ts files in the workspace packages also compile just fine without it.

Adding"include": ["src/**/*.ts", "src/**/*.tsx"] to a tsconfig file in bar didn't change anything

I thought for a moment the tsx file was excluded by return /node_modules/.test(path) at https://github.com/zeit/next.js/blob/canary/packages/next/build/webpack-config.ts#L766 since with the plugin webpack reported a resolution in node_modules so I reverted the tsconfig-path-webpack-plugin for a minute and added the following and the

 config.resolve.modules = [
      path.join(__dirname, ".."),
      ...config.resolve.modules,
    ];
config.resolve.alias = {
      ...config.resolve.alias,
      baz: "baz/src",
    };

to the webpack config.

The foo, bar and baz modules are now resolved to ../foo,../bar and ../baz instead of <rootDir>/node_module/{foo,bar,baz} but bar still fails to compile.

../bar/index.tsx 1:18
Module parse failed: Unexpected token (1:18)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
> const Bar = () => <strong>bar</strong>
| 
| export default Bar

Dumping all the paths handled by the exclude rule on next-babel-loader is also confusing since none of foo, bar or baz files seem to be handled by it, or at least not directly (I'll pass on how unhelpful the webpack error message seems to be :) )

My next best guess was that there is some kind of config in babel loader that knows how to handle .ts files but not .tsx

I tried a custom babel.config.json[1] with the following content

  "presets": [
    [
      "next/babel",
      {
        "preset-typescript": { 
            "isTsx": true , 
            "allExtensions": true,
        }
      }
    ]
  ]
}

Since I wasn't sure the config was being applied I also checked the next babel preset and added both options right in there but that still failed.

That's as far as I was able to go, I guess i'll let the next-transpile-modules be for a while longer.
Hopefully people who come across a similar issue later will find this issue, my explorations and be able to understand and propose a fix for that last wart.

[1] I'm not sure why the documentation recommends .babelrc over babel.config.json when the babel documentation recommends the latter (but I admit that the use case section at the top doesn't make much sense to me)

[jeantil]

next-transpile-modules counter-attacks

In the previous episode, resolving the typescript module baz proved difficult but the resistance the developper was able to successfully blow up the death star successfully build the project by fixing relative paths or using the force a webpack typescript plugin.

In this new episode the developer has turned a bit more to the dreaded dark side of the force typescript and used a forbidden feature: enum this leads to a new compile error which seems to demonstrate that the initial fix was not enough and that the corrupt republic framework was lying when it said the project compiled fine, that was only because the typescript code was processable as js code and not using actual typescript features.

TLRD;

Using next-transpile-modules works and the build completes

The longer read

In an interesting plot twist:

• putting the enum in a ts file in packages/web-app/components or in packages/web-app/pages and using a relative import to that file in pages/index.tsxdoes compile fine
• putting the enum in a ts file in packages/web-app/components or in packages/web-app/pages and using a relative import to that file in pages/index.tsxdoes compile fine
• putting the enum in a ts file in packages/baz/src/index.ts and using a relative file import in pages/index.tsx to ../../baz/src/index does not compile
• putting the enum in packages/baz/src/index.ts and using an import alias to module baz does not compile
  So it looks like something, somewhere is configured to only transpile files in or below the directory where next.config.js resides.
  Any import even absolute outside of that directory will not be transpiled.

digging further into the default build config, it looks like the baseUrl resolution mecanism, doesn't honor typescript's extends directive. Therefore if the baseUrl and paths have been defined in an extended base configuration file it will not be detected by next.

a workaround is to manually recreate a next-babel-loader which includes the baseUrl and duplicates the existing next-babel-loader, this works but is quite brittle. updates to next base configuration would have to be actively replicated. identifying the existing rule to inject the baseUrl is not trivial either as rules seem to be highly polymorphic which makes locating the loader name pretty hard.

In any case it looks like not including resolvedBaseUrl in https://github.com/zeit/next.js/blob/canary/packages/next/build/webpack-config.ts#L761 is a mistake but it wouldn't be enough to fix the specific issue demonstrated in the repo.