Watchman doesn't work with react-native anymore

[mlabrum]

Hey,

I've grabbed the latest watchman from https://ci.appveyor.com/api/projects/wez/watchman/artifacts/watchman.zip?branch=master&job=Environment:+WATCHMAN_WIN7_COMPAT%3D

React-native version: 0.44.2 (latest)
Windows 10

Running react-native with react-native start

Looking for JS files in
   C:\tmp\test\test


React packager ready.

Loading dependency graph, done.
error: bundling: Error
    at DependencyGraph._getAbsolutePath (C:/tmp/test/test/node_modules/react-native/packager/src/node-haste/index.js:272:11)
    at DependencyGraph.getDependencies (C:/tmp/test/test/node_modules/react-native/packager/src/node-haste/index.js:216:26)
    at Resolver.getDependencies (C:/tmp/test/test/node_modules/react-native/packager/src/Resolver/index.js:106:27)
    at _resolverPromise.then.resolver (C:/tmp/test/test/node_modules/react-native/packager/src/Bundler/index.js:561:62)
    at process._tickCallback (internal/process/next_tick.js:109:7)
Bundling `index.android.js`  0.0% (0/1), failed.

React native on device gives the following error
Cannot find entry file index.android.js in any of the roots ["c:/tmp/test/test/"]

Watchman logs

2017-06-05T09:27:39,625: [listener] Watchman 4.9.0 <no build info set> starting up on NTCSRF206
2017-06-05T09:27:39,632: [listener] failed to parse json from C:\Users\Matt\AppData\Local\Temp/Matt-state/state: unable to open C:\Users\Matt\AppData\Local\Temp/Matt-state/state: No such file or directory
2017-06-05T09:27:39,632: [listener] waiting for pipe clients on \\.\pipe\watchman-Matt
2017-06-05T09:27:39,656: [client=000001D8D9454810:stm=000001D8D9452770:pid=0] path C:/tmp/test/test is on filesystem type NTFS
2017-06-05T09:27:39,658: [client=000001D8D9454810:stm=000001D8D9452770:pid=0] root C:/tmp/test/test using watcher mechanism win32 (auto was requested)
2017-06-05T09:27:41,398: [client=000001D8D9454810:stm=000001D8D9452770:pid=0] PERF: {"start_time": 1496620659.6559999, "version": "4.9.0", "description": "dispatch_command:watch-project", "meta": {"root": {"path": "C:/tmp/test/test", "watcher": "win32", "ticks": 1, "recrawl_count": 0, "case_sensitive": false, "number": 1}, "args": ["watch-project", "C:/tmp/test/test"], "client": {"pid": 0}}, "pid": 12892, "elapsed_time": 1.742}
2017-06-05T09:27:41,398: [io 000001D8D946DC10 C:/tmp/test/test] crawl complete
2017-06-05T09:27:41,398: [io 000001D8D946DC10 C:/tmp/test/test] PERF: {"start_time": 1496620659.6689999, "version": "4.9.0", "description": "full-crawl", "meta": {"root": {"path": "C:/tmp/test/test", "watcher": "win32", "ticks": 2, "recrawl_count": 0, "case_sensitive": false, "number": 1}}, "pid": 12892, "elapsed_time": 1.7290000000000001}

I can confirm watchman 4.7.0 worked (though it had the symlink issue on windows)

Thanks,
I'm currently digging through RN code trying to see if I can find anything...

[wez]

Can you try running:

watchman --server-encoding=json --persistent log-level debug > debug.log

in another window while you try running react native, and share the debug.log file with me? That should show us more detail about what watchman is doing.

[mlabrum]

Hey,
I've done some hacking and I think I've found the issue and it seems to be due to path separators

I've modified node_modules\jest-haste-map\build\crawlers\watchman.js:110 and added
console.log(name)

Watchman 4.9.0
Gives

C:/tmp/test/test\node_modules/qs/bower.json
C:/tmp/test/test\node_modules/random-bytes/package.json
C:/tmp/test/test\node_modules/randomatic/package.json
C:/tmp/test/test\node_modules/range-parser/package.json
C:/tmp/test/test\node_modules/raw-body/package.json
C:/tmp/test/test\node_modules/react/package.json
C:/tmp/test/test\node_modules/react-clone-referenced-element/package.json
C:/tmp/test/test\node_modules/react-deep-force-update/package.json
C:/tmp/test/test\node_modules/react-devtools-core/package.json
C:/tmp/test/test\node_modules/react-native/package.json
C:/tmp/test/test\node_modules/react-native/jest-preset.json
C:/tmp/test/test\node_modules/react-proxy/package.json

Watchman 4.7.0
Gives

C:\tmp\test\test\node_modules\sprintf-js\demo\angular.html
C:\tmp\test\test\node_modules\serve-index\public\directory.html
C:\tmp\test\test\node_modules\rebound\browser_test\index.html
C:\tmp\test\test\node_modules\rebound\docs\rebound.html
C:\tmp\test\test\node_modules\rebound\examples\index.html
C:\tmp\test\test\node_modules\errorhandler\public\error.html
C:\tmp\test\test\node_modules\clone\test.html
C:\tmp\test\test\node_modules\clone\test-apart-ctx.html
C:\tmp\test\test\node_modules\jsbn\example.html
C:\tmp\test\test\node_modules\rebound\index.html
C:\tmp\test\test\node_modules\sax\LICENSE-W3C.html
C:\tmp\test\test\node_modules\through2\LICENSE.html

So it seems newer versions of watchman are returning different file paths?

(I can still attach that debug log if required)

[wez]

if you hack that javascript file to replace / with \\ does it make react native happy again?

[wez]

... I'm not really a javascript guy; presumably something like name = name.replace('/', '\\'); would do the job

[mlabrum]

Yes it does work

[wez]

For the sake of clarifying what the problem is, can you also try normalizing it the other way?

name = name.replace('\\', '/');

[wez]

also: how did you track it down to this? Can you share the error you found and where you found it?

[wez]

FWIW, f257233 is where this behavior was changed

[wez]

The reason I'm asking the above is to assess whether we need to go back to the old behavior to minimize headaches, or whether it is just that react-native's code needs to be fixed up.

[mlabrum]

replacing it all with / doesn't work.

I tracked this down by doing the following....

Running: react-native start

Looking for JS files in
   C:\tmp\test\test


React packager ready.

Loading dependency graph, done.
C:\tmp\test\test\index.android.js
(node:3152) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 2): NotFoundError: Cannot find entry file index.android.js in any of the roots: ["C:\\tmp\\test\\test"]
(node:3152) DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
Bundling `index.android.js`  ░░░░░░░░░░░░░░░░  0.0% (0/1)
error: bundling: Error
    at DependencyGraph._getAbsolutePath (C:/tmp/test/test/node_modules/react-native/packager/src/node-haste/index.js:276:11)
    at DependencyGraph.getDependencies (C:/tmp/test/test/node_modules/react-native/packager/src/node-haste/index.js:216:26)
    at Resolver.getDependencies (C:/tmp/test/test/node_modules/react-native/packager/src/Resolver/index.js:106:27)
    at _resolverPromise.then.resolver (C:/tmp/test/test/node_modules/react-native/packager/src/Bundler/index.js:561:62)
    at process._tickCallback (internal/process/next_tick.js:109:7)
Bundling `index.android.js`  0.0% (0/1), failed.

So I jumped to the first file in the exception
C:/tmp/test/test/node_modules/react-native/packager/src/node-haste/index.js:276

Found it was trying to build an absolute path for the file it was trying to find, so I added a console.log to to figure out what potentialAbsPath was
and it gave me "C:\tmp\test\test\index.android.js"
then I worked out what this._hasteFS.exists(potentialAbsPath) was calling and hasteFS seems to do a lookup against the file cache
I found a method on hasteFS called getAllFiles so I added a console.log(this._hasteFS.getAllFiles() and found that it listed files with the wrong path separator

then it was just a hunt to figure out how react-native received the files from watchman node_modules\jest-haste-map\build\crawlers\watchman.js and testing replacing the file separator.

I guess this could just be fixed in react-native, but I do wonder what else in the watchman ecosystem depends on windows file paths matching....

[wez]

@cpojer (picking you because you touched that portion of RN most recently!) can you help me (or find me someone that can help me!) understand where the sensitivity to the slashes is coming from within the hasteFS code?

Forward slashes and mixtures of forward slashes are fine on windows (you'll notices that node itself is using them in the stack trace in the comment above), so I'd prefer to make RN do the right thing here.

[cpojer]

This is the code that is shared between Jest and react-native-packager: https://github.com/facebook/jest/blob/master/packages/jest-haste-map/src/index.js

[wez]

We're chatting about this internally; at the moment we're learning towards making a change in the javascript side of haste/react-native