[Draft] feat: use yarn 4 in monorepo

[Saadnajmi]

Summary:

Staying as draft as I do some testing for viability. Once done, I'll submit an RFC if that is helpful.

I am aware this may not work well for Meta internally. I wanted to see how far I could get, and whether I could take some of the diffs and keep them in React Native macOS as well.

The Yarn package manager is currently on major version 4, but React Native still uses version 1 for itself. Most of the community has moved to Yarn 3+ at this point, I personally make that my first change in every repo I touch. There are several benefits to Yarn 4:

1. Increased speed

Yarn 4 is way faster than yarn 1, at both cold and hot installs. I'll add some timing data once I have it.

2. Workspace protocol

With modern package managers, you can use the workspace protocol to specify dependencies between projects in the monorepo. That means you can do something like this:

  "name": "react-native",
  "version": "1000.0.0",
  "dependencies": {
    ...
-    "@react-native/codegen": "0.80.0-main*",
+    "@react-native/codegen": "workspace:*",
}

You wouldn't need that arbitrary marker of 0.xx-main that is used on the main branch that we need scripts/releases/set-version.jsto go and update a bunch ofpackage.jsonfiles to set, we can just specifyworkspace:*and atyarn npm publish` time, the publish command will replace the versions appropriately. Less manual scripts means a more stable release!

3. Support for pnpm mode

Yarn 4 supports multiple linker layouts. By default, Yarn 4 uses the standard node_modules linker for layout of dependent modules. This is the tried-and-tested standard for most of the JS ecosystem, but is vulnerable to phantom dependencies. In contrast, Yarn 4 also supports pnpm layout which is much more strict at making sure that a package only has access to the dependencies it lists, and not others that happen to be in the monorepo due to transitive dependencies. We use pnpm layout for other repos that depends on react-native like rnx-kit, and often have to go upstream fixes in React Native where pnpm caught a phantom dependency. See:

• chore: add jest-config to monorepo dependencies #50280
• fix: correctly forward dependency on @react-native-community/cli-server-api #49325
• fix(dev-middleware): add missing invariant dependency #49047
• fix: fix semver not being found in pnpm setups #47310
• fix: fix @react-native-community/cli-platform-* packages not being found in monorepos #47308
• fix: fix @react-native-community/cli not being found in monorepos #47304
• fix(react-native-codegen): scripts require yargs #45995
• fix(react-native): fix codegen failing in a pnpm monorepo setup #45993
• fix(dev-middleware): fix react-native config failing in pnpm setups #42907

3. A much cleaner yarn.lock file.

The yarn.lock file is now in yaml, and also specifies what dependencies are local to the monorepo and what isn't. It also has tools like yarn dedupe built in to deduplicate dependencies, which has been very useful elsewhere.

3. Support for plugins

Yarn 4 has a very strong plugin ecosystem. We at Microsoft have built plugins for simple things like patching boost to download from not-the-jfrog cdn to supporting custom protocols we only use internally. This seems to be a large differediffere

Changelog:

[INTERNAL] [CHANGED] - Use Yarn 4 for monorepo

Test Plan:

Double check that releases-ci is fine