🦁 Yarn 2.0 Working Group

[arcanis]

Alright @yarnpkg/core (and community!). Let's discuss what we want for the 2.0 release 🙂

Ideas "en vrac":

• Drop support for Node 4 (build(node4): Drop Node 4 builds which has expired #5736)

• Revamp the command line implementation (switch from Commander to Yargs, remove the complicated custom logic as much as possible)

• Harmonize the command line (an example in mind is the --pattern option, which sometimes is an option like for yarn list, sometimes a -P,--pattern option like for yarn upgrade, sometimes a --scope option that only takes the scope like for yarn upgrade-interactive, and sometimes an argument that doesn't glob like for yarn outdated)

• Change the lockfile syntax to a YAML-like file format (YAML-like because we would just slightly tweak our parser to emit something YAML-compatible, but we wouldn't support any advanced YAML feature)

• Remove the code that deals with multiple registries (yarn / npm, cf src/registries). The yarn registry being a mere mirror of the npm registry, this doesn't make sense.

Anything else you think would be important to ship during this version? Especially in term of breaking changes requiring a semver-major bump?

[imsnif]

About the YAML lockfile - I think it's great. Maybe we can do some revamping of the format while we're there? eg. lose shasum in favor of integrity?
Actually, maybe we can implement this: https://github.com/yarnpkg/rfcs/blob/master/accepted/0000-registry-url-in-lock-file.md
What does everyone think?

[cpojer]

As you can imagine, I like the CLI stuff, that is ripe for a large cleanup for sure. The lockfile change I'm less excited about because I don't really see the clear value that it would bring. Can you explain that a bit more?

What other large features should we consider for a Yarn 2.0 release that we haven't thought of yet?

[arcanis]

Can you explain that a bit more?

It's a bit clunky in term of interop to have our own file format. Sure we have the @yarnpkg/lockfile package to parse/write it, but I feel like it would be better if we didn't have to reinvent the wheel when there's fine solutions outside.

Iirc the initial reasoning behind the custom lockfile format was that JSON isn't convenient to read for humans, and that YAML parsers are too big. I kinda agree with both but:

• If size is a concern, then JSON.parse is native and would remove the parser from the equation (750 lines ... we would still keep the conflict resolver so let's say 700). It depends on much it's a concern vs readability.

• Not using a YAML parser doesn't mean we can't have a YAML-compatible syntax. We're almost compatible, except for a few quotes and colons here and there. It makes me sad to be so close but so far 🙂

The only reason for me not to change how things currently work is that the lockfile file format is a core component, so we would have to make sure that the 2.0 would be able to convert the old format into the new format, whether it's JSON or YAML. Then we could remove this logic starting from 3.0 or 4.0. It's a bit of work, so maybe it's not worth it, but I still think it's worth being considered 🙂

[arcanis]

Actually, maybe we can implement this: https://github.com/yarnpkg/rfcs/blob/master/accepted/0000-registry-url-in-lock-file.md

Yup, that sounds like a good idea (I have a few comments on the proposal itself, but I'll post it to yarnpkg/rfcs#84).

[arcanis]

I'd also like to remove the part of the code that deals with multiple registries (not talking about registry urls - we have branches for the "yarn registry" and for the "npm registry", which doesn't make sense since the yarn registry is effectively the npm registry).

Looking at the code, I think it was originally used to have a different configuration between npm and yarn, but I haven't seen it used anywhere, even after a quick search on Github, and it complexifies a lot the codebase (especially since it isn't actually implemented everywhere and isn't well tested, so I'm not even sure it works ...).

[cpojer]

I feel like there isn't a good enough justification to change the lockfile format: it would churn all existing lockfiles because we believe aligning it with YAML is a good thing. In hindsight we should have probably gone with YAML but now the cost of changing doesn't seem worth it if the upside is only hypothetical.

[rally25rs]

The lockfile change I'm less excited about because I don't really see the clear value that it would bring.

I believe the complaint from users is that @yarnpkg/lockfile is nice to have, but only if you are implementing a JS utility. I remember someone saying that they wanted to write something in Python that does some processing of the lockfile.

---

This might be a good opportunity to deprecate link: dependency type. It's a bit flakey to actually use (if the package you link to has a node_modules dir then node will resolve in there, instead of in the parent package that is including the link) and I think the intent behind it has been supplanted by workspaces. Alternatively, we instruct people to use yarn link instead of link: dependencies.

---

As we did with yarn v1, I think we should include all the high-priority tagged issues and try to resolve them (there are 35 currently).

[rally25rs]

Also, we could use a GitHub "Project" to track v2 tasks, like we did for v1. I thought that worked well.

[rally25rs]

Oh I've got one to add to the list: automated build & release, and defining a release cadence. Right now it feels sort of random of when we release a new version, and it leads to a lot of "when will this PR be released" questions.

It's be nice to do something like "on the first of every month we will release a bug fix increment"

[Daniel15]

We have the capability to have automated releases, I was just worried about releases being unstable. Maybe it's not so bad though. Most parts of the release are already automated, the only piece we're missing is automatic tagging of releases, and automatic changelogs. :) Sent from my phone.

…

On Fri, May 4, 2018, 12:21 AM Jeff Valore ***@***.***> wrote: Oh I've got one to add to the list: automated build & release, and defining a release cadence. Right now it feels sort of random of when we release a new version, and it leads to a lot of "when will this PR be released" questions. It's be nice to do something like "on the first of every month we will release a bug fix increment" — You are receiving this because you are on a team that was mentioned. Reply to this email directly, view it on GitHub <#5773 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAFnHWXtOcIS3P9Ozu-zG2tN1qbXMsIRks5tuxJigaJpZM4TvnPl> .

[BYK]

There's also this that I really want to address: #4147 and #4379.

I'll share my proposal on this soon on that ticket.

The lockfile change I'm less excited about because I don't really see the clear value that it would bring.

The custom format of the lockfile is a constant source of confusion for people. It is not well-documented, is not supported by syntax highlighters (they treat it as YAML), and we need to maintain our own parser package which we don't really do a good job of.

If the lockfile was a limited subset of YAML it may resolve most of the issues I listed above. @imsnif's suggestion is also about the lock file but more about its contents rather than the format.

What other large features should we consider for a Yarn 2.0 release that we haven't thought of yet?

I think 2.0 is about breaking changes, not necessarily large changes.

[peterjanes]

Has yarn stabilized to the point where it could expose an API (#2740)? It's not a breaking change per se, more the opposite.

[kaylie-alexa]

Definitely +1 for cleaning up command line arguments as they're really confusing. As a user I understand @cpojer 's concern about having to basically recommit your entire lockfile with the upgrade but I think dropping the customer parser and exposing it as YAML api would be better for future's sake.

Few other thoughts I had in mind:

1. Come up with a documentation-driven model for development in Yarn -- it's hard to sync between code changes, cli options, and docs which live on a separate repo. Maybe start with every PR needs an accompanying docs change unless it's something like a chore?
2. A new command yarn run-parallel that allows you to run multiple commands together like yarn run-parallel lint test. Also possibly implement the --pattern flag to run commands, e.g. yarn run-parallel --pattern build?
3. Yarn check has not been kept to up-to-date with the new features of yarn and there seems to be confusion/lack of documentation about --verify-tree, --integrity, etc. Yarn check [--integrity [--verbose]] fail and messages aren't helpful in the slightest #2287