Upgrade devDependencies, add lockfile

[jgonggrijp]

Since there was no lockfile yet, I decided to pull out yarn. Yarn gave me lots of warnings, so it's clearly time for some upgrades.

[ogonkov]

Is yarn really necessary?

[jgonggrijp]

No, of course every other package manager will work as well. Would you prefer another one, and if so, which and why?

[ogonkov]

I guess npm is good enough in current state. An im one of this people, that believe that lock file is not required for library.

[jgonggrijp]

Now we seem to be discussing two topics.

Whether to have a lockfile or not: as far as I know, all current package managers (at the very least npm and yarn) give you a lockfile by default, creating it on the spot if it didn't exist yet. So the question is only whether to commit it or not. Given that every single person who installs Backbone's devDependencies is going to have a lockfile anyway, we might as well all have the same one, no? In fact, I think that is one of the main purposes of a lockfile: ensuring that everyone on the team is working on the exact same thing.

Which package manager to use: sure, npm is good enough, but so is yarn. On top of that, yarn is faster and deterministic and it deduplicates dependencies. I find it convenient. I can be convinced to use another package manager, but the argument will have to be stronger than "good enough".

[ogonkov]

package-lock.json will not be packaged. If we commit it, that would be for our own CI/developer team, user will end with different package-lock (that the reason why i never use it in my libs).

[ogonkov]

I'm not pushing here to avoid package-lock, just opinion

[jgonggrijp]

I appreciate your opinion, and your active participation in general. Please continue sharing your opinion.

Yes, the lockfile is for our team and for CI. I consider that enough reason to commit it. Let's put that part of the discussion aside, because I believe there is really no added value in keeping it out of Git.

Regarding whether to use npm or yarn, I'm still leaning towards yarn, but I'll allow some time for more people to chime in with opinions and arguments. I'm not in a hurry.

[paulfalgout]

I typically prefer npm to yarn, but as @jgonggrijp is the most active contributor at this point, I'd say dealers choice. However I do think keeping backbone and underscore in similar tooling is ideal, and it looks like underscore is using npm? Is it switching to yarn?

[jgonggrijp]

Why do you typically prefer npm over yarn, @paulfalgout? I would be really interested to learn that.

You are right that Underscore is still using npm. I abstained from the transition there, because it already had a package-lock.json. I might eventually switch Underscore to yarn as well, though.

I like the consistency argument, but that consistency has already been lost. Or it is already leaning towards yarn, depending on how you look at it. Underscore-contrib and Underscore Fusion (which I still need to publish) are using yarn already. So is another library I am currently working towards publishing, which I expect to be somewhat "associated" with Underscore and Backbone as well.

[paulfalgout]

Nothing overly particular re: yarn. I think at one point yarn was clearly ahead of npm. Then at one point once npm improved I migrated just about everything from yarn and gulp etc etc etc to npm and webpack or rollup to reduce tooling dependencies. Most of my repos have an .nvmrc so anyone running nvm will already have the correct (or close enough) version of npm for that version of node and nothing additional to install. Yarn is just 1 additional complexity.. typically with everything.. most CIs require some additional yarn step etc.. but I no longer see a particular benefit to that complexity.

All that said, it's not that big of a deal to support yarn, nor is it hard to switch back. ATM you seem to be the foremost contributor to the backbone-ecosystem so whatever your most comfortable with seems best.

[jgonggrijp]

Generally, I am much more interested in making a good choice than in making a comfortable choice. I appreciate your kind gesture, though.

You wrote that npm had improved, which @ogonkov had already hinted at as well. I suspected I had probably missed something about npm's recent developments, so I investigated. I ended up looking at this, this, this, this and this. I learned a couple of things:

• Apparently, npm 7 with a lockfile is deterministic, too.
• Apparently, npm 7 will consult and update a yarn.lock if present, alongside its own package-lock.json.
• The author of the npm blog made a convincing case that npm's package-lock.json is more complete than yarn.lock.
• Yarn can still be expected to be substantially faster than npm for big tasks.
• For small tasks (nearly everything was already installed, or adding or removing a single dependency), yarn has no obvious performance advantage, though small tasks are faster anyway, so probably less important to compare.
• Contrary to what I believed, yarn does not flatten the dependencies by default. This requires an opt-in by passing the --flat option.
• NPM can deduplicate dependencies if you opt-in, too (but the option is called --prefer-dedupe instead of --flat).
• NPM might produce a smaller installation size by default.

For a moment I was tempted to suggest adding both a yarn.lock and a package-lock.json and letting contributors decide for themselves whether to use yarn or npm 7+. I rejected the idea, because I realized that the scripts in the package.json assume a particular package manager. It would also somewhat neutralize the purpose of having everyone on the team work on exactly the same environment.

I think I'll compare the combined size of the node_modules and the lockfile with and without the --flat/--prefer-dedupe flag. If no clear winner comes out of that comparison, I'll go with npm v7, on the grounds that the scripts already assume npm.

[jgonggrijp]

Most of my repos have an .nvmrc so anyone running nvm will already have the correct (or close enough) version of npm for that version of node and nothing additional to install.

Maybe Backbone should do that too? Could you give an example of how that works?

Yarn is just 1 additional complexity.. typically with everything.. most CIs require some additional yarn step etc.. but I no longer see a particular benefit to that complexity.

In yarn's defense, the Node.js images from Docker Hub, GH Actions, GitLab, Travis etcetera nowadays all seem to include yarn by default, so this additional complexity seems to have been eliminated.

[paulfalgout]

.nvmrc is just the node version. So most of my projects are just that file containing 14 or 16 stored in the root. Then you can run nvm use for a project. I have a vscode plugin that just runs it automatically when I open the terminal for a project. It doesn't enforce it, but it's a help.

Essentially the work of up-keeping node is keeping CI and the .nvmrc files up to date.

It's more of a tool like .editorconfig that saves steps for devs that use the plugins.

Yeah yarn is typically installed on docker instances.. however for some projects I'm on, the docker image for CI is php and we then install node.. at that point you're always having to also install yarn... again.. negligible, but I didn't see any benefit from yarn any more.