Duplicate of #9161

Maybe we could unlock #9161? It's frozen I can't upvote the issue. This way, you could gather more feedback from the community.

/CC @nodejs/user-feedback

Well, looking at https://stats.yarnpkg.com/, yarn accounts for 0.02% of the downloads, it's probably too early.

It became impossible for us to get stats since npm moved to Cloudflare, circa ~june 2018. Back then, Yarn had 48% of the package downloads, in constant augmentation from ~20% in April 2017. Your guess is as good as mine as to where we are now 🙂

I would like to get this re-opened because of the reason that alternatives become more popular and are faster than npm (besides all that was mentioned in the #9161 discussion). The title should be Allow users to choose a default package manager.

• npm (default)
• pnpm (my choice)
• yarn 2 (berry)

I would definitely champion work in this direction. There are a couple important bits:

1. For the Node.js installers, users should be presented a choice of package manager to install. Which package managers to include in last list can be a sensitive topic. We would need some baseline criteria for adding a package manager to the options list.

2. For the tar ball distributions, we would need a script that effectively does the same -- providing the ability to select a package manager to download and install.

3. Rather than bundling the npm client into the nodejs/node source tree, it should be removed. It might still be that we provide a distribution of the client, but it would not exist in the source tree.

I am in favor of a user-selectable package manager, with the default one that is:

• better debug-able (error logs alone sufficient to figure out what went wrong)
• better documented (features easily understood and align with what it does)
• better modularized (less side effects and dependencies)
• well defined caching, and environment variables

than others. In addition, visible evidence of / promise on:

• an LTS policy aligning with node.js
• an open governance and community based project development and support model
• well defined and documented release cadence
• an agreed upon secure engineering policy

re-opening, as the linked issue (to which this is a duplicate of) is closed and locked.

One important question that we should first agree on is: what workflow do we want to improve? There are two different ones, and the solutions aren't the same for both.

1. Either we want to make it easier for each developer to create and work on their projects (so the solution would be to make it easier to work with a package manager of their choice),

2. Or we want to make it easier for all developers to work on all Node projects (so the solution would be to make it easier to work with any project, regardless of their package manager)

The difference may look subtle but let me show a practical example. VSCode uses Yarn as package manager (they have a yarn.lock checked-in, and they may use Yarn-specific features). Lodash uses npm (they have a package-lock.json, and they may also use npm-specific features). As such, it becomes clear that the choice of package manager isn't up to the users, but rather up to the projects! Indeed, the package manager (and its version) can be seen as the very first dependency for any project.

For this reason, I'm not convinced by, let's say, an install prompt that would allow choosing the package manager to install. In my mind, it'll simply fall into the same trap of making it harder to work with the package managers that didn't get picked. It may help new users a bit, but power users will end up still needing to install the other package managers manually.

So what do I suggest? I obviously really want to solve this. First, to summarize my ideal solution:

1. Package managers shouldn't necessarily be shipped by default,
2. I'd like projects to decide the package manager (not users)
3. I'd like the package managers install process to be transparent

Before going further, let me tell you about a feature of Yarn, something called yarnPath. When the global yarn CLI spawns, it looks for a .yarnrc.yml file into the current directory and greps it for a yarnPath key. If it finds it, it shells out to it. Basically, the idea is you can store your Yarn version in your repository, preventing cases where multiple members of a team use different versions of Yarn. Updating the version can be done through a dedicated command, yarn set version 2, which will download the latest release from our website, store it into the repository, and update yarnPath.

So .. what's my point? Simple: this workflow proves that we don't need the full package managers to be shipped within Node. What we do need are the binary names to be shipped with Node. For example for Yarn, all we really need is for a global yarn binary to support this yarnPath and yarn set version logic. If we have this, then it becomes up to each project to select the version they want to use, and we'll tick everything on the checklist:

1. The full Yarn binary wouldn't be shipped - for Yarn we would provide a few-KB source file that would only support the yarnPath and yarn set version code path, and for npm/pnpm I believe we could write a very minimal wrapper that would follow the workflow I showcase below.

2. Each project would be able to define the version of the package manager they want to use, and having multiple projects with multiple different versions wouldn't be a problem.

3. Most users wouldn't even have to be aware of this, because the details would be abstracted out.

And it's not Yarn-specific! Generally speaking, all we need is to mock the package managers binaries to "jumpers" - very simple binaries that would throw an error if the version they detect should be used isn't installed. For example, here are a few examples of the workflow I have in mind:

$ mkdir my-project && cd my-project

$ npm install
No npm selected. Do you want to use 6.14.1 (latest)? [Y/n]
Installing npm 6.14.1... Done.
Installing the project... Done.

$ npm --version
6.14.1

$ jq .engines package.json
{
    "engines": {
        "pm": "npm@6.14.1"
    }
}

$ git clone git@github.com:arcanis/my-pnpm-project my-pnpm-project
$ cd my-pnpm-project

$ jq .engines package.json
{
    "engines": {
        "pm": "pnpm@4.11.1"
    }
}

$ pnpm install
Installing pnpm 4.11.1... Done.
Installing the project... Done.

$ pnpm --version
4.11.1

$ pnpm install
Installing the project... Done.

$ git clone git@github.com:arcanis/my-yarn-project my-yarn-project
$ cd my-yarn-project

$ cat .yarnrc.yml
yarnPath: .yarn/releases/yarn-2.0.0.js

$ yarn --version
2.0.0

$ yarn install
Installing the project... Done.

I like your proposal. However, the user still has to know the appropriate CLI command for the package.json's engine/pm field.

What if we had a generic command for that aswell, e.g.:

node packages install
# checks package.json, installs pm engine if not available, uses it to install deps

What if we had a generic command for that aswell, e.g.:

There are a few ways we could solve it. For example, we could simply make the jumpers warn when used on incompatible projects:

$ jq .engines package.json
{
  "engines": {
    "pm": "pnpm@4.11.0"
  }
}

$ npm install
Error: This project is configured to be used with pnpm.
Please run `pnpm install` instead.

This way each project would keep their identity, and users wouldn't be confused as much by a command having different behaviors on different projects (my main worry with having a single command).

LGTM @arcanis ! What about global installed packages? Some prefer npm, some prefer yarn and if both are used, incompatibilities could appear (had this issue with NVM, NPM and pnpm). To your suggestion I would add this:

Users can select a default package manager, this would be picked by default for global packages and newly created projects. For example, when I create a new project I now do node init which creates a package.json and installs the global default version into my project. Then we are no longer do npm install oder yarn add but more likely node add eslint and node add -g eslint which would automatically select the correct package manager to use.
Node would need to add command aliases like @ionic/cli does it.

Additionally, I would cache and symlink the PM versions, so the user is not downloading duplicated versions, when they already got the same in another project.

While aliases might be useful in the future, I think this should be a follow-up and considered out-of-scope for this discussion. Multiple reasons for that:

• Having node aliases isn't a prerequisite for having the regular binary jumpers (whereas having the jumpers is a prerequisite for having aliases).

• I suspect adding commands to the node binary will be a tougher sell, so I'd prefer to avoid this part of the discussion (the worst case would be that it reaches a dead end, blocking as a result the whole idea or at least delaying it significantly).

  • Another angle is that I suspect there is no right or wrong position on this, so keeping the discussion open and productive might be difficult. By comparison I think everyone is now aware that the npm binary isn't the exclusive JS package manager anymore, so we'll be able to spend more time on the "how" than debating the "why".

• It's not clear how much would be aliased, so more discussion would be needed. Only install? Or add too? What about --version? Or workspaces info? What about the CLI options? (No need to answer, it's just an example of the kind of complexity that this would add)

• There are identity concerns with aliasing the package manager versions under common names. For example, let's say that node install crashes, where do we expect users to fill the bug reports? Etc, etc.

I'd prefer we stay focused on global binaries, and later on we'll have the time to think about a Node integration - in a separate project dedicated to this goal.

I have two follow-ups on @arcanis’s proposal:

1. What makes a package manager “good enough” to ship a stub with Node.js? For example, if user X comes in and asks us to ship a Bower stub, should we? What about the 100 other new package managers this will spawn? Where do we draw the line and how does that evolve over time?
2. What happens when a package manager is unmaintained, dead, or otherwise undeveloped? How do we manage the legacy stubs? Are we stuck with them forever, like we are with many OG bits of core?

Looking into the future I see exponential growth of projects that are added to this, leading to a more fractured ecosystem and overall worse DX for new and novice users. I am extremely cautious about shoving this burden on users.

Additionally, to both @jasnell and @arcanis, how do you envision your potential solutions being effective for FaaS platforms, Docker images, and PaaS platforms without tremendous additional overhead? (I am aware that this has been a request from FaaS folks who are running code that doesn’t need npm!)

all I’m gonna say about that gist: ugh.

Thanks for putting this out there, @jasnell.

This survey is great but not 100% accurate. Some users might not even know what package manager they are using.

If they use Rush, do they know pnpm or Yarn is executed under the hood? (Rush supports all 3 package managers).

If they create a new remix on Glitch, do they know that pnpm installs the dependencies on the backend? (Glitch uses an old version of pnpm to reduce disk space usage)

At my company, there are almost 200 developers that use pnpm. I am not sure all of them know that it is not the official package manager. They don't care because our internal wikis teach to use it for installing dependencies. And almost none of them use Twitter (we are mostly from Ukraine) to see and vote in your poll.

I am not saying that pnpm is more popular than 4% but even that 4% is a lot.

This survey is great but not 100% accurate. Some users might not even know what package manager they are using.

Without a doubt! If nothing else, it reinforces the point that Node.js core should be fairly deliberate about how it moves forward with default package management... if the decision ultimately is to maintain the status quo we should have a definitive reason that's more than just it's the status quo.

BTW I appreciate the balanced consideration of pros/cons, data collecting, and interesting progression of ideas in this GitHub thread. I apologize for jumping to the conclusion that other people had jumped to a conclusion.

I took a few hours this week-end and built a prototype for the jumper proposal described in #15244 (comment), since it seemed like one of the possible picks. The result is here: arcanis/pmm. I didn't hit particular blockers, although there's a slight runtime overhead I'd like to address in the future.

@jasnell What do you think would be the next step in this discussion?

I still think that needing a specific package manager to install a package highlights that a package manager is doing something wrong, not that node is doing something wrong. A package using yarn's pnp thing that hijacks require should still run if you use npm and don't hijack require, because in both cases the needed dependencies should be in package.json. i don't want to get too much into differing package locks, but i'd say any package intended to be installed top-level that i can't npm i -g package-name on is inherently broken. All that is left here is developing a package (you've cloned the repo), which already requires development specific tooling (babel, eslint, etc), and I don't developing packages is a big enough use case to make node ship a package manager manager.

Another idea that I think wasn't mentioned. Maybe it needs to be a package manager manager + node manager. Maybe it should be done on the nvm/nvs level as different projects are also expecting a specific version of Node, not only a specific version of the package manager.

developing packages is a big enough use case to make node ship a package manager manager

I think it is less about developing packages and more about developing large enterprise projects. Small package authors don't really care about which package manager they use. Large projects might need specific features of specific package managers.

@arcanis ... that's awesome. Give me about a day to have time to check out your POC then get back to you on it.

@devsnek ... I definitely agree with what you're saying but currently, it's just not clear who actually owns the definition of How a Node.js Module Actually Works. I think we need to put some thought in to addressing that.

I would be highly in favor of having stubs that install them on demand.

I read the entire thread and first of all I'm thankful, that this discussion has started because I think it is really important to have this discussed.
An I'm thankful that so many people contribute to this discussion with opinions and possible solutions.

To give my two cents to this discussion:
I'm just a casual user of npm and would love to use yarn per default in projects.
BUT in my opinion should Node.JS shipped with npm per default (or as default package manager - if you like to describe it that way), simply for that reason, that NPM is not only a package manager but also the registry that is used by all other (bigger) package managers per default.

How about no package manager at all like both Deno and browser handle it? With absolute http urls!
Would be nice to just import x from 'https://example.com/lib.js' without having to install anything and supporting lazy loading/caching as needed.

Obvious we would need more security options like disable/sandboxing fs, net, process & env
wish i would like to have either way (whatever we decide to support)

Would be nice to just import x from 'https://example.com/lib.js' without having to install anything and supporting lazy loading/caching as needed.

What is Deno's story for versioning? How do you distinguish SemVer major/minor/patch bumps? How do you express a peer dependency relationship?

Also, suppose:

• A depends on B and C
• B depends on D@^1.0.0, but
• C asks for D@~1.2.3

How would Deno ensure that a single copy of D@1.2.5 gets shared between B and C, instead of side-by-side installations of D@1.3.0 and D@1.2.5? This is what package managers are all about.

That can be handled by some CDN servers url structure
Eg https://unpkg.com/deepmerge@1.4.4/dist/cjs.js

Googles Hosted Libraries have done it even since before node's was introduced

denoland/deno#47

Import maps is also possible in deno that can help with dependencies

@jasnell

There is another aspect to all of this that is important and that has not yet been discussed. Yarn 2 introduces the pnp concept of package resolution. npm, Inc. has indicated that their future directions around tink also intend to move in that direction, and while there is a good proportion of the ecosystem that appears to favor that model, a number of major enterprise users of Node.js actually do not -- and do not want to see pnp become the default model for package management in Node.js. Should a decision be made here to stick with npm as the default client, we'd need to have a discussion around this regardless of any other clients being used by the ecosystem if the intent is still to move to the tink/pnp approach.

In other words, Node.js core needs to come to agreement around one very specific question: What do we (Node.js core) wish to define as being the minimal default package management experience out of the box with Node.js core. I would argue strongly that pnp is not it. Node.js core should provide a minimal experience for installing and publishing modules. Nothing more, nothing less.

There's a pretty fundamental difference between tink and pnp, afaik. Tink does not change or override the Node.js module resolution process. Instead, it creates a virtual filesystem, such that the Node.js module resolver just works like it always has. When it gets ENOENT errors from fs operations, these operations are re-tried against the appropriate file found in a centralized cache.

We fully appreciate that enterprise (or other) users may prefer to install without this kind of experimental magic. When we do implement it (which won't be for a while yet), it'll be opt-in for at least one major release, and assuming all goes well, opt-out thereafter. We have no plans to ever ship a version of npm that can't install your dependencies right into node_modules as normal files, and may back away from this entire endeavor if it turns out to be a bad idea in practice.

npm is at least as committed to backwards compatibility as Node.js, even to a fault. And we're of course happy to keep the lines of communication open and make sure we're on the same page about the DX that npm and Node.js provide for our users.