[BUG] npm link and dependency hoisting don't play nice?

[markmckenna]

I am not 100% sure if this is a bug or I'm just misunderstanding the proper behaviour, but I don't really see a clear way to resolve it. I'm also having trouble building a minimal reproducing case because I don't seem to be able to play the npm link trick without first publishing the package I'm trying to cross-link.

Explanation:

Imagine a repo like this:

project-a/package.json:
  { "name": "project-a",
    "workspaces": [ "subproject-a" ] }
project-a/subproject-a/package.json:
  { "name": "subproject-a",
    "dependencies": { "webpack-merge": "^4.2.2" }}
project-b/package.json
  { "name": "project-b",
    "dependencies": { "subproject-a": "0.0.1" }}

I want to do local development on everything together, so I do:

( cd project-a/subproject-a && npm link )
cd project-b
npm link subproject-a
npm install

If I do something in project-b that requires webpack-merge, it generally won't be able to find the dependency, because it isn't anywhere in the upstream of subproject-a as it appears in project-b/node_modules.

I also observed the issue (already logged) that npm link updates project-b/package.lock with a file: reference.

EDIT: Also note that if I break ranks and run npm install in subproject-a directly, everything works according to plan.

Versions:

npm@7.0.6

[Gameghostify]

#2034 is related/duplicate

[wesleytodd]

I am seeing something very similar:

$ cd opta && npm i && npm link
& cd ../opta-doc && npm i && npm link opta
$ npm ls yargs
opta-doc@0.0.0 ./opta-doc
├─┬ mocha@8.2.0
│ └── yargs@13.3.2
└─┬ opta@0.0.6 -> ./opta
  └─┬ mocha@7.2.0
    ├─┬ yargs-unparser@1.6.0
    │ └── yargs@13.3.2
    └── yargs@13.3.2
$ cd ../opta && npm ls yargs
opta@0.0.6 ./opta
└─┬ mocha@7.2.0
  ├─┬ yargs-unparser@1.6.0
  │ └── yargs@13.3.2
  └── yargs@13.3.2
$ cat package.json | grep -n5 "yargs\":"                     
29-  "dependencies": {
30-    "inquirer": "^7.1.0",
31:     "yargs": "^15.3.1"
32-  },
33-  "devDependencies": {
$ npm test
# ...
     Cannot find module 'yargs/yargs'
Require stack:
- ./opta/index.js
- ./opta-doc/index.js
- ./opta-doc/test/index.js
- ./opta-doc/node_modules/mocha/lib/esm-utils.js
- ./opta-doc/node_modules/mocha/lib/mocha.js
- ./opta-doc/node_modules/mocha/lib/cli/one-and-dones.js
- ./opta-doc/node_modules/mocha/lib/cli/options.js
- ./opta-doc/node_modules/mocha/bin/mocha

[markmckenna]

#2034 I don't think is a duplicate. It talks about modifying package.json, whereas this is about the general dependency locating mechanism.

I also encountered another issue with locating of dependencies; hoisted dependencies can't be found by npm exec when the lower-layer package.json has its own node_modules. The search stops as soon as it sees any .bin directory.