Installing a dependency from git attempts to install all transitive dependencies even those that aren't in the lock file

[SayakMukhopadhyay]

This is a very particular issue that I am facing and I will have to go into the details to explain so please bear with me.

Background

I am working on a Hugo project with the Docsy theme and Docsy has a dev dependency on the npm library hugo-extended. This library has a post install scripts which downloads the Hugo binary pertaining to the current platform and attempts to execute it once to check if the correct version was downloaded.

If I install Docsy with npm install github:google/docsy#semver:0.11.0, the resulting package-lock.json doesn't include hugo-extended at all, which is what I want (we install Hugo ourselves and don't want the transitive dependency of Hugo Docsy brings). Installing Docsy like this works just fine (at least on the surface) on my Windows system and on most Linux based containers.

The Issue

But the issue crops up when trying to install Docsy in an Alpine linux based container. The error that crops up is due to the downloaded binary not being compatible with musl based systems like Alpine.

But, that is not the issue. The issue here is that running npm install github:google/docsy#semver:0.11.0 results in an attempt to install hugo-extended. Since hugo-extended is not in my package-lock.json, there should be no attempt to install hugo-extended. In fact using yarn instead, I don't face this issue as (seemingly) yarn doesn't try to install hugo-extended.

The diagnosis

Initially, I spent a lot of time trying to get the hugo binary working with alpine but as I mentioned before, I kept thinking that npm shouldn't even be attempting to install hugo-extended in the first place and I found the yarn actually doesn't. Which led me to try and find if the issue is with npm itself. Running npm install github:google/docsy#semver:0.11.0 gives a particular error message among others

git dep preparation failed

Searching around this issue led me to
https://github.com/npm/pacote/blob/bf1f60f58bb61f053262f54724edcacaadb221ce/lib/git.js#L152-L191
Here, I can see that npm tries to run npm install on any git based dependencies which results in attempting to install everything in the package.json of that git based dependency, *even if it isn't present in our package-lock.json`.

To me, this is an unpredictable behaviour. npm shouldn't be attempting to install the whole package.json of this dependency, even if it is doing that in a temp folder.

Note

We have workarounds for this but the fact that this issue doesn't happen with yarn is indicative that it's npm's implementation that is leading to this issue. I am not raising this issue as

1. I don't know if it should go to npm/cli or npm/pacote
2. Maybe some discussion is needed first

Here's the related issue and discussion which led to this post: kubernetes/website#49460 (comment)

[ebndev]

Hi @SayakMukhopadhyay, thanks for being a part of the GitHub Community!

Have you tested this with different npm versions to see if it’s a recent change?

npm/pacote might be the right place for an issue but for now, I'll label this as a bug so the Product team can keep track of it.

[SayakMukhopadhyay]

Thanks, I have tried with multiple versions, new and old and it's all the same.

[ebndev]

Okay, good to know!

[github-actions[bot]]

💬 Your Product Feedback Has Been Submitted 🎉

Thank you for taking the time to share your insights with us! Your feedback is invaluable as we build a better GitHub experience for all our users.

Here's what you can expect moving forward ⏩

• Your input will be carefully reviewed and cataloged by members of our product teams. 
  • Due to the high volume of submissions, we may not always be able to provide individual responses.
  • Rest assured, your feedback will help chart our course for product improvements.
• Other users may engage with your post, sharing their own perspectives or experiences. 
• GitHub staff may reach out for further clarification or insight. 
  • We may 'Answer' your discussion if there is a current solution, workaround, or roadmap/changelog post related to the feedback.

Where to look to see what's shipping 👀

• Read the Changelog for real-time updates on the latest GitHub features, enhancements, and calls for feedback.
• Explore our Product Roadmap, which details upcoming major releases and initiatives.

What you can do in the meantime 💻

• Upvote and comment on other user feedback Discussions that resonate with you.
• Add more information at any point! Useful details include: use cases, relevant labels, desired outcomes, and any accompanying screenshots.

As a member of the GitHub community, your participation is essential. While we can't promise that every suggestion will be implemented, we want to emphasize that your feedback is instrumental in guiding our decisions and priorities.

Thank you once again for your contribution to making GitHub even better! We're grateful for your ongoing support and collaboration in shaping the future of our platform. ⭐