[BUG][8.4.0][Windows] npm i -g npm@8 fails in GitHub Actions

[AviVahl]

Is there an existing issue for this?

• I have searched the existing issues

This issue exists in the latest npm version

• I am using the latest npm

Current Behavior

This issue appears to have started with the release of [email protected].

https://github.com/wixplosives/file-services/runs/4976059479?check_suite_focus=true
https://github.com/wixplosives/pleb/runs/4973815604?check_suite_focus=true

Happens on every repository where I'm using npm@8 in GitHub Actions.
I realize this can be a (permission?) issue with GitHub Actions containers, but I wouldn't rule out a regression in npm itself.

Expected Behavior

npm i -g npm@8 should pass on Windows in GitHub Actions

Steps To Reproduce

1. In Github Actions
2. With this (super simple) config: https://github.com/wixplosives/file-services/blob/master/.github/workflows/tests.yml
3. Run 'npm i -g npm@8'
4. See error:

2022-01-28T04:02:14.0117198Z npm ERR! code EPERM
2022-01-28T04:02:14.0117959Z npm ERR! syscall rename
2022-01-28T04:02:14.0119521Z npm ERR! path C:\npm\prefix\node_modules\npm\node_modules\.node-gyp.DELETE\node_modules\@tootallnate
2022-01-28T04:02:14.0121582Z npm ERR! dest C:\npm\prefix\node_modules\npm\node_modules\node-gyp\node_modules\@tootallnate
2022-01-28T04:02:14.0123300Z npm ERR! errno -4048
2022-01-28T04:02:14.0163074Z npm ERR! Error: EPERM: operation not permitted, rename 'C:\npm\prefix\node_modules\npm\node_modules\.node-gyp.DELETE\node_modules\@tootallnate' -> 'C:\npm\prefix\node_modules\npm\node_modules\node-gyp\node_modules\@tootallnate'
2022-01-28T04:02:14.0165302Z npm ERR!  [OperationalError: EPERM: operation not permitted, rename 'C:\npm\prefix\node_modules\npm\node_modules\.node-gyp.DELETE\node_modules\@tootallnate' -> 'C:\npm\prefix\node_modules\npm\node_modules\node-gyp\node_modules\@tootallnate'] {
2022-01-28T04:02:14.0167111Z npm ERR!   cause: [Error: EPERM: operation not permitted, rename 'C:\npm\prefix\node_modules\npm\node_modules\.node-gyp.DELETE\node_modules\@tootallnate' -> 'C:\npm\prefix\node_modules\npm\node_modules\node-gyp\node_modules\@tootallnate'] {
2022-01-28T04:02:14.0167983Z npm ERR!     errno: -4048,
2022-01-28T04:02:14.0168501Z npm ERR!     code: 'EPERM',
2022-01-28T04:02:14.0169020Z npm ERR!     syscall: 'rename',
2022-01-28T04:02:14.0170019Z npm ERR!     path: 'C:\\npm\\prefix\\node_modules\\npm\\node_modules\\.node-gyp.DELETE\\node_modules\\@tootallnate',
2022-01-28T04:02:14.0171280Z npm ERR!     dest: 'C:\\npm\\prefix\\node_modules\\npm\\node_modules\\node-gyp\\node_modules\\@tootallnate'
2022-01-28T04:02:14.0174531Z npm ERR!   },
2022-01-28T04:02:14.0175411Z npm ERR!   errno: -4048,
2022-01-28T04:02:14.0176225Z npm ERR!   code: 'EPERM',
2022-01-28T04:02:14.0176835Z npm ERR!   syscall: 'rename',
2022-01-28T04:02:14.0177871Z npm ERR!   path: 'C:\\npm\\prefix\\node_modules\\npm\\node_modules\\.node-gyp.DELETE\\node_modules\\@tootallnate',
2022-01-28T04:02:14.0179144Z npm ERR!   dest: 'C:\\npm\\prefix\\node_modules\\npm\\node_modules\\node-gyp\\node_modules\\@tootallnate',
2022-01-28T04:02:14.0179809Z npm ERR!   parent: 'npm'
2022-01-28T04:02:14.0180347Z npm ERR! }
2022-01-28T04:02:14.0180781Z npm ERR! 
2022-01-28T04:02:14.0181378Z npm ERR! The operation was rejected by your operating system.
2022-01-28T04:02:14.0182244Z npm ERR! It's possible that the file was already in use (by a text editor or antivirus),
2022-01-28T04:02:14.0182979Z npm ERR! or that you lack permissions to access it.
2022-01-28T04:02:14.0183459Z npm ERR! 
2022-01-28T04:02:14.0184264Z npm ERR! If you believe this might be a permissions issue, please double-check the
2022-01-28T04:02:14.0185116Z npm ERR! permissions of the file and its containing directories, or try running
2022-01-28T04:02:14.0185804Z npm ERR! the command again as root/Administrator.
2022-01-28T04:02:14.0563679Z 
2022-01-28T04:02:14.0565244Z npm ERR! A complete log of this run can be found in:
2022-01-28T04:02:14.0566097Z npm ERR!     C:\npm\cache\_logs\2022-01-28T04_02_14_018Z-debug.log
2022-01-28T04:02:15.2381520Z ##[error]Process completed with exit code 1.

Environment

• npm: 8..4.0
• Node.js: saw it happening in latest 12/14
• OS Name: win32
• System Model Name: container in GitHub Actions
• npm config: <it's CI, so a bit harder to just execute commands>

[AviVahl]

Just to clarify, this worked up until yesterday's release.

[sirber]

8.4.0 is broken for me too, general use on Windows (ntfs). Same node-gyp error. 8.3.x works though.

[aslushnikov]

This is affecting us as well at http://github.com/microsoft/playwright

We'll downgrade to 8.3 for now.

[ChristopherHaws]

This effects our team as well, downgrading to 8.3.x fixed the issue for us :(

[Linkgoron]

This also happens for us on CircleCI windows orbs as well.

[nlf]

we've seen this happen too. there's a bug in npm versions prior to 7.5.4 that causes some complications when doing a global update of npm in Windows. while it's strange that this didn't occur with [email protected] but does with [email protected], starting with a baseline of [email protected] or newer resolves the issue.

we've worked around this for our own CI by manually updating npm to 7.5.4 in windows before we attempt to update to npm@latest
npm/template-oss#36

i'm going to do some investigation and see if i can determine what exactly it was in 8.4.0 that changed that caused this breakage to rear its ugly head again, but in the mean time updating to any version newer than 7.5.4 that will install for you should allow you to update to the latest release without issue.

[Linkgoron]

we've seen this happen too. there's a bug in npm versions prior to 7.5.4 that causes some complications when doing a global update of npm in Windows. while it's strange that this didn't occur with [email protected] but does with [email protected], starting with a baseline of [email protected] or newer resolves the issue.

we've worked around this for our own CI by manually updating npm to 7.5.4 in windows before we attempt to update to npm@latest npm/template-oss#36

i'm going to do some investigation and see if i can determine what exactly it was in 8.4.0 that changed that caused this breakage to rear its ugly head again, but in the mean time updating to any version newer than 7.5.4 that will install for you should allow you to update to the latest release without issue.

I think that what caused this to pop-up is that the cli is now dependant on both @tootallnate/once version 1 and version 2 (because of [email protected])

[sirber]

with 8.5.0, it works on my main project. I use "npm ci" to clean and install my packages.

sirbe@sirber-e5470 MINGW64 ~/Projects/scms/scms/webapps/Scms (develop)
$ npm i npm@latest -g

removed 176 packages, changed 14 packages, and audited 39 packages in 6s

1 moderate severity vulnerability

To address all issues, run:
  npm audit fix

Run `npm audit` for details.

sirbe@sirber-e5470 MINGW64 ~/Projects/scms/scms/webapps/Scms (develop)
$ npm ci
npm WARN deprecated @types/[email protected]: This is a stub types definition for error-stack-parser (https://github.com/stacktracejs/error-stack-parser). error-stack-parser provides its own type definitions, so you don't need @types/error-stack-parser installed!
npm WARN deprecated [email protected]: See https://github.com/lydell/source-map-url#deprecated
npm WARN deprecated [email protected]: Please see https://github.com/lydell/urix#deprecated
npm WARN deprecated [email protected]: See https://github.com/lydell/source-map-resolve#deprecated
npm WARN deprecated [email protected]: https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated [email protected]: gulp-util is deprecated - replace it, following the guidelines at https://medium.com/gulpjs/gulp-util-ca3b1f9f9ac5
npm WARN deprecated [email protected]: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.

added 1017 packages, and audited 1021 packages in 30s

78 packages are looking for funding
  run `npm fund` for details

12 vulnerabilities (6 moderate, 3 high, 3 critical)

To address issues that do not require attention, run:
  npm audit fix

Some issues need review, and may require choosing
a different dependency.

Run `npm audit` for details.

sirbe@sirber-e5470 MINGW64 ~/Projects/scms/scms/webapps/Scms (develop)
$ npm run dev

> [email protected] dev
> webpack --mode=development --config=./webpack-config.js

assets by path *.js 8.64 MiB
  asset vendors.js 5.28 MiB [emitted] (name: vendors) (id hint: vendors)
  asset bundle.js 1.75 MiB [emitted]
  asset app.js 1.61 MiB [emitted] (name: app) 1 related asset
asset app.css 255 KiB [emitted] (auxiliary name: app)
asset mix-manifest.json 256 bytes [emitted]
Entrypoint app 6.89 MiB (1.71 MiB) = vendors.js 5.28 MiB app.js 1.61 MiB 2 auxiliary assets
runtime modules 4.47 KiB 9 modules
modules by path ./node_modules/ 4.57 MiB 807 modules
modules by path ./Scripts/ 929 KiB 322 modules
modules by path *.js$/ 7.04 KiB
  ./Scripts/components/ sync \.js$/ 6.68 KiB [built] [code generated]
  ./Scripts/bindings/ sync \.js$/ 363 bytes [built] [code generated]
./Styles/sass/app.sass 51 bytes [built] [code generated]
./Definitions/Traductions.csv 192 KiB [built] [code generated]
webpack 5.66.0 compiled successfully in 13439 ms

[danielhjacobs]

@sirber What version of npm are you starting with? As mentioned in #4341 (comment), starting with a baseline of [email protected] or newer resolves the issue, but the issue still exists in version 8.5 from what I've seen in https://github.com/ruffle-rs/ruffle/runs/5199647826?check_suite_focus=true.

[AviVahl]

@sirber does it also work for you when executed via powershell/cmd? Native win32 path handling is usually something that breaks, and we wouldn't see that when running in an emulated posix-like env (where the Windows paths are exposed as posix ones).

Also, it breaks in CIs that have Node 12/14 preinstalled with npm@6. It might specific to people moving from npm6 -> npm8 on Windows.

The error message appears to be pointing to mishandling node-gyp -> http-proxy-agent -> @tootallnate/once. AFAICT, http-proxy-agent uses v1 while npm itself upgraded to v2. Maybe it's the bundled dependencies behavior that blows up? Maybe @npmcli/arborist is acting up?

[sirber]

@sirber What version of npm are you starting with? As mentioned in #4341 (comment), starting with a baseline of [email protected] or newer resolves the issue, but the issue still exists in version 8.5 from what I've seen in https://github.com/ruffle-rs/ruffle/runs/5199647826?check_suite_focus=true.

8.3.x: works
8.4.0 fails
8.5.0 works

[sirber]

@sirber does it also work for you when executed via powershell/cmd? Native win32 path handling is usually something that breaks, and we wouldn't see that when running in an emulated posix-like env (where the Windows paths are exposed as posix ones).

I'm with git bash, on windows 10.

Here's with powershell:

PS C:\Users\sirbe\Projects\scms\scms\webapps\Scms> npm i npm@latest -g

removed 176 packages, changed 14 packages, and audited 39 packages in 6s

1 moderate severity vulnerability

To address all issues, run:
  npm audit fix

Run `npm audit` for details.
PS C:\Users\sirbe\Projects\scms\scms\webapps\Scms> npm --version
8.5.0
PS C:\Users\sirbe\Projects\scms\scms\webapps\Scms> npm ci
npm WARN deprecated @types/[email protected]: This is a stub types definition for error-stack-parser (https://github.com/stacktracejs/error-stack-parser). error-stack-parser provides its own type definitions, so you don't need @types/error-stack-parser installed!
npm WARN deprecated [email protected]: See https://github.com/lydell/source-map-url#deprecated
npm WARN deprecated [email protected]: Please see https://github.com/lydell/urix#deprecated
npm WARN deprecated [email protected]: See https://github.com/lydell/source-map-resolve#deprecated
npm WARN deprecated [email protected]: https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated [email protected]: gulp-util is deprecated - replace it, following the guidelines at https://medium.com/gulpjs/gulp-util-ca3b1f9f9ac5
npm WARN deprecated [email protected]: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.

added 1017 packages, and audited 1021 packages in 32s

78 packages are looking for funding
  run `npm fund` for details

12 vulnerabilities (6 moderate, 3 high, 3 critical)

To address issues that do not require attention, run:
  npm audit fix

Some issues need review, and may require choosing
a different dependency.

Run `npm audit` for details.
PS C:\Users\sirbe\Projects\scms\scms\webapps\Scms> npm run dev

> [email protected] dev
> webpack --mode=development --config=./webpack-config.js

assets by path *.js 8.64 MiB
  asset vendors.js 5.28 MiB [compared for emit] (name: vendors) (id hint: vendors)
  asset bundle.js 1.75 MiB [compared for emit]
  asset app.js 1.61 MiB [compared for emit] (name: app) 1 related asset
asset app.css 255 KiB [compared for emit] (auxiliary name: app)
asset mix-manifest.json 256 bytes [emitted] [compared for emit]
Entrypoint app 6.89 MiB (1.71 MiB) = vendors.js 5.28 MiB app.js 1.61 MiB 2 auxiliary assets
runtime modules 4.47 KiB 9 modules
modules by path ./node_modules/ 4.57 MiB 807 modules
modules by path ./Scripts/ 929 KiB 322 modules
modules by path *.js$/ 7.04 KiB
  ./Scripts/components/ sync \.js$/ 6.68 KiB [built] [code generated]
  ./Scripts/bindings/ sync \.js$/ 363 bytes [built] [code generated]
./Styles/sass/app.sass 51 bytes [built] [code generated]
./Definitions/Traductions.csv 192 KiB [built] [code generated]
webpack 5.66.0 compiled successfully in 13090 ms

[AviVahl]

Just checked 8.5.0. Still failing with same error:
https://github.com/wixplosives/mocha-play/runs/5204402271?check_suite_focus=true

[sirber]

What if you completly remove "C:\npm\prefix\node_modules" ?

[nlf]

To restate the things we know:

Updating npm globally in Windows using a command such as npm install --global npm when starting with an npm version below 7.5.4 is known to fail for some releases in the 7.x and 8.x ranges.

This bug has been fixed as of npm 7.5.4. We do not intend to back port a fix to npm 6 at this time.

nodejs ships npm@6 with node versions 12 and 14. this version is known to fail for global updates to some 7.x and 8.x releases for our Windows users.

To correct the problem, you must update your current npm release to a minimum of version 7.5.4 before attempting to install the latest version. this can be done in a few ways, but the recommended ones are:

1. Update to nodejs@16, which includes npm@8 and is not affected by this bug
2. Manually update npm to a minimum of version 7.5.4 before attempting to update with npm itself.

A manual update can be performed with the following steps:

1. Download https://registry.npmjs.org/npm/-/npm-7.5.4.tgz
2. Extract the file downloaded above, this step will vary depending on your environment
3. After extraction, overwrite your current npm installation with the contents of the package directory.

To locate the current npm installation if npm is currently working run npm prefix -g, append node_modules\npm to the result.

If npm is not working, run where npm this should give you two results found in the same directory (for example, on my Windows laptop using nvs I get the results C:\Users\username\AppData\Local\nvs\default\npm and C:\Users\username\AppData\Local\nvs\default\npm.cmd). Remove the filename from the first path, and append node_modules\npm. In my example the result would be C:\Users\username\AppData\Local\nvs\default\node_modules\npm

Copy the entire contents of the package directory from the extracted tarball to this directory. Run npm -v to confirm you are now on version 7.5.4. You may now update to the latest npm release with npm install --global npm.

[ybiquitous]

@nlf Thanks for the detailed explanation. But I have a question.

Currently, the windows-latest image includes [email protected]. Even in this case, does npm install -g npm fail?

[Linkgoron]

To restate the things we know:

Updating npm globally in Windows using a command such as npm install --global npm when starting with an npm version below 7.5.4 is known to fail for some releases in the 7.x and 8.x ranges.

This bug has been fixed as of npm 7.5.4. We do not intend to back port a fix to npm 6 at this time.

nodejs ships npm@6 with node versions 12 and 14. this version is known to fail for global updates to some 7.x and 8.x releases for our Windows users.

To correct the problem, you must update your current npm release to a minimum of version 7.5.4 before attempting to install the latest version. this can be done in a few ways, but the recommended ones are:

1. Update to nodejs@16, which includes npm@8 and is not affected by this bug
2. Manually update npm to a minimum of version 7.5.4 before attempting to update with npm itself.

A manual update can be performed with the following steps:

1. Download https://registry.npmjs.org/npm/-/npm-7.5.4.tgz
2. Extract the file downloaded above, this step will vary depending on your environment
3. After extraction, overwrite your current npm installation with the contents of the package directory.

To locate the current npm installation if npm is currently working run npm prefix -g, append node_modules\npm to the result.

If npm is not working, run where npm this should give you two results found in the same directory (for example, on my Windows laptop using nvs I get the results C:\Users\username\AppData\Local\nvs\default\npm and C:\Users\username\AppData\Local\nvs\default\npm.cmd). Remove the filename from the first path, and append node_modules\npm. In my example the result would be C:\Users\username\AppData\Local\nvs\default\node_modules\npm

Copy the entire contents of the package directory from the extracted tarball to this directory. Run npm -v to confirm you are now on version 7.5.4. You may now update to the latest npm release with npm install --global npm.

There is a degradation here though, after 8.4. With 8.3 I could install a package globally (which is not NPM) even when it had a dependency on npm - with version 8.4 I can't do that anymore.

[SnoopyDevelops]

To restate the things we know:

Updating npm globally in Windows using a command such as npm install --global npm when starting with an npm version below 7.5.4 is known to fail for some releases in the 7.x and 8.x ranges.

This bug has been fixed as of npm 7.5.4. We do not intend to back port a fix to npm 6 at this time.

nodejs ships npm@6 with node versions 12 and 14. this version is known to fail for global updates to some 7.x and 8.x releases for our Windows users.

To correct the problem, you must update your current npm release to a minimum of version 7.5.4 before attempting to install the latest version. this can be done in a few ways, but the recommended ones are:

1. Update to nodejs@16, which includes npm@8 and is not affected by this bug
2. Manually update npm to a minimum of version 7.5.4 before attempting to update with npm itself.

A manual update can be performed with the following steps:

1. Download https://registry.npmjs.org/npm/-/npm-7.5.4.tgz
2. Extract the file downloaded above, this step will vary depending on your environment
3. After extraction, overwrite your current npm installation with the contents of the package directory.

To locate the current npm installation if npm is currently working run npm prefix -g, append node_modules\npm to the result.

If npm is not working, run where npm this should give you two results found in the same directory (for example, on my Windows laptop using nvs I get the results C:\Users\username\AppData\Local\nvs\default\npm and C:\Users\username\AppData\Local\nvs\default\npm.cmd). Remove the filename from the first path, and append node_modules\npm. In my example the result would be C:\Users\username\AppData\Local\nvs\default\node_modules\npm

Copy the entire contents of the package directory from the extracted tarball to this directory. Run npm -v to confirm you are now on version 7.5.4. You may now update to the latest npm release with npm install --global npm.

After copying the entire contents of the package directory, running npm -v to confirms I am now on version 7.5.4.
But after that, trying to update with npm install --global npm results the error of Cannnot find module 'retry'

PS C:\Users\Administrator> npm -v
7.5.4
PS C:\Users\Administrator> npm list -g
C:\Program Files\nodejs -> C:\Users\Administrator\AppData\Roaming\nvm\v14.21.1
`-- [email protected]

PS C:\Users\Administrator> npm install -g npm

removed 489 packages, changed 2 packages, and audited 28 packages in 3s

found 0 vulnerabilities
PS C:\Users\Administrator> npm -v
C:\Users\Administrator\AppData\Roaming\nvm\v14.21.1\node_modules\npm\lib\cli.js:48
    throw err
    ^

Error: Cannot find module 'retry'