npm hangs on Node 6.4.0

[NullDivision]

When running npm i the process hangs with message:

npm iextract:webpack: sill doParallel extract 1000

[mattkime]

which OS? whats does your package.json look like? this is likely due to your setup.

[NullDivision]

Happens an Arch Linux with all packages up to date. Install works fine if I install each package separately but not when installing all of them.

Rolled back to version 6.3.1 and now install works fine.

[codejunkienick]

I can confirm that. I am also having this problem on archlinux but with node 6.4

[NullDivision]

Today I tried doing another update, this time got Node 6.5.0.

Output:

npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: lodash@<3.0.0 is no longer maintained. Upgrade to lodash@^4.0.0.
npm WARN deprecated [email protected]: graceful-fs v3.0.0 and before will fail on node releases >= v7.0. Please update to graceful-fs@^4.0.0 as soon as possible. Use 'npm ls graceful-fs' to find it in the tree.
npm WARN deprecated [email protected]: cross-spawn no longer requires a build toolchain, use it instead!
[             .....] / extract:webpack-dev-server: sill doParallel extract 1112

Version output:

/home/nulldivision>node -v && npm -v
v6.5.0
3.10.6

EDIT: Also tried clearing NPM cache but it made no difference.

[leafi]

I'm also experiencing the same issue, same distro, same versions.

$ npm i
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
          .....] | extract:webpack-validator: sill doParallel extract 914

Npm sits there pegging a core to 100%. I've tried leaving it overnight to no avail.

[leaf@unicorn ~]$ node -v && npm -v
v6.5.0
3.10.6

[NullDivision]

Ok. So far we have confirmation on Arch. Someone should confirm this on another OS/distro.

[addaleax]

Npm sits there pegging a core to 100%.

Can you try attaching a debugger to the process when it’s stuck? gdb -p pid & running backtrace inside gdb should give a proper stack trace, and the output of strace -p pid might also be helpful.

[iscekic]

Same issue, also on Arch.

node --version
v6.5.0
npm --version
3.10.6

My package.json: https://github.com/Torwori/ts-mobx-react-starter-kit/blob/update-deps/package.json

The "hanging":

[             .....] | extract:webpack-dev-server: sill doParallel extract 886

gdb backtrace output:

(gdb) backtrace
#0  0x00007f488a5c9171 in write () from /usr/lib/libpthread.so.0
#1  0x00007f488a9eff85 in ?? () from /usr/lib/libuv.so.1
#2  0x00007f488a9f041f in ?? () from /usr/lib/libuv.so.1
#3  <signal handler called>
#4  0x0000000000989f95 in v8::internal::FieldType::AsClass() ()
#5  0x000000000098a0f4 in v8::internal::FieldType::Convert(v8::internal::Zone*) ()
#6  0x0000000000787c8b in v8::internal::compiler::AccessInfoFactory::ComputePropertyAccessInfo(v8::internal::Handle<v8::internal::Map>, v8::internal::Handle<v8::internal::Name>, v8::internal::compiler::AccessMode, v8::internal::compiler::PropertyAccessInfo*) ()
#7  0x000000000078831e in v8::internal::compiler::AccessInfoFactory::ComputePropertyAccessInfos(v8::internal::List<v8::internal::Handle<v8::internal::Map>, v8::internal::FreeStoreAllocationPolicy> const&, v8::internal::Handle<v8::internal::Name>, v8::internal::compiler::AccessMode, v8::internal::ZoneVector<v8::internal::compiler::PropertyAccessInfo>*) ()
#8  0x00000000007f9566 in v8::internal::compiler::JSNativeContextSpecialization::ReduceNamedAccess(v8::internal::compiler::Node*, v8::internal::compiler::Node*, v8::internal::List<v8::internal::Handle<v8::internal::Map>, v8::internal::FreeStoreAllocationPolicy> const&, v8::internal::Handle<v8::internal::Name>, v8::internal::compiler::AccessMode, v8::internal::LanguageMode, v8::internal::compiler::Node*) ()
#9  0x00000000007fb3af in v8::internal::compiler::JSNativeContextSpecialization::ReduceNamedAccess(v8::internal::compiler::Node*, v8::internal::compiler::Node*, v8::internal::FeedbackNexus const&, v8::internal::Handle<v8::internal::Name>, v8::internal::compiler::AccessMode, v8::internal::LanguageMode) ()
#10 0x00000000007fb43f in v8::internal::compiler::JSNativeContextSpecialization::ReduceJSLoadNamed(v8::internal::compiler::Node*) ()
#11 0x00000000007fda15 in v8::internal::compiler::JSNativeContextSpecialization::Reduce(v8::internal::compiler::Node*) ()
#12 0x00000000007ca506 in v8::internal::compiler::GraphReducer::Reduce(v8::internal::compiler::Node*) ()
#13 0x00000000007caf5c in v8::internal::compiler::GraphReducer::ReduceTop() ()
#14 0x00000000007cb088 in v8::internal::compiler::GraphReducer::ReduceNode(v8::internal::compiler::Node*) ()
#15 0x00000000008325ed in v8::internal::compiler::Pipeline::GenerateCode() ()
#16 0x00000000008895a2 in v8::internal::OptimizedCompileJob::CreateGraph() ()
#17 0x000000000088a945 in ?? ()
#18 0x000000000088b6d9 in v8::internal::Compiler::CompileOptimized(v8::internal::Handle<v8::internal::JSFunction>, v8::internal::Compiler::ConcurrencyMode) ()
#19 0x0000000000b63e05 in v8::internal::Runtime_CompileOptimized_Concurrent(int, v8::internal::Object**, v8::internal::Isolate*) ()
#20 0x000024abc6e092a7 in ?? ()
#21 0x000024abc6e091e1 in ?? ()
#22 0x00007ffc39673740 in ?? ()
#23 0x0000000300000000 in ?? ()
#24 0x00007ffc39673798 in ?? ()
#25 0x000024abc6e3ba38 in ?? ()
#26 0x000037f4771a6cd9 in ?? ()
#27 0x000025b36eb04381 in ?? ()
#28 0x000037f4771a6cd9 in ?? ()
#29 0x0000000000000000 in ?? ()

strace output:

--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x38} ---
read(3, "*", 1)                         = 1
write(7, "X\3072\2\0\0\0\0\v\0\0\0\0\0\0\0", 16) = -1 EAGAIN (Resource temporarily unavailable)
write(4, "*", 1)                        = 1
rt_sigreturn({mask=[]})                 = 0
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x38} ---
read(3, "*", 1)                         = 1
write(7, "X\3072\2\0\0\0\0\v\0\0\0\0\0\0\0", 16) = -1 EAGAIN (Resource temporarily unavailable)
write(4, "*", 1)                        = 1
rt_sigreturn({mask=[]})                 = 0
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x38} ---
... and so on

[addaleax]

@Torwori That looks like something different… your process doesn’t hang with 100 % CPU, does it?

Your problem looks like nodejs/node#8310 (where more information is certainly likely to be useful, too) … I assume that only occurs since Node v6.5.0?

[iscekic]

@addaleax It uses 100%. I left it running for an hour and then had to SIGKILL it.

I'll try Node v6.4.0 and let you know.

[addaleax]

@Torwori oh, that seems like useful information on why the process hangs then… would be really good to know if the same stack trace appears with v6.4.0.

Also, do you happen to have anything installed that handles segmentation faults? Can you run your npm install command wrapped in strace, like this: strace -fe execve npm install?

I really appreciate your help with digging into this!

[iscekic]

@addaleax Just tried npm install with Node v6.4.0, works fine.

Also, do you happen to have anything installed that handles segmentation faults?

Not that I know of.

strace -fe execve npm install (Node v6.5.0) output:

execve("/usr/bin/npm", ["npm", "install"], [/* 46 vars */]) = 0
execve("/usr/local/sbin/node", ["node", "/usr/bin/npm", "install"], [/* 46 vars */]) = -1 ENOENT (No such file or directory)
execve("/usr/local/bin/node", ["node", "/usr/bin/npm", "install"], [/* 46 vars */]) = -1 ENOENT (No such file or directory)
execve("/usr/bin/node", ["node", "/usr/bin/npm", "install"], [/* 46 vars */]) = 0
strace: Process 3227 attached
strace: Process 3226 attached
strace: Process 3225 attached
strace: Process 3224 attached
strace: Process 3223 attached
strace: Process 3231 attached
strace: Process 3230 attached
strace: Process 3229 attached
strace: Process 3228 attached
[pid  3222] --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x38} ---
[pid  3222] --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x38} ---
[pid  3222] --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x38} ---
[pid  3222] --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x38} ---
... until I stopped it

[addaleax]

So… repeated SIGSEGVs usually mean that something is trying to handle the segmentation fault (which would otherwise just crash the program) and tries to continue executing… I’ll try to reproduce this myself, maybe it’ll “work” in a VM.

A more or less brute-force way to inspect the situation would be looking at the disassembly in one of the stack frames above the signal handler (the #2 0x00007f488a9f041f in ?? () from /usr/lib/libuv.so.1 lines from your gdb output)… if you don’t mind, you could run

(gdb) frame 2
(gdb) disassemble $rip-40,$rip+40

which shows what exactly is being executed after each SIGSEGV.

[iscekic]

@addaleax Output:

Dump of assembler code from 0x7fe70e3bf308 to 0x7fe70e3bf358:
   0x00007fe70e3bf308:  xor    0x28,%rsi
   0x00007fe70e3bf310:  jne    0x7fe70e3bf329
   0x00007fe70e3bf312:  add    $0xa8,%rsp
   0x00007fe70e3bf319:  retq   
   0x00007fe70e3bf31a:  nopw   0x0(%rax,%rax,1)
   0x00007fe70e3bf320:  mov    %rdx,%rax
   0x00007fe70e3bf323:  jmp    0x7fe70e3bf2f4
   0x00007fe70e3bf325:  xor    %eax,%eax
   0x00007fe70e3bf327:  jmp    0x7fe70e3bf2ff
   0x00007fe70e3bf329:  callq  0x7fe70e3b33f0 <__stack_chk_fail@plt>
   0x00007fe70e3bf32e:  xchg   %ax,%ax
=> 0x00007fe70e3bf330:  push   %r14
   0x00007fe70e3bf332:  push   %r13
   0x00007fe70e3bf334:  mov    %edi,%r13d
   0x00007fe70e3bf337:  push   %r12
   0x00007fe70e3bf339:  push   %rbp
   0x00007fe70e3bf33a:  push   %rbx
   0x00007fe70e3bf33b:  sub    $0x20,%rsp
   0x00007fe70e3bf33f:  mov    %fs:0x28,%rax
   0x00007fe70e3bf348:  mov    %rax,0x18(%rsp)
   0x00007fe70e3bf34d:  xor    %eax,%eax
   0x00007fe70e3bf34f:  callq  0x7fe70e3b3160 <__errno_location@plt>
   0x00007fe70e3bf354:  movq   $0x0,(%rsp)
End of assembler dump.

[NullDivision]

@addaleax Since I reported it with title "6.4.0", yes. The issue started in Node 6.4.0.

[addaleax]

@NullDivision Sorry, didn’t mean to sound like I doubted you. My question was directed at @Torwori … his stacktrace points to a possible failure in V8, which is extensively updated in Node v6.5.0, so that seemed a natural question to ask.

The 100 % CPU usage in your case might very well have the same cause – strace -fe execve npm install should verify that for you, too. The underlying segmentation fault itself would probably be different in your case, though.

Either of you… this looks like something is calling into libuv’s uv_process_kill during the segfault. Is there any chance the segfault-handler package is installed and loaded while npm is running? That seems like a reasonable candidate for causing problems here…

[iscekic]

@addaleax I don't have that package installed globally.

[addaleax]

Okay … puh. One last thing I can suggest to try before I go to bed: ldd $(which node) to see which native libraries are loaded when node is running.

btw, I’ve set up an Arch Linux VM and installed the current versions of node & npm, but couldn’t reproduce anything so far, unfortunately…

[moki]

@NullDivision Had same issue after upgrading to 6.5, rolling back to 6.4 solves the issue.

[iscekic]

@addaleax

ldd $(which node)
    linux-vdso.so.1 (0x00007ffdb753b000)
    libz.so.1 => /usr/lib/libz.so.1 (0x00007fc424ce7000)
    libhttp_parser.so.2.7.1 => /usr/lib/libhttp_parser.so.2.7.1 (0x00007fc424adf000)
    libuv.so.1 => /usr/lib/libuv.so.1 (0x00007fc4248bb000)
    librt.so.1 => /usr/lib/librt.so.1 (0x00007fc4246b3000)
    libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007fc424496000)
    libdl.so.2 => /usr/lib/libdl.so.2 (0x00007fc424292000)
    libssl.so.1.0.0 => /usr/lib/libssl.so.1.0.0 (0x00007fc424021000)
    libcrypto.so.1.0.0 => /usr/lib/libcrypto.so.1.0.0 (0x00007fc423baa000)
    libicui18n.so.57 => /usr/lib/libicui18n.so.57 (0x00007fc423730000)
    libicuuc.so.57 => /usr/lib/libicuuc.so.57 (0x00007fc423388000)
    libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007fc423000000)
    libm.so.6 => /usr/lib/libm.so.6 (0x00007fc422cfc000)
    libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007fc422ae5000)
    libc.so.6 => /usr/lib/libc.so.6 (0x00007fc422747000)
    libnsl.so.1 => /usr/lib/libnsl.so.1 (0x00007fc42252f000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fc424efd000)
    libicudata.so.57 => /usr/lib/libicudata.so.57 (0x00007fc420ab3000)

[addaleax]

That looks, unfortunately, pretty normal … I’m headed for sleep now, but thanks everyone for their time so far. Maybe I’ll be able to reproduce some of this at some of these problems, and maybe someone else comes up with ideas… If you think you have a way to make this problem occur reliably on other Arch machines, let me know.

[leafi]

It might be related to the project itself.

I've set up an Arch Linux VM on DigitalOcean with a pretty small project (empty except package.json) that exhibits the issue. I'll attempt to Direct Message you on Twitter with login details.

Otherwise, here's a .zip containing a package.json where doing 'npm i' causes the problem.

failproj.zip

[addaleax]

Ok, so, thanks to @leafi’s server access, I know at least now why npm doesn’t segfault properly… a nested dependency of npm, signal-exit, registers a handler for SIGSEGV on the JS side. Commenting out the corresponding line in /usr/lib/node_modules/npm/node_modules/npmlog/node_modules/gauge/node_modules/signal-exit/signals.js makes npm crash properly.

Ping @bcoe … why does signal-exit listen to SIGSEGV, SIGILL, SIGBUS? That doesn’t sound like a terribly good idea to me – this bug demonstrates quite well that these events leave the application in an undefined state, and in my experience almost certainly not a state that’s good enough to (try to) perform calls into the JS runtime.

[addaleax]

I’m pretty sure this is an actual V8 bug, see nodejs/node#8310 (comment). For now, everyone here should be good by waiting a couple of hours or so until the official Node.js binaries are out, or using Node 6.4.0 (which seems to only reduce the frequency of the bug, not eliminate it), or compiling 6.5.0 from the sources using gcc < 6.

The symptom of hanging when encountering a SIGSEGV is definitely an npm problem, though.

[othiym23]

I'm content to let this conversation play out between @addaleax (thank you so much for your help here!) and @bcoe -- I know there are sensible reasons for what signal-exit does in the context of tap; it would be interesting to hear from @iarna why gauge is using signal-exit, and if that's something that should be used for every npm call. Or, perhaps, whether signal-exit should be made semi-configurable so that it stops trying to trap fatal signals in some cases. Iono?

[gabx]

@addaleax I can confirm running the mentioned line in my terminal fixed this issue. Arch and node 6.5. I was able to run $ npm install at least.
Thank you for this fix

[addaleax]

Cool! Also, Node v6.6.0 is out, the bug has been fixed in that release – You can just use the normal Arch binaries again!

@othiym23 I’d say this can be closed. =)

[gabx]

archlinux
% node -v
v6.6.0
%make
......
no issue till the end.
I think this issue can be closed, yes

[arnaudoff]

Had this issue with node v6.5.0 and npm 3.10.7, I confirm that upgrading to v6.6.0 solved it on my ArchLinux system.

[paklei]

Same issue with
uname -a

Linux archpc 4.7.2-1-ARCH #1 SMP PREEMPT Sat Aug 20 23:02:56 CEST 2016 x86_64 GNU/Linux

node -v

v6.5.0

npm -v

3.10.8

After update the system that problem was solved

node -v && npm -v
v6.6.0
3.10.8

[NullDivision]

Tested with v6.6.0. Issue is gone. We can mark it as closed I think. Thank you to the node team for their hard work.

[kenany]

Closing as resolved :)

[erquhart]

@kenany I'm seeing this same issue running Node 6.6.0 on Ubuntu 14.04 via Vagrant/VirtualBox on a Mac. Same silent failure every time npm install runs:

extract:time-grunt: sill doParallel extract 1226

Any chance it's related?

[addaleax]

@erquhart Hm, it would be very unlikely. Can you give more information, like:

• npm -v
• node -p process.versions
• Where did you get the Node.js executable from? Did you compile it yourself and if so, with that compiler?
• Does the process terminate when Ctrl+C is pressed?
• If no: Can you try do do strace -fe execve npm install and put the output into a gist?

(edit: to clarify, these questions are somewhat laid out to verify whether this is the same bug or not)

[erquhart]

@addaleax installed from Debian NodeSource using the package manager instructions on nodejs.org.

The process doesn't hang, in case you were thinking that, but I am able to terminate via ctrl+c.

$ npm -v && node -p process.versions
3.10.3
{ http_parser: '2.7.0',
  node: '6.6.0',
  v8: '5.1.281.83',
  uv: '1.9.1',
  zlib: '1.2.8',
  ares: '1.10.1-DEV',
  icu: '56.1',
  modules: '48',
  openssl: '1.0.2h' }

[addaleax]

Yeah, that’s something unrelated then. I guess the best thing to do is update npm (npm install -g npm@next) and verify that it’s still a problem, then open a new issue here?

[erquhart]

@addaleax one guess what error I got while attempting to install the latest npm lol. Opening a new issue. Thanks for your help!

[spsinghats]

I am getting this issue on node 6.9.4, npm 3.10.10,

[alchialchi]

same on 7.5

[leodelasoul]

What helped for me is that i made extra swap and ram space for my virtual machine so the extracting process would not be killed.

[lwmonster]

So, what is the final solution?

Still have the problem on:

npm -v
3.10.10
node -v
v6.10.2

[addaleax]

@lwmonster Can you update npm to the latest version ([sudo] npm install -g npm@latest) and check whether the problem persists? If it does, please open a new issue here (and feel free to @mention me in it). The problem that this particular issue is about has been fixed a while ago.

[lwmonster]

@addaleax Still the same problem:

npm install
npm WARN deprecated [email protected]: wrench.js is deprecated! You should check out fs-extra (https://github.com/jprichardson/node-fs-extra) for any operations you were using wrench for. Thanks for all the usage over the years.
npm WARN deprecated [email protected]: Use uuid module instead
已杀死           ..] / extract:mongoomise: sill gunzTarPerm extractEntry fiveby-config.json

[root@localhost m-mall-admin]# npm -v
4.4.4
[root@localhost m-mall-admin]# node -v
v6.10.2

OS version:
cat /proc/version
Linux version 2.6.32-042stab116.1 ([email protected]) (gcc version 4.4.6 20120305 (Red Hat 4.4.6-4) (GCC) ) #1 SMP Wed May 4 16:21:02 MSK 2016

[cuihairu]

$ npm -v && node -p process.versions

3.10.10
{ http_parser: '2.7.0',
  node: '6.11.2',
  v8: '5.1.281.103',
  uv: '1.11.0',
  zlib: '1.2.11',
  ares: '1.10.1-DEV',
  icu: '58.2',
  modules: '48',
  openssl: '1.0.2l' }

[azarus]

@NullDivision I still experience this problem on Centos7
Hardware: VPS 512MB 20GB disk space

$ npm install
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: graceful-fs v3.0.0 and before will fail on node releases >= v7.0. Please update to graceful-fs@^4.0.0 as soon as possible. Use 'npm ls graceful-fs' to find it in the tree.
npm WARN deprecated [email protected]: Please update to the latest object-keys
npm WARN deprecated [email protected]: Use uuid module instead
[              ....] \ extract:vinyl-source-stream: sill doParallel extract 934

the vps is froozen by the npm install.

[azarus]

@addaleax bump, i think this issue still persist as you can read from the comments after your message.

[azarus]

related:
#18448

[phoniks]

I think I am also seeing this issue, but not sure. Any parallel extraction seems to fail on digitialocean's ubuntu setup.