-
-
Notifications
You must be signed in to change notification settings - Fork 157
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory #403
Comments
Thank you for creating this issue. However, issues need to follow one of our templates so that we can clearly understand your particular circumstances. Please help us help you by recreating the issue using one of our templates. Please respect time of other developers, thanks |
@alexander-akait Problem was terser plugin eats up a lot of memory after release 5.1.2. / I use terser plugin with webpack. My configuration Terser plugin.
I think in the https://github.com/webpack-contrib/terser-webpack-plugin/releases/tag/v5.1.2 was commit which which have leak memory |
This commit fixes memory usage, with 5.1.1 all fine? |
@alexander-akait Yes in the 5.1.1 all fine. Could you check pls version 5.1.2 ? |
@sergeu90 Can you provide reproducible example? I want to check memory usage, because it should fix memory usage |
@alexander-akait on my real project I don't share source code.I think you can take my configuration and compare 5.1.1 and 5.1.2 between |
Sorry I can find the problem using comparing, because it is fix the parallel problem and memory usage |
@alexander-akait maybe change status issue from close to open |
@sergeu90 please provide example of the problem, if you need i can sign NDA |
@alexander-akait Above i send configuration terser plugin. I think you can compare two version. |
@sergeu90 I compare and don't see any problem with memory, if we have big problem(s) with memory, we will have a lot of issue here, but you can see it is not, what is the problem to help? You can share repo only for me and then close access after we fix the problem |
@alexander-akait Sorry but i can't share project and give access |
@sergeu90 Unfortunately I can't help here, maybe related webpack/webpack#13550 |
I recommend to pin terser-webpack-plugin to v5.1.1 right now, look like jest-worker has memory leak |
@alexander-akait Thanks. Now I downgraded to 5.1.1 and works fine. |
I can confirm we experience the same issue both on angular storybook builds and angular builds. they both point to the @5.1.2. |
@Tallyb Please provide reproducible test repo |
@alexander-akait |
@Tallyb Please open an issue in |
- I've been to hell and back, and haven't managed to pinpoint all of the exact dependencies that cause the build to leak (or more likely: attempt to over-allocate) memory but after extensive testing, this is the set of dependencies that manages to build in an environment with max 3GB of memory available - It could very well be that some of the dependencies blocked by this commit's locks simply take the build's memory consumption over the 3GB limit and aren't part of the same issue of over allocating memory -- but this is a decent compromise that ensures that the build should no longer intermittently fail in CI (which has 4GB, of which roughly 3,3GB is usable) - The test setup: - `docker run --rm -it --volume $(pwd):/home/circleci/project --memory=3G cimg/node:14.15` (same image as in CI, and 3 GB of memory ensures that the build will always work on the 4 GB available in CI) 1. `cd .. && cp -r project frontend` (just to ignore any file permission issues without touching the executor image too much) 2. `rm -rf dist node_modules` 3. `yarn install --immutable` 4. `yarn build` 5. -> repeat at least 3 times and if all pass (& CI passes) -> OK set of dependency versions - 8 parallel CI jobs running to increase sample size with `/sys/fs/cgroup/memory/memory.max_usage_in_bytes` logged - Findings: - _Something_ (or a combination of things) attempts to over allocate memory (specifically new heap space) when running with `master`'s versions of frontend dependencies (likely culprits described below) - With the versions used in this commit, the build will use all memory available to it but GC is early enough to never actually go over the limit **but** something in the newer dependencies goes over the line, causing reaping of the build processes and crashing the build - The most likely explanation would be something "outside" the Node.js process (i.e. something creates additional processes), which doesn't obey the same heap limits - TypeScript 4.3 has a few issues with relating to complex types which lead to at least significantly higher memory usage (likely not the same over-allocation issue as below) -> **lock to ~4.2.x** - Related: microsoft/TypeScript#44299 and microsoft/TypeScript#44281 - This is actually a good idea anyway as TS makes breaking changes in "minor" releases (e.g. 4.2.x -> 4.3.x) and we shouldn't blindly update it - webpack's dependency `[email protected]` updates `jest-worker` 26 -> 27 which appears to cause memory over allocation, likely related to the way it creates new processes and/or threads for workers based on incorrect assumptions about the available amount of CPU cores inside a Docker container (`os.cpus()` vs. what's actually allocated in cgroups) -> **lock to 5.1.1** - There's also a big issue with shared dependencies with `@storybook/*` modules also using webpack (but *4.x.x* instead of 5.x.x), which makes using `resolutions` pretty difficult without locking absolutely everything - Related: webpack/webpack#13550 and webpack-contrib/terser-webpack-plugin#403 - `@types/[email protected]` is incompatible with Jest 27 (jestjs/jest#11640) -> update only to 15.x.x - `[email protected]` updates _some_ dependency that either causes a similar memory over-allocation or simply increases the build's memory usage over the set 3GB limit - My money's on `webpack-sources` as based on webpack's changelogs it appears to have had some memory leak issues between the versions used by `[email protected]` and `[email protected]` (latest at the time of this commit) - `[email protected]` could also be related but didn't manage to find anything conclusive here - Additionally, update all dependencies _that don't break the build / cause memory issues_ to the latest non-major versions - Excluded Chart.js (& related) dependencies as that whole repo seems in a very weird place now - 3.x.x released a while ago and there hasn't been any release notes, tags or anything - 3.0.0 diff is pretty massive with very non-descriptive commit/PR messages - Excluded `react-select` and `react-datepicker` as they're going to be dropped at some point and both of them appeared to have pretty major changes - Excluded Testcafe as that's always a pain & we're going to drop it when everything's been re-written for Playwright - Update webpack loaders to their latest (incl. major) versions -- nothing breaking for us
Hello everyone after update terser plugin to 5.1.2 i have error
I downgrade to 5.1.1 and works.
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
1: 0xa222f0 node::Abort() [webpack]
2: 0x96411f node::FatalError(char const*, char const*) [webpack]
3: 0xb97f1e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [webpack]
4: 0xb98297 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [webpack]
5: 0xd52fd5 [webpack]
6: 0xd53b5f [webpack]
7: 0xd61beb v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [webpack]
8: 0xd657ac v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [webpack]
9: 0xd33e5b v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [webpack]
10: 0x107c5af v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [webpack]
11: 0x1423279 [webpack]
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] build:
cross-env NODE_ENV=production webpack --config internals/webpack/webpack.prod.babel.js --color --progress
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
The text was updated successfully, but these errors were encountered: