runtime: throttle heap growth #14735
Comments
What would GOGC mean in that context? It would no longer be the case that the next GC cycle triggers "when the ratio of freshly allocated data to live data remaining after the previous collection reaches this percentage."
Don't we return unused pages to the OS (e.g. via MADV_DONTNEED) after a spike subsides? To cap the amount of additional CPU we'd probably need to stop doing that (which I'd argue is fine, but I would expect to encounter many users who vehemently disagree). |
I believe that nothing is changed for end-user. GOGC is expressed in terms of live heap during a GC, and you generally don't know when GC is triggered. With concurrent marking and sweeping, these things are blurred even more. Let's just pretend that GC has higher chances to be triggered at a more fortunate point.
This is a good point. I feel that both things are useful and need to coexist. I don't have a ready answer. |
|
Interesting. Are you basically proposing that we low-pass filter the heap goal? (Or at least low-pass filter it when it's going up.) I'm curious for more details about v8's GC scheduling if you have them or have a pointer.
If anything, we need to return memory to the OS more aggressively. In this case, returning it more aggressively will make a heap spike less costly in terms of RSS over time. But in general, there's no such thing as "unused memory". If it's not being used by an application, it's being used for something else like OS file caches or, on mobile, whole other applications. There's obviously a balance to be struck since we need to amortize the cost of returning memory to the OS and then potentially asking for it back. We don't want to return a page the instant we're not using it, but there is most certainly value in returning memory to the OS. |
|
@dvyukov, what you are describing is the exact opposite of the problem I see most often when looking at GC overheads. What I see is programs like the compiler that tend to have long phases where they mostly allocate and don't give up much garbage from those allocations. In those phases, you'd ideally want the goal to be even higher, because the GC isn't going to reclaim much anyway. What you're describing would move the goal in the opposite direction, making it grow even more slowly and therefore making GC cost even more. That is, you are optimizing space at the cost of increased GC overhead. Often people want the opposite. I am not convinced we should change the goal calculation. There are clearly scenarios where it's too big and where it's too small, but at least it's understandable, and GOGC gives control in the cases where it absolutely needs changing. |
Yes, low-pass filter when heap is growing.
Can't find anything now. Probably it was just personal communication.
Completely agree.
I wonder if we can do heap growth throttling for long running server and client programs without compromising compiler... I looked at compiler gctrace, and the steady growth is easily recognizable. So the simplest heuristic would be to throttle gc goal once, if during the next gc reachable memory continues to grow, give up with throttling.
The absolutely need is more-or-less always related to reduction of GOGC value (you have infinite amount of CPU time, but limited memory). So temporary reducing GOGC should not be an issue. And the throttling should correlate with GOGC (e.g. GOGC/2), so you still can control it with GOGC. FTR, here are compiler traces for net package with different values of GOGC: |
|
The opposite optimization can also make sense, and would allow to win back some CPU time. If we have a long-running app where reachable heap fluctuates between 800MB and 1GB, RSS still will be at 2GB. But if we set GC goal to 2 GB most of the time, we can save 10% of GC overhead on average continuously. |
|
Is it possible to adjust heap goal according to current (This is because saving CPU and saving memory are For example, we can query getrusage(2) periodically, |
IMO, this again falls prey to the fallacy of thinking of memory as "unused." All memory is used all of the time, just for different things. Having an application use all of the system's memory is never the best policy. As an extreme example, at the very least you want to leave space for the application's own binary in the OS file cache; if you put the system under more pressure, the application will slow down, even though you're giving it more memory. In most applications there are other things in memory that are helping out the application, but don't count toward its RSS. I have no idea how to compute the optimal trade-off here, hence heuristics that try to keep the memory footprint within the realm of the minimum possible footprint. Hierarchical memory management is actually an area I feel should be ripe for research, but I've never seen anything on it. When different layers can adjust their resource usage in different ways, it really feels like there should be a cooperative way to iterate toward an optimum.
Linux doesn't actually report this. Even if it did, a lot of systems (particularly server systems) will kill your process before you ever start swapping. And even if the system does swap, this won't tell you if your application is causing other applications to swap. |
|
A spike in memory usage could also signal a phase change in the program. As long as the user is running with the default GOMAXPROCS and GOGC then On Thu, Mar 10, 2016 at 5:01 AM, Minux Ma [email protected] wrote:
|
gopherbot
added
the
compiler/runtime
GC goal is calculated as:
This works good in steady state and it is not a problem when heap is shrinking. But it is a problem when reachable heap size has episodic short-term spikes.
Consider that typical reachable heap size is 1GB, so heap grows to 2GB (provided GOGC=100). Now consider that there is a short spike of reachable heap size to 1.5GB. If GC is triggered after the spike, everything is fine. However if we are unlucky and GC observes 1.5GB of reachable heap, it will set goal to 3GB. And at this point we will necessary consume these 3GB and increase RSS by 1.5x unnecessary.
The proposal is to throttle GC goal if we see that the new goal will increase max heap size. We can either use, say, GOGC/2 in such cases; or maybe set heap goal to current max heap size. In the above example, if we set goal to 2GB we can avoid growing heap at all. This way we pay a fixed amount of CPU during heap growth, but can get lower RSS in return. Note that heap growth can't continue infinitely, so the amount of additional CPU is limited.
FWIW client-side GC-based systems (most notably javascript), go to great effort to throttle heap growth in such scenarios. AFAIK, v8 uses something like GOGC=10% during growth just to not over-allocate.
The text was updated successfully, but these errors were encountered: