Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding stock gc trigger and set it as default #222

Merged
merged 4 commits into from
Jan 23, 2025

Conversation

udesou
Copy link
Contributor

@udesou udesou commented Jan 23, 2025

This PR implements Julia's Stock GC heuristics for triggering GC and resizing the heap. It also updates mmtk-core to the latest version.

@udesou
Copy link
Contributor Author

udesou commented Jan 23, 2025

julia-version
JULIA_REPO=udesou/Julia
JULIA_REF=9f8df7b5d1c6760c0488525dea0a7efa0f609f53

mmtk/src/gc_trigger.rs Outdated Show resolved Hide resolved
mmtk/src/gc_trigger.rs Show resolved Hide resolved
if alloc_mem != 0 && alloc_time != 0 && gc_mem != 0 && gc_time != 0 {
let alloc_rate = alloc_mem as f64 / alloc_time as f64;
let gc_rate = gc_mem as f64 / gc_time as f64;
target_allocs = (heap_size as f64 * alloc_rate / gc_rate).sqrt() * TUNING_FACTOR;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The computation looks like mem balancer. If this is Julia's current heuristics, I assume Julia is using a variant of mem balancer (e.g. the thrashing detection).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's right. I think it does, and that's what I wanted to say in the MMTk meeting.
However, looking at TUNING_FACTOR from Julia, it looks much different than what mmtk-core implements? But again, the equation they use seems different...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We did not 'tune' for mmtk's mem balancer. They multiply with the TUNING_FACTOR, we divide by the TUNING_FACTOR. But set this aside, it looks like that they are using a variant of mem balancer.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, our heuristics started with stock membalancer and then we changed the quite a bit, notably by allowing it to scale a lot faster when small

@udesou udesou force-pushed the adding-stock-gc-trigger branch from 8bfb290 to c5af30d Compare January 23, 2025 02:55
@udesou udesou requested a review from qinsoon January 23, 2025 02:55
@udesou
Copy link
Contributor Author

udesou commented Jan 23, 2025

FYI, here is some rough data (in seconds) about time spent doing some of the steps for building Julia and running make test-gc and how it compares when using Membalancer, MMTK with Stock GC's trigger and Stock.

                                                 membal       mmtk + stock's trigger         stock
Base.Compiler ────                               199.87                 191.10               193.89

Sysimage built. Summary:
Base ────────                                    52.54                  42.17                41.35                                                   
Stdlibs ─────                                    20.97                  19.58                19.46                                           
Total ───────                                    73.51                  61.75                60.82                                                

Precompilation complete. Summary:
Total ───────                                    97.21                  70.55                68.22
Outputting sysimage file...
Output ──────                                    157.13                 120.83               128.78

Precompiling packages finished.
  106 dependency precompiled in                  281                    154                  138
  
make test-gc                                     1168.33                233.25               68.98 

Not sure why there's a huge discrepancy in the gc tests (maybe because we only support full heap collections?), but that seems much more reasonable than what we get from MemBalancer.

@udesou udesou merged commit 2310eba into mmtk:master Jan 23, 2025
8 checks passed
oscardssmith pushed a commit to JuliaLang/julia that referenced this pull request Jan 27, 2025
This PR sets MMTk to use the stock GC heuristics for heap resizing by
default. It also updates the `mmtk-julia` version to incorporate the
changes that support the new gc triggering heuristics (see
mmtk/mmtk-julia#222).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants