karmada-metrics-adapter: reduce memory usage #4796
Conversation
karmada-bot
added
the
size/S
karmada-bot
added
size/L
Codecov ReportAll modified and coverable lines are covered by tests ✅
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## master #4796 +/- ##
==========================================
- Coverage 51.78% 51.76% -0.02%
==========================================
Files 250 250
Lines 24989 24980 -9
==========================================
- Hits 12940 12931 -9
+ Misses 11340 11339 -1
- Partials 709 710 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
chaunceyjiang
force-pushed
the
pods
branch
3 times, most recently
from
23bdd83 to
d60be38
Compare
When there is a large amount of pod usage in the member cluster, metrics-adapter will consume a lot of memory. The reason is that it caches all the information of all pods in the cluster. However, we don't need all this information, so we trim some of the information to reduce memory usage. Signed-off-by: chaunceyjiang <[email protected]>
|
We have tested on such environment: 2 member clusters, each cluster have 500 node, 10000 pod in all. before this PR introduced: karmada-metrics-adapter cost 1.3GB memory at least (and may probabilistically rise), memory overhead is mainly due to the line after this PR introduced: only cost about 300MB. This is a significant performance optimization ! ٩(๑❛ᴗ❛๑)۶ |
|
Now, can you add more explaination in PR description to summarize how does this PR works to reduce the memory? You only said you trimmed some of the information, may be you can explain more about why |
|
/LGTM |
|
It's probably more of a feature. |
karmada-bot
added
the
kind/feature
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: RainbowMango The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
karmada-bot
added
the
approved
When there is a large amount of pod usage in the member cluster, metrics-adapter will consume a lot of memory. The reason is that it caches all the information of all pods in the cluster. However, we don't need all this information, so we trim some of the information to reduce memory usage.
What type of PR is this?
/kind bug
What this PR does / why we need it:
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
The core idea of this PR is to use
TransformFuncto trim the fields that pod and node need to buffer. Currently, it only retains fields such asname,namespace,labels, etc. Other fields are not needed, so we do not need to cache them.Does this PR introduce a user-facing change?: