Allow simulator to persist changes in cluster snapshot

[x13n]

Which component this PR applies to?

cluster-autoscaler

What type of PR is this?

/kind cleanup
/kind feature

What this PR does / why we need it:

One more refactor of existing scale down code in order to make parallel scale down (#5079) possible.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

This is a follow-up to #5118. Only last 2 commits should be reviewed as a part of this PR. It will be on hold until the previous one is merged.

/hold

Does this PR introduce a user-facing change?

NONE

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

/assign @MaciekPytel
/assign @yaroslava-serdiuk

[x13n]

/retest

[x13n]

FWIW, looks like tests are failing due to #5134

[x13n]

/test test-and-verify

[k8s-ci-robot]

@x13n: No presubmit jobs available for kubernetes/autoscaler@master

In response to this:

/test test-and-verify

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[x13n]

/hold cancel

[MaciekPytel]

/lgtm
/approve
Happy to see resource limits moved away to a separate file, I really hated how they were mixed up with scale-down code before.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: MaciekPytel, x13n

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• cluster-autoscaler/OWNERS [MaciekPytel]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment