MANOPD-81038 MANOPD-78302 Refactoring and fixes of OS/package versions

[ilia1243]

Description

1. cluster_finalized.yaml no longer caches versions of custom packages from services.packages.install section after [PSUPCLPL-9592] packages versions #168
2. Changes in enrichment procedure in [MANOPD-76880] Fix cache package version #181 related to cache_versions property are doubtful and should be reverted/reworked.
   • If the user customizes an association, its caching is skipped during add_node.
     Instead, turning the caching off should be done intentionally via the same cache_versions=false property for each necessary association.
   • If the user uses cache_versions=false for all associations, we do not use our recommended package versions.
     We should do use recommended versions.
     The user will still be able to not follow our recommendations by explicitly redefining of package_name section of each necessary association.
3. Detection and work with OS versions are weird and should be reworked.

Fixes #262

Solution

• Enrichment procedure is changed as per the p. 2 of the Description.
• cluster_finalized.yaml now caches package versions independently on cache_versions property.
  If any package is installed with different versions on different nodes, warning is printed and the package is taken as-is from the inventory.
  If multiple OS families or versions detected on nodes, packages detection is skipped.
• Major refactoring of OS and package versions detection, enrichment and usage.
  • Enrichment of packages now happens independently on whether it is add_node or any other procedure.
    Customized packages are moved to OS family specific section and later used from there.
  • It is no longer possible to customize associations in global section for clusters in multiple OS.
    The customization should be moved to the corresponding OS specific section before OS migration.
  • cluster.context['os'] variable is removed. Now cluster global OS is detected using cluster.get_os_family().
    It does not depend on the procedure with the only exception that global OS family is calculated based on final nodes only.
  • NodeGroup.get_nodes_os() no longer accepts any flag.
    It returns OS family of really those group on which it is called and does not throw exceptions for unsupported or multiple OS families.
• Fixed unnecessary audit installation if balancer is added.
• Resolving of cluster_finalized.yaml is rewritten to enrichment-like style.
• Other fixes of cluster_finalized.yaml resolving if multiple OS families or versions detected on nodes or if some nodes are not sudoers.
• Fixes in check_paas procedure for clusters on multiple OS.
• Other refactoring and fixes.

Test Cases

TestCase 1

Test Configuration:

• Hardware: Any
• OS: All nodes have the same OS
• Inventory: Any with non-empty services.packages.install section. Some of packages have equal version on all nodes and some of them do not.

Steps:

1. Run any procedure

Results:

Before	After
cluster_finalized.yaml does not contain versions for services.packages.install section	Packages that have equal version on all nodes are cached. For other packages there are warnings in logs.

Regression Tests

• upgrade and migrate_cri procedures with custom packages sections.
• Audit installation on cluster with debian nodes.
• OS migration
• add_node when initial nodes having different associations packages versions and if cache_versions option is specified.

Checklist

• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation.
• Integration CI passed
• Unit tests. If Yes list of new/changed tests with brief description.
• There is no merge conflicts

Unit tests

• test_cluster.py: added tests for cluster.get_os_family()
• test_audit.py, test_haproxy.py, test_keepalived.py: fixes according to new enrichment approach
• test_migrate_cri.py, test_upgrade.py: added tests for packages enrichment and finalization
• test_packages.py: added many tests for packages enrichment, detection and caching.

[shmo1218]

The doc file is reviewed.