MANOPD-83734 Actualize and ensure mandatory thirdparty packages

[ilia1243]

Description

• KubeMarine and Kubernetes require the list of mandatory thirdparty packages, but they are not installed by default.

Solution

• All mandatory packages are now installed by default.
  • Audit is now installed on any OS, and not only on Debian nodes.
  • Added iptables to the list of mandatory packages.
  • semanage is installed only on RHEL nodes.
  • unzip is installed only if it is required by services.thirdparties.
  • conntrack, iptables are installed only on control-plane, worker nodes.
  • The remained openssl, curl, and kmod packages are installed on all nodes.
• Mandatory packages are managed the same as other system packages (docker, haproxy, etc.) and installed during prepare.package_manager.manage_packages task.
• Added services.packages.mandatory section that allows to turn mandatory packages off.
• Added mandatory_packages_off patch of existing inventories to turn automatic managing of mandatory packages off for backward compatibility.
• Fixed JSON schema for upgrade procedure. Now it allows packages associations only for docker and containerd because there is no process to upgrade other system packages. It now also prohibits package_manager section.
• Fixed package detection process to take only relevant nodes into account (for example, control-plane nodes are not checked when detecting of haproxy).
• Default packages associations are partially moved to globals.yaml for deduplication.
• check_iaas checks that mandatory packages are available.
• Added tasks to check_paas to check that audit and mandatory packages have equal versions on the relevant nodes.

How to apply

Run migrate_kubemarine --force-apply mandatory_packages_off

Test Cases

TestCase 1

Test Configuration:

• Hardware: Any
• OS: Any
• Inventory: Empty services.packages.install section.

Steps:

1. Run install.

Results:

Before	After
Installation fails with lack of packages	Installation succeeds with possible warnings in deploy.kubernetes.init task.

TestCase 2

Test Configuration:

• Hardware: Any
• OS: Any
• Inventory: Turn off some mandatory packages in services.packages.mandatory.

Steps:

1. Run install.

Results:

Before	After
Not applicable	All except the explicitly turned off mandatory packages are installed. Installation might fail if the omitted package is not installed but required for correct KubeMarine/Kubernetes work.

TestCase 3

Test Configuration:

• Hardware: Any
• OS: Any
• Inventory: Any

Steps:

1. Run kubemarine_migrate --force-apply mandatory_packages_off.

Results:

Before	After
Not applicable	Inventory is patched and all mandatory packages are turned off

TestCase 4

Test Configuration:

• Hardware: Any
• OS: Any
• Inventory: Any

Steps:

1. Run any procedure.

Results:

Before	After
Not applicable	Mandatory package versions are detected and cached in cluster_finalized.yaml.

TestCase 5

Test Configuration:

• Hardware: Any
• OS: Any
• Inventory: Any

Steps:

1. Run check_iaas, check_paas.

Results:

Before	After
Not applicable	check_iaas checks that mandatory packages are available in repositories. check_paas checks that mandatory packages and audit have equal versions.

Checklist

• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• Integration CI passed
• Unit tests. If Yes list of new/changed tests with brief description
• There is no merge conflicts

Unit tests

test_executor.py - added tests for minor fixes/improvements of RemoteExecutor.
test_audit.py - added tests for installation on centos.
test_install.py - new tests for mandatory packages installation.
test_packages.py - added tests for improved packages versions detection, including support of mandatory packages.

[Imadzuma]

Please also exclude packages to install from ci/default_config.yaml to handle case with mandatory packages from CI

[ilia1243]

Please also exclude packages to install from ci/default_config.yaml to handle case with mandatory packages from CI

Done

[koryaga]

@shmo1218 pls proceed