MANOPD-81794 JSON schema implementation

[ilia1243]

Description

• Inventory files lack of validation especially if variables / sections are specified in unexpected places.

Fixes #304

Solution

• Added new dependency on jsonschema and JSON schemas to validate inventory file of cluster and of each Kubemarine procedure.
  The JSON schemas are used during KubeMarine run and can be used in IDEs that support YAML validation, i. e. PyCharm, Visual Studio Code.
• Implemented extended heuristics to show most suitable errors.
• Added new values section to hold custom variables to be used in Jinja2 templates.
• Added option --ignore-schema-errors that continues work of Kubemarine even if validation is failed. The option is to be removed in next release.
• Added "default" and "description" annotations to the JSON schemas.
• Fixes in list merging strategy related to merging of the procedure inventory and the cluster inventory.
• Packages enrichment is split into different enrichment procedures to support more robust merging of upgrade procedure inventory and the cluster inventory.
• Partially revert of [Hot fix]Build and check binary files #305 to fix build with new 3rd-party. Now binary build is again created using custom Dockerfile instead of GitHub action.
• Different changes in documentation.
• Fix to make node.name parameter optional.

How to apply

It is necessary to verify inventory file:

• If it contains artifacts from old KubeMarine versions which are no longer processed by KubeMarine, such section should be adopted to the new version, or removed.
• If there are custom variables which are dynamically used in Jinja2 templates, such variables should be moved to the new dedicated values section. Usages of the variables should be adopted correspondingly. See Limitations

Test Cases

TestCase 1

Test Configuration:

• Hardware: Any
• OS: Any
• Inventory: Any

Steps:

1. Make cluster.yaml contain sections or variables that are not known to KubeMarine.
2. Run any procedure.

Results:

Before	After
successful	failed with validation error

TestCase 2

Test Configuration:

• Hardware: Any
• OS: Any
• Inventory: Any

Steps:

1. Make inventory file for any procedure contain sections or variables that are not known to KubeMarine.
2. Run this procedure.

Results:

Before	After
successful	failed with validation error

TestCase 3

Test Configuration:

• Hardware: Any
• OS: Any
• Inventory: Any

Steps:

1. Make cluster.yaml or inventory file for any procedure contain sections or variables that are not known to KubeMarine.
2. Run the procedure with --ignore-schema-errors option.

Results:

Before	After
successful	successful with warnings

Regression Testing

• Need to test all procedures with different parameters for regression.
• Re-test KubeMarine binary delivery.

Checklist

• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• Integration CI passed
• Unit tests. If Yes list of new/changed tests with brief description
• There is no merge conflicts

Unit tests

1. test_upgrade.py - refactoring, fixed and added tests for the changed packages enrichment flow.
2. test_schema.py:
   • checks that finalized inventory is valid against the schema.
   • checks that examples are actual
   • extended schema validation errors heuristics
3. test_coredns.py - added tests for some not obvious configmap generation features.
4. test_inventory.py - allow to omit node names
5. test_manage_pss.py - manage_pss procedure enrichment and finalization.
6. test_packages.py - refactoring only