Fix ModelViolationError while parsing repo files #1266

tomasfratrik · 2024-07-17T11:28:30Z

This error occurs when repo file has invalid definition, specifically the 'name' entry of the config file.

After this, we get rid of the traceback error, and the following errors are shown

2024-06-17 13:35:00.974356 [ERROR] Actor: system_facts
Message: Invalid repository definition: invalid-repo in: /etc/yum.repos.d/invalid.repo: The value of "name" field is None, but this is not allowed
Summary:
    Hint: For more directions on how to resolve the issue, see: https://access.redhat.com/solutions/6969001.
2024-06-17 13:35:01.545642 [ERROR] Actor: repositories_blacklist
Message: Actor didn't receive required messages: RepositoriesFact

Jira: RHEL-19249

github-actions · 2024-07-17T11:28:41Z

Thank you for contributing to the Leapp project!

Please note that every PR needs to comply with the Leapp Guidelines and must pass all tests in order to be mergeable.
If you want to request a review or rebuild a package in copr, you can use following commands as a comment:

review please @oamg/developers to notify leapp developers of the review request
/packit copr-build to submit a public copr build using packit

Packit will automatically schedule regression tests for this PR's build and latest upstream leapp build.
However, here are additional useful commands for packit:

/packit test to re-run manually the default tests
/packit retest-failed to re-run failed tests manually
/packit test oamg/leapp#42 to run tests with leapp builds for the leapp PR#42 (default is latest upstream - master - build)

Note that first time contributors cannot run tests automatically - they need to be started by a reviewer.

It is possible to schedule specific on-demand tests as well. Currently 2 test sets are supported, beaker-minimal and kernel-rt, both can be used to be run on all upgrade paths or just a couple of specific ones.
To launch on-demand tests with packit:

/packit test --labels kernel-rt to schedule kernel-rt tests set for all upgrade paths
/packit test --labels beaker-minimal-8.10to9.4,kernel-rt-8.10to9.4 to schedule kernel-rt and beaker-minimal test sets for 8.10->9.4 upgrade path

See other labels for particular jobs defined in the .packit.yaml file.

Please open ticket in case you experience technical problem with the CI. (RH internal only)

Note: In case there are problems with tests not being triggered automatically on new PR/commit or pending for a long time, please contact leapp-infra.

tomasfratrik · 2024-07-17T12:40:31Z

/packit retest-failed

matejmatuska · 2024-07-18T10:29:19Z

/packit retest-failed

matejmatuska · 2024-07-18T11:26:45Z

/packit retest-failed

matejmatuska

The overall approach is ok, but there are some problems.

repos/system_upgrade/common/actors/systemfacts/libraries/systemfacts.py

matejmatuska

Just one small suggestion :) code-wise ok, I am going to test it now

repos/system_upgrade/common/libraries/repofileutils.py

matejmatuska

The PR description is outdated, from my testing on RHEL 8 the output looks like this, which is good:

2024-08-02 12:17:22.052435 [ERROR] Actor: system_facts
Message: Invalid repository definition: copr:copr.fedorainfracloud.org:group_oamg:leapp in: /etc/yum.repos.d/leapp-copr.repo: The value of "name" field is None, but this is not allowed
Summary:
    Hint: For more directions on how to resolve the issue, see: https://access.redhat.com/solutions/6969001.
2024-08-02 12:17:33.302747 [ERROR] Actor: repositories_blacklist
Message: Actor didn't receive required messages: RepositoriesFacts

After the PR desc is updated, squash the commits and briefly describe the changes in the final one, otherwise LGTM.

DO NOT MERGE YET - FOR NOW IT'S NOT TARGETED FOR THE UPCOMING RELEASE

tomasfratrik · 2024-08-15T07:34:09Z

The latest push squashes commits.

fernflower · 2024-08-15T08:36:11Z

Could you please add a unit test?

matejmatuska · 2024-08-19T07:57:58Z

Could you please add a unit test?

Agree, I don't know how I forgot that, sorry @tomasfratrik.

tomasfratrik · 2024-08-29T07:35:19Z

Could you please add a unit test?

I added it!

matejmatuska

Tests has been added and it works as expected. LGTM

STILL DO NOT MERGE

tomasfratrik · 2024-08-29T11:20:49Z

The latest push squashed commits.

repos/system_upgrade/common/libraries/repofileutils.py

matejmatuska

@pirat89 pointed out that a unit test was added, but only for the system_facts actor. Tests are still missing for the parse_repofile and potentially _parse_repository functions.

Also docstrings have to be included on public members in shared libraries (it would be okay in an actor lib).

repos/system_upgrade/common/libraries/repofileutils.py

pirat89 · 2024-09-16T17:29:56Z

/packit copr-build

matejmatuska

I would add a little more test cases, otherwise ok.

repos/system_upgrade/common/libraries/tests/test_repofileutils.py

tomasfratrik · 2024-10-29T13:15:26Z

/packit copr-build

This error occurs when repo file has invalid definition, specifically when the 'name' entry of the config files is invalid. Also add tests. Jira: RHEL-19249

matejmatuska

Integration tests on RHEL7 are broken everywhere, not cause by these changes.

LGTM

pirat89 · 2024-11-01T09:15:54Z

/packit copr-build

pirat89 · 2024-11-01T10:58:43Z

@matejmatuska seems that the non-els error we discussed has been caused just by infra issue. merging.

## Announce This is the last upstream release covering IPU 7 -> 8. The system_upgrade_el7toel8 repository is going to be removed from further releases. Other artifacts related to IPU 7 -> 8 are going to be removed too. In the same time it's the last release compatible with Python 2.7. Since now we are going to develope and test the code only for Python 3.6 and newer. ## Packaging - [IPU 9 -> 10] Require libdb-utils on RHEL 9 (oamg#1289) - Require leapp-framework 6.0+ (oamg#1142) - Update leapp-deps package to satisfy leapp_framework_deps 6 (oamg#1142) ## Upgrade handling ### Fixes - Activate LVM VGs with `--sysinit` option to correct the use in the upgrade initramfs (oamg#1329) - Cap max size of the sparse files to 1TiB when storage with large amount of space (oamg#1294) - Fix IPU being blocked by resource limitations (oamg#1256) - Fix pes events scanner crashing when there are duplicate packages in the received instructions (oamg#1296) - Fix pes events scanner not respecting user’s transaction configuration (oamg#1296) - Fix storage scanner crashing when command outputs contain colon character (oamg#1258) - Fix the report when handling broken parsing of kernel cmdline (oamg#1290) - Generate proper error message instead ModelViolationError when parsing invalid repository definition (oamg#1266) - Handle default kernel cmdline when multiple boot entries for the default kernel are defined (oamg#1302) - Load all available obsoleted keys (oamg#1286) - Prevent a possible crash with LiveMode when adding the upgrade boot entry on LVM systems (oamg#1313) - Prevent failure for upgrade paths when obsoleted keys are not defined (oamg#1285) - [IPU 7 -> 8] Ignore invalid firewalld configuration (oamg#1341) - [IPU 7 -> 8] scangrubdevpartitionlayout: Skip warning msgs from fdisk (oamg#1306) - [IPU 8 -> 9] Fix problems with the bootloader when upgrading to RHEL 9.6 on ARM (oamg#1275, oamg#1331, oamg#1339) - [IPU 9 -> 10] Fix output of commands executed inside systemd-nspawn containers (oamg#1323) - [IPU 9 -> 10] Fix remediation instructions for deprecated NM configuration (oamg#1330) - [IPU 9 -> 10] Obsolete RHEL9 GPG key signed with SHA1 (oamg#1325) ### Enhancements - Add RHEL 9.6 and 10.0 product certificates (oamg#1287, oamg#1309) - Introduce new IPU paths 8.10 -> 9.6 and 9.6 -> 10.0 (oamg#1309) - Add possibility to use net.naming-scheme (for IPU 8 -> 9 only for now) (oamg#1215, oamg#1312) - Drop upgrade paths related to RHEL 8.8, 9.2, and 9.5 (oamg#1309, oamg#1344) - Enable upgrade for EL8+ systems with LUKS bound to Clevis with TPM 2.0 token (oamg#1200) - Enable upgrade with RHUI on Alibaba cloud for ARM machines (oamg#1277) - Introduce a possibility to configure leapp actors covering RHUI on clouds (oamg#1142) - Minor improvements in preupgrade reports (oamg#1315, oamg#1326, oamg#1342) - Raise an inhibitor if unsupported target version supplied instead of error (oamg#1328) - Skip checking of (PKI) `directory-hash` dir to speedup the upgrade process and clean logs (oamg#1297) - Update leapp upgrade data files (oamg#1307) - [IPU 9 -> 10] Cover upgrades for MySQL and PostgreSQL databases (oamg#1316, oamg#1343) - [IPU 9 -> 10] Detect OpenSSL engines configured in /etc/pki/tls/openssl.cnf (oamg#1338) - [IPU 9 -> 10] Detect XFS file systems with problematic parameters (oamg#1318) - [IPU 9 -> 10] Detect deprecated network-scripts (ifcfg) files (oamg#1332, oamg#1347) - [IPU 9 -> 10] Detect whether subscribed systems are using SCA (oamg#1333) - [IPU 9 -> 10] Handle correctly switch of symlink to directory when upgrading Ruby IRB (oamg#1304) - [IPU 9 -> 10] Inform user about Libdb removal if present (oamg#1319) - [IPU 9 -> 10] Inhibit the upgrade for Intel CPUs with microarchitecture x86_64-v2 (oamg#1196) - [IPU 9 -> 10] Move crypto-policies related actors to the system_upgrade_common repository to apply it for IPU 9 -> 10 as well - [IPU 9 -> 10] Update checks for SAP HANA (oamg#1346) - [IPU 9 -> 10] Update pam_userdb database backend for RHEL10 (oamg#1289)

## Announce This is the last upstream release covering IPU 7 -> 8. The system_upgrade_el7toel8 repository is going to be removed from further releases. Other artifacts related to IPU 7 -> 8 are going to be removed too. In the same time it's the last release compatible with Python 2.7. Since now we are going to develope and test the code only for Python 3.6 and newer. ## Packaging - [IPU 9 -> 10] Require libdb-utils on RHEL 9 (oamg#1289) - Require leapp-framework 6.0+ (oamg#1142) - Update leapp-deps package to satisfy leapp_framework_deps 6 (oamg#1142) ## Upgrade handling ### Fixes - Activate LVM VGs with `--sysinit` option to correct the use in the upgrade initramfs (oamg#1329) - Cap max size of the sparse files to 1TiB when storage with large amount of space (oamg#1294) - Fix IPU being blocked by resource limitations (oamg#1256) - Fix pes events scanner crashing when there are duplicate packages in the received instructions (oamg#1296) - Fix pes events scanner not respecting user’s transaction configuration (oamg#1296) - Fix storage scanner crashing when command outputs contain colon character (oamg#1258) - Fix the report when handling broken parsing of kernel cmdline (oamg#1290) - Generate proper error message instead ModelViolationError when parsing invalid repository definition (oamg#1266) - Handle default kernel cmdline when multiple boot entries for the default kernel are defined (oamg#1302) - Load all available obsoleted keys (oamg#1286) - Prevent a possible crash with LiveMode when adding the upgrade boot entry on LVM systems (oamg#1313) - Prevent failure for upgrade paths when obsoleted keys are not defined (oamg#1285) - [IPU 7 -> 8] Ignore invalid firewalld configuration (oamg#1341) - [IPU 7 -> 8] scangrubdevpartitionlayout: Skip warning msgs from fdisk (oamg#1306) - [IPU 8 -> 9] Fix problems with the bootloader when upgrading to RHEL 9.6 on ARM (oamg#1275, oamg#1331, oamg#1339) - [IPU 9 -> 10] Fix output of commands executed inside systemd-nspawn containers (oamg#1323) - [IPU 9 -> 10] Fix remediation instructions for deprecated NM configuration (oamg#1330) - [IPU 9 -> 10] Obsolete RHEL9 GPG key signed with SHA1 (oamg#1325) ### Enhancements - Add RHEL 9.6 and 10.0 product certificates (oamg#1287, oamg#1309) - Introduce new IPU paths 8.10 -> 9.6 and 9.6 -> 10.0 (oamg#1309) - Add possibility to use net.naming-scheme (for IPU 8 -> 9 only for now) (oamg#1215, oamg#1312) - Drop upgrade paths related to RHEL 8.8, 9.2, and 9.5 (oamg#1309, oamg#1344) - Enable upgrade for EL8+ systems with LUKS bound to Clevis with TPM 2.0 token (oamg#1200) - Enable upgrade with RHUI on Alibaba cloud for ARM machines (oamg#1277) - Introduce a possibility to configure leapp actors covering RHUI on clouds (oamg#1142) - Minor improvements in preupgrade reports (oamg#1315, oamg#1326, oamg#1342) - Raise an inhibitor if unsupported target version supplied instead of error (oamg#1328) - Skip checking of (PKI) `directory-hash` dir to speedup the upgrade process and clean logs (oamg#1297) - Update leapp upgrade data files (oamg#1307) - [IPU 9 -> 10] Cover upgrades for MySQL and PostgreSQL databases (oamg#1316, oamg#1343) - [IPU 9 -> 10] Detect OpenSSL engines configured in /etc/pki/tls/openssl.cnf (oamg#1338) - [IPU 9 -> 10] Detect XFS file systems with problematic parameters (oamg#1318) - [IPU 9 -> 10] Detect deprecated network-scripts (ifcfg) files (oamg#1332, oamg#1347) - [IPU 9 -> 10] Detect whether subscribed systems are using SCA (oamg#1333) - [IPU 9 -> 10] Handle correctly switch of symlink to directory when upgrading Ruby IRB (oamg#1304) - [IPU 9 -> 10] Inform user about Libdb removal if present (oamg#1319) - [IPU 9 -> 10] Inhibit the upgrade for Intel CPUs with microarchitecture x86_64-v2 (oamg#1196) - [IPU 9 -> 10] Move crypto-policies related actors to the system_upgrade_common repository to apply it for IPU 9 -> 10 as well - [IPU 9 -> 10] Update checks for SAP HANA (oamg#1346) - [IPU 9 -> 10] Update pam_userdb database backend for RHEL10 (oamg#1289) ## Additional changes interesting for devels - Introducing new upstream documentation focused on the leapp-repository project at https://leapp-repository.readthedocs.io/introduce-docs/. The documentation is still under reconstruction at this time, however we consider it a big step. All PRs impacting stuff that should be documented will be required to do so before they could be merged.

## Announce This is the last upstream release covering IPU 7 -> 8. The system_upgrade_el7toel8 repository is going to be removed from further releases. Other artifacts related to IPU 7 -> 8 are going to be removed too. In the same time it's the last release compatible with Python 2.7. Since now we are going to develope and test the code only for Python 3.6 and newer. ## Packaging - [IPU 9 -> 10] Require libdb-utils on RHEL 9 (#1289) - Require leapp-framework 6.0+ (#1142) - Update leapp-deps package to satisfy leapp_framework_deps 6 (#1142) ## Upgrade handling ### Fixes - Activate LVM VGs with `--sysinit` option to correct the use in the upgrade initramfs (#1329) - Cap max size of the sparse files to 1TiB when storage with large amount of space (#1294) - Fix IPU being blocked by resource limitations (#1256) - Fix pes events scanner crashing when there are duplicate packages in the received instructions (#1296) - Fix pes events scanner not respecting user’s transaction configuration (#1296) - Fix storage scanner crashing when command outputs contain colon character (#1258) - Fix the report when handling broken parsing of kernel cmdline (#1290) - Generate proper error message instead ModelViolationError when parsing invalid repository definition (#1266) - Handle default kernel cmdline when multiple boot entries for the default kernel are defined (#1302) - Load all available obsoleted keys (#1286) - Prevent a possible crash with LiveMode when adding the upgrade boot entry on LVM systems (#1313) - Prevent failure for upgrade paths when obsoleted keys are not defined (#1285) - [IPU 7 -> 8] Ignore invalid firewalld configuration (#1341) - [IPU 7 -> 8] scangrubdevpartitionlayout: Skip warning msgs from fdisk (#1306) - [IPU 8 -> 9] Fix problems with the bootloader when upgrading to RHEL 9.6 on ARM (#1275, #1331, #1339) - [IPU 9 -> 10] Fix output of commands executed inside systemd-nspawn containers (#1323) - [IPU 9 -> 10] Fix remediation instructions for deprecated NM configuration (#1330) - [IPU 9 -> 10] Obsolete RHEL9 GPG key signed with SHA1 (#1325) ### Enhancements - Add RHEL 9.6 and 10.0 product certificates (#1287, #1309) - Introduce new IPU paths 8.10 -> 9.6 and 9.6 -> 10.0 (#1309) - Add possibility to use net.naming-scheme (for IPU 8 -> 9 only for now) (#1215, #1312) - Drop upgrade paths related to RHEL 8.8, 9.2, and 9.5 (#1309, #1344) - Enable upgrade for EL8+ systems with LUKS bound to Clevis with TPM 2.0 token (#1200) - Enable upgrade with RHUI on Alibaba cloud for ARM machines (#1277) - Introduce a possibility to configure leapp actors covering RHUI on clouds (#1142) - Minor improvements in preupgrade reports (#1315, #1326, #1342) - Raise an inhibitor if unsupported target version supplied instead of error (#1328) - Skip checking of (PKI) `directory-hash` dir to speedup the upgrade process and clean logs (#1297) - Update leapp upgrade data files (#1307) - [IPU 9 -> 10] Cover upgrades for MySQL and PostgreSQL databases (#1316, #1343) - [IPU 9 -> 10] Detect OpenSSL engines configured in /etc/pki/tls/openssl.cnf (#1338) - [IPU 9 -> 10] Detect XFS file systems with problematic parameters (#1318) - [IPU 9 -> 10] Detect deprecated network-scripts (ifcfg) files (#1332, #1347) - [IPU 9 -> 10] Detect whether subscribed systems are using SCA (#1333) - [IPU 9 -> 10] Handle correctly switch of symlink to directory when upgrading Ruby IRB (#1304) - [IPU 9 -> 10] Inform user about Libdb removal if present (#1319) - [IPU 9 -> 10] Inhibit the upgrade for Intel CPUs with microarchitecture x86_64-v2 (#1196) - [IPU 9 -> 10] Move crypto-policies related actors to the system_upgrade_common repository to apply it for IPU 9 -> 10 as well - [IPU 9 -> 10] Update checks for SAP HANA (#1346) - [IPU 9 -> 10] Update pam_userdb database backend for RHEL10 (#1289) ## Additional changes interesting for devels - Introducing new upstream documentation focused on the leapp-repository project at https://leapp-repository.readthedocs.io/introduce-docs/. The documentation is still under reconstruction at this time, however we consider it a big step. All PRs impacting stuff that should be documented will be required to do so before they could be merged.

tomasfratrik force-pushed the fix-modelviolation-error branch 3 times, most recently from 3d49f24 to b294a98 Compare July 17, 2024 12:03

matejmatuska requested changes Jul 19, 2024

View reviewed changes

repos/system_upgrade/common/actors/systemfacts/libraries/systemfacts.py Outdated Show resolved Hide resolved

repos/system_upgrade/common/actors/systemfacts/libraries/systemfacts.py Outdated Show resolved Hide resolved

tomasfratrik force-pushed the fix-modelviolation-error branch from 2a0f448 to 3bd08f8 Compare July 19, 2024 13:52

tomasfratrik marked this pull request as ready for review July 24, 2024 07:49

matejmatuska requested changes Aug 2, 2024

View reviewed changes

repos/system_upgrade/common/libraries/repofileutils.py Outdated Show resolved Hide resolved

matejmatuska requested changes Aug 2, 2024

View reviewed changes

tomasfratrik force-pushed the fix-modelviolation-error branch from 3bd08f8 to 49708b5 Compare August 6, 2024 08:38

pirat89 added this to the 8.10/9.6 milestone Aug 6, 2024

pirat89 added the bug Something isn't working label Aug 7, 2024

tomasfratrik force-pushed the fix-modelviolation-error branch from 49708b5 to 9ed7217 Compare August 15, 2024 07:30

tomasfratrik force-pushed the fix-modelviolation-error branch from 9ed7217 to 880fe49 Compare August 28, 2024 14:03

matejmatuska approved these changes Aug 29, 2024

View reviewed changes

tomasfratrik force-pushed the fix-modelviolation-error branch from 880fe49 to 94ffbb8 Compare August 29, 2024 11:19

pirat89 reviewed Aug 29, 2024

View reviewed changes

repos/system_upgrade/common/libraries/repofileutils.py Outdated Show resolved Hide resolved

matejmatuska requested changes Aug 29, 2024

View reviewed changes

repos/system_upgrade/common/libraries/repofileutils.py Outdated Show resolved Hide resolved

tomasfratrik force-pushed the fix-modelviolation-error branch from 94ffbb8 to 5a584b8 Compare September 3, 2024 14:45

matejmatuska requested changes Oct 25, 2024

View reviewed changes

repos/system_upgrade/common/libraries/tests/test_repofileutils.py Outdated Show resolved Hide resolved

tomasfratrik force-pushed the fix-modelviolation-error branch from 5a584b8 to 0a55672 Compare October 28, 2024 22:23

tomasfratrik force-pushed the fix-modelviolation-error branch from 0a55672 to 3af8eec Compare October 29, 2024 13:37

Raise proper error when ModelViolationError occurs

ad4dbc3

This error occurs when repo file has invalid definition, specifically when the 'name' entry of the config files is invalid. Also add tests. Jira: RHEL-19249

tomasfratrik force-pushed the fix-modelviolation-error branch from 3af8eec to ad4dbc3 Compare October 30, 2024 13:10

matejmatuska approved these changes Oct 31, 2024

View reviewed changes

pirat89 merged commit ec07824 into oamg:main Nov 1, 2024
22 of 26 checks passed

pirat89 added the changelog-checked The merger/reviewer checked the changelog draft document and updated it when relevant label Nov 1, 2024

pirat89 mentioned this pull request Feb 14, 2025

Release 0.22.0 #1348

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix ModelViolationError while parsing repo files #1266

Fix ModelViolationError while parsing repo files #1266

tomasfratrik commented Jul 17, 2024 •

edited

Loading

github-actions bot commented Jul 17, 2024

tomasfratrik commented Jul 17, 2024

matejmatuska commented Jul 18, 2024

matejmatuska commented Jul 18, 2024

matejmatuska left a comment

matejmatuska left a comment

matejmatuska left a comment

tomasfratrik commented Aug 15, 2024

fernflower commented Aug 15, 2024

matejmatuska commented Aug 19, 2024 •

edited

Loading

tomasfratrik commented Aug 29, 2024

matejmatuska left a comment •

edited

Loading

tomasfratrik commented Aug 29, 2024

matejmatuska left a comment •

edited

Loading

pirat89 commented Sep 16, 2024

matejmatuska left a comment

tomasfratrik commented Oct 29, 2024

matejmatuska left a comment

pirat89 commented Nov 1, 2024

pirat89 commented Nov 1, 2024

Fix ModelViolationError while parsing repo files #1266

Fix ModelViolationError while parsing repo files #1266

Conversation

tomasfratrik commented Jul 17, 2024 • edited Loading

github-actions bot commented Jul 17, 2024

Thank you for contributing to the Leapp project!

tomasfratrik commented Jul 17, 2024

matejmatuska commented Jul 18, 2024

matejmatuska commented Jul 18, 2024

matejmatuska left a comment

Choose a reason for hiding this comment

matejmatuska left a comment

Choose a reason for hiding this comment

matejmatuska left a comment

Choose a reason for hiding this comment

tomasfratrik commented Aug 15, 2024

fernflower commented Aug 15, 2024

matejmatuska commented Aug 19, 2024 • edited Loading

tomasfratrik commented Aug 29, 2024

matejmatuska left a comment • edited Loading

Choose a reason for hiding this comment

tomasfratrik commented Aug 29, 2024

matejmatuska left a comment • edited Loading

Choose a reason for hiding this comment

pirat89 commented Sep 16, 2024

matejmatuska left a comment

Choose a reason for hiding this comment

tomasfratrik commented Oct 29, 2024

matejmatuska left a comment

Choose a reason for hiding this comment

pirat89 commented Nov 1, 2024

pirat89 commented Nov 1, 2024

tomasfratrik commented Jul 17, 2024 •

edited

Loading

matejmatuska commented Aug 19, 2024 •

edited

Loading

matejmatuska left a comment •

edited

Loading

matejmatuska left a comment •

edited

Loading