Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

properly scope try/except when loading obsoleted keys #1286

Merged
merged 1 commit into from
Aug 30, 2024

Conversation

evgeni
Copy link
Member

@evgeni evgeni commented Aug 21, 2024

We want to load all possible keys, even after a KeyError happenend

Fixes: 7e0fb44

Copy link

Thank you for contributing to the Leapp project!

Please note that every PR needs to comply with the Leapp Guidelines and must pass all tests in order to be mergeable.
If you want to request a review or rebuild a package in copr, you can use following commands as a comment:

  • review please @oamg/developers to notify leapp developers of the review request
  • /packit copr-build to submit a public copr build using packit

Packit will automatically schedule regression tests for this PR's build and latest upstream leapp build.
However, here are additional useful commands for packit:

  • /packit test to re-run manually the default tests
  • /packit retest-failed to re-run failed tests manually
  • /packit test oamg/leapp#42 to run tests with leapp builds for the leapp PR#42 (default is latest upstream - master - build)

Note that first time contributors cannot run tests automatically - they need to be started by a reviewer.

It is possible to schedule specific on-demand tests as well. Currently 2 test sets are supported, beaker-minimal and kernel-rt, both can be used to be run on all upgrade paths or just a couple of specific ones.
To launch on-demand tests with packit:

  • /packit test --labels kernel-rt to schedule kernel-rt tests set for all upgrade paths
  • /packit test --labels beaker-minimal-8.10to9.4,kernel-rt-8.10to9.4 to schedule kernel-rt and beaker-minimal test sets for 8.10->9.4 upgrade path

See other labels for particular jobs defined in the .packit.yaml file.

Please open ticket in case you experience technical problem with the CI. (RH internal only)

Note: In case there are problems with tests not being triggered automatically on new PR/commit or pending for a long time, please contact leapp-infra.

@evgeni
Copy link
Member Author

evgeni commented Aug 21, 2024

@pirat89 damn, the first PR was not as good as I thought yesterday. This one is better scoped. Sorry!

TL;DR:

>>> things = {'7': 'seven', '9': 'nine'}
>>> keys = []
>>> try:
...     for i in ['7', '8', '9']:
...         keys.append(things[i])
... except KeyError:
...     pass
... 
>>> keys
['seven']
>>> things = {'7': 'seven', '9': 'nine'}
>>> keys = []
>>> for i in ['7', '8', '9']:
...     try:
...         keys.append(things[i])
...     except KeyError:
...         pass
...     
... 
>>> keys
['seven', 'nine']

Copy link
Member

@pirat89 pirat89 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ouch. I missed that too. can you please add unit-tests that will cover the changed behaviour?

@evgeni evgeni force-pushed the keyerror branch 12 times, most recently from 5daf43f to 775693b Compare August 23, 2024 19:37
@evgeni
Copy link
Member Author

evgeni commented Aug 23, 2024

@pirat89 done!

@evgeni evgeni force-pushed the keyerror branch 2 times, most recently from a1744ec to f575b00 Compare August 24, 2024 06:37
We want to load all possible keys, even *after* a KeyError happenend

Fixes: 7e0fb44
@evgeni
Copy link
Member Author

evgeni commented Aug 24, 2024

testing farm seems to have a bad day:

"stderr": "The server rate limit has been exceeded, please try again later."

@pirat89
Copy link
Member

pirat89 commented Aug 26, 2024

testing farm seems to have a bad day:

"stderr": "The server rate limit has been exceeded, please try again later."

there has been an internal outage. re-running

@pirat89
Copy link
Member

pirat89 commented Aug 26, 2024

/packit copr-build

@evgeni
Copy link
Member Author

evgeni commented Aug 28, 2024

/packit retest-failed

@pirat89
Copy link
Member

pirat89 commented Aug 28, 2024

@evgeni ignore failed tests with "ondemand" substring. We are aware that these are failing most likely due to some changes in tests / test tooling / configuration which we missed.

@evgeni
Copy link
Member Author

evgeni commented Aug 28, 2024

Okay, then this is as green as it gets I guess

@pirat89 pirat89 merged commit bf302fc into oamg:master Aug 30, 2024
29 of 32 checks passed
@pirat89 pirat89 added the changelog-checked The merger/reviewer checked the changelog draft document and updated it when relevant label Aug 30, 2024
pirat89 added a commit to pirat89/leapp-repository that referenced this pull request Feb 14, 2025
## Announce
This is the last upstream release covering IPU 7 -> 8. The
system_upgrade_el7toel8 repository is going to be removed from further
releases. Other artifacts related to IPU 7 -> 8 are going to be removed too.

In the same time it's the last release compatible with Python 2.7. Since now
we are going to develope and test the code only for Python 3.6 and newer.

## Packaging
- [IPU 9 -> 10] Require libdb-utils on RHEL 9 (oamg#1289)
- Require leapp-framework 6.0+ (oamg#1142)
- Update leapp-deps package to satisfy leapp_framework_deps 6 (oamg#1142)

## Upgrade handling
### Fixes
- Activate LVM VGs with `--sysinit` option to correct the use in the upgrade initramfs (oamg#1329)
- Cap max size of the sparse files to 1TiB when storage with large amount of space (oamg#1294)
- Fix IPU being blocked by resource limitations (oamg#1256)
- Fix pes events scanner crashing when there are duplicate packages in the received instructions (oamg#1296)
- Fix pes events scanner not respecting user’s transaction configuration (oamg#1296)
- Fix storage scanner crashing when command outputs contain colon character (oamg#1258)
- Fix the report when handling broken parsing of kernel cmdline (oamg#1290)
- Generate proper error message instead ModelViolationError when parsing invalid repository definition (oamg#1266)
- Handle default kernel cmdline when multiple boot entries for the default kernel are defined (oamg#1302)
- Load all available obsoleted keys (oamg#1286)
- Prevent a possible crash with LiveMode when adding the upgrade boot entry on LVM systems (oamg#1313)
- Prevent failure for upgrade paths when obsoleted keys are not defined (oamg#1285)
- [IPU 7 -> 8] Ignore invalid firewalld configuration (oamg#1341)
- [IPU 7 -> 8] scangrubdevpartitionlayout: Skip warning msgs from fdisk (oamg#1306)
- [IPU 8 -> 9] Fix problems with the bootloader when upgrading to RHEL 9.6 on ARM (oamg#1275, oamg#1331, oamg#1339)
- [IPU 9 -> 10] Fix output of commands executed inside systemd-nspawn containers (oamg#1323)
- [IPU 9 -> 10] Fix remediation instructions for deprecated NM configuration (oamg#1330)
- [IPU 9 -> 10] Obsolete RHEL9 GPG key signed with SHA1 (oamg#1325)

### Enhancements
- Add RHEL 9.6 and 10.0 product certificates (oamg#1287, oamg#1309)
- Introduce new IPU paths 8.10 -> 9.6 and 9.6 -> 10.0 (oamg#1309)
- Add possibility to use net.naming-scheme (for IPU 8 -> 9 only for now) (oamg#1215, oamg#1312)
- Drop upgrade paths related to RHEL 8.8, 9.2, and 9.5 (oamg#1309, oamg#1344)
- Enable upgrade for EL8+ systems with LUKS bound to Clevis with TPM 2.0 token (oamg#1200)
- Enable upgrade with RHUI on Alibaba cloud for ARM machines (oamg#1277)
- Introduce a possibility to configure leapp actors covering RHUI on clouds (oamg#1142)
- Minor improvements in preupgrade reports (oamg#1315, oamg#1326, oamg#1342)
- Raise an inhibitor if unsupported target version supplied instead of error (oamg#1328)
- Skip checking of (PKI) `directory-hash` dir to speedup the upgrade process and clean logs (oamg#1297)
- Update leapp upgrade data files (oamg#1307)
- [IPU 9 -> 10] Cover upgrades for MySQL and PostgreSQL databases (oamg#1316, oamg#1343)
- [IPU 9 -> 10] Detect OpenSSL engines configured in /etc/pki/tls/openssl.cnf (oamg#1338)
- [IPU 9 -> 10] Detect XFS file systems with problematic parameters (oamg#1318)
- [IPU 9 -> 10] Detect deprecated network-scripts (ifcfg) files (oamg#1332, oamg#1347)
- [IPU 9 -> 10] Detect whether subscribed systems are using SCA (oamg#1333)
- [IPU 9 -> 10] Handle correctly switch of symlink to directory when upgrading Ruby IRB (oamg#1304)
- [IPU 9 -> 10] Inform user about Libdb removal if present (oamg#1319)
- [IPU 9 -> 10] Inhibit the upgrade for Intel CPUs with microarchitecture x86_64-v2 (oamg#1196)
- [IPU 9 -> 10] Move crypto-policies related actors to the system_upgrade_common repository to apply it for IPU 9 -> 10 as well
- [IPU 9 -> 10] Update checks for SAP HANA (oamg#1346)
- [IPU 9 -> 10] Update pam_userdb database backend for RHEL10 (oamg#1289)
pirat89 added a commit to pirat89/leapp-repository that referenced this pull request Feb 14, 2025
## Announce
This is the last upstream release covering IPU 7 -> 8. The
system_upgrade_el7toel8 repository is going to be removed from further
releases. Other artifacts related to IPU 7 -> 8 are going to be removed too.

In the same time it's the last release compatible with Python 2.7. Since now
we are going to develope and test the code only for Python 3.6 and newer.

## Packaging
- [IPU 9 -> 10] Require libdb-utils on RHEL 9 (oamg#1289)
- Require leapp-framework 6.0+ (oamg#1142)
- Update leapp-deps package to satisfy leapp_framework_deps 6 (oamg#1142)

## Upgrade handling
### Fixes
- Activate LVM VGs with `--sysinit` option to correct the use in the upgrade initramfs (oamg#1329)
- Cap max size of the sparse files to 1TiB when storage with large amount of space (oamg#1294)
- Fix IPU being blocked by resource limitations (oamg#1256)
- Fix pes events scanner crashing when there are duplicate packages in the received instructions (oamg#1296)
- Fix pes events scanner not respecting user’s transaction configuration (oamg#1296)
- Fix storage scanner crashing when command outputs contain colon character (oamg#1258)
- Fix the report when handling broken parsing of kernel cmdline (oamg#1290)
- Generate proper error message instead ModelViolationError when parsing invalid repository definition (oamg#1266)
- Handle default kernel cmdline when multiple boot entries for the default kernel are defined (oamg#1302)
- Load all available obsoleted keys (oamg#1286)
- Prevent a possible crash with LiveMode when adding the upgrade boot entry on LVM systems (oamg#1313)
- Prevent failure for upgrade paths when obsoleted keys are not defined (oamg#1285)
- [IPU 7 -> 8] Ignore invalid firewalld configuration (oamg#1341)
- [IPU 7 -> 8] scangrubdevpartitionlayout: Skip warning msgs from fdisk (oamg#1306)
- [IPU 8 -> 9] Fix problems with the bootloader when upgrading to RHEL 9.6 on ARM (oamg#1275, oamg#1331, oamg#1339)
- [IPU 9 -> 10] Fix output of commands executed inside systemd-nspawn containers (oamg#1323)
- [IPU 9 -> 10] Fix remediation instructions for deprecated NM configuration (oamg#1330)
- [IPU 9 -> 10] Obsolete RHEL9 GPG key signed with SHA1 (oamg#1325)

### Enhancements
- Add RHEL 9.6 and 10.0 product certificates (oamg#1287, oamg#1309)
- Introduce new IPU paths 8.10 -> 9.6 and 9.6 -> 10.0 (oamg#1309)
- Add possibility to use net.naming-scheme (for IPU 8 -> 9 only for now) (oamg#1215, oamg#1312)
- Drop upgrade paths related to RHEL 8.8, 9.2, and 9.5 (oamg#1309, oamg#1344)
- Enable upgrade for EL8+ systems with LUKS bound to Clevis with TPM 2.0 token (oamg#1200)
- Enable upgrade with RHUI on Alibaba cloud for ARM machines (oamg#1277)
- Introduce a possibility to configure leapp actors covering RHUI on clouds (oamg#1142)
- Minor improvements in preupgrade reports (oamg#1315, oamg#1326, oamg#1342)
- Raise an inhibitor if unsupported target version supplied instead of error (oamg#1328)
- Skip checking of (PKI) `directory-hash` dir to speedup the upgrade process and clean logs (oamg#1297)
- Update leapp upgrade data files (oamg#1307)
- [IPU 9 -> 10] Cover upgrades for MySQL and PostgreSQL databases (oamg#1316, oamg#1343)
- [IPU 9 -> 10] Detect OpenSSL engines configured in /etc/pki/tls/openssl.cnf (oamg#1338)
- [IPU 9 -> 10] Detect XFS file systems with problematic parameters (oamg#1318)
- [IPU 9 -> 10] Detect deprecated network-scripts (ifcfg) files (oamg#1332, oamg#1347)
- [IPU 9 -> 10] Detect whether subscribed systems are using SCA (oamg#1333)
- [IPU 9 -> 10] Handle correctly switch of symlink to directory when upgrading Ruby IRB (oamg#1304)
- [IPU 9 -> 10] Inform user about Libdb removal if present (oamg#1319)
- [IPU 9 -> 10] Inhibit the upgrade for Intel CPUs with microarchitecture x86_64-v2 (oamg#1196)
- [IPU 9 -> 10] Move crypto-policies related actors to the system_upgrade_common repository to apply it for IPU 9 -> 10 as well
- [IPU 9 -> 10] Update checks for SAP HANA (oamg#1346)
- [IPU 9 -> 10] Update pam_userdb database backend for RHEL10 (oamg#1289)

## Additional changes interesting for devels
- Introducing new upstream documentation focused on the leapp-repository
  project at https://leapp-repository.readthedocs.io/introduce-docs/.
  The documentation is still under reconstruction at this time, however we
  consider it a big step. All PRs impacting stuff that should be documented
  will be required to do so before they could be merged.
@pirat89 pirat89 mentioned this pull request Feb 14, 2025
pirat89 added a commit to pirat89/leapp-repository that referenced this pull request Feb 14, 2025
## Announce
This is the last upstream release covering IPU 7 -> 8. The
system_upgrade_el7toel8 repository is going to be removed from further
releases. Other artifacts related to IPU 7 -> 8 are going to be removed too.

In the same time it's the last release compatible with Python 2.7. Since now
we are going to develope and test the code only for Python 3.6 and newer.

## Packaging
- [IPU 9 -> 10] Require libdb-utils on RHEL 9 (oamg#1289)
- Require leapp-framework 6.0+ (oamg#1142)
- Update leapp-deps package to satisfy leapp_framework_deps 6 (oamg#1142)

## Upgrade handling
### Fixes
- Activate LVM VGs with `--sysinit` option to correct the use in the upgrade initramfs (oamg#1329)
- Cap max size of the sparse files to 1TiB when storage with large amount of space (oamg#1294)
- Fix IPU being blocked by resource limitations (oamg#1256)
- Fix pes events scanner crashing when there are duplicate packages in the received instructions (oamg#1296)
- Fix pes events scanner not respecting user’s transaction configuration (oamg#1296)
- Fix storage scanner crashing when command outputs contain colon character (oamg#1258)
- Fix the report when handling broken parsing of kernel cmdline (oamg#1290)
- Generate proper error message instead ModelViolationError when parsing invalid repository definition (oamg#1266)
- Handle default kernel cmdline when multiple boot entries for the default kernel are defined (oamg#1302)
- Load all available obsoleted keys (oamg#1286)
- Prevent a possible crash with LiveMode when adding the upgrade boot entry on LVM systems (oamg#1313)
- Prevent failure for upgrade paths when obsoleted keys are not defined (oamg#1285)
- [IPU 7 -> 8] Ignore invalid firewalld configuration (oamg#1341)
- [IPU 7 -> 8] scangrubdevpartitionlayout: Skip warning msgs from fdisk (oamg#1306)
- [IPU 8 -> 9] Fix problems with the bootloader when upgrading to RHEL 9.6 on ARM (oamg#1275, oamg#1331, oamg#1339)
- [IPU 9 -> 10] Fix output of commands executed inside systemd-nspawn containers (oamg#1323)
- [IPU 9 -> 10] Fix remediation instructions for deprecated NM configuration (oamg#1330)
- [IPU 9 -> 10] Obsolete RHEL9 GPG key signed with SHA1 (oamg#1325)

### Enhancements
- Add RHEL 9.6 and 10.0 product certificates (oamg#1287, oamg#1309)
- Introduce new IPU paths 8.10 -> 9.6 and 9.6 -> 10.0 (oamg#1309)
- Add possibility to use net.naming-scheme (for IPU 8 -> 9 only for now) (oamg#1215, oamg#1312)
- Drop upgrade paths related to RHEL 8.8, 9.2, and 9.5 (oamg#1309, oamg#1344)
- Enable upgrade for EL8+ systems with LUKS bound to Clevis with TPM 2.0 token (oamg#1200)
- Enable upgrade with RHUI on Alibaba cloud for ARM machines (oamg#1277)
- Introduce a possibility to configure leapp actors covering RHUI on clouds (oamg#1142)
- Minor improvements in preupgrade reports (oamg#1315, oamg#1326, oamg#1342)
- Raise an inhibitor if unsupported target version supplied instead of error (oamg#1328)
- Skip checking of (PKI) `directory-hash` dir to speedup the upgrade process and clean logs (oamg#1297)
- Update leapp upgrade data files (oamg#1307)
- [IPU 9 -> 10] Cover upgrades for MySQL and PostgreSQL databases (oamg#1316, oamg#1343)
- [IPU 9 -> 10] Detect OpenSSL engines configured in /etc/pki/tls/openssl.cnf (oamg#1338)
- [IPU 9 -> 10] Detect XFS file systems with problematic parameters (oamg#1318)
- [IPU 9 -> 10] Detect deprecated network-scripts (ifcfg) files (oamg#1332, oamg#1347)
- [IPU 9 -> 10] Detect whether subscribed systems are using SCA (oamg#1333)
- [IPU 9 -> 10] Handle correctly switch of symlink to directory when upgrading Ruby IRB (oamg#1304)
- [IPU 9 -> 10] Inform user about Libdb removal if present (oamg#1319)
- [IPU 9 -> 10] Inhibit the upgrade for Intel CPUs with microarchitecture x86_64-v2 (oamg#1196)
- [IPU 9 -> 10] Move crypto-policies related actors to the system_upgrade_common repository to apply it for IPU 9 -> 10 as well
- [IPU 9 -> 10] Update checks for SAP HANA (oamg#1346)
- [IPU 9 -> 10] Update pam_userdb database backend for RHEL10 (oamg#1289)

## Additional changes interesting for devels
- Introducing new upstream documentation focused on the leapp-repository
  project at https://leapp-repository.readthedocs.io/introduce-docs/.
  The documentation is still under reconstruction at this time, however we
  consider it a big step. All PRs impacting stuff that should be documented
  will be required to do so before they could be merged.
pirat89 added a commit that referenced this pull request Feb 14, 2025
## Announce
This is the last upstream release covering IPU 7 -> 8. The
system_upgrade_el7toel8 repository is going to be removed from further
releases. Other artifacts related to IPU 7 -> 8 are going to be removed too.

In the same time it's the last release compatible with Python 2.7. Since now
we are going to develope and test the code only for Python 3.6 and newer.

## Packaging
- [IPU 9 -> 10] Require libdb-utils on RHEL 9 (#1289)
- Require leapp-framework 6.0+ (#1142)
- Update leapp-deps package to satisfy leapp_framework_deps 6 (#1142)

## Upgrade handling
### Fixes
- Activate LVM VGs with `--sysinit` option to correct the use in the upgrade initramfs (#1329)
- Cap max size of the sparse files to 1TiB when storage with large amount of space (#1294)
- Fix IPU being blocked by resource limitations (#1256)
- Fix pes events scanner crashing when there are duplicate packages in the received instructions (#1296)
- Fix pes events scanner not respecting user’s transaction configuration (#1296)
- Fix storage scanner crashing when command outputs contain colon character (#1258)
- Fix the report when handling broken parsing of kernel cmdline (#1290)
- Generate proper error message instead ModelViolationError when parsing invalid repository definition (#1266)
- Handle default kernel cmdline when multiple boot entries for the default kernel are defined (#1302)
- Load all available obsoleted keys (#1286)
- Prevent a possible crash with LiveMode when adding the upgrade boot entry on LVM systems (#1313)
- Prevent failure for upgrade paths when obsoleted keys are not defined (#1285)
- [IPU 7 -> 8] Ignore invalid firewalld configuration (#1341)
- [IPU 7 -> 8] scangrubdevpartitionlayout: Skip warning msgs from fdisk (#1306)
- [IPU 8 -> 9] Fix problems with the bootloader when upgrading to RHEL 9.6 on ARM (#1275, #1331, #1339)
- [IPU 9 -> 10] Fix output of commands executed inside systemd-nspawn containers (#1323)
- [IPU 9 -> 10] Fix remediation instructions for deprecated NM configuration (#1330)
- [IPU 9 -> 10] Obsolete RHEL9 GPG key signed with SHA1 (#1325)

### Enhancements
- Add RHEL 9.6 and 10.0 product certificates (#1287, #1309)
- Introduce new IPU paths 8.10 -> 9.6 and 9.6 -> 10.0 (#1309)
- Add possibility to use net.naming-scheme (for IPU 8 -> 9 only for now) (#1215, #1312)
- Drop upgrade paths related to RHEL 8.8, 9.2, and 9.5 (#1309, #1344)
- Enable upgrade for EL8+ systems with LUKS bound to Clevis with TPM 2.0 token (#1200)
- Enable upgrade with RHUI on Alibaba cloud for ARM machines (#1277)
- Introduce a possibility to configure leapp actors covering RHUI on clouds (#1142)
- Minor improvements in preupgrade reports (#1315, #1326, #1342)
- Raise an inhibitor if unsupported target version supplied instead of error (#1328)
- Skip checking of (PKI) `directory-hash` dir to speedup the upgrade process and clean logs (#1297)
- Update leapp upgrade data files (#1307)
- [IPU 9 -> 10] Cover upgrades for MySQL and PostgreSQL databases (#1316, #1343)
- [IPU 9 -> 10] Detect OpenSSL engines configured in /etc/pki/tls/openssl.cnf (#1338)
- [IPU 9 -> 10] Detect XFS file systems with problematic parameters (#1318)
- [IPU 9 -> 10] Detect deprecated network-scripts (ifcfg) files (#1332, #1347)
- [IPU 9 -> 10] Detect whether subscribed systems are using SCA (#1333)
- [IPU 9 -> 10] Handle correctly switch of symlink to directory when upgrading Ruby IRB (#1304)
- [IPU 9 -> 10] Inform user about Libdb removal if present (#1319)
- [IPU 9 -> 10] Inhibit the upgrade for Intel CPUs with microarchitecture x86_64-v2 (#1196)
- [IPU 9 -> 10] Move crypto-policies related actors to the system_upgrade_common repository to apply it for IPU 9 -> 10 as well
- [IPU 9 -> 10] Update checks for SAP HANA (#1346)
- [IPU 9 -> 10] Update pam_userdb database backend for RHEL10 (#1289)

## Additional changes interesting for devels
- Introducing new upstream documentation focused on the leapp-repository
  project at https://leapp-repository.readthedocs.io/introduce-docs/.
  The documentation is still under reconstruction at this time, however we
  consider it a big step. All PRs impacting stuff that should be documented
  will be required to do so before they could be merged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
changelog-checked The merger/reviewer checked the changelog draft document and updated it when relevant
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants