-
Notifications
You must be signed in to change notification settings - Fork 152
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enable 8>9 upgrades with FIPS enabled #1053
Conversation
Thank you for contributing to the Leapp project!Please note that every PR needs to comply with the Leapp Guidelines and must pass all tests in order to be mergeable.
To launch regression testing public members of oamg organization can leave the following comment:
Please open ticket in case you experience technical problem with the CI. (RH internal only) Note: In case there are problems with tests not being triggered automatically on new PR/commit or pending for a long time, please consider rerunning the CI by commenting leapp-ci build (might require several comments). If the problem persists, contact leapp-infra. |
0fa75d6
to
f2074b2
Compare
repos/system_upgrade/common/actors/initramfs/checkfipsenabled/actor.py
Outdated
Show resolved
Hide resolved
df0dbd2
to
62afa46
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One minor nitpick, the rest looks good to me.
repos/system_upgrade/common/actors/initramfs/checkfipsenabled/actor.py
Outdated
Show resolved
Hide resolved
2ef89fb
to
bc94a87
Compare
14dbde8
to
035486f
Compare
/rerun |
Copr build succeeded: https://copr.fedorainfracloud.org/coprs/build/5709680 |
Testing Farm request for RHEL-8.6-rhui/5709680 regression testing has been created. |
Testing Farm request for RHEL-7.9-rhui/5709680 regression testing has been created. |
Testing Farm request for RHEL-8.6.0-Nightly/5709680 regression testing has been created. |
Testing Farm request for RHEL-8.8.0-Nightly/5709680 regression testing has been created. |
Testing Farm request for RHEL-7.9-ZStream/5709680 regression testing has been created. |
Testing Farm request for RHEL-7.9-ZStream/5709680 regression testing has been created. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am yet to finish my testing but so far looks good to me. I suggest squashing some comments (not all of them as a good part of them makes sense). Thanks o/
...ade/common/actors/initramfs/upgradeinitramfsgenerator/libraries/upgradeinitramfsgenerator.py
Outdated
Show resolved
Hide resolved
Everything seems to be working fine after testing. @pirat89 mentioned he would take a look tomorrow. |
f1d17cd
to
857a257
Compare
@Rezney I have reordered and cleaned up the git history. I was keeping the history in a messy state in case further touches would be required during reviews. Furthermore, commit hash is included in the leapp built I have used during testing, so you can identify the PR state directly from the logs attaches to the ticket. |
Discussed one potential issue in person (under investigation now). Except that, everything else is good to go - just the code review without manual testing. |
/packit build |
2612e1d
to
c795a64
Compare
c795a64
to
f5ede13
Compare
/packit build |
@pirat89 I have rerun the upgrade after my latest changes removing some of the unnecessary logic, the upgrade went OK. |
# This is the 1st commit message: Fix dead link in checkipaserver actor # This is the commit message oamg#2: checkhybridimage: Fix the produce of the report The actor can produce the report, however the report is never stored / accepted by the leapp framework as the Report has not been set in the `produces` tuple. Fixing the actor. Signed-off-by: Petr Stodulka <[email protected]> # This is the commit message oamg#3: Upgrade packit.yaml config to have integration tests This commit introduces the execution of leapp-repository integration tests as a packit job. Signed-off-by: Rodolfo Olivieri <[email protected]> # This is the commit message oamg#4: Update packit config to match the leapp-repositoyr tests Signed-off-by: Rodolfo Olivieri <[email protected]> # This is the commit message oamg#5: Add new environment variable to 8.8to9.2 Signed-off-by: Rodolfo Olivieri <[email protected]> # This is the commit message oamg#6: Stop mentioning the "releasever" file removal Do not mention the "releasever" file removal in order to not confuse users. # This is the commit message oamg#7: Fix trace with impossible LEAPP_DEVEL_TARGET_RELEASE With this change the (pre)upgrade will correctly handle impossible target release version, no more ugly trace will be shown. OAMG-8651 # This is the commit message oamg#8: Make copr-build functioning again After some unknown changes around COPR, the building command and the used COPR configuration file needs to be updated. OAMG-8876 # This is the commit message oamg#9: Add tag in packit.yaml to enable cost metrics collection Now all tft tests run by packit should be marked accordingly with a sst-upgrades tag. OAMG-8892 # This is the commit message oamg#10: Workaround packit#2010 issue Looks like tf_post_install_script and environment override does not play nice together yet, so let's workaround it for now. OAMG-8892 # This is the commit message oamg#11: Improve the "checkgrubcore" report message No action is needed in case Leapp is able to detect the GRUB2 device # This is the commit message oamg#12: Update pr-welcome-msg with packit tests info Let's mention the big leap for leapp officially together with a command to (re-)trigger tests. # This is the commit message oamg#13: Further tune welcome-bot message - Do not use master, use latest upstream - Add precise command to request review from oamg-developers # This is the commit message oamg#14: Remove note about leapp-ci build As leapp packages are built by packit now, this is not used anymore. # This is the commit message oamg#15: Fix false positive non-utf symlinks reported Because of botched up check for python2 valid utf symlinks were reported as non-utf ones. OAMG-8629 # This is the commit message oamg#16: Refactor rootscanner to use library Also introduce tests for the nonutf symlinks # This is the commit message oamg#17: update .pylintrc # This is the commit message oamg#18: Set encoding for tests # This is the commit message oamg#19: Introduce leapp data in the RPM & repository In the past it was needed to obtain the data by * automatic download of data files from RH Insights (cloud.redhat.com) - which required access to the server & have the system registered * manual download from the article: https://access.redhat.com/articles/3664871 which required to login to the portal, download the archive and install its content manually on the system Additional problem was with the syncing of data, as because of the separation of data from the code, people had ensure they are using the right combination of data and SW manually - in case the data has not been downloaded automatically from RH Insights. Having the data in the RPM makes our lives easier so let's install them to `/etc/leapp/files/` via the package directly. Set /etc/leapp/files/* as configuration files to ensure that modified files will be backed up with *.rpmsave suffix as expected for configuration files. This could be important in case users manually updated e.g. repomap.json file to reflect their internal settings of the satellite server. The complete table how the configuration files are handled during rpm installation/upgrade/... is here: https://www.cl.cam.ac.uk/~jw35/docs/rpm_config.html Keeping original functionality still available, so if people remove the files from the system, data can be still downloaded from the server automatically. It seems it does not make sense, but we could still use the possibility to download more up-to-date data from RH Insights. Configure codespell to ignore .etc/leapp/files # This is the commit message oamg#20: Add el8toel9 actor to handle directory -> symlink with ruby IRB. The `/usr/share/ruby/irb/` directory is a symlink in RHEL 9. Simply remove the directory to then let the RPM mechanism create the correct symlink on update. Users should not expect to ever retain anything in this directory. # This is the commit message oamg#21: Enable 8>9 upgrades with FIPS enabled (oamg#1053) Short story long: ============== FIPS refers to a set of security standards governing many aspects of how information should be handled by computers and by people. FIPS in context of RHEL typically means FIPS 140, a single document defining rules for the use of encryption and cryptographic services. In essence, it defines requirements for cryptographic modules (e.g. what algorithms can be used, thus preventing the use of insecure ones) manipulating sensitive information. From the point of view of upgrades there are 5 components that are certified under FIPS: kernel, OpenSSL, GnuTLS, NSS, and libgcrypt. As for the kernel, fips=1 needs to be present on the cmdline in order to enable FIPS in the kernel. Kernel offers a /proc/sys/crypto/fips_enabled virtual file containing the information about whether the kernel has booted with FIPS enabled. According to FIPS, the userspace components need to verify that they were not tampered with, and thus they have to have some sort of checksum either in a special section of a binary, or in a separate file. The components read /proc/sys/crypto/fips_enabled to know when to switch into the FIPS enabled mode. OpenSSL does things a bit differently, by not including support for FIPS directly but via a special fips.so module. Furthermore, for OpenSSL to check whether FIPS is enabled a configuration file must be present at /etc/pki/tls/openssl.cnf, because the code checking for FIPS is a part of configuration parsing. As every userspace component has different implementation of FIPS-mode, and a different way for the user to check whether the component believes that the system it is running on has FIPS enabled, there is no straightforward way for us to make really sure that all of the components run in FIPS enabled inside the target userspace container. Brief summary of changes in the code: * scanfips: split scanning for fips into a separate actor * checkfips: inhibit the IPU only on 7>8 * in case of FIPS for IPU 8 -> 9, copy related files into the target container to be able to generate FIPS compliant initramfs * checkfipsenabled: check for fips in upgrade initramfs (LastTestsPhase); interrupt the upgrade if FIPS is not enabled in the upgrade environment * upgradeinitramfsgenerator: refactor library and tests due to FIPS * create upgrade kernel hmac unconditionallly --------- Co-authored-by: Michal Hecko <[email protected]> # This is the commit message oamg#22: Change the upgrade paths for SAP HANA - Drop 7.9 to 8.2 - Add 7.9 to 8.8, but keep 7.9 to 8.6 as default - Add 8.8 to 9.2 - Drop SAP HANA version check for the target releases >=8.8 and >=9.2 - Correct actor.py docstring to support ppc64le for 8to9 upgrade (see PR1042) # This is the commit message oamg#23: Inhibit unsupported x86-64 microarchitecture RHEL9 As per [x86-64-ABI][1] In addition to the AMD64 baseline architecture, several micro-architecture levels implemented by later CPU modules have been defined, starting at level ``x86-64-v2``. RHEL9 has a higher CPU requirement than older versions, it now requires a CPU compatible with ``x86-64-v2`` instruction set or higher. Until now, there was no check for this and the upgrade crashed unexpectedly. This commit handles this issue and provides the user with a report explaining the problem. The CPU Features are gathered using the ``lscpu`` command by way of using the ``ScanCPU`` actor. The ``ScanCPU`` actor had to be also modified to provide the required flags. The mapping of CPU Features to flags provided by ``lscpu`` has been determined by using the ``/arch/x86/include/asm/cpufeatures.h`` file from the linux kernel. [1]: https://gitlab.com/x86-psABIs/x86-64-ABI.git
checkhybridimage: Fix the produce of the report The actor can produce the report, however the report is never stored / accepted by the leapp framework as the Report has not been set in the `produces` tuple. Fixing the actor. Signed-off-by: Petr Stodulka <[email protected]> Upgrade packit.yaml config to have integration tests This commit introduces the execution of leapp-repository integration tests as a packit job. Signed-off-by: Rodolfo Olivieri <[email protected]> Update packit config to match the leapp-repositoyr tests Signed-off-by: Rodolfo Olivieri <[email protected]> Add new environment variable to 8.8to9.2 Signed-off-by: Rodolfo Olivieri <[email protected]> Stop mentioning the "releasever" file removal Do not mention the "releasever" file removal in order to not confuse users. Fix trace with impossible LEAPP_DEVEL_TARGET_RELEASE With this change the (pre)upgrade will correctly handle impossible target release version, no more ugly trace will be shown. OAMG-8651 Make copr-build functioning again After some unknown changes around COPR, the building command and the used COPR configuration file needs to be updated. OAMG-8876 Add tag in packit.yaml to enable cost metrics collection Now all tft tests run by packit should be marked accordingly with a sst-upgrades tag. OAMG-8892 Workaround packit#2010 issue Looks like tf_post_install_script and environment override does not play nice together yet, so let's workaround it for now. OAMG-8892 Improve the "checkgrubcore" report message No action is needed in case Leapp is able to detect the GRUB2 device Update pr-welcome-msg with packit tests info Let's mention the big leap for leapp officially together with a command to (re-)trigger tests. Further tune welcome-bot message - Do not use master, use latest upstream - Add precise command to request review from oamg-developers Remove note about leapp-ci build As leapp packages are built by packit now, this is not used anymore. Fix false positive non-utf symlinks reported Because of botched up check for python2 valid utf symlinks were reported as non-utf ones. OAMG-8629 Refactor rootscanner to use library Also introduce tests for the nonutf symlinks update .pylintrc Set encoding for tests Introduce leapp data in the RPM & repository In the past it was needed to obtain the data by * automatic download of data files from RH Insights (cloud.redhat.com) - which required access to the server & have the system registered * manual download from the article: https://access.redhat.com/articles/3664871 which required to login to the portal, download the archive and install its content manually on the system Additional problem was with the syncing of data, as because of the separation of data from the code, people had ensure they are using the right combination of data and SW manually - in case the data has not been downloaded automatically from RH Insights. Having the data in the RPM makes our lives easier so let's install them to `/etc/leapp/files/` via the package directly. Set /etc/leapp/files/* as configuration files to ensure that modified files will be backed up with *.rpmsave suffix as expected for configuration files. This could be important in case users manually updated e.g. repomap.json file to reflect their internal settings of the satellite server. The complete table how the configuration files are handled during rpm installation/upgrade/... is here: https://www.cl.cam.ac.uk/~jw35/docs/rpm_config.html Keeping original functionality still available, so if people remove the files from the system, data can be still downloaded from the server automatically. It seems it does not make sense, but we could still use the possibility to download more up-to-date data from RH Insights. Configure codespell to ignore .etc/leapp/files Add el8toel9 actor to handle directory -> symlink with ruby IRB. The `/usr/share/ruby/irb/` directory is a symlink in RHEL 9. Simply remove the directory to then let the RPM mechanism create the correct symlink on update. Users should not expect to ever retain anything in this directory. Enable 8>9 upgrades with FIPS enabled (oamg#1053) Short story long: ============== FIPS refers to a set of security standards governing many aspects of how information should be handled by computers and by people. FIPS in context of RHEL typically means FIPS 140, a single document defining rules for the use of encryption and cryptographic services. In essence, it defines requirements for cryptographic modules (e.g. what algorithms can be used, thus preventing the use of insecure ones) manipulating sensitive information. From the point of view of upgrades there are 5 components that are certified under FIPS: kernel, OpenSSL, GnuTLS, NSS, and libgcrypt. As for the kernel, fips=1 needs to be present on the cmdline in order to enable FIPS in the kernel. Kernel offers a /proc/sys/crypto/fips_enabled virtual file containing the information about whether the kernel has booted with FIPS enabled. According to FIPS, the userspace components need to verify that they were not tampered with, and thus they have to have some sort of checksum either in a special section of a binary, or in a separate file. The components read /proc/sys/crypto/fips_enabled to know when to switch into the FIPS enabled mode. OpenSSL does things a bit differently, by not including support for FIPS directly but via a special fips.so module. Furthermore, for OpenSSL to check whether FIPS is enabled a configuration file must be present at /etc/pki/tls/openssl.cnf, because the code checking for FIPS is a part of configuration parsing. As every userspace component has different implementation of FIPS-mode, and a different way for the user to check whether the component believes that the system it is running on has FIPS enabled, there is no straightforward way for us to make really sure that all of the components run in FIPS enabled inside the target userspace container. Brief summary of changes in the code: * scanfips: split scanning for fips into a separate actor * checkfips: inhibit the IPU only on 7>8 * in case of FIPS for IPU 8 -> 9, copy related files into the target container to be able to generate FIPS compliant initramfs * checkfipsenabled: check for fips in upgrade initramfs (LastTestsPhase); interrupt the upgrade if FIPS is not enabled in the upgrade environment * upgradeinitramfsgenerator: refactor library and tests due to FIPS * create upgrade kernel hmac unconditionallly --------- Co-authored-by: Michal Hecko <[email protected]> Change the upgrade paths for SAP HANA - Drop 7.9 to 8.2 - Add 7.9 to 8.8, but keep 7.9 to 8.6 as default - Add 8.8 to 9.2 - Drop SAP HANA version check for the target releases >=8.8 and >=9.2 - Correct actor.py docstring to support ppc64le for 8to9 upgrade (see PR1042) Inhibit unsupported x86-64 microarchitecture RHEL9 As per [x86-64-ABI][1] In addition to the AMD64 baseline architecture, several micro-architecture levels implemented by later CPU modules have been defined, starting at level ``x86-64-v2``. RHEL9 has a higher CPU requirement than older versions, it now requires a CPU compatible with ``x86-64-v2`` instruction set or higher. Until now, there was no check for this and the upgrade crashed unexpectedly. This commit handles this issue and provides the user with a report explaining the problem. The CPU Features are gathered using the ``lscpu`` command by way of using the ``ScanCPU`` actor. The ``ScanCPU`` actor had to be also modified to provide the required flags. The mapping of CPU Features to flags provided by ``lscpu`` has been determined by using the ``/arch/x86/include/asm/cpufeatures.h`` file from the linux kernel. [1]: https://gitlab.com/x86-psABIs/x86-64-ABI.git Add kernel drivers Add debug msg Query kernel version through RPM Query kernel version through RPM Query kernel version through RPM Query kernel version through RPM Query kernel version through RPM Add kernel drivers Test add kernel modules Compress modules Remove modules Add kernel module Debug Debug Add tests for checkinitramfstasks Add tests for upgradeinitramfsgenerator Remove testing module Fix lint
## Packaging - Requires leapp-framework 5.0 ## Upgrade handling ### Fixes - Add el8toel9 actor to handle directory -> symlink with ruby IRB. (oamg#1076) - Do not try to update GRUB core on IBM Z systems (oamg#1117) - Fix failing upgrades with devtmpfs file systems specified in FSTAB (oamg#1090) - Fix the calculation of the required free space on each partitions/volume for the upgrade transactions (oamg#1097) - Fix the generation of the report about hybrid images (oamg#1064) - Handle correctly the installed certificates to allow upgrades with custom repositories using HTTPs with enabled SSL verification (oamg#1106) - Minor improvements and fixes of various reports (oamg#1066, oamg#1067, oamg#1085) - Update error messages about leapp data files to inform user how to obtain valid data files (oamg#1121) - Update links in various reports (oamg#1062, oamg#1086) - Update the repomap data to cover changed repoids in RHUI Azure (oamg#1087) - [IPU 7 -> 8] Fix false positive report about invalid symlinks on RHEL 7 (oamg#1052) - [IPU 8 -> 9] Inhibit the upgrade when unsupported x86-64 microarchitecture is detected (oamg#1059) ### Enhancements - Include updated leapp data files in the RPM (oamg#1046, oamg#1092, oamg#1119) - Update the set of supported upgrade paths (oamg#1077): - RHEL with SAP HANA 7.9 -> 8.6, 8.8 (default: 8.6) - RHEL with SAP HANA 8.8 -> 9.2 - Introduce new upgrade paths: - RHEL 7.9 -> 8.9 (default) - RHEL 8.9 -> 9.3 - Correctly update grub2 when /boot resides on multiple devices aggregated in RAID (oamg#1093, oamg#1115) - Enable upgrades for machines using RHUI on AlibabaCloud (oamg#1088) - Introduce possibility to add kernel drivers to initramfs (oamg#1081) - Redesign handling of information about kernel (booted and target) in preparation for new changes in RHEL 9 (oamg#1107) - Redesign source system overlay to use disk images backed by sparse files to optimize disk space consumption (oamg#1097, oamg#1103) - Requires leapp-framework 5.0 (oamg#1061, oamg#1116) - Use new leapp CLI API which provides better report summary output (oamg#1061, oamg#1116) - [IPU 8 -> 9] Detect and report use of deprecated Xorg drivers (oamg#1078) - [IPU 8 -> 9] Introduce IPU for systems with FIPS enabled (oamg#1053) ## Additional changes interesting for devels - Deprecated `GrubInfo.orig_device_name` field in the `GrubInfo` model (replaced by `GrubInfo.orig_devices`) (oamg#1093) - Deprecated `InstalledTargetKernelVersion` model (replaced by `InstalledTargetKernelInfo`) (oamg#1107) - Deprecated `leapp.libraries.common.config.version.is_rhel_realtime` (check the type in msg `KernelInfo`, field `type`) (oamg#1107) - Deprecated `leapp.libraries.common.grub.get_grub_device()` (replaced by `leapp.libraries.common.grub.get_grub_devices()`) (oamg#1093) - Introduced new devel envar LEAPP_DEVEL_KEEP_DISK_IMGS=1 to skip the removal of the created disk images for OVL. That's sometimes handy for the debugging. (oamg#1097)
## Packaging - Requires leapp-framework 5.0 ## Upgrade handling ### Fixes - Add el8toel9 actor to handle directory -> symlink with ruby IRB. (#1076) - Do not try to update GRUB core on IBM Z systems (#1117) - Fix failing upgrades with devtmpfs file systems specified in FSTAB (#1090) - Fix the calculation of the required free space on each partitions/volume for the upgrade transactions (#1097) - Fix the generation of the report about hybrid images (#1064) - Handle correctly the installed certificates to allow upgrades with custom repositories using HTTPs with enabled SSL verification (#1106) - Minor improvements and fixes of various reports (#1066, #1067, #1085) - Update error messages about leapp data files to inform user how to obtain valid data files (#1121) - Update links in various reports (#1062, #1086) - Update the repomap data to cover changed repoids in RHUI Azure (#1087) - [IPU 7 -> 8] Fix false positive report about invalid symlinks on RHEL 7 (#1052) - [IPU 8 -> 9] Inhibit the upgrade when unsupported x86-64 microarchitecture is detected (#1059) ### Enhancements - Include updated leapp data files in the RPM (#1046, #1092, #1119) - Update the set of supported upgrade paths (#1077): - RHEL with SAP HANA 7.9 -> 8.6, 8.8 (default: 8.6) - RHEL with SAP HANA 8.8 -> 9.2 - Introduce new upgrade paths: - RHEL 7.9 -> 8.9 (default) - RHEL 8.9 -> 9.3 - Correctly update grub2 when /boot resides on multiple devices aggregated in RAID (#1093, #1115) - Enable upgrades for machines using RHUI on AlibabaCloud (#1088) - Introduce possibility to add kernel drivers to initramfs (#1081) - Redesign handling of information about kernel (booted and target) in preparation for new changes in RHEL 9 (#1107) - Redesign source system overlay to use disk images backed by sparse files to optimize disk space consumption (#1097, #1103) - Requires leapp-framework 5.0 (#1061, #1116) - Use new leapp CLI API which provides better report summary output (#1061, #1116) - [IPU 8 -> 9] Detect and report use of deprecated Xorg drivers (#1078) - [IPU 8 -> 9] Introduce IPU for systems with FIPS enabled (#1053) ## Additional changes interesting for devels - Deprecated `GrubInfo.orig_device_name` field in the `GrubInfo` model (replaced by `GrubInfo.orig_devices`) (#1093) - Deprecated `InstalledTargetKernelVersion` model (replaced by `InstalledTargetKernelInfo`) (#1107) - Deprecated `leapp.libraries.common.config.version.is_rhel_realtime` (check the type in msg `KernelInfo`, field `type`) (#1107) - Deprecated `leapp.libraries.common.grub.get_grub_device()` (replaced by `leapp.libraries.common.grub.get_grub_devices()`) (#1093) - Introduced new devel envar LEAPP_DEVEL_KEEP_DISK_IMGS=1 to skip the removal of the created disk images for OVL. That's sometimes handy for the debugging. (#1097)
Add support for in-place 8>9 upgrades of FIPS-enabled systems
FIPS refers to a set of security standards governing many aspects of how information should be handled by computers and by people. FIPS in context of RHEL typically means FIPS 140, a single document defining rules for the use of encryption and cryptographic services. In essence, it defines requirements for cryptographic modules (e.g. what algorithms can be used, thus preventing the use of insecure ones) manipulating sensitive information.
From the point of view of upgrades there are 5 components that are certified under FIPS: kernel, OpenSSL, GnuTLS, NSS, and libgcrypt. As for the kernel,
fips=1
needs to be present on the cmdline in order to enable FIPS in the kernel. Kernel offers a/proc/sys/crypto/fips_enabled
virtual file containing the information about whether the kernel has booted with FIPS enabled.According to FIPS, the userspace components need to verify that they were not tampered with, and thus they have to have some sort of checksum either in a special section of a binary, or in a separate file. The components read
/proc/sys/crypto/fips_enabled
to know when to switch into the FIPS enabled mode. OpenSSL does things a bit differently, by not including support for FIPS directly but via a specialfips.so
module. Furthermore, for OpenSSL to check whether FIPS is enabled a configuration file must be present at/etc/pki/tls/openssl.cnf
, because the code checking for FIPS is a part of configuration parsing.As every userspace component has different implementation of FIPS-mode, and a different way for the user to check whether the component believes that the system it is running on has FIPS enabled, there is no straightforward way for us to make really sure that all of the components run in FIPS enabled inside the target userspace container.
Working around a bug in dracut
As @neverpanic discovered during his initial investigation of 8>9 upgrades with FIPS enabled, dracut currently contains a bug due to which it does not include some necessary files in the initramfs. Until the issue is resolved in dracut, a temporary workaround consisting of manually specifying files to be included in the initramfs is implemented.
Finally, this PR, including the majority of the above description (modulo possible mistakes introduced by @MichalHe), is based on the great work done by @neverpanic. Thanks!
ref: OAMG-7824