Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Respawn modules to use the system python interpreter #460

Merged

Conversation

gotmax23
Copy link
Contributor

@gotmax23 gotmax23 commented May 13, 2023

SUMMARY

The seboolean, selinux, firewalld, and firewalld_info modules depend on
system bindings that are only available for the default system python
interpreter. ansible-core is not packaged for the default system python
interpreter on RHEL 8 and 9. When automatic interpreter discovery does
not occur (e.g. when using implicit localhost [1]), ansible-core will
not use the system interpreter to run ansible modules and the
aforementioned modules will not work even if the bindings are installed.
The RHEL ansible-core maintainers as well as the EPEL ansible and
ansible-collection-* package maintainers (inc. me) have gotten multiple
bug reports about this. We have been telling people to fix their setup
to use the correct Python interpreter. Fortunately, ansible-core 2.11
and above have a module utility that'll respawn modules to use the
correct system interpreter.

[1] https://docs.ansible.com/ansible/latest/inventory/implicit_localhost.html

ISSUE TYPE
  • Feature Pull Request
COMPONENT NAME

seboolean
selinux
firewalld
firewalld_info

@softwarefactory-project-zuul
Copy link
Contributor

Build failed.
https://ansible.softwarefactory-project.io/zuul/buildset/4ac97d5b9bf142c9b8543800014b1b50

✔️ ansible-changelog-fragment SUCCESS in 14s
ansible-test-sanity-docker-devel FAILURE in 8m 27s (non-voting)
✔️ ansible-test-sanity-docker-milestone SUCCESS in 10m 48s (non-voting)
✔️ ansible-test-sanity-docker-stable-2.9 SUCCESS in 12m 23s
ansible-test-sanity-docker-stable-2.10 FAILURE in 8m 44s
✔️ ansible-test-sanity-docker-stable-2.11 SUCCESS in 11m 09s
✔️ ansible-test-sanity-docker-stable-2.12 SUCCESS in 11m 09s
✔️ ansible-test-sanity-docker-stable-2.13 SUCCESS in 8m 34s
✔️ ansible-test-sanity-docker-stable-2.14 SUCCESS in 12m 24s
✔️ ansible-test-units-posix-python39 SUCCESS in 5m 32s
✔️ ansible-test-units-posix-python310 SUCCESS in 8m 09s
✔️ ansible-galaxy-importer SUCCESS in 3m 34s
✔️ build-ansible-collection SUCCESS in 6m 39s

@gotmax23
Copy link
Contributor Author

recheck

@softwarefactory-project-zuul
Copy link
Contributor

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/8ba35e72273641aea63b78dc3090c6c9

✔️ ansible-changelog-fragment SUCCESS in 16s
ansible-test-sanity-docker-devel FAILURE in 8m 30s (non-voting)
✔️ ansible-test-sanity-docker-milestone SUCCESS in 12m 08s (non-voting)
✔️ ansible-test-sanity-docker-stable-2.9 SUCCESS in 10m 53s
✔️ ansible-test-sanity-docker-stable-2.10 SUCCESS in 9m 33s
✔️ ansible-test-sanity-docker-stable-2.11 SUCCESS in 13m 28s
✔️ ansible-test-sanity-docker-stable-2.12 SUCCESS in 8m 28s
✔️ ansible-test-sanity-docker-stable-2.13 SUCCESS in 8m 20s
✔️ ansible-test-sanity-docker-stable-2.14 SUCCESS in 10m 44s
✔️ ansible-test-units-posix-python39 SUCCESS in 4m 56s
✔️ ansible-test-units-posix-python310 SUCCESS in 5m 15s
✔️ ansible-galaxy-importer SUCCESS in 4m 18s
✔️ build-ansible-collection SUCCESS in 6m 38s

@gotmax23
Copy link
Contributor Author

The FreeBSD tests timed out

@gotmax23
Copy link
Contributor Author

/azp run

@azure-pipelines
Copy link

Commenter does not have sufficient privileges for PR 460 in repo ansible-collections/ansible.posix

@gotmax23 gotmax23 closed this May 13, 2023
@gotmax23 gotmax23 reopened this May 13, 2023
@softwarefactory-project-zuul
Copy link
Contributor

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/eb6d55f42eda4c679640afd37bf08027

✔️ ansible-changelog-fragment SUCCESS in 14s
ansible-test-sanity-docker-devel FAILURE in 11m 18s (non-voting)
✔️ ansible-test-sanity-docker-milestone SUCCESS in 9m 47s (non-voting)
✔️ ansible-test-sanity-docker-stable-2.9 SUCCESS in 13m 23s
✔️ ansible-test-sanity-docker-stable-2.10 SUCCESS in 9m 26s
✔️ ansible-test-sanity-docker-stable-2.11 SUCCESS in 9m 35s
✔️ ansible-test-sanity-docker-stable-2.12 SUCCESS in 8m 27s
✔️ ansible-test-sanity-docker-stable-2.13 SUCCESS in 8m 17s
✔️ ansible-test-sanity-docker-stable-2.14 SUCCESS in 7m 09s
✔️ ansible-test-units-posix-python39 SUCCESS in 5m 56s
✔️ ansible-test-units-posix-python310 SUCCESS in 5m 28s
✔️ ansible-galaxy-importer SUCCESS in 3m 20s
✔️ build-ansible-collection SUCCESS in 6m 39s

@gotmax23
Copy link
Contributor Author

I marked this as draft. I'd like to refactor this so that there's less duplicated code.

@gotmax23 gotmax23 marked this pull request as draft May 14, 2023 05:27
gotmax23 added 5 commits May 14, 2023 05:44
The seboolean, selinux, firewalld, and firewalld_info modules depend on
system bindings that are only available for the default system python
interpreter. ansible-core is not packaged for the default system python
interpreter on RHEL 8 and 9. When automatic interpreter discovery does
not occur (e.g. when using implicit localhost [1]), ansible-core will
not use the system interpreter to run ansible modules and the
aforementioned modules will not work even if the bindings are installed.
The RHEL ansible-core maintainers as well as the EPEL ansible and
ansible-collection-* package maintainers (inc. me) have gotten multiple
bug reports about this. We have been telling people to fix their setup
to use the correct Python interpreter. Fortunately, ansible-core 2.11
and above have a module utility that'll respawn modules to use the
correct system interpreter.

[1] https://docs.ansible.com/ansible/latest/inventory/implicit_localhost.html
@softwarefactory-project-zuul
Copy link
Contributor

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/c555e312e4fb44c78976869441a82d75

✔️ ansible-changelog-fragment SUCCESS in 15s
ansible-test-sanity-docker-devel FAILURE in 8m 25s (non-voting)
✔️ ansible-test-sanity-docker-milestone SUCCESS in 9m 23s (non-voting)
✔️ ansible-test-sanity-docker-stable-2.9 SUCCESS in 10m 28s
✔️ ansible-test-sanity-docker-stable-2.10 SUCCESS in 9m 24s
✔️ ansible-test-sanity-docker-stable-2.11 SUCCESS in 10m 10s
✔️ ansible-test-sanity-docker-stable-2.12 SUCCESS in 7m 17s
✔️ ansible-test-sanity-docker-stable-2.13 SUCCESS in 8m 57s
✔️ ansible-test-sanity-docker-stable-2.14 SUCCESS in 8m 07s
✔️ ansible-test-units-posix-python39 SUCCESS in 4m 45s
✔️ ansible-test-units-posix-python310 SUCCESS in 5m 14s
✔️ ansible-galaxy-importer SUCCESS in 3m 27s
✔️ build-ansible-collection SUCCESS in 6m 41s

@gotmax23 gotmax23 marked this pull request as ready for review May 14, 2023 16:05
@gotmax23
Copy link
Contributor Author

Ready for review

Copy link
Collaborator

@maxamillion maxamillion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is amazing, thank you! 👍

@maxamillion
Copy link
Collaborator

rebuild_merge

@maxamillion maxamillion added the mergeit Gate PR in Zuul CI label Nov 30, 2023
Copy link
Contributor

Build succeeded (gate pipeline).
https://ansible.softwarefactory-project.io/zuul/buildset/df34ab80342a461781fc8d24f0d4df19

✔️ ansible-changelog-fragment SUCCESS in 16s
ansible-test-sanity-docker-devel FAILURE in 17m 42s (non-voting)
ansible-test-sanity-docker-milestone FAILURE in 11m 20s (non-voting)
✔️ ansible-test-sanity-docker-stable-2.9 SUCCESS in 10m 37s
✔️ ansible-test-sanity-docker-stable-2.10 SUCCESS in 12m 39s
✔️ ansible-test-sanity-docker-stable-2.11 SUCCESS in 19m 40s
✔️ ansible-test-sanity-docker-stable-2.12 SUCCESS in 7m 54s
✔️ ansible-test-sanity-docker-stable-2.13 SUCCESS in 10m 04s
✔️ ansible-test-sanity-docker-stable-2.14 SUCCESS in 17m 59s
✔️ ansible-test-units-posix-python39 SUCCESS in 6m 22s
✔️ ansible-test-units-posix-python310 SUCCESS in 6m 06s
✔️ ansible-galaxy-importer SUCCESS in 3m 17s
✔️ build-ansible-collection SUCCESS in 7m 50s

@softwarefactory-project-zuul softwarefactory-project-zuul bot merged commit 6f95c8b into ansible-collections:main Nov 30, 2023
83 of 86 checks passed
lumiere-bot bot referenced this pull request in coolguy1771/home-ops Sep 16, 2024
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
|
[ansible.posix](https://github.com/ansible-collections/ansible.posix)
| galaxy-collection | minor | `1.5.4` -> `1.6.0` |

---

### Release Notes

<details>
<summary>ansible-collections/ansible.posix (ansible.posix)</summary>

###
[`v1.6.0`](https://github.com/ansible-collections/ansible.posix/blob/HEAD/CHANGELOG.rst#v160)

[Compare
Source](https://github.com/ansible-collections/ansible.posix/compare/1.5.4...1.6.0)

\======

## Release Summary

This is the minor release of the `ansible.posix` collection.
This changelog contains all changes to the modules and plugins
in this collection that have been added after the release of
`ansible.posix` 1.5.4.

## Major Changes

- Dropping support for Ansible 2.9, ansible-core 2.15 will be minimum
required version for this release

## Minor Changes

- Add summary_only parameter to profile_roles and profile_tasks
callbacks.
- firewalld - add functionality to set forwarding
([https://github.com/ansible-collections/ansible.posix/pull/548](https://github.com/ansible-collections/ansible.posix/pull/548)).
- firewalld - added offline flag implementation
([https://github.com/ansible-collections/ansible.posix/pull/484](https://github.com/ansible-collections/ansible.posix/pull/484))
- firewalld - respawn module to use the system python interpreter when
the `firewall` python module is not available for
`ansible_python_interpreter`
([https://github.com/ansible-collections/ansible.posix/pull/460](https://github.com/ansible-collections/ansible.posix/pull/460)).
- firewalld_info - Only warn about ignored zones, when there are zones
ignored.
- firewalld_info - respawn module to use the system python interpreter
when the `firewall` python module is not available for
`ansible_python_interpreter`
([https://github.com/ansible-collections/ansible.posix/pull/460](https://github.com/ansible-collections/ansible.posix/pull/460)).
- mount - add no_log option for opts parameter
([https://github.com/ansible-collections/ansible.posix/pull/563](https://github.com/ansible-collections/ansible.posix/pull/563)).
- seboolean - respawn module to use the system python interpreter when
the `selinux` python module is not available for
`ansible_python_interpreter`
([https://github.com/ansible-collections/ansible.posix/pull/460](https://github.com/ansible-collections/ansible.posix/pull/460)).
- selinux - respawn module to use the system python interpreter when the
`selinux` python module is not available for
`ansible_python_interpreter`
([https://github.com/ansible-collections/ansible.posix/pull/460](https://github.com/ansible-collections/ansible.posix/pull/460)).

## Removed Features (previously deprecated)

- skippy - Remove skippy pluglin as it is no longer
supported([https://github.com/ansible-collections/ansible.posix/issues/350](https://github.com/ansible-collections/ansible.posix/issues/350)).

## Bugfixes

- Bugfix in the documentation regarding the path option for
authorised_key([https://github.com/ansible-collections/ansible.posix/issues/483](https://github.com/ansible-collections/ansible.posix/issues/483)).
-   seboolean - make it work with disabled SELinux
- synchronize - maintain proper formatting of the remote paths
([https://github.com/ansible-collections/ansible.posix/pull/361](https://github.com/ansible-collections/ansible.posix/pull/361)).
- sysctl - fix sysctl to work properly on symlinks
([https://github.com/ansible-collections/ansible.posix/issues/111](https://github.com/ansible-collections/ansible.posix/issues/111)).

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Renovate
Bot](https://github.com/renovatebot/renovate).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOC43Ny4wIiwidXBkYXRlZEluVmVyIjoiMzguNzcuMCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsicmVub3ZhdGUvYW5zaWJsZSIsInR5cGUvbWlub3IiXX0=-->

Co-authored-by: lumiere-bot[bot] <98047013+lumiere-bot[bot]@users.noreply.github.com>
@richm
Copy link
Contributor

richm commented Nov 15, 2024

SUMMARY

The seboolean, selinux, firewalld, and firewalld_info modules depend on system bindings that are only available for the default system python interpreter. ansible-core is not packaged for the default system python interpreter on RHEL 8 and 9.

@gotmax23 can you explain what this means - "ansible-core is not packaged for the default system python interpreter on RHEL 8 and 9"?
Does this mean "ansible-core is packaged in such a way as to use a python module by default instead of the standard system python e.g. /usr/bin/python3.9 on RHEL 9"?
Does this mean "ansible-core does not have built in to its list of python interpreters to use for modules on managed nodes the correct python versions for RHEL 8 and RHEL 9"?

I'm wondering if this is the same problem as https://access.redhat.com/solutions/6726561:
"Issue

When running a playbook utilizing the implicit localhost that calls a RHEL System Role on RHEL 8.6 or later with Ansible Core, the playbook fails with messages similar to:
TASK [redhat.rhel_system_roles.selinux : Set SELinux booleans] *****************************************************************************
...
"msg": "Failed to import the required Python library (libselinux-python) on host01's Python /usr/bin/python3.11. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter"

If this is the problem - what we tell RHEL System Roles customers is this.

"
Root Cause

RHEL 8.6 uses Ansible Core 2.12 to support RHEL System Roles. Ansible Core 2.12 utilizes Python 3.8 which does not contain all of the required Python modules (for example the blivet module for the storage RHEL System Role).

"

and these are the workarounds:
"

Resolution

Choose one of the options below to workaround the issue:

  • Create an inventory file that lists localhost with the ansible_connection=local option.
    For example, an inventory file with:
    localhost ansible_connection=local
    Run ansible-playbook and specify that this inventory file should be used:
    ansible-playbook -i inventory

  • Create an inventory file that lists localhost.
    Note that this will result in ansible-playbook connecting to the localhost over SSH with SSH key authentication, which must have previously been configured.
    For example, an inventory file with:
    localhost
    Run ansible-playbook and specify that this inventory file should be used:
    ansible-playbook -i inventory

  • Use implicit localhost, with the ansible_python_interpreter variable set to use platform-python
    For example:
    ansible-playbook -e 'ansible_python_interpreter=/usr/libexec/platform-python'
    "

Unfortunately the solution in this PR is causing a problem with the way we 'vendor' in the ansible.posix modules for RHEL customers (since ansible.posix is unsupported for RHEL customers). There is some deep failure when the python code is loaded, before the module is even executed. I've been beating my head against the wall trying to figure out why this line (as part of the vendoring, we rewrite the module) is causing the problems:

from ansible.module_utils.selinux_lsr._respawn import respawn_module, HAS_RESPAWN_UTIL

Also note the older legacy role form (we vendor the module into the rhel-system-roles.selinux role). We do the same sort of vendoring for lots of other modules in lots of other system roles, and I do not have an issue.

At any rate, if this is the same problem as I linked to above https://access.redhat.com/solutions/6726561 - then I think I can just remove the respawn import line, and create mock respawn_module and HAS_RESPAWN_UTIL

When automatic interpreter discovery does not occur (e.g. when using implicit localhost [1]), ansible-core will not use the system interpreter to run ansible modules and the aforementioned modules will not work even if the bindings are installed. The RHEL ansible-core maintainers as well as the EPEL ansible and ansible-collection-* package maintainers (inc. me) have gotten multiple bug reports about this. We have been telling people to fix their setup to use the correct Python interpreter. Fortunately, ansible-core 2.11 and above have a module utility that'll respawn modules to use the correct system interpreter.

[1] https://docs.ansible.com/ansible/latest/inventory/implicit_localhost.html

ISSUE TYPE
* Feature Pull Request
COMPONENT NAME

seboolean selinux firewalld firewalld_info

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
mergeit Gate PR in Zuul CI
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants