Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add rockstor-build systemd service #2793 #2795

Conversation

phillxnet
Copy link
Member

@phillxnet phillxnet commented Feb 1, 2024

Move build.sh execution from within rpm %posttrans script to it's own dedicated rockstor-build.service. Enabling greater fidelity and control over the environment and timing; and eases development and user feedback on build.sh failures in the future.

Partnered with rockstor.spec changes in rockstor-rpmbuild repo.

Includes

  • New rockstor-build.service file.
  • After= & Requires= entries in rockstor-pre on rockstor-build, to extend our service cascade.
  • Trivial build.sh and pkg_mgmt.py comment updates.
  • Add the new rockstor-build.service to initrock.py to assist in asserting the service akin to all other rockstor services. Mostly redundant given our rpm service management, but nice-to-have.

Fixes #2793


Partner Pull Request:

Move build.sh execution from within rpm %posttrans script to
it's own dedicated rockstor-build.service. Enabling greater
fidelity and control over the environment and timing; and eases
development and user feedback on build.sh failures in the future.

Partnered with rockstor.spec changes in rockstor-rpmbuild repo.

## Includes
- New rockstor-build.service file.
- After= & Requires= entries in rockstor-pre on rockstor-build,
to extend our service cascade.
- Trivial build.sh and pkg_mgmt.py comment updates.
- Add the new rockstor-build.service to initrock.py to assist
in asserting the service akin to all other rockstor services.
Mostly redundant given our rpm service management, but nice-to-have.
@phillxnet
Copy link
Member Author

Testing

An rpm build with these proposed changes and those in the linked/partnered rpmbuild repo successfully installed and started all service: creating our .venv with the newly added rockstor-build.servcie:

systemctl status rockstor-build.service 
● rockstor-build.service - Build Rockstor
     Loaded: loaded (/usr/lib/systemd/system/rockstor-build.service; disabled; vendor preset: disabled)
     Active: active (exited) since Fri 2024-02-02 16:31:26 WET; 1min 43s ago
   Main PID: 4326 (code=exited, status=0/SUCCESS)

Feb 02 16:31:26 rleap15-4 build.sh[4326]: ROCKSTOR BUILD SCRIPT COMPLETED
Feb 02 16:31:26 rleap15-4 build.sh[4326]: If installing from source, from scratch, for development; i.e. NOT via RPM:
Feb 02 16:31:26 rleap15-4 build.sh[4326]: Note GnuPG & password-store ExecStartPre steps in /opt/rockstor/conf/rockstor-pre.service
Feb 02 16:31:26 rleap15-4 build.sh[4326]: 1. Run 'cd /opt/rockstor'.
Feb 02 16:31:26 rleap15-4 build.sh[4326]: 2. Run 'systemctl start postgresql'.
Feb 02 16:31:26 rleap15-4 build.sh[4326]: 3. Run 'export DJANGO_SETTINGS_MODULE=settings'.
Feb 02 16:31:26 rleap15-4 build.sh[4326]: 4. Run 'export PASSWORD_STORE_DIR=/root/.password-store'.
Feb 02 16:31:26 rleap15-4 build.sh[4326]: 5. Run 'poetry run initrock' as root (equivalent to rockstor-pre.service ExecStart).
Feb 02 16:31:26 rleap15-4 build.sh[4326]: 6. Run 'systemctl enable --now rockstor-bootstrap'.
Feb 02 16:31:26 rleap15-4 systemd[1]: Finished Build Rockstor.

And after a subsequent reboot we have the following status output from the same new service:

systemctl status rockstor-build.service 
○ rockstor-build.service - Build Rockstor
     Loaded: loaded (/usr/lib/systemd/system/rockstor-build.service; disabled; vendor preset: disabled)
     Active: inactive (dead)
  Condition: start condition failed at Fri 2024-02-02 16:36:26 WET; 59s ago

Feb 02 16:36:26 rleap15-4 systemd[1]: Condition check resulted in Build Rockstor being skipped.

Indicating that we have successful conditional .venv re-construction via systemd now.

A re-test of elements of the above, to prove .venv rebuild when the directory does not exist:

systemctl stop rockstor*
rm -rf /opt/rockstor/.venv/
systemctl start rockstor-bootstrap.service

Succeeded in all services starting successfully and a new .venv having been constructed from Poetries cached downloads from PyPi.

Poetry reinstall:

To prove that our new systemd service and also install our requried poetrywe have:

systemctl stop rockstor*
rm -rf /opt/rockstor/.venv/
which poetry
/usr/local/bin/poetry
export PIPX_HOME=/opt/pipx
export PIPX_BIN_DIR=/usr/local/bin
export PIPX_MAN_DIR=/usr/local/share/man
pipx uninstall poetry
uninstalled poetry! ✨ 🌟 ✨
which poetry
which: no poetry in (/sbin:/usr/sbin:/usr/local/sbin:/root/.local/bin:/root/bin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin)
# And to ensure we have a working boot after this, complete with Poetry install and .venv build:
reboot

On the subsequent instance we have our new poetry install and it having created our new .venv, with all services working as expected.

@phillxnet
Copy link
Member Author

phillxnet commented Feb 2, 2024

Updating

A prior V10 DB rockstor instance (15.3 stable installer derived but dup'ed to 15.4) also successfully updated to this new rpm. Successuffly rebuilding the venv and instantiating the new Poetry. This test also helps with proving no adverse interations with a long %posttrans (db_upgrade.sh) and our new rockstor-build.service.

Also note that we now have the more accessible system journal (rather than zypper history log) to view our new services run of build.sh outcome:

Feb 02 17:09:43 installer systemd[1]: Starting Build Rockstor...
Feb 02 17:09:43 installer build.sh[15688]: which: no poetry in (/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin)
Feb 02 17:09:43 installer build.sh[15687]: Unset VIRTUAL_ENV
Feb 02 17:09:43 installer build.sh[15687]: build.sh has PATH=/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Feb 02 17:09:43 installer build.sh[15687]: Adopting installs' LANG=en_GB.UTF-8
Feb 02 17:09:43 installer build.sh[15689]: /usr/local/bin is already in PATH.
Feb 02 17:09:43 installer build.sh[15689]: ⚠  All pipx binary directories have been added to PATH. If you are sure you
Feb 02 17:09:43 installer build.sh[15689]: want to proceed, try again with the '--force' flag.
Feb 02 17:09:43 installer build.sh[15689]: Otherwise pipx is ready to go! ✨ 🌟 ✨Feb 02 17:09:44 installer build.sh[15691]: creating virtual environment...
Feb 02 17:09:44 installer build.sh[15691]: creating shared libraries...
Feb 02 17:09:48 installer build.sh[15691]: upgrading shared libraries...
Feb 02 17:09:53 installer build.sh[15691]: installing poetry from spec 'poetry==1.7.1'...
Feb 02 17:09:53 installer chronyd[715]: Selected source 91.209.16.78 (1.opensuse.pool.ntp.org)
Feb 02 17:10:10 installer build.sh[15691]: done! ✨ 🌟 ✨Feb 02 17:10:10 installer build.sh[15691]:   installed package poetry 1.7.1, installed using Python 3.11.5
Feb 02 17:10:10 installer build.sh[15691]:   These apps are now globally available
Feb 02 17:10:10 installer build.sh[15691]:     - poetry
Feb 02 17:10:10 installer build.sh[15732]: venvs are in /opt/pipx/venvs
Feb 02 17:10:10 installer build.sh[15732]: apps are exposed on your $PATH at /usr/local/bin
Feb 02 17:10:10 installer build.sh[15732]:    package poetry 1.7.1, installed using Python 3.11.5
Feb 02 17:10:10 installer build.sh[15732]:     - poetry
Feb 02 17:10:27 installer build.sh[15687]: Creating jslibs/js/lib & populating from rockstor-jslibs.tar.gz
...
Feb 02 17:10:27 installer build.sh[16125]: rockstor-jslibs-5.0.7/underscore.js
Feb 02 17:10:27 installer build.sh[16127]: gpg: keybox '/root/.gnupg/pubring.kbx' created
Feb 02 17:10:29 installer build.sh[16127]: gpg: /root/.gnupg/trustdb.gpg: trustdb created
Feb 02 17:10:29 installer build.sh[16127]: gpg: key A8A8E55F6CC7EC74 marked as ultimately trusted
Feb 02 17:10:29 installer build.sh[16127]: gpg: directory '/root/.gnupg/openpgp-revocs.d' created
Feb 02 17:10:29 installer build.sh[16127]: gpg: revocation certificate stored as '/root/.gnupg/openpgp-revocs.d/A1D46B054D63C19F48183904A8A8E55F6CC7EC74.rev'
Feb 02 17:10:29 installer build.sh[16140]: mkdir: created directory '/root/.password-store/'
Feb 02 17:10:29 installer build.sh[16133]: Password store initialized for rockstor@localhost
Feb 02 17:10:29 installer build.sh[16153]: mkdir: created directory '/root/.password-store/python-keyring'
Feb 02 17:10:29 installer build.sh[16153]: mkdir: created directory '/root/.password-store/python-keyring/rockstor'
Feb 02 17:10:29 installer build.sh[16158]: tr: write error: Broken pipe
Feb 02 17:10:29 installer build.sh[16158]: tr: write error
Feb 02 17:10:29 installer build.sh[16147]: The generated password for python-keyring/rockstor/SECRET_KEY is:
Feb 02 17:10:29 installer build.sh[16147]: 5S7I8eUGM0AHYIykHSVKc0F2fQ51U3i3As8sW9HeNJVJPhiyFf8Fq2QRGYf0FJEFC4BpX1PFLBw7GFvKMWFrlxkgyOGwSTwmXJ2m
Feb 02 17:10:30 installer build.sh[16222]: Error: python-keyring/rockstor/SECRET_KEY_FALLBACK is not in the password store.
Feb 02 17:10:30 installer build.sh[16227]: Error: python-keyring/rockstor/CLIENT_SECRET is not in the password store.
Feb 02 17:10:32 installer build.sh[16161]: No SECRET_KEY_FALLBACK - rotated on reboot / rockstor services restart.
Feb 02 17:10:32 installer build.sh[16161]: Copying '/opt/rockstor/.venv/lib/python3.11/site-packages/django/contrib/admin/static/admin/css/autocomplete.css'
...
Feb 02 17:10:33 installer build.sh[16161]: 515 static files copied to '/opt/rockstor/static', 517 post-processed.
Feb 02 17:10:33 installer build.sh[15687]: ROCKSTOR BUILD SCRIPT COMPLETED
Feb 02 17:10:33 installer build.sh[15687]: If installing from source, from scratch, for development; i.e. NOT via RPM:
Feb 02 17:10:33 installer build.sh[15687]: Note GnuPG & password-store ExecStartPre steps in /opt/rockstor/conf/rockstor-pre.service
Feb 02 17:10:33 installer build.sh[15687]: 1. Run 'cd /opt/rockstor'.
Feb 02 17:10:33 installer build.sh[15687]: 2. Run 'systemctl start postgresql'.
Feb 02 17:10:33 installer build.sh[15687]: 3. Run 'export DJANGO_SETTINGS_MODULE=settings'.
Feb 02 17:10:33 installer build.sh[15687]: 4. Run 'export PASSWORD_STORE_DIR=/root/.password-store'.
Feb 02 17:10:33 installer build.sh[15687]: 5. Run 'poetry run initrock' as root (equivalent to rockstor-pre.service ExecStart).
Feb 02 17:10:33 installer build.sh[15687]: 6. Run 'systemctl enable --now rockstor-bootstrap'.
Feb 02 17:10:33 installer systemd[1]: Finished Build Rockstor.
Feb 02 17:10:33 installer systemd[1]: Starting Tasks required prior to starting Rockstor...

With our expected pass output of:

pass
Password Store
└── python-keyring
    └── rockstor
        ├── CLIENT_SECRET
        ├── SECRET_KEY_FALLBACK
        └── SECRET_KEY

@phillxnet
Copy link
Member Author

Given this is yet-another partnered pull request with one rockstor-rpmbuild the testing procedure is somewhat labour-some.

But from the above comments this looks to be good to go. Easing he way for our pending Samba fix and further updates from this new rockstor*.services arrangement. But to me this feels a lot more appropriate that what we had: i.e. a super stretched use of the %posttrans scriptlet in rpm. Which has proven to be difficult for folks to repo on where failures are concerned, i.e. systemd is way more familiar than zypper logs. And examination/experimentation likewise far more familiar knowledge.

I'll go ahead and merge this; a little overly architectural for RC phase of testing, but this way we start our next stable channel we mean to go on. Plus we have had way too many sensitivities re our build.sh working in terminal and not in %posttrans. Systemd gives us way more flexibility to modify our env for build.sh where-as rpm provides very little.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant