deploy: configure sd-whonix by package

[cfm]

Status

Ready for review

• Blocked on build: package securedrop-qubesdb and securedrop-whonix-config securedrop-client#2032

Description of Changes

Closes #1039 by configuring sd-whonix using freedomofpress/securedrop-client#2032's securedrop-whonix-config rather than Salt.

Closes #936.

Testing

If testing after freedomofpress/securedrop-client#2032:

1. make clone && make dev

...and if testing in conjunction with freedomofpress/securedrop-client#2032:

2. On failure in sd-base-bookworm-template: manually install securedrop-qubesdb
3. make dev again
4. On failure in whonix-gateway-17: manually install securedrop-qubesdb and securedrop-whonix-config
5. make dev again

Either way, finally:

6. You're able to log into the Client.

Checklist

If you have made changes to the provisioning logic

• All tests (make test) pass in dom0

If you have added or removed files

• I have updated MANIFEST.in and rpm-build/SPECS/securedrop-workstation-dom0-config.spec

[legoktm]

The debs are now on apt-test so I've rebased to kick off CI.

[legoktm]

Do we need a cleanup stage to uninstall the packages? Admittedly I can't really see anything that was cleaning it up before...

[legoktm]

Running make dev doesn't invoke the newly added salt thing, I had to manually run sudo qubesctl --show-output --skip-dom0 --targets whonix-gateway-17 state.highstate and then it all worked (and I could log in and auth_private was provisioned correctly).

I think this is because the template is not tagged as sd-workstation so it needs an explicit salt run thing.

We also need a salt state to remove the packages (purge ideally) to restore it to original state on uninstall.

[cfm]

Thanks, all. Let me tackle the rebase after #1048 and then I'll turn this back over to you, unless you'd rather I address #1051 (comment) at the same time.

[legoktm]

and then I'll turn this back over to you, unless you'd rather I address #1051 (comment) at the same time.

No, I double checked with Ro as well and we can defer it to after 4.2 (but before GA).

[cfm]

Rebased from main after #1048.

[legoktm]

LGTM, just waiting on a green check from SDW CI.

[cfm]

https://ws-ci-runner.securedrop.org/2024-06-03-233933015542-5628be940faa556ca8480ec1c7d8715289c0ba61-Qubes_4.2_B-update_20240603044409.log.txt:

INFO:[2024-06-04-00:08:49:204996]             ID: update-apt-cache-with-stable-change
INFO:[2024-06-04-00:08:49:205027]       Function: cmd.run
INFO:[2024-06-04-00:08:49:205055]           Name: apt-get update --allow-releaseinfo-change
INFO:[2024-06-04-00:08:49:205092]         Result: False
INFO:[2024-06-04-00:08:49:205123]        Comment: Command "apt-get update --allow-releaseinfo-change" run
INFO:[2024-06-04-00:08:49:205152]        Started: 00:06:47.633926
INFO:[2024-06-04-00:08:49:205185]       Duration: 1199.845 ms
INFO:[2024-06-04-00:08:49:205213]        Changes:   
INFO:[2024-06-04-00:08:49:205255]                 ----------
INFO:[2024-06-04-00:08:49:205289]                 pid:
INFO:[2024-06-04-00:08:49:205317]                     1823
INFO:[2024-06-04-00:08:49:205349]                 retcode:
INFO:[2024-06-04-00:08:49:205382]                     100
INFO:[2024-06-04-00:08:49:205410]                 stderr:
INFO:[2024-06-04-00:08:49:205442]                     E: Could not get lock /var/lib/apt/lists/lock. It is held by process 1578 (apt-get)
INFO:[2024-06-04-00:08:49:205475]                     E: Unable to lock directory /var/lib/apt/lists/
INFO:[2024-06-04-00:08:49:205507]                 stdout:
INFO:[2024-06-04-00:08:49:205539]                     Reading package lists...

[cfm]

Hmm. @legoktm, in the latest CI failure you flagged:

INFO:[2024-06-04-00:44:18:944941] ERROR: test_v3_auth_private_file (test_sd_whonix.SD_Whonix_Tests.test_v3_auth_private_file)
INFO:[2024-06-04-00:44:18:944972] ----------------------------------------------------------------------
INFO:[2024-06-04-00:44:18:945006] Traceback (most recent call last):
INFO:[2024-06-04-00:44:18:945038]   File "/home/user/securedrop-workstation/tests/test_sd_whonix.py", line 24, in test_v3_auth_private_file
INFO:[2024-06-04-00:44:18:945071]     self.assertFileHasLine("/var/lib/tor/authdir/app-journalist.auth_private", line)
INFO:[2024-06-04-00:44:18:945099]   File "/home/user/securedrop-workstation/tests/base.py", line 139, in assertFileHasLine
INFO:[2024-06-04-00:44:18:945133]     remote_content = self._get_file_contents(remote_path)
INFO:[2024-06-04-00:44:18:945165]                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INFO:[2024-06-04-00:44:18:945202]   File "/home/user/securedrop-workstation/tests/base.py", line 98, in _get_file_contents
INFO:[2024-06-04-00:44:18:945251]     return subprocess.check_output(cmd).decode("utf-8")
INFO:[2024-06-04-00:44:18:945287]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INFO:[2024-06-04-00:44:18:945321]   File "/usr/lib64/python3.11/subprocess.py", line 466, in check_output
INFO:[2024-06-04-00:44:18:945354]     return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
INFO:[2024-06-04-00:44:18:945391]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INFO:[2024-06-04-00:44:18:945424]   File "/usr/lib64/python3.11/subprocess.py", line 571, in run
INFO:[2024-06-04-00:44:18:945452]     raise CalledProcessError(retcode, process.args,
INFO:[2024-06-04-00:44:18:945487] subprocess.CalledProcessError: Command '['qvm-run', '-p', 'sd-whonix', 'sudo /bin/cat /var/lib/tor/authdir/app-journalist.auth_private']' returned non-zero exit status 1.
INFO:[2024-06-04-00:44:18:945517]

I've tacked on 9835669 to test the "Qubes feature to QubesDB variable" path more rigorously. But I also see one of our rebases lost the bash -c from the securedrop-whonix-config service unit's ExecStartPost, which (as I learned and thought I had fixed last week!) doesn't run in a shell. I'll push that fix up now, but I won't be able to test it locally until tomorrow:

• freedomofpress/securedrop-client@2014070

[cfm]

freedomofpress/securedrop-client#2056 may help here; with it I'm not able to reproduce this failure locally. However, Workstation CI does not appear to be running on new commits here....

[legoktm]

Since CI isn't cooperating I'm just going to verify it manually and then merge it.

[legoktm]

Unfortunately, I found a race condition, the /var/lib/tor/authdir directory is created by anon-gw-anonymizer-config.service, so we need a Requires/After (don't remember of the top of my head which one). Otherwise it'll error that the directory doesn't exist.

see https://github.com/Whonix/anon-gw-anonymizer-config/blob/master/etc/torrc.d/65_gateway.conf#L32-L39 and https://github.com/Whonix/anon-gw-anonymizer-config/blob/master/usr/libexec/anon-gw-anonymizer-config/make-sure-torrc-exist#L72.

Once that lands this should be ready to go, 🤞🏾

[cfm]

Thanks! On it.

[cfm]

@legoktm in #1051 (review):

Unfortunately, I found a race condition, the /var/lib/tor/authdir directory is created by anon-gw-anonymizer-config.service, so we need a Requires/After (don't remember of the top of my head which one). Otherwise it'll error that the directory doesn't exist.

see https://github.com/Whonix/anon-gw-anonymizer-config/blob/master/etc/torrc.d/65_gateway.conf#L32-L39 and https://github.com/Whonix/anon-gw-anonymizer-config/blob/master/usr/libexec/anon-gw-anonymizer-config/make-sure-torrc-exist#L72.

Once that lands this should be ready to go, 🤞🏾

Great catch: it looks like this was the cause of the original CI failure I wasn't able to reproduce locally. Fixed proposed and explained (with one FIXME for follow-up) in freedomofpress/securedrop-client#2058.

[legoktm]