Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates Whonix-based templates 14 -> 15 #358

Merged
merged 9 commits into from
Dec 11, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 14 additions & 61 deletions dom0/sd-dom0-files.sls
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,12 @@
# over time. These scripts should be ported to an RPM package.
##

include:
# Import the upstream Qubes-maintained anon-whonix settings.
# The anon-whoni config pulls in sys-whonix and sys-firewall,
# as well as ensures the latest versions of Whonix are installed.
- qvm.anon-whonix

dom0-rpm-test-key:
file.managed:
# We write the pubkey to the repos config location, because the repos
Expand Down Expand Up @@ -87,79 +93,26 @@ dom0-securedrop-icon:
- require:
- file: dom0-securedrop-icons-directory

dom0-enabled-apparmor-on-whonix-gw-14-template:
dom0-enabled-apparmor-on-whonix-gw-template:
qvm.vm:
- name: whonix-gw-14
- name: whonix-gw-15
- prefs:
- kernelopts: "nopat apparmor=1 security=apparmor"
- require:
- sls: qvm.anon-whonix

dom0-enabled-apparmor-on-whonix-ws-14-template:
dom0-enabled-apparmor-on-whonix-ws-template:
qvm.vm:
- name: whonix-ws-14
- name: whonix-ws-15
- prefs:
- kernelopts: "nopat apparmor=1 security=apparmor"
- require:
- sls: qvm.anon-whonix

dom0-create-opt-securedrop-directory:
file.directory:
- name: /opt/securedrop

# Temporary workaround to ensure the whonix templateVMs have their whonix repos
# disabled. While they are no longer supported by Whonix, they should still
# receive upstream Debian updates). Broken apt list prevents these updates from
# being applied. sd-whonix uses whonix-14-gw directly, so we must update that
# template. We must also used the whonix_repository tool, as otherwise the
# repos may reappear. sudo whonix_repository --enable to bring them back.
dom0-whonix-gw-disable-apt-list:
cmd.run:
- name: >
test -f /opt/securedrop/whonix-gw-14-ths-repo-disabled ||
qvm-run -a whonix-gw-14
"sudo whonix_repository --disable" &&
qvm-shutdown --wait whonix-gw-14 &&
touch /opt/securedrop/whonix-gw-14-ths-repo-disabled
- require:
- file: dom0-create-opt-securedrop-directory

# We need to disable the whonix apt sources for the python-futures installation
# for ws as well, for the python-futures package to be properly installed
dom0-whonix-ws-disable-apt-list:
cmd.run:
- name: >
test -f /opt/securedrop/whonix-ws-14-ths-repo-disabled ||
qvm-run -a whonix-ws-14
"sudo whonix_repository --disable" &&
qvm-shutdown --wait whonix-ws-14 &&
touch /opt/securedrop/whonix-ws-14-ths-repo-disabled
- require:
- file: dom0-create-opt-securedrop-directory

# Temporary workaround to bootstrap Salt support on target.
dom0-whonix-gw-14-install-python-futures:
cmd.run:
- name: >
test -f /opt/securedrop/whonix-gw-14-python-futures ||
qvm-run -a whonix-gw-14
"python -c 'import concurrent.futures' ||
{ sudo apt-get update && sudo apt-get install -qq python-futures ; }" &&
qvm-shutdown --wait whonix-gw-14 &&
touch /opt/securedrop/whonix-gw-14-python-futures
- require:
- file: dom0-create-opt-securedrop-directory
- cmd: dom0-whonix-gw-disable-apt-list

dom0-whonix-ws-14-install-python-futures:
cmd.run:
- name: >
test -f /opt/securedrop/whonix-ws-14-python-futures ||
qvm-run -a whonix-ws-14
"python -c 'import concurrent.futures' ||
{ sudo apt-get update && sudo apt-get install -qq python-futures ; }" &&
qvm-shutdown --wait whonix-ws-14 &&
touch /opt/securedrop/whonix-ws-14-python-futures
- require:
- file: dom0-create-opt-securedrop-directory
- cmd: dom0-whonix-ws-disable-apt-list

{% set gui_user = salt['cmd.shell']('groupmems -l -g qubes') %}

dom0-login-autostart-directory:
Expand Down
10 changes: 6 additions & 4 deletions dom0/sd-proxy.sls
Original file line number Diff line number Diff line change
Expand Up @@ -10,32 +10,34 @@
##

include:
- qvm.template-whonix-ws
- sd-whonix
- sd-upgrade-templates

sd-proxy-template:
qvm.vm:
- name: sd-proxy-template
- name: sd-proxy-buster-template
- clone:
- source: whonix-ws-14
- source: whonix-ws-15
- label: blue
- tags:
- add:
- sd-workstation
- sd-buster
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we must require sls sd-upgrade-templates here , to ensure the template update was successful before cloning whonix-ws-15 to sd-proxy-buster-template


sd-proxy:
qvm.vm:
- name: sd-proxy
- present:
- template: sd-proxy-template
- label: blue
- prefs:
- template: sd-proxy-buster-template
- netvm: sd-whonix
- kernelopts: "nopat apparmor=1 security=apparmor"
- autostart: true
- tags:
- add:
- sd-workstation
- sd-buster
- require:
- qvm: sd-whonix
- qvm: sd-proxy-template
Expand Down
27 changes: 27 additions & 0 deletions dom0/sd-sys-whonix-vms.sls
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# -*- coding: utf-8 -*-
# vim: set syntax=yaml ts=2 sw=2 sts=2 et :

include:
# Import the upstream Qubes-maintained anon-whonix settings.
# The anon-whoni config pulls in sys-whonix and sys-firewall,
# as well as ensures the latest versions of Whonix are installed.
- qvm.anon-whonix

# The Qubes logic is too polite about enforcing template
# settings, using "present" rather than "prefs". Below
# we force the template updates.
sys-whonix-template-config:
qvm.vm:
- name: sys-whonix
- prefs:
- template: whonix-gw-15
- require:
- sls: qvm.anon-whonix

anon-whonix-template-config:
qvm.vm:
- name: anon-whonix
- prefs:
- template: whonix-ws-15
- require:
- sls: qvm.anon-whonix
11 changes: 7 additions & 4 deletions dom0/sd-whonix.sls
Original file line number Diff line number Diff line change
Expand Up @@ -10,23 +10,26 @@
##

include:
- qvm.template-whonix-gw
- qvm.sys-firewall
# Import the upstream Qubes-maintained anon-whonix settings.
# The anon-whoni config pulls in sys-whonix and sys-firewall,
# as well as ensures the latest versions of Whonix are installed.
- qvm.anon-whonix

sd-whonix:
qvm.vm:
- name: sd-whonix
- present:
- template: whonix-gw-14
- label: purple
- mem: 500
- prefs:
- template: whonix-gw-15
- provides-network: true
- netvm: "sys-firewall"
- autostart: true
- kernelopts: "nopat apparmor=1 security=apparmor"
- tags:
- add:
- sd-workstation
- sd-buster
- require:
- qvm: sys-firewall
- sls: qvm.anon-whonix
3 changes: 2 additions & 1 deletion dom0/sd-workstation.top
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ base:
- sd-workstation-template
- sd-upgrade-templates
- sd-dom0-qvm-rpc
- sd-sys-whonix-vms
- sd-export
- sd-gpg
- sd-proxy
Expand All @@ -19,7 +20,7 @@ base:
- sd-export-files
sd-gpg:
- sd-gpg-files
sd-proxy-template:
sd-proxy-buster-template:
- sd-proxy-template-files
sd-svs:
- sd-svs-config
Expand Down
133 changes: 81 additions & 52 deletions dom0/securedrop-handle-upgrade
Original file line number Diff line number Diff line change
Expand Up @@ -12,61 +12,90 @@ TASK=${1:-default}
# 2. The AppVM must not be a DispVM template that used as the default DispVM
# for an AppVM, nor the system default DispVM.
if [[ $TASK == "prepare" ]]; then
# sd-svs, we simply shutdown the machine as we want to preserve the data
if qvm-check sd-svs --quiet; then
BASE_TEMPLATE=$(qvm-prefs sd-svs template)
if [[ ! $BASE_TEMPLATE =~ "buster" ]]; then
if qvm-check --running sd-svs; then
qvm-shutdown --wait sd-svs
fi
fi
fi
# sd-svs, we simply shutdown the machine as we want to preserve the data
if qvm-check sd-svs --quiet; then
BASE_TEMPLATE=$(qvm-prefs sd-svs template)
if [[ ! $BASE_TEMPLATE =~ "buster" ]]; then
if qvm-check --running sd-svs; then
qvm-shutdown --wait sd-svs
fi
fi
fi

# For sd-svs-disp and sd-export-usb-dvm, DispVM templates. We can delete both
# VMs since they contain no persistent data. The installer will re-create them
# as part of the provisioning process.
# We set the default DispVM to empty string to ensure nothing is opened in an
# insecure (unmanaged or not yet updated) or networked vm, until the
# provisioning process runs again and sets that value to sd-svs-disp
if qvm-check --quiet sd-svs-disp; then
BASE_TEMPLATE=$(qvm-prefs sd-svs-disp template)
if [[ ! $BASE_TEMPLATE =~ "buster" ]]; then
qubes-prefs default_dispvm ''
qvm-shutdown --wait sd-svs-disp
qvm-remove -f sd-svs-disp
fi
fi
# For sd-svs-disp and sd-export-usb-dvm, DispVM templates. We can delete both
# VMs since they contain no persistent data. The installer will re-create them
# as part of the provisioning process.
# We set the default DispVM to empty string to ensure nothing is opened in an
# insecure (unmanaged or not yet updated) or networked vm, until the
# provisioning process runs again and sets that value to sd-svs-disp
if qvm-check --quiet sd-svs-disp; then
BASE_TEMPLATE=$(qvm-prefs sd-svs-disp template)
if [[ ! $BASE_TEMPLATE =~ "buster" ]]; then
qubes-prefs default_dispvm ''
qvm-shutdown --wait sd-svs-disp
qvm-remove -f sd-svs-disp
fi
fi

if qvm-check --quiet sd-export-usb; then
BASE_TEMPLATE=$(qvm-prefs sd-export-usb-dvm template)
if [[ ! $BASE_TEMPLATE =~ "buster" ]]; then
qvm-shutdown --wait sd-export-usb
qvm-shutdown --wait sd-export-usb-dvm
qvm-remove -f sd-export-usb
qvm-remove -f sd-export-usb-dvm
fi
fi
if qvm-check --quiet sd-export-usb; then
BASE_TEMPLATE=$(qvm-prefs sd-export-usb-dvm template)
if [[ ! $BASE_TEMPLATE =~ "buster" ]]; then
qvm-shutdown --wait sd-export-usb
qvm-shutdown --wait sd-export-usb-dvm
qvm-remove -f sd-export-usb
qvm-remove -f sd-export-usb-dvm
fi
fi

# Finally for sd-gpg, we simply shutdown the machine
if qvm-check --quiet sd-gpg; then
BASE_TEMPLATE=$(qvm-prefs sd-gpg template)
if [[ ! $BASE_TEMPLATE =~ "buster" ]]; then
qvm-shutdown --wait sd-gpg
fi
fi
# For Whonix VMs, shut them down, so we can upate the TemplateVM settings.
# We shut down sd-proxy before sd-whonix, since its netvm is sd-whonix, which won't
# shutdown if a client is connected.
if qvm-check --quiet sd-proxy; then
BASE_TEMPLATE=$(qvm-prefs sd-proxy template)
if [[ ! $BASE_TEMPLATE =~ "buster" ]]; then
qvm-shutdown --wait sd-proxy
fi
fi

if qvm-check --quiet sd-whonix; then
BASE_TEMPLATE=$(qvm-prefs sd-whonix template)
if [[ ! $BASE_TEMPLATE =~ "15" ]]; then
qvm-shutdown --wait sd-whonix
fi
fi

# Kill sys-whonix, to make sure connected clients don't prevent shutdown.
if qvm-check --quiet sys-whonix; then
BASE_TEMPLATE=$(qvm-prefs sys-whonix template)
if [[ ! $BASE_TEMPLATE =~ "15" ]]; then
if qvm-check --quiet --running sys-whonix; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: weird indentation for sys-whonix here, makes it a bit hard to read

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@emkll Ack, this is tabs-versus-spaces. Most of the scripts in the repo use 4 spaces for bash scripts, but this one and only this one uses tabs instead—I wasn't careful about making sure to force use of tabs when editing the script. Agreed, we should indeed clean it up to avoid large amounts of frustration. 😃

qvm-kill sys-whonix
# Wait for machine to stop fully, since qvm-kill doesn't block
sleep 5
fi
fi
fi

# Finally for sd-gpg, we simply shutdown the machine
if qvm-check --quiet sd-gpg; then
BASE_TEMPLATE=$(qvm-prefs sd-gpg template)
if [[ ! $BASE_TEMPLATE =~ "buster" ]]; then
qvm-shutdown --wait sd-gpg
fi
fi
elif [[ $TASK == "remove" ]]; then
# For each template, ensure the TemplateVM exists, that it is shut down
# before deleting it.
for template in sd-svs-template sd-svs-disp-template sd-export-template
do
if qvm-check "${template}" --quiet; then
if qvm-check --running "${template}"; then
qvm-shutdown --wait "${template}"
fi
qvm-remove -f "${template}"
fi
done
# For each template, ensure the TemplateVM exists, that it is shut down
# before deleting it.
for template in sd-svs-template sd-svs-disp-template sd-export-template sd-proxy-template
do
if qvm-check "${template}" --quiet; then
if qvm-check --running "${template}"; then
qvm-shutdown --wait "${template}"
fi
qvm-remove -f "${template}"
fi
done
else
echo "Please specify prepare or remove"
exit 1
echo "Please specify prepare or remove"
exit 1
fi
9 changes: 6 additions & 3 deletions tests/test_dom0_config.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
import subprocess
import unittest

STRETCH_TEMPLATES = ["sd-svs-template",
"sd-svs-disp-template",
"sd-export-template"]
STRETCH_TEMPLATES = [
"sd-svs-template",
"sd-svs-disp-template",
"sd-export-template",
"sd-proxy-template",
]


class SD_Qubes_Dom0_Templates_Tests(unittest.TestCase):
Expand Down
9 changes: 7 additions & 2 deletions tests/test_proxy_vm.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,13 @@ def test_sd_proxy_yaml_config(self):
for line in wanted_lines:
self.assertFileHasLine("/etc/sd-proxy.yaml", line)

def test_whonix_ws_14_repo_disabled(self):
assert self._fileExists(self.whonix_apt_list) is False
def test_whonix_ws_repo_enabled(self):
"""
During Whonix 14 -> 15 migration, we removed the apt list file
(because the repo wasn't serving, due to EOL status). Let's
make sure it's there, since we're past 14 now.
"""
assert self._fileExists(self.whonix_apt_list)


def load_tests(loader, tests, pattern):
Expand Down
Loading