Leaner provision-all

Removes hardcoding of VM names to leverage bits of the Admin API (via
`qvm-ls --tags`) where appropriate. Updated comments throughout.

Trying to lean on Salt's parallel execution as much as possible,
to keep runtimes low. The step-by-step execution of Salt (sys-firewall,
then dom0, then VMs) could be handled by Salt orchestrators, but not
implementing such a substantial refactor right now.

[0] https://docs.saltstack.com/en/latest/topics/orchestrate/orchestrate_runner.html

Makefile

@@ -122,9 +122,6 @@ prep-dom0: prep-salt # Copies dom0 config files for VM updates
 	sudo qubesctl top.enable sd-dom0-files
 	sudo qubesctl --show-output --targets dom0 state.highstate
 
-list-vms: ## Prints all Qubes VMs managed by Workstation salt config
-	@./scripts/list-vms
-
 destroy-all: ## Destroys all VMs managed by Workstation salt config
 	qubes-prefs default_dispvm fedora-30-dvm
 	./scripts/destroy-vm --all

scripts/provision-all

@@ -5,25 +5,22 @@ set -u
 set -o pipefail
 
 
-# Format list of all VMs comma-separated, for use as qubesctl target
-all_sdw_vms_target="$(./scripts/list-vms | perl -npE 's/\n/,/g' | perl -npE 's/,$//' )"
-
+# The max concurrency reduction (4->2) was required to avoid "did not return clean data"
+# errors from qubesctl. It may be possible to raise this again.
+max_concurrency="2"
 
 echo "Configure Fedora-based system VMs"
-sudo qubesctl --show-output --targets dom0 state.sls sd-sys-vms
+sudo qubesctl --show-output state.sls sd-sys-vms
 sudo qubesctl --show-output --skip-dom0 --targets sys-firewall state.sls sd-sys-firewall-files
 
-echo "Create base Template to be used by others"
-sudo qubesctl --show-output --targets dom0 state.sls sd-workstation-template
+# Running only against dom0, to ensure the VMs are created (but not yet configured)
+echo "Set up dom0 config files, including RPC policies, and create VMs"
+sudo qubesctl --show-output state.highstate
 
-echo "Configure packages inside base Template"
-sudo qubesctl --show-output --skip-dom0 --targets securedrop-workstation state.sls sd-workstation-template-files
+# Format list of all VMs comma-separated, for use as qubesctl target
+# We run this after dom0's highstate, so that the VMs are available for listing by tag.
+all_sdw_vms_target="$(qvm-ls --tags sd-workstation --raw-list | perl -npE 's/\n/,/g' | perl -npE 's/,$//' )"
 
-echo "Set up dom0 config files, including RPC policies, and create VMs"
-# The dom0 config runs implicitly via qubesctl (unless `--skip-dom0` is passed), so the VM
-# creation logic will be run before the states adding files inside the VMs.
-#sudo qubesctl --show-output --targets dom0 state.highstate
+# We skip dom0 in the task below, since dom0 highstate was enforced in the previous command.
 echo "Provision all SecureDrop Workstation VMs with service-specific configs"
-# The max concurrency reduction (4->2) was required to avoid "did not return clean data"
-# errors from qubesctl. It may be possible to raise this again.
-sudo qubesctl --show-output --max-concurrency 2 --targets "$all_sdw_vms_target" state.highstate
+sudo qubesctl --show-output --max-concurrency "$max_concurrency" --skip-dom0 --targets "$all_sdw_vms_target" state.highstate