[xenial] Verify Trusty backup -> Xenial recovery story

[eloquence]

We should ensure that a SecureDrop backup completed on 14.04 can successfully be recovered on 16.04. Whether we ask admins to perform this step manually, or whether we automate it, it may be a required part of the Xenial migration and will certainly be highly recommended.

If clean upgrades to Xenial are not yet implemented one should complete this ticket by following these steps instead:

1. Create a backup on 14.04 server
2. Create fresh install on 16.04 and then attempt to run the restore

Part of #3204, may result in follow-up issues.

[kushaldas]

As of today with commit 10359094085e6c1839490e74e827c3cac1151b8d from develop, this just works 👍

[zenmonkeykstop]

This failed for me while reviewing PR #4080 for 3961-separate-subdir-per-distro-for-deb-pkg-buildson prod VMs

• did a clean 14.04 install
• took a backup
• did a fresh install on 16.04
• ran ./securedrop-admin restore <filename here>
• backup completed successfully

After backup, ssh app fails, as /var/lib/tor/services is overwritten. Copying var/lib/tor/services/ssh/hostname from the tarball to install_files/ansible-base/app-ssh-aths and rerunning ./securedrop-admin tailsconfig didn't restore access, and now ssh to app or mon fails immediately.

[conorsch]

Thanks for the report, @zenmonkeykstop . To clarify, you say:

Copying var/lib/tor/services/ssh/hostname from the tarball to install_files/ansible-base/app-ssh-aths

Did you run a literal cp on that file? Recall that the contents of app-ssh-aths must be prefixed with the HidServAuth declaration, as handled by this tiny template: https://github.com/freedomofpress/securedrop/blob/003b01721de8b711869d06af7ec4477456a06e79/install_files/ansible-base/roles/restrict-direct-access/templates/ths_config.j2

Other things to try:

• confirm Tails torrc has the correct HidServAuth line for the SSH services (according to tarball contents)
• rm -rf ~/.ansible/ on Tails; it's possible ControlPersist files still exist there, but point to a dead socket on the remote host
• console into the app/prod VMs and inspect tor service status, as well as tor/ssh logs

[zenmonkeykstop]

D'oh, didn't add HidServAuth, working fine with that addition.