Updated restore playbook to preserve existing server-side SSH configuration

[zenmonkeykstop]

Status

Ready for review

Description of Changes

Fixes #5833 .

This PR:

• Updates restore ansible role, splitting out tasks into includes by functionality
• Excludes SSH configuration from the restore process

Preserving the existing SSH configuration simplifies the restore process and prevents a known error where the process fails when reloading the Application Server tor configuration.

Testing

prerequisites

• a v3 ssh-over-tor prod instance (VMs or hardware)
• an admin workstation with this branch checked out.
• backup files from a previous ssh-over-tor instance with different onion service addresses (v2, v2+v3, v3 if possible), in ~/Persistent/securedrop/install_files/ansible-base.

v2-only:

• via an admin workstation terminal, attempt to restore from the v2-only backup (v2-only.tar.gz) with the following commands:

  cd ~/Persistent/securedrop
  ./securedrop-admin --force restore install_files/ansible-base/v2-only.tar.gz
  

  • verify that the restore fails immediately, with a message indicating that a data-only backup can be performed.
• attempt a data-only restore from the v2-only backup with the following commands:

   cd ~/Persistent/securedrop
  ./securedrop-admin --force restore --preferve-tor-config install_files/ansible-base/v2-only.tar.gz
  

  • verify that the restore completes successfully, without running the reload Tor task.
  • verify that all onion service addresses are unchanged and that the data from the backup is present.

v2+v3:

• copy the admin workstation ~/.ssh/config file to a temporary location, i.e. ~/sshconfig.bak

• via an admin workstation terminal, attempt to restore from the v2+v3 backup (v2nv3.tar.gz) with the following commands:

  cd ~/Persistent/securedrop
  ./securedrop-admin --force restore install_files/ansible-base/v2nv3.tar.gz
  

  • verify that the restore playbook completes successfully and does not error out on the reload Tor task.

• wait a few minutes for Tor to restart on the server, and then complete the restore process with the following commands:

  ./securedrop-admin install
  ./securedrop-admin tailsconfig
  

  • verify that the application and monitor servers are available over SSH (ssh app uptime && ssh mon uptime)
  • verify that the SI and JI shortcuts now point to the addresses from the backup and are accessible
  • verify that v2 services from the backup are not available
  • verify that the SSH onion service addresses in ~/.ssh/config and ~/sshconfig.bak are identical

v3-only:

• copy the admin workstation ~/.ssh/config file to a temporary location, i.e. ~/sshconfig.bak

• via an admin workstation terminal, attempt to restore from the v3-only backup (v3-only.tar.gz) with the following commands:

  cd ~/Persistent/securedrop
  ./securedrop-admin --force restore install_files/ansible-base/v3-only.tar.gz
  

  • verify that the restore playbook completes successfully and does not error out on the reload Tor task.

• wait a few minutes for Tor to restart on the server, and then complete the restore process with the following commands:

  ./securedrop-admin install
  ./securedrop-admin tailsconfig
  

  • verify that the application and monitor servers are available over SSH (ssh app uptime && ssh mon uptime)
  • verify that the SI and JI shortcuts now point to the addresses from the backup and are accessible
  • verify that the SSH onion service addresses in ~/.ssh/config and ~/sshconfig.bak are identical

Deployment

Deployed with GUI or manual update of admin workstation - no server-side change.

Checklist

If you made non-trivial code changes:

• [ x I have written a test plan and validated it for this PR

Choose one of the following:

• I have opened a PR in the docs repo for these changes, or will do so later
• I would appreciate help with the documentation
• These changes do not require documentation

[rmol]

v2-only:

• via an admin workstation terminal, attempt to restore from the v2-only backup (v2-only.tar.gz) with the following commands:

  cd ~/Persistent/securedrop
  ./securedrop-admin --force restore install_files/ansible-base/v2-only.tar.gz
  

  • verify that the restore fails immediately, with a message indicating that a data-only backup can be performed.
• attempt a data-only restore from the v2-only backup with the following commands:

   cd ~/Persistent/securedrop
  ./securedrop-admin --force restore --preserve-tor-config install_files/ansible-base/v2-only.tar.gz
  

  • verify that the restore completes successfully, without running the reload Tor task.
  • verify that all onion service addresses are unchanged and that the data from the backup is present.

v2+v3:

• copy the admin workstation ~/.ssh/config file to a temporary location, i.e. ~/sshconfig.bak

• via an admin workstation terminal, attempt to restore from the v2+v3 backup (v2nv3.tar.gz) with the following commands:

  cd ~/Persistent/securedrop
  ./securedrop-admin --force restore install_files/ansible-base/v2nv3.tar.gz
  

  • verify that the restore playbook completes successfully and does not error out on the reload Tor task.

• wait a few minutes for Tor to restart on the server, and then complete the restore process with the following commands:

  ./securedrop-admin --force install
  ./securedrop-admin --force tailsconfig
  

  • verify that the application and monitor servers are available over SSH (ssh app uptime && ssh mon uptime)
  • verify that the SI and JI shortcuts now point to the addresses from the backup and are accessible
  • verify that v2 services from the backup are not available
  • verify that the SSH onion service addresses in ~/.ssh/config and ~/sshconfig.bak are identical

v3-only:

• copy the admin workstation ~/.ssh/config file to a temporary location, i.e. ~/sshconfig.bak

• via an admin workstation terminal, attempt to restore from the v3-only backup (v3-only.tar.gz) with the following commands:

  cd ~/Persistent/securedrop
  ./securedrop-admin --force restore install_files/ansible-base/v3-only.tar.gz
  

  • verify that the restore playbook completes successfully and does not error out on the reload Tor task.

• wait a few minutes for Tor to restart on the server, and then complete the restore process with the following commands:

  ./securedrop-admin --force install
  ./securedrop-admin --force tailsconfig
  

  • verify that the application and monitor servers are available over SSH (ssh app uptime && ssh mon uptime)
  • verify that the SI and JI shortcuts now point to the addresses from the backup and are accessible
  • verify that the SSH onion service addresses in ~/.ssh/config and ~/sshconfig.bak are identical

[rmol]

The reorganization looks all right, and the test plan produced the expected results in each scenario.