Skip to content

Workstation Acceptance Tests

Jump to bottom
deeplow edited this page Jul 15, 2024 · 53 revisions

SecureDrop Workstation test scenarios

Some scenarios require a supported printer. We're tracking compatible printers available to team members here.

Also see https://drive.google.com/drive/u/0/folders/1lyk2V15e7amI9OxvgUK_YhYIUVNDWJo0 for different types of file submissions we'll want to test for printing and display (mostly pulled from https://file-examples.com/).

Possible way to break up testing:

Creative/exploratory testing: Beyond the test plan, it's helpful to be a bit creative in a way that simulates a normal user - with emphasis on the last part. (Clicking a button a few times to see if there are any adverse effects on the UI - helpful! Using an automated framework to spam the GUI with hundreds of clicks per second to see what happens - less helpful!). Document any exploratory testing, impressions and observations, and ideally, include steps for others to reproduce your findings.

Some examples:

  • Qubes-related interactions: How's updating? How's attaching, removing, and interacting with peripheral devices? Do our docs clearly state how to recover from scenarios such as failed network connection, failed updates, etc? What happens if you disconnect the network in the middle of an update, use toxiproxy to simulate slow downloads, etc?
  • Large file issues: Can large files be downloaded and/or moved between VMs? Can conditions be introduced that would make some large file downloads problematic (eg: a several-megabyte file)?
  • Large volume of submissions: What happens when there are lots of submissions on the server?
  • Impatient/hurried user: What are your observations if you approach your testing session from the point of view of being in a hurry, short on time, rapidly clicking, etc

Some of the feedback may be bug reports and/or UX observations that aren't directly bug reports.

Internationalization (reference)

At least one tester should check that:

  • Under the LANG=en_XA pseudolocale, all GUI strings like foo are correctly wrapped like [!foo!]. (Any string not so wrapped is missing its gettext() wrapper.)
  • Generate a conversation transcript and export it. Inspect to make sure it displays correctly in the specified locale.

Qubes scenarios

Some of these may be a bit time-consuming, so we typically have a subset of devs test these specific scenarios. In these scenarios, we try to validate the behavior of different components of the system. There's a separate section with detail on Client functionality specifically.

The Qubes scenarios are tracked in the requirements tracing document. This is an effort to ensure most (if not all system functionality) is covered by QA / automated tests. This temporarily lives in a google doc, but the goal to move it to the repository itself and have each requirement mapped to its implementation and respective test with the help of a tool like OpenFastTrace.

Client scenarios

Some client tests do not require being run in a SecureDrop Workstation environment. The list of scenarios below is a complete list for full acceptance testing, which should be done in the supported target Qubes environment.

Scenario: Offline mode without existing data

Prerequisites:

  • server is available and contains source test data
  • test data includes at least one previously downloaded submission
  • test data includes at least one undownloaded submission
  • ~/.securedrop_client/data in sd-app is empty, and ~/.securedrop_client/svs.sqlite does not exist (do not delete the entire ~/.securedrop_client directory)
  • the sd-devices VM is not running (shut down manually if necessary)
  • a supported printer is available, but not attached.

Offline to Online

  • When SecureDrop desktop icon is double-clicked, preflight updater is displayed
  • After preflight updater runs, when user clicks Continue, login dialog is displayed
  • When user clicks Work Offline, login dialog closes and main window opens
  • after startup:
    • there is no sync attempt with the server
    • the source list is empty
  • When the user clicks the top-left user icon and chooses Sign in:
    • the login dialog is displayed over the main window
  • When the user enters valid login details and clicks Log in:
    • the login dialog closes
    • The user icon is updated to reflect the user's details
    • the client is synced with the server and the source list is updated
  • When the user selects a source with submissions from the source list:
    • the conversation view is populated with the source conversation
    • the reply panel is active
    • a reply can be sent to the source
    • a submission can be downloaded
    • a downloaded submission can be exported
  • When the user clicks the main window close button:
    • the client exits.

Scenario: Offline mode with existing data

Prerequisites:

  • server is available and contains source test data
  • test data includes at least one previously downloaded submission
  • test data includes at least one undownloaded submission
  • client data directory has been synced with server in a previous login
  • the sd-devices VM is not running (shut down manually if necessary)
  • a supported printer is available, but not attached.

Offline to Online

  • When SecureDrop desktop icon is double-clicked, preflight updater is displayed
  • After preflight updater runs, when user clicks Continue, login dialog is displayed
  • When user clicks Work Offline, login dialog closes and main window opens
  • after startup:
    • there is no sync attempt with the server
    • the source list is populated with contents of last server sync
  • When the user selects a source with submissions from the source list:
    • the conversation view is populated with the source conversation
    • the reply panel is inactive, with a "Sign in" message
    • a previously downloaded submission can be exported
    • a previously downloaded submission can be printed
    • When the user clicks Download on an undownloaded submission, a message is displayed instructing the user to sign in to perform the download
  • When the user clicks the top-left user icon and chooses Sign in:
    • the login dialog is displayed over the main window
  • When the user enters valid login details and clicks Log in:
    • the login dialog closes
    • The user icon is updated to reflect the user's details
    • source data is synced with the server
  • When the user selects a source with submissions from the source list:
    • the conversation view is populated with the source conversation
    • the reply panel is active
    • When the user replies to a source, the reply is added to the source conversation
    • When the user clicks Download on an undownloaded submission, the submission is downloaded and decrypted
    • When the user clicks Export on a submission, the export process can be completed
    • When the user clicks Print on a submission, the print process can be completed
  • When the user clicks the main window close button:
    • the client exits.

Scenario: Client and Journalist Interface both in use

Note: this scenario requires access to the Journalist Interface (JI) via Tor Browser. If the scenario is being tested on Qubes, the JI address can be found in sd-whonix in /usr/local/etc/torrc.d/50_user.conf. See https://github.com/freedomofpress/securedrop-workstation/wiki/Developer-Tips#how-to-connect-to-the-journalist-interface-in-qubes for instructions on how to connect to the JI in a VM.

Prerequisites:

  • server is available and contains source test data
  • client data directory is empty

Login

  • when SecureDrop desktop icon is double-clicked, preflight updater is displayed
  • After preflight updater runs, when user clicks Continue, login dialog is displayed
  • after valid login to client:
    • the login dialog closes
    • source data is downloaded and source list is populated
    • user is prompted for GPG key access
    • submissions and replies are decrypted
    • the source list is displayed but no sources are selected by default
    • the conversation view is not populated
  • when the JI address is visited in Tor Browser:
    • JI login page is displayed
  • after valid login to JI using same account as for client:
    • sources page is displayed, containing the same sources as the client (order may differ)

Sources, replies, submissions

  • when a source is starred in the client:
    • the source is also starred in the JI after a page reload
  • when a starred source is unstarred in the JI:
    • the source is also unstarred in the client after next sync.
  • when a reply is sent to a source via the client:
  • the reply is visible in the JI and can be viewed by the source in the Source Interface
  • when a reply is sent to a source via the JI:
    • the reply is visible in the source conversation view after next sync
  • when the journalist account used to reply is deleted by an admin in the JI:
    • the next sync is successful
    • the reply is visible in the conversation view
    • the journalist's details are deleted from the client database
  • when a reply is deleted by a source:
  • when an individual file submission is deleted in the JI:
    • the submission is no longer listed in the conversation view
    • the submission files are deleted from the client data directory
  • when an individual message is deleted in the JI:
    • the message is no longer listed in the conversation view
    • the messages are deleted from the client database
  • when a source is deleted in the JI:
    • the source is no longer listed in the client after next sync
    • files associated with the source are no longer present in the client data directory
  • when a source is deleted in the client:
    • the source is no longer listed in the JI after a page reload

Scenario: Large dataset

Prerequisites:

  • server is available and contains large source test dataset (256 sources, submission sizes ranging from 1-500MB)
  • client data directory is empty

Sources

  • after valid login:
    • the login dialog closes
    • all source data is downloaded and source list is populated
    • user can scroll to bottom of source list
    • user is prompted for GPG key access
    • submissions and replies are decrypted
    • the source list is displayed but no sources are selected by default
    • the conversation view is not populated
  • when a source is selected in source list:
    • conversation view is populated with source conversation
    • a source message containing HTML is displayed as unformatted text
    • source submissions have an active Download button
    • source submission compressed file size is displayed accurately
  • when the upper right 3-dot button is clicked:
    • a menu is displayed with a delete source account option
    • when delete source account is selected:
      • the source is deleted from the source list and the converation view is blanked
      • the source is deleted from the server and not restored on next sync
      • source submissions and messages are removed from the client's data directory
  • when a source is starred in source list, and the client is closed and reopened in Online mode:
    • the source is still starred in the source list

Replies

  • when a source is selected in the source list:
    • the reply panel is available for use and there is no message asking the user to sign in
    • a reply can be added to the conversations
    • a reply containing HTML is displayed as unformatted text
    • two replies added immediately after each other are ordered correctly

Submissions

Preview
  • when Download is clicked on a submission:
    • the submission is downloaded and decrypted
    • the Download button is replaced with Print and Export options
    • the submission filename is displayed.
  • For a DOC submission:
    • when the submission filename is clicked, a disposable VM (dispVM) is started.
    • after the dispVM starts, the submission is displayed in LibreOffice
    • when LibreOffice is closed, the dispVM shuts down
  • For a PDF submission:
    • when the submission filename is clicked, a dispVM is started.
    • after the dispVM starts, the submission is displayed in evince
    • when evince is closed, the dispVM shuts down
  • For a JPEG submission:
    • when the submission filename is clicked, a dispVM is started.
    • after the dispVM starts, the submission is displayed in Image Viewer
    • when evince is closed, the dispVM shuts down

Release-specific test plans

Some of the tests below should be incorporated into main test plan after the release, while others will not need to be re-tested with each release.

SecureDrop Workstation 0.3.0

Moved to https://github.com/freedomofpress/securedrop-workstation/issues/548

SecureDrop Client 0.2.0

Moved to https://github.com/freedomofpress/securedrop-client/pull/1083

Clone this wiki locally