PaX flags are unset during install time

[conorsch]

Description

The grsecurity pax flag checks on the grub binaries are currently failing. Those don't run in CI—yet; see #3909—so we haven't been carefully observing them.

Steps to Reproduce

1. Provision local staging: make staging.
2. Run testinfra suite: molecule verify -s libvirt-staging (substitute virtualbox-staging if appropriate)
3. Observe PaX flag errors
4. Re-run staging logic: molecule converge -s libvirt-staging
5. Re-run testinfra suite: molecule verify -s libvirt-staging (substitute virtualbox-staging if appropriate)
6. Observe all tests pass [0]

Expected Behavior

The fresh install story configures PaX flags in a persistent manner.

Actual Behavior

The fresh install story sets PaX flags, but the are overwritten if updates to grub are available.

Comments

We can resolve this problem by moving the "common" role before the "grsecurity" role; it should be the first in the play. We'll upgrade all packages, then after updates to grub are installed, the PaX flags will be set.

In #2741 we moved the grsecurity role up to fail faster. That minor convenience isn't worth the risk of misconfiguration. Since #2703, we've been managing PaX flags on grub for existing installs—but we still have an install time window where they can be overwritten.

[0] Regarding "all tests pass", I actually saw one still fail, but it's unrelated:

``` =================================== FAILURES =================================== ____ test_securedrop_application_test_journalist_key[ansible://app-staging] ____ [gw3] linux2 -- Python 2.7.13 /home/conor/.virtualenvs/sd/bin/python2

File = <class 'testinfra.modules.base.GNUFile'>, Sudo = <sudo>

    def test_securedrop_application_test_journalist_key(File, Sudo):
        """
        Ensure the SecureDrop Application GPG public key file is present.
        This is a test-only pubkey provided in the repository strictly for testing.
        """
        pubkey_file = File("{}/test_journalist_key.pub".format(
            securedrop_test_vars.securedrop_data))
        # Sudo is only necessary when testing against app hosts, since the
        # permissions are tighter. Let's elevate privileges so we're sure
        # we can read the correct file attributes and test them.
        with Sudo():
            assert pubkey_file.is_file
>           assert pubkey_file.user == "root"
E           AssertionError: assert 'www-data' == 'root'
E             - www-data
E             + root

../testinfra/staging/app-code/test_securedrop_app_code.py:61: AssertionError
=== 1 failed, 423 passed, 13 skipped, 3 xfailed, 1 xpassed in 190.45 seconds ===

</details>

[kushaldas]

molecule verify -s {libvirt,virtualbox}-staging-xenial fails with the following errors

   =================================== FAILURES ===================================
    __________ test_pax_flags[ansible://app-staging-/usr/sbin/grub-probe] __________
    [gw1] linux2 -- Python 2.7.13 /home/kdas/code/sd/bin/python2
    [XPASS(strict)] PaX flags unset at install time, see issue #3916
    _______ test_pax_flags[ansible://app-staging-/usr/sbin/grub-mkdevicemap] _______
    [gw1] linux2 -- Python 2.7.13 /home/kdas/code/sd/bin/python2
    [XPASS(strict)] PaX flags unset at install time, see issue #3916
    _______ test_pax_flags[ansible://app-staging-/usr/bin/grub-script-check] _______
    [gw1] linux2 -- Python 2.7.13 /home/kdas/code/sd/bin/python2
    [XPASS(strict)] PaX flags unset at install time, see issue #3916
    __________ test_pax_flags[ansible://mon-staging-/usr/sbin/grub-probe] __________
    [gw0] linux2 -- Python 2.7.13 /home/kdas/code/sd/bin/python2
    [XPASS(strict)] PaX flags unset at install time, see issue #3916
    _______ test_pax_flags[ansible://mon-staging-/usr/sbin/grub-mkdevicemap] _______
    [gw1] linux2 -- Python 2.7.13 /home/kdas/code/sd/bin/python2
    [XPASS(strict)] PaX flags unset at install time, see issue #3916
    _______ test_pax_flags[ansible://mon-staging-/usr/bin/grub-script-check] _______
    [gw0] linux2 -- Python 2.7.13 /home/kdas/code/sd/bin/python2
    [XPASS(strict)] PaX flags unset at install time, see issue #3916
    === 6 failed, 447 passed, 13 skipped, 2 xfailed, 2 xpassed in 211.42 seconds ===

This is while testing #4080

[conorsch]

Turns out the base box used for Xenial testing (and therefore presumably any VM, Trusty included) can result in different behavior here. Examples below:

Using a base box dated 2018-08-24 (admittedly rather old):

$ vagrant box list | grep ubuntu-16
bento/ubuntu-16.04 (libvirt, 201808.24.0)
bento/ubuntu-16.04 (virtualbox, 201808.24.0)
bento/ubuntu-16.04 (virtualbox, 201812.27.0)

All config tests were passing under Xenial:

======== 446 passed, 13 skipped, 8 xfailed, 2 xpassed in 148.99 seconds ========

After upgrading the box base, however, to one dated

$ vagrant box update --box bento/ubuntu-16.04 --provider virtualbox
$ vagrant mutate bento/ubuntu-16.04 libvirt
$ vagrant box list | grep ubuntu-16
bento/ubuntu-16.04 (libvirt, 201808.24.0)
bento/ubuntu-16.04 (virtualbox, 201808.24.0)
bento/ubuntu-16.04 (libvirt, 201812.27.0)
bento/ubuntu-16.04 (virtualbox, 201812.27.0)

Then the PaX tests are seen to fail:

    =================================== FAILURES ===================================
    __________ test_pax_flags[ansible://app-staging-/usr/sbin/grub-probe] __________
    [gw2] linux2 -- Python 2.7.13 /home/conor/.virtualenvs/sd/bin/python2
    [XPASS(strict)] PaX flags unset at install time, see issue #3916
    _______ test_pax_flags[ansible://app-staging-/usr/sbin/grub-mkdevicemap] _______
    [gw2] linux2 -- Python 2.7.13 /home/conor/.virtualenvs/sd/bin/python2
    [XPASS(strict)] PaX flags unset at install time, see issue #3916
    _______ test_pax_flags[ansible://app-staging-/usr/bin/grub-script-check] _______
    [gw2] linux2 -- Python 2.7.13 /home/conor/.virtualenvs/sd/bin/python2
    [XPASS(strict)] PaX flags unset at install time, see issue #3916
    __________ test_pax_flags[ansible://mon-staging-/usr/sbin/grub-probe] __________
    [gw3] linux2 -- Python 2.7.13 /home/conor/.virtualenvs/sd/bin/python2
    [XPASS(strict)] PaX flags unset at install time, see issue #3916
    _______ test_pax_flags[ansible://mon-staging-/usr/sbin/grub-mkdevicemap] _______
    [gw3] linux2 -- Python 2.7.13 /home/conor/.virtualenvs/sd/bin/python2
    [XPASS(strict)] PaX flags unset at install time, see issue #3916
    _______ test_pax_flags[ansible://mon-staging-/usr/bin/grub-script-check] _______

Same as @kushaldas reported above, and in #4080, and most recently in #4099. While a role run order change would indeed resolve in the clean install story, we should consider managing PaX over time, via paxctld or otherwise, to ensure a stable system state over time.