TestPantsDaemonIntegration.test_pantsd_sigterm is flaky

[cosmicexplorer]

To my great shame, this test, added in #6574, has flaked: see https://travis-ci.org/pantsbuild/pants/jobs/510547772.

                     ==================== FAILURES ====================
                     _ TestPantsDaemonIntegration.test_pantsd_sigterm _
                     
                     self = <pants_test.pantsd.test_pantsd_integration.TestPantsDaemonIntegration testMethod=test_pantsd_sigterm>
                     
                         def test_pantsd_sigterm(self):
                           self._assert_pantsd_keyboardinterrupt_signal(
                             signal.SIGTERM,
                     >       ['Signal {signum} (SIGTERM) was raised. Exiting with failure.'.format(signum=signal.SIGTERM)])
                     
                     .pants.d/pyprep/sources/62676ab0dd65698bc674cce2191134fbe580f110/pants_test/pantsd/test_pantsd_integration.py:464: 
                     _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
                     .pants.d/pyprep/sources/62676ab0dd65698bc674cce2191134fbe580f110/pants_test/pantsd/test_pantsd_integration.py:450: in _assert_pantsd_keyboardinterrupt_signal
                         self.assertIn(msg, waiter_run.stderr_data)
                     E   AssertionError: 'Signal 15 (SIGTERM) was raised. Exiting with failure.' not found in 'INFO] waiting for file /home/travis/build/pantsbuild/pants/.pants.d/tmp/tmpnxjru4og.pants.d/.pids/pantsd/pid to appear...\nInterrupted by user:\nInterrupted by user over pailgun client!\n'

Not sure how to repro this yet, but running it enough times locally might work. I believe this happens when the timeout for reading more content from the nailgun client completes before all the output from the subprocess is written. It's not clear why the timeout being hit isn't causing a warn log, as it is expected to.

[cosmicexplorer]

Happened again. I think the answer might actually be the opposite of the above, in that the subprocess likely completes before the timeout completes, so the self._exit_reason is raised, but there's no signal sent to the pantsd-runner subprocess. How can we stop this subprocess from completing before our timeout? We could either make this timeout smaller (probably the right idea), or we could figure out how to make our subprocess wait longer.