More intermittent Behat failures

[arknoll]

My system information:

• Operating system type: Travis
• Operating system version: BLT Travis integration
• BLT version: 8.6.14

Travis is stalling out while running behat tests. It is specifically stalling during javascript tests. This likely has to do with chromedriver failing. See #1126

@security @api
Feature: Site Security
  As a project owner
  I want my site to be safe and secure
  So I do not have my site compromised
  @users
  Scenario: The Super User does not have the default name                 # features/security.feature:8
    Given I am logged in as a user with the "administer users" permission # Drupal\DrupalExtension\Context\DrupalContext::assertLoggedInWithPermissions()
    When I visit "user/1/edit"                                            # Drupal\DrupalExtension\Context\MinkContext::assertAtPath()
    Then the "Username" field should contain "nidcr_admin"                # Drupal\DrupalExtension\Context\MinkContext::assertFieldContains()
  @mollom
  Scenario: Verify that Mollom module is enabled                           # features/security.feature:14
    Given I am logged in as a user with the "administer mollom" permission # Drupal\DrupalExtension\Context\DrupalContext::assertLoggedInWithPermissions()
    When I visit "admin/config/content/mollom/settings"                    # Drupal\DrupalExtension\Context\MinkContext::assertAtPath()
    Then I should see the heading "Mollom settings"                        # Drupal\DrupalExtension\Context\MinkContext::assertHeading()
  @tfa
  Scenario: Verify that Two-factor authentication is enabled            # features/security.feature:20
    Given I am logged in as a user with the "access content" permission # Drupal\DrupalExtension\Context\DrupalContext::assertLoggedInWithPermissions()
    And I am on the homepage                                            # Drupal\DrupalExtension\Context\MinkContext::iAmOnHomepage()
    When I click "My account"                                           # Drupal\DrupalExtension\Context\MinkContext::assertClick()
    And I follow "Security"                                             # Drupal\DrupalExtension\Context\MinkContext::clickLink()
    Then I should see the heading "TFA application"                     # Drupal\DrupalExtension\Context\MinkContext::assertHeading()
@staff-contact @api
Feature: Staff Contact content type
  As an content editor I should be able
  to create nodes of staff contact
  @content-type @content-create @javascript
  Scenario: To check if we can create a node of type staff contact with extramural option selected                   # features/staff_contact.feature:8
No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.
Check the details on how to adjust your build configuration on: https://docs.travis-ci.com/user/common-build-problems/#Build-times-out-because-no-output-was-received
The build has been terminated

[geerlingguy]

I'm starting to hit this now and then as well:

  @javascript
  Scenario: Ensure that the WYSIWYG editor is present.         # features/Content.feature:18
Starting ChromeDriver 2.24.417424 (c5c5ea873213ee72e3d0929b47482681555340c3) on port 6922
Only local connections are allowed.
No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.
Check the details on how to adjust your build configuration on: https://docs.travis-ci.com/user/common-build-problems/#Build-times-out-because-no-output-was-received

Seems entirely random, but between this and the frequent MySQL build failures on Travis, we're now at a rate where I could flip a coin as to whether a build on Travis will succeed on the first run :(

[geerlingguy]

Inside Drupal VM, it seems that any new builds are running into this issue... e.g. if you go:

$ vagrant ssh
vagrant $ cd /var/www/projectname
vagrant $ blt tests:behat

Then you get the error in the above comment.

From a chat somewhere, I remember seeing a mention of:

to switch back to phantomjs, set behat.launch-selenium: false and behat.launch-phantomjs: true

Checking whether this works OOTB for people using Drupal VM with BLT.

[geerlingguy]

...and it looks like it does work splendidly. Just modify the behat section in your blt/project.yml file with the following:

behat:
  paths:
    - your-paths-go-here
  launch-selenium: false
  launch-phantomjs: true

[geerlingguy]

Note that this makes most things work... however, there's a particular Lightning test that seems to fail on PhantomJS that works via Chromedriver—the Feature: Scheduled updates to content scenario Scenario: Publishing a node that is scheduled to be published in the past. It fails here:

    And I visit "/admin/config/workflow/schedule-updates/run"               # Drupal\DrupalExtension\Context\MinkContext::assertAtPath()
    And I press "Run Updates"                                               # Drupal\DrupalExtension\Context\MinkContext::pressButton()
    And I should see "Results: 1 update(s) were performed"                  # Drupal\DrupalExtension\Context\MinkContext::assertPageContainsText()
      The text "Results: 1 update(s) were performed" was not found anywhere in the text of the current page. (Behat\Mink\Exception\ResponseTextException)

[grasmash]

This issue will not be resolved in BLT since it stems from issues in the tools themselves.

As of #1705, BLT will offer selenium, phantomjs, and headless chrome support. Hopefully one of these tools will meet all of your needs.