Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow a single retry on flaky scenarios #1792

Merged
merged 3 commits into from
Dec 22, 2022
Merged

Allow a single retry on flaky scenarios #1792

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e142a9a
Allow a single retry on flaky scenarios
twometresteve Dec 22, 2022
a1aaa5d
Too flaky to reinstate
twometresteve Dec 22, 2022
ee29a62
Merge branch 'next' into tms/flaky-retries
twometresteve Dec 22, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion features/full_tests/in_foreground.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,5 @@ Feature: In foreground field populates correctly
And I send the app to the background for 5 seconds
Then I wait to receive an error
And the error is valid for the error reporting API version "4.0" for the "Android Bugsnag Notifier" notifier
And the event "app.inForeground" is false
# PLAT-9155 Flaky: the event "app.inForeground" is false
And the error is correct for "InForegroundScenario" or I allow a retry
6 changes: 4 additions & 2 deletions features/full_tests/native_event_tracking.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@ Feature: Synchronizing app/device metadata in the native layer
And I send the app to the background for 5 seconds
And I wait to receive an error
Then the error payload contains a completed handled native report
And the event "app.inForeground" is false
# PLAT-9155 Flaky: the event "app.inForeground" is false
And the error is correct for "CXXBackgroundNotifyScenario" or I allow a retry
And the event "app.durationInForeground" equals 0
And the event "app.duration" is greater than 0
And the event "context" string is empty
Expand All @@ -43,7 +44,8 @@ Feature: Synchronizing app/device metadata in the native layer
And I configure Bugsnag for "CXXDelayedCrashScenario"
And I wait to receive an error
Then the error payload contains a completed handled native report
And the event "app.inForeground" is false
# PLAT-9155 Flaky: the event "app.inForeground" is false
And the error is correct for "CXXDelayedCrashScenario" or I allow a retry
And the event "app.duration" is greater than 0
And the event "context" string is empty
And the event "unhandled" is true
7 changes: 7 additions & 0 deletions features/steps/android_steps.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -261,5 +261,12 @@ def click_if_present(element)
when 'MultiThreadedStartupScenario'
Maze.dynamic_retry = true if message == 'You must call Bugsnag.start before any other Bugsnag methods'
Maze.check.equal 'Scenario complete', message
when 'InForegroundScenario', 'CXXBackgroundNotifyScenario', 'CXXDelayedCrashScenario'
begin
step 'the event "app.inForeground" is false'
rescue Test::Unit::AssertionFailedError
Maze.dynamic_retry = true
raise
end
end
end