Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(tests): add 10ms delay after we are done with stubr #408

Merged
merged 3 commits into from
Feb 19, 2024

Conversation

argl
Copy link
Contributor

@argl argl commented Jan 26, 2024

  • Make sure we drop any stubr instance deliberately, both with the attribute macro and with manual setup.
  • Drop stubr using our own async helper function drop_stubr(stubr: Stubr)that drops it and adds 10ms of delay.
  • Remove all older check/wait/sleep functions around stubr.
  • Make sure we only use a single instance of stubr inside a test by splitting tests up.

I ran the tests in a loop on my local machine, this version did not fail after around 130 invocations which is where I stopped it.

Shell script to run tests it in a loop:

LOGFILE=testlog.log
export RUST_LOG=rumba:info
export MDN_SETTINGS=.settings.test.toml
echo "$(date -Iseconds) START" >$LOGFILE
while true; do
	cargo test --all -- --test-threads=1 --nocapture; 
	if [ $? -ne 0 ]; then 
		echo "$(date -Iseconds) ===== The test failed. ======" | tee -a $LOGFILE; 
		break; 
	else 
		echo "$(date -Iseconds) OK" >>$LOGFILE
		sleep 0.1; 
		clear; 
	fi; 
done

…selves in all cases, removed old check/wait functions around stubr
Copy link
Contributor

It looks like this is your first pull request. 🎉
Thank you for your contribution! One of the project maintainers will triage
and assign the pull request for review. We appreciate your patience. To
safeguard the health of the project, please take a moment to read our
code of conduct.

@argl argl marked this pull request as ready for review January 26, 2024 14:01
@argl argl requested review from mdn-bot and a team as code owners January 26, 2024 14:01
@caugner caugner changed the title add 10ms delay after we are done with stubr fix(tests): add 10ms delay after we are done with stubr Jan 29, 2024
Copy link
Contributor

@fiji-flo fiji-flo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks looks good. And thanks for changing the sleep 👍

@fiji-flo fiji-flo merged commit 18d8fda into main Feb 19, 2024
3 checks passed
@fiji-flo fiji-flo deleted the fix-flaky-tests branch February 19, 2024 15:35
Copy link
Contributor

Congratulations on your first merged pull request. 🎉 Thank you for your contribution!
Did you know we have a project board with high-impact contribution opportunities?
We look forward to your next contribution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants