Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

[BUGFIX] [1.x] Backport of "Avoid logging seed for skipped tests #19322" #19327

Closed

Conversation

DickJC123
Copy link
Contributor

Description

This fixes issue #19255, where it was noted that when a unittest was skipped, the skip-exception caused the with_seed() decorator to log the seed. While the 1.x branch uses nosetests, not pytest, the same behavior is seen on 1.x. With this PR the seed is no longer logged.

This PR doesn't have any testing added- I will inspect the CI log for skipped tests such as test_operator.py:test_activation to verify that the seed logging has been successfully eliminated.

@leezu @samskalicky

Checklist

Essentials

  • [X ] PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
  • [X ] Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage
  • [X ] Code is well-documented

@mxnet-bot
Copy link

Hey @DickJC123 , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

  • To trigger all jobs: @mxnet-bot run ci [all]
  • To trigger specific jobs: @mxnet-bot run ci [job1, job2]

CI supported jobs: [windows-gpu, sanity, clang, miscellaneous, windows-cpu, website, centos-cpu, centos-gpu, edge, unix-gpu, unix-cpu]


Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

@DickJC123
Copy link
Contributor Author

I can verify the PR has fixed the problem. In the centos-cpu log of this PR:

[2020-10-10T02:44:13.929Z] test_operator.test_rnn_with_new_param ... ok (0.1284s)
[2020-10-10T02:44:13.929Z] test_operator.test_activation ... SKIP: test fails intermittently. temporarily disabled till it gets fixed. tracked at https://github.com/apache/incubator-mxnet/issues/13915
[2020-10-10T02:44:13.929Z] test_operator.test_laop_6 ... SKIP: Test crashes https://github.com/apache/incubator-mxnet/issues/15975
[2020-10-10T02:44:13.929Z] test_operator.test_sequence_last ... SKIP: Flaky test: https://github.com/apache/incubator-mxnet/issues/11395
[2020-10-10T02:44:28.755Z] test_operator.test_lstm_sym ... /work/mxnet/python/mxnet/rnn/rnn_cell.py:674: UserWarning: NTC layout detected. Consider using TNC for FusedRNNCell for faster speed

while in a log from an early 1.x PR #19296 without this fix:

[2020-10-06T02:22:39.700Z] test_operator.test_rnn_with_new_param ... ok (0.1109s)
[2020-10-06T02:22:39.700Z] test_operator.test_activation ... [WARNING] Error seen with seeded test, use MXNET_TEST_SEED=1946711394 to reproduce.
[2020-10-06T02:22:39.700Z] Error seen with seeded test, use MXNET_TEST_SEED=1946711394 to reproduce.
[2020-10-06T02:22:39.700Z] SKIP: test fails intermittently. temporarily disabled till it gets fixed. tracked at https://github.com/apache/incubator-mxnet/issues/13915
[2020-10-06T02:22:39.700Z] test_operator.test_laop_6 ... [WARNING] Error seen with seeded test, use MXNET_TEST_SEED=2941211 to reproduce.
[2020-10-06T02:22:39.700Z] Error seen with seeded test, use MXNET_TEST_SEED=2941211 to reproduce.
[2020-10-06T02:22:39.700Z] SKIP: Test crashes https://github.com/apache/incubator-mxnet/issues/15975
[2020-10-06T02:22:39.700Z] test_operator.test_sequence_last ... [WARNING] Error seen with seeded test, use MXNET_TEST_SEED=1784308719 to reproduce.
[2020-10-06T02:22:39.700Z] Error seen with seeded test, use MXNET_TEST_SEED=1784308719 to reproduce.
[2020-10-06T02:22:39.700Z] SKIP: Flaky test: https://github.com/apache/incubator-mxnet/issues/11395
[2020-10-06T02:22:54.544Z] test_operator.test_lstm_sym ... /work/mxnet/python/mxnet/rnn/rnn_cell.py:674: UserWarning: NTC layout detected. Consider using TNC for FusedRNNCell for faster speed

@mseth10 mseth10 added pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test and removed pr-work-in-progress PR is still work in progress labels Mar 14, 2022
@DickJC123 DickJC123 closed this Mar 15, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
pr-awaiting-testing PR is reviewed and waiting CI build and test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants