Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mark tests requiring network access and add a test workflow that disables network access #88

Merged
merged 2 commits into from
Sep 28, 2021

Conversation

jwodder
Copy link
Member

@jwodder jwodder commented Sep 27, 2021

Closes #87.

@jwodder jwodder added the tests Add or improve existing tests label Sep 27, 2021
@codecov
Copy link

codecov bot commented Sep 27, 2021

Codecov Report

Merging #88 (9d8a47d) into master (d4c1378) will increase coverage by 6.80%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #88      +/-   ##
==========================================
+ Coverage   89.47%   96.28%   +6.80%     
==========================================
  Files          14       16       +2     
  Lines        1378     1399      +21     
==========================================
+ Hits         1233     1347     +114     
+ Misses        145       52      -93     
Flag Coverage Δ
unittests 96.28% <100.00%> (+6.80%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
dandischema/tests/conftest.py 100.00% <100.00%> (ø)
dandischema/tests/test_datacite.py 100.00% <100.00%> (+53.48%) ⬆️
dandischema/tests/test_metadata.py 100.00% <100.00%> (ø)
dandischema/tests/utils.py 100.00% <100.00%> (ø)
dandischema/datacite.py 95.32% <0.00%> (+43.92%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d4c1378...9d8a47d. Read the comment docs.

@yarikoptic
Copy link
Member

are test fails against dandi-cli the same as recently encountered in dandisets (dandi/dandisets@7ec49de) requiring to wait for it to become "valid"? I even wonder now if such logic should be somehow incorporated into .publish with some time out (e.g. of 5 seconds, which should be sufficient for unittesting) since it seems to become a common pattern and ideally should not be duplicated across codebase?

@jwodder
Copy link
Member Author

jwodder commented Sep 27, 2021

@yarikoptic That failure reason only applies to one test (fixed in dandi/dandi-cli@024b93b). Most of the failing tests are likely due to some mismatch in dandischema versions, and the failure is only exacerbated by the fact that trying to show the validation errors fails because dandi-cli's master branch expects version validation errors to be reported in a key that was removed from the API without telling me (addressed in dandi/dandi-cli@45a4215).

@yarikoptic
Copy link
Member

is only exacerbated by the fact that trying to show the validation errors fails because dandi-cli's master branch expects version validation errors to be reported in a key that was removed from the API without telling me (addressed in dandi/dandi-cli@45a4215).

shouldn't we have caught it somehow by integration testing in dandi-api against dandi-cli -- https://github.com/dandi/dandi-api/actions/workflows/cli-integration.yml seems to stay green

@jwodder
Copy link
Member Author

jwodder commented Sep 27, 2021

@yarikoptic We don't have any tests that expect Dandiset validation to fail.

@yarikoptic
Copy link
Member

Thank you @jwodder, let's proceed.

@TheChymera , set DANDI_TESTS_NONETWORK=1 env var while running tests and those which require outbound network will be skipped

@yarikoptic yarikoptic merged commit 45986b2 into master Sep 28, 2021
@yarikoptic yarikoptic deleted the gh-87 branch September 28, 2021 14:37
@TheChymera
Copy link

thanks, it worked ^^

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tests Add or improve existing tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

dandischema/tests/test_metadata.py::test_migrate_041 requires network access
3 participants