Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for newer HDMF #1191

Merged
merged 2 commits into from
Jan 24, 2023
Merged

Fix for newer HDMF #1191

merged 2 commits into from
Jan 24, 2023

Conversation

TheChymera
Copy link
Contributor

@TheChymera TheChymera commented Jan 23, 2023

From: hdmf-dev/hdmf#817 (comment)

Short summary (full story in the issue above):

=hdmf-3.5.0 breaks our dandi/tests/test_files.py::test_validate_bogus test. This is because a new deletion method was introduced, which assumes all HDF objects have _HDF5IO__open_links, which some constructors don't create. Upstream has fixed this, but this will only take effect in future versions. However, this will still not fix our test, because upstream would prefer to still raise a warning, which pytest detects as a failure. In order to be able to use future HDMF versions, we need this PR.

Since there is no patched HDMF on pypi available for our use, the test suite below doesn't actually test the case which this PR addresses, but I tested it locally with the patched HDMF package. I would propose, if we agree on the fix, we just ping upstream for a new patch release 3.5.1.

@yarikoptic

@codecov
Copy link

codecov bot commented Jan 23, 2023

Codecov Report

Base: 89.08% // Head: 89.09% // Increases project coverage by +0.00% 🎉

Coverage data is based on head (9ad66c4) compared to base (d958555).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1191   +/-   ##
=======================================
  Coverage   89.08%   89.09%           
=======================================
  Files          76       76           
  Lines        9448     9451    +3     
=======================================
+ Hits         8417     8420    +3     
  Misses       1031     1031           
Flag Coverage Δ
unittests 89.09% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
dandi/tests/test_files.py 100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@TheChymera TheChymera marked this pull request as ready for review January 23, 2023 19:41
warnings.filterwarnings(
"ignore", message="HDF5IO was not fully initialized before close"
)
errors = dandi_file(path).get_validation_errors()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please add a comment on why needs to be filtered, point to hdmf issue.

PR should be accompanied with the revert of 6d065ec to ensure that we test it with current/buggy hdmf

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commenting improved in 9ad66c4.

@yarikoptic
As for the second part, sorry if my explanation in the first post was not clear enough, but we specifically do not want to revert that commit. The issue is a real bug in 3.5.0, which we cannot easily circumvent in DANDI. This PR simply leverages the upstream patch, which will become available upon the next release, hopefully one we can ask for right now if you agree this has been properly addressed for our use case (I do).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I think I got it. But is this the only place where such warning might be raised? If not, we might need to add it to https://github.com/dandi/dandi-cli/blob/master/tox.ini#L48 instead of to some other tests.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, this is the only test with that issue. Not sure if it's a good idea to move the catching to tox.ini — as that will not be caught when running straight-up pytest... or maybe I'm misunderstanding what you're suggesting? I predominantly use pytest directly to test locally, and none of the tox.ini errors ever popped up 🤔

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In any case upstream said a new patch release for this sounds good.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

confusingly enough pytest reads (as well) config from tox.ini, see https://docs.pytest.org/en/7.1.x/reference/customize.html#tox-ini , that is why you didn't see any error. Ok, if you say that it is the only test -- let's proceed.

@yarikoptic yarikoptic merged commit 10921e3 into master Jan 24, 2023
@yarikoptic yarikoptic deleted the hdmfix branch January 24, 2023 01:26
@jwodder jwodder added the tests Add or improve existing tests label Feb 10, 2023
@github-actions
Copy link

🚀 PR was released in 0.49.0 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
released tests Add or improve existing tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants