Give validate command an --allow-any-path option

[jwodder]

Closes #780.

[codecov]

Codecov Report

Merging #783 (f911a2c) into master (27dd3e7) will decrease coverage by 0.17%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #783      +/-   ##
==========================================
- Coverage   85.22%   85.05%   -0.18%     
==========================================
  Files          59       59              
  Lines        6039     6041       +2     
==========================================
- Hits         5147     5138       -9     
- Misses        892      903      +11     

Flag	Coverage Δ
unittests	85.05% <100.00%> (-0.18%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
dandi/cli/cmd_validate.py	85.00% <100.00%> (+0.38%)	⬆️
dandi/validate.py	53.48% <100.00%> (+0.54%)	⬆️
dandi/support/generatorify.py	0.00% <0.00%> (-16.10%)	⬇️
dandi/dandiarchive.py	82.23% <0.00%> (-1.53%)	⬇️
dandi/download.py	81.69% <0.00%> (-0.55%)	⬇️
dandi/dandiapi.py	92.46% <0.00%> (-0.20%)	⬇️
dandi/tests/test_metadata.py	100.00% <0.00%> (ø)
dandi/metadata.py	82.17% <0.00%> (+2.50%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 27dd3e7...f911a2c. Read the comment docs.

[yarikoptic]

FWIW, I thought to suggest adding exclude_datalad=True but since we already default to exclude_dotdirs = True I guess that doesn't matter.

[yarikoptic]

need to think whether we would like to make validate consistent with upload and gain devel option --allow-any-path (so not just always do allow_any_path=True)... otherwise we are making validate and upload not behave analogously and that could confuse folks. WDYT?

[jwodder]

@yarikoptic Done.

[yarikoptic]

Thank you @jwodder ... let's proceed with this and release to please dandischema testing with all the fixes we piled up.

[yarikoptic]

note -- there is one failing test run (ubuntu-18.04, 3.7, dev-deps) with

=================================== FAILURES ===================================
_____________ test_download_000027_resume[0.210831.2033-<lambda>2] _____________
dandi/tests/test_download.py:90: in test_download_000027_resume
    download(url, tmp_path, get_metadata=False)
dandi/download.py:59: in download
    parsed_url = parse_dandi_url(urls[0])
dandi/dandiarchive.py:421: in parse
    new_url = cls.follow_redirect(url)
dandi/dandiarchive.py:534: in follow_redirect
    raise NotFoundError(url)
E   dandi.exceptions.NotFoundError: https://dandiarchive.org/dandiset/000027/0.210831.2033
------------------------------ Captured log call -------------------------------
DEBUG    dandi:dandiarchive.py:402 Parsing url https://dandiarchive.org/dandiset/000027/0.210831.2033

but it can't relate to this PR, so proceeding