Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/python311 #89

Merged
merged 17 commits into from
Nov 15, 2022
Merged

Feat/python311 #89

merged 17 commits into from
Nov 15, 2022

Conversation

justindujardin
Copy link
Owner

@justindujardin justindujardin commented Nov 15, 2022

Python 3.11 dropped the use of Accessor classes (🎉), but it broke Pathy.

This PR refactors away from the Accessor class to using BucketClients directly from the Pathy class. This change calls out the one bit of technical debt that the project has: its dependency on pathlib internals. Pathy relies explicitly on the base pathlib class to do glob matches against scandir results. This isn't great because the project breaks when internals change, but it keeps the library simple by not reimplementing wildcard matching functionality.

Changes:

  • drop the use of _Accessor class from the internals of pathlib.
  • Detect python < 3.11 and provide the base needed accessor method "scandir"
  • Detect python >= 3.11 and provide the Path._scandir function
  • Move accessor methods to the Pathy class

Future Work

  • Remove the use of pathlib scandir entirely and replace it with an inline glob matching system to provide this functionality.

 - drop the use of _Accessor class from the internals of pathlib. It was removed in python 3.11 which broke... everything.
 - Detect python < 3.11 and provide the base needed accessor method "scandir"
 - Detect python >= 3.11 and provide the Path._scandir function
 - Move accessor methods to the Pathy class
 - Remove a few _accessor tests, and update those that referenced the accessor directly
 - TODO: Remove the use of pathlib scandir entirely, and replace it with an inlined glob matching system to provide this functionality. This is the only tech debt the project has left at this point that I can see.
 - not sure why I'm getting _accessor is readonly errors. stack overflow suggests it might be this. use the working code from main branch
 - file not fs 😅
 - better detect s3 and gcs test capabilities
 - the mypy version with a fix for numpy errors doesn't support the older python 3.6 builds
 - the mypy build that works with 3.6 doesn't work with python 3.11
 - the build that works with 3.10+ doesn't work with 3.6 since it's unpublished for that version.
@codecov
Copy link

codecov bot commented Nov 15, 2022

Codecov Report

Base: 100.00% // Head: 100.00% // No change to project coverage 👍

Coverage data is based on head (5c94e6d) compared to base (8c5f092).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff            @@
##            master       #89   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            5         5           
  Lines         1030      1004   -26     
=========================================
- Hits          1030      1004   -26     
Impacted Files Coverage Δ
pathy/__init__.py 100.00% <100.00%> (ø)
pathy/about.py 100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

 - instead of has_gcs or has_s3, add gcs_installed and gcs_testable where installed means the deps are there, and testable means the env vars are there to support it.
BREAKING CHANGE: Pathy.exists() no longer enumerates buckets if given a path with no root.
 - Accidental removal
@justindujardin justindujardin merged commit c2a0586 into master Nov 15, 2022
@justindujardin justindujardin deleted the feat/python311 branch November 15, 2022 04:00
justindujardin added a commit that referenced this pull request Nov 15, 2022
I missed this bit while adding support for python 3.11 in #89
@github-actions
Copy link

🎉 This PR is included in version 0.7.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

justindujardin added a commit that referenced this pull request Nov 15, 2022
I missed this bit while adding support for python 3.11 in #89

* chore: keep codecov from reporting early 

Builds look failed until all the tasks are done. Codecov's site says that `require_ci_to_pass` is default yes. 🤷 

Add after_n_builds set to the number of build tasks we have.
github-actions bot pushed a commit that referenced this pull request Nov 15, 2022
## [0.7.1](v0.7.0...v0.7.1) (2022-11-15)

### Bug Fixes

* **pypi:** add classifiers for python 3.10 / 3.11 ([515cb5d](515cb5d)), closes [#89](#89)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant