Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Added a generic FileStream (still in active development!) #2654

Merged
merged 45 commits into from
Oct 2, 2024
Merged

refactor: Added a generic FileStream (still in active development!) #2654

Changes from 1 commit
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
a45d34d
Initial impl
edgarrmondragon Sep 6, 2024
20b8edc
Merge branch 'main' into 2648-feat-add-a-generic-filestream-interface
edgarrmondragon Sep 10, 2024
914520c
Implement a simple local file and directory
edgarrmondragon Sep 10, 2024
3ebea11
Fix types
edgarrmondragon Sep 10, 2024
bf66f7f
Fix order
edgarrmondragon Sep 10, 2024
bd5c138
Creation and modification time
edgarrmondragon Sep 10, 2024
e1ffcc0
Implement S3 file and directory
edgarrmondragon Sep 11, 2024
6c9a423
Implement filesystem root
edgarrmondragon Sep 11, 2024
fba831c
Read bytes
edgarrmondragon Sep 11, 2024
fca4f2d
Use a different caching approach
edgarrmondragon Sep 11, 2024
8ea6652
Deptry DEP001
edgarrmondragon Sep 11, 2024
79707cc
Change S3 list_contents implementation
edgarrmondragon Sep 11, 2024
939e544
Merge branch 'main' into 2648-feat-add-a-generic-filestream-interface
edgarrmondragon Sep 11, 2024
8881efa
Add boto3 as an extra dep
edgarrmondragon Sep 11, 2024
ce600f6
Simpler interface
edgarrmondragon Sep 12, 2024
97b27d7
Merge branch 'main' into 2648-feat-add-a-generic-filestream-interface
edgarrmondragon Sep 17, 2024
0fd556a
Use fsspec
edgarrmondragon Sep 17, 2024
5a24cad
Start abstracting away a `FileStream`
edgarrmondragon Sep 17, 2024
742524c
Move implementation upstream
edgarrmondragon Sep 18, 2024
83c6754
Use preferred fsspec abstraction
edgarrmondragon Sep 18, 2024
ecabf25
Use fsspec to list directory contents
edgarrmondragon Sep 18, 2024
b85739e
Test incremental extraction
edgarrmondragon Sep 18, 2024
33c8c04
Fix types
edgarrmondragon Sep 18, 2024
982b6d8
Clean up
edgarrmondragon Sep 18, 2024
3d08c62
Increase coverage
edgarrmondragon Sep 18, 2024
2aabcc4
Move tap logic upstream
edgarrmondragon Sep 18, 2024
8795b7a
Fix imports
edgarrmondragon Sep 18, 2024
e82da7b
Add no cover comment
edgarrmondragon Sep 18, 2024
f0b2707
Configure filesystem name
edgarrmondragon Sep 18, 2024
1b9f4ed
Update settings docs
edgarrmondragon Sep 18, 2024
8144fe2
CSV-specific settings
edgarrmondragon Sep 18, 2024
22b4fbe
Test custom delimiter
edgarrmondragon Sep 18, 2024
a01449a
Merge branch 'main' into 2648-feat-add-a-generic-filestream-interface
edgarrmondragon Sep 23, 2024
92f1522
Merge branch 'main' into 2648-feat-add-a-generic-filestream-interface
edgarrmondragon Sep 26, 2024
e5a4a4f
Merge branch 'main' into 2648-feat-add-a-generic-filestream-interface
edgarrmondragon Sep 26, 2024
95d8b82
Implement schema discovery
edgarrmondragon Sep 26, 2024
cbf69c9
Add pragma: no cover
edgarrmondragon Sep 26, 2024
9fd33fe
Move ReadMode
edgarrmondragon Sep 26, 2024
3b11ac4
Implement FTP and SFTP configurations
edgarrmondragon Sep 27, 2024
e20567f
Make deptry happy
edgarrmondragon Sep 27, 2024
4628468
Link to docs
edgarrmondragon Sep 27, 2024
47ba446
Merge branch 'main' into 2648-feat-add-a-generic-filestream-interface
edgarrmondragon Oct 2, 2024
d6c4cc9
Update pyproject.toml
edgarrmondragon Oct 2, 2024
6f2e30c
Merge branch 'main' into 2648-feat-add-a-generic-filestream-interface
edgarrmondragon Oct 2, 2024
e527cee
Implement primary keys
edgarrmondragon Oct 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merge branch 'main' into 2648-feat-add-a-generic-filestream-interface
  • Loading branch information
@edgarrmondragon
edgarrmondragon committed Oct 2, 2024
commit 6f2e30cf67aa5194936cec2faa81058ea001287c

This merge commit was added into this branch cleanly.

There are no new changes to show, but you can still view the diff.