Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Roadmap #3

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Roadmap #3

wants to merge 3 commits into from

Conversation

warpfork
Copy link
Member

Drafted a roadmap with my current thoughts on what we could do with a bunch of ADLs. It's taking the idea of the code here now and running with it.

I think I see roughly 4 or 5 things that could be a discrete useful ADL, and by putting them all together, I think we'd have a really slick way to handle unixfsv1 data.

I think these are things that would make sense to co-locate all in one repo, but this PR is also a discussion-starter for that, I guess.

@warpfork
Copy link
Member Author

warpfork commented Apr 6, 2021

If I understand correctly, a couple of the directory read ADL checkboxes will also be done in the very near future already 👼

Comment on lines +9 to +15
- ADL to make files appear as `kind`==`bytes`, where interally layout=trickledag:
- [ ] Readable
- [ ] Writable
- [ ] Chunker as a callback/interface
- (may choose to implement this as mostly shared code with the other file ADL, where the internal layout is just a parameter; undecided)
- [ ] Upstream work to determine how `ipld.Node` and `ipld.NodeAssembler` should work for data in the size scale that requires streaming and seeking
- ADL to make files appear as `kind`==`bytes`, where interally layout=balanceddag:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where interally layout=trickledag
where interally layout=balanceddag

Hm... the wording confuses me: Is the above only applicable to the writer ( for obvious reasons ), or does this apply to the reader as well? That is - can an ADL be written which reads arbitrary arrangements of UnixFS nodes, as originally intended way way back?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably the writers are different and the readers end up being the same. But I didn't want to be prescriptive about how it's implemented. Easy enough to check two boxes at once if the implementer ends up nailing both goals with the same code.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Postel violation though. I want go-ipfs to continue being able reading dags created by something that's not go-ipfs.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand what in this wording gives the impression that that will somehow get harder as a result of this work, but if there is such a miswording, I'm open to fixing it.

@warpfork
Copy link
Member Author

I think this may need an updated to check more of the boxes that have now already become done, but I would otherwise like to merge this :) (I think think it may become something to link to in near-future project proposals, too.)

These are just my current thoughts on what we could do with a bunch of
ADLs, taking the idea of the code here now and running with it.
(It's possible that work can be done in the upstream before getting to
it here... but equally if not more likely that the work in this repo
should push the interface design, since that way we'll really know
what we need from it.)
Copy link
Collaborator

@hannahhoward hannahhoward left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@warpfork sorry I never followed up here but this looks LGTM though we should go back and update the check marks -- a lot of them are filled out now :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants