Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add fetcher config top level dependency #8081

Merged
merged 2 commits into from
Apr 21, 2021
Merged

Add fetcher config top level dependency #8081

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5077fd3
feat(node): add fetcher config top level dependency
hannahhoward Apr 20, 2021
63b6593
fix(deps): update to latest go-path
hannahhoward Apr 21, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 11 additions & 9 deletions core/coreapi/coreapi.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,15 @@ import (
"fmt"

bserv "github.com/ipfs/go-blockservice"
"github.com/ipfs/go-ipfs-blockstore"
"github.com/ipfs/go-ipfs-exchange-interface"
blockstore "github.com/ipfs/go-ipfs-blockstore"
exchange "github.com/ipfs/go-ipfs-exchange-interface"
offlinexch "github.com/ipfs/go-ipfs-exchange-offline"
"github.com/ipfs/go-ipfs-pinner"
"github.com/ipfs/go-ipfs-provider"
pin "github.com/ipfs/go-ipfs-pinner"
provider "github.com/ipfs/go-ipfs-provider"
offlineroute "github.com/ipfs/go-ipfs-routing/offline"
ipld "github.com/ipfs/go-ipld-format"
dag "github.com/ipfs/go-merkledag"
"github.com/ipfs/go-path/resolver"
coreiface "github.com/ipfs/interface-go-ipfs-core"
"github.com/ipfs/interface-go-ipfs-core/options"
ci "github.com/libp2p/go-libp2p-core/crypto"
Expand Down Expand Up @@ -54,9 +55,9 @@ type CoreAPI struct {
baseBlocks blockstore.Blockstore
pinning pin.Pinner

blocks bserv.BlockService
dag ipld.DAGService

blocks bserv.BlockService
dag ipld.DAGService
resolver *resolver.Resolver
peerstore pstore.Peerstore
peerHost p2phost.Host
recordValidator record.Validator
Expand Down Expand Up @@ -165,8 +166,9 @@ func (api *CoreAPI) WithOptions(opts ...options.ApiOption) (coreiface.CoreAPI, e
baseBlocks: n.BaseBlocks,
pinning: n.Pinning,

blocks: n.Blocks,
dag: n.DAG,
blocks: n.Blocks,
dag: n.DAG,
resolver: n.Resolver,

peerstore: n.Peerstore,
peerHost: n.PeerHost,
Expand Down
6 changes: 1 addition & 5 deletions core/coreapi/path.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,10 @@ import (
gopath "path"

"github.com/ipfs/go-ipfs/namesys/resolve"
"github.com/ipfs/go-unixfsnode"

"github.com/ipfs/go-cid"
ipld "github.com/ipfs/go-ipld-format"
ipfspath "github.com/ipfs/go-path"
"github.com/ipfs/go-path/resolver"
coreiface "github.com/ipfs/interface-go-ipfs-core"
path "github.com/ipfs/interface-go-ipfs-core/path"
)
Expand Down Expand Up @@ -53,9 +51,7 @@ func (api *CoreAPI) ResolvePath(ctx context.Context, p path.Path) (path.Resolved
return nil, fmt.Errorf("unsupported path namespace: %s", p.Namespace())
}

r := resolver.NewBasicResolver(api.blocks)
r.FetchConfig.NodeReifier = unixfsnode.Reify
node, rest, err := r.ResolveToLastNode(ctx, ipath)
node, rest, err := api.resolver.ResolveToLastNode(ctx, ipath)
if err != nil {
return nil, err
}
Expand Down
25 changes: 21 additions & 4 deletions core/node/core.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ import (
"github.com/ipfs/go-blockservice"
"github.com/ipfs/go-cid"
"github.com/ipfs/go-datastore"
"github.com/ipfs/go-fetcher"
bsfetcher "github.com/ipfs/go-fetcher/impl/blockservice"
"github.com/ipfs/go-filestore"
blockstore "github.com/ipfs/go-ipfs-blockstore"
exchange "github.com/ipfs/go-ipfs-exchange-interface"
Expand All @@ -21,6 +23,10 @@ import (
"github.com/ipfs/go-path/resolver"
"github.com/ipfs/go-unixfs"
"github.com/ipfs/go-unixfsnode"
dagpb "github.com/ipld/go-codec-dagpb"
"github.com/ipld/go-ipld-prime"
basicnode "github.com/ipld/go-ipld-prime/node/basic"
"github.com/ipld/go-ipld-prime/schema"
"github.com/libp2p/go-libp2p-core/host"
"github.com/libp2p/go-libp2p-core/routing"
"go.uber.org/fx"
Expand Down Expand Up @@ -84,11 +90,22 @@ func (s *syncDagService) Session(ctx context.Context) format.NodeGetter {
return merkledag.NewSession(ctx, s.DAGService)
}

// FetcherConfig returns a fetcher config that can build new fetcher instances
func FetcherConfig(bs blockservice.BlockService) fetcher.Factory {
Comment on lines +93 to +94
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hannahhoward @Stebalien do you think these lines of code for adding in UnixFS support are something that people are going to be missing?

Embedding go-ipfs as a library is not the friendliest at the moment so I'm wondering if people are expecting UnixFS support to live elsewhere and if this should live in a helper package somewhere.

Not a huge deal since we can wait until the requests come in to extract this code out, but if there are users who would be confused (or if we're going to feel the need to put this code in the release notes for go-path or go-merkledag) then we could put it somewhere else (e.g. a helper package in unixfsnode or go-merkledag)

Copy link
Contributor Author

@hannahhoward hannahhoward Apr 21, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it was in go-unixfsnode, people objected...

Ultimately I agree and I would rather it live here.

fc := bsfetcher.NewFetcherConfig(bs)
fc.NodeReifier = unixfsnode.Reify
fc.PrototypeChooser = dagpb.AddSupportToChooser(func(lnk ipld.Link, lnkCtx ipld.LinkContext) (ipld.NodePrototype, error) {
if tlnkNd, ok := lnkCtx.LinkNode.(schema.TypedLinkNode); ok {
return tlnkNd.LinkTargetNodePrototype(), nil
}
return basicnode.Prototype.Any, nil
})
return fc
}

// Resolver returns a resolver that's configured to look up unixfs paths
func Resolver(bs blockservice.BlockService) *resolver.Resolver {
rs := resolver.NewBasicResolver(bs)
rs.FetchConfig.NodeReifier = unixfsnode.Reify
return rs
func Resolver(fetcherFactory fetcher.Factory) *resolver.Resolver {
return resolver.NewBasicResolver(fetcherFactory)
}

// Dag creates new DAGService
Expand Down
1 change: 1 addition & 0 deletions core/node/groups.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,7 @@ func Offline(cfg *config.Config) fx.Option {
var Core = fx.Options(
fx.Provide(BlockService),
fx.Provide(Dag),
fx.Provide(FetcherConfig),
fx.Provide(Resolver),
fx.Provide(Pinning),
fx.Provide(Files),
Expand Down
2 changes: 1 addition & 1 deletion fuse/readonly/readonly_unix.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,13 @@ package readonly
import (
"context"
"fmt"
"github.com/ipfs/go-cid"
"io"
"os"
"syscall"

fuse "bazil.org/fuse"
fs "bazil.org/fuse/fs"
"github.com/ipfs/go-cid"
core "github.com/ipfs/go-ipfs/core"
ipld "github.com/ipfs/go-ipld-format"
logging "github.com/ipfs/go-log"
Expand Down
5 changes: 3 additions & 2 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ require (
github.com/ipfs/go-ds-flatfs v0.4.5
github.com/ipfs/go-ds-leveldb v0.4.2
github.com/ipfs/go-ds-measure v0.1.0
github.com/ipfs/go-fetcher v1.3.0
github.com/ipfs/go-filestore v0.0.3
github.com/ipfs/go-fs-lock v0.0.6
github.com/ipfs/go-graphsync v0.8.0
Expand All @@ -51,14 +52,14 @@ require (
github.com/ipfs/go-metrics-interface v0.0.1
github.com/ipfs/go-metrics-prometheus v0.0.2
github.com/ipfs/go-mfs v0.1.2
github.com/ipfs/go-path v0.0.10-0.20210405201800-40f1060226f7
github.com/ipfs/go-path v0.0.10-0.20210421213242-802a897dfcd8
github.com/ipfs/go-pinning-service-http-client v0.1.0
github.com/ipfs/go-unixfs v0.2.4
github.com/ipfs/go-unixfsnode v1.1.1
github.com/ipfs/go-verifcid v0.0.1
github.com/ipfs/interface-go-ipfs-core v0.4.1-0.20210326022702-98763dda3e52
github.com/ipld/go-car v0.2.1-0.20210312021557-7afab98d034f
github.com/ipld/go-codec-dagpb v1.2.1-0.20210405170603-d0b86f7623c2 // indirect
github.com/ipld/go-codec-dagpb v1.2.1-0.20210405170603-d0b86f7623c2
github.com/ipld/go-ipld-prime v0.9.1-0.20210402181957-7406578571d1
github.com/jbenet/go-is-domain v1.0.5
github.com/jbenet/go-random v0.0.0-20190219211222-123a90aedc0c
Expand Down
17 changes: 4 additions & 13 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -387,10 +387,9 @@ github.com/ipfs/go-ds-leveldb v0.4.2 h1:QmQoAJ9WkPMUfBLnu1sBVy0xWWlJPg0m4kRAiJL9
github.com/ipfs/go-ds-leveldb v0.4.2/go.mod h1:jpbku/YqBSsBc1qgME8BkWS4AxzF2cEu1Ii2r79Hh9s=
github.com/ipfs/go-ds-measure v0.1.0 h1:vE4TyY4aeLeVgnnPBC5QzKIjKrqzha0NCujTfgvVbVQ=
github.com/ipfs/go-ds-measure v0.1.0/go.mod h1:1nDiFrhLlwArTME1Ees2XaBOl49OoCgd2A3f8EchMSY=
github.com/ipfs/go-fetcher v1.1.0 h1:2tWDIPMLNkPCoW3VdFVaM9ZhvESoLYWOTyhyJI0hhwg=
github.com/ipfs/go-fetcher v1.1.0/go.mod h1:RHp10iwAdiJCsxqNvz6JMv1S7tajHSpC36oR6mJuF5M=
github.com/ipfs/go-fetcher v1.2.0 h1:2eRF8JQCkCwsXn8c0VmGrl9ksCLG1Ll36WSId7Ld30Q=
github.com/ipfs/go-fetcher v1.2.0/go.mod h1:RFvn2LiuWIfyXt3ChONqIvQfsbEhBYWhwUeatYjFPIA=
github.com/ipfs/go-fetcher v1.3.0 h1:XlF4GCg9LkdAfQk3Kdd3JD0dZ5TgqfydJ7+gog/Eotc=
github.com/ipfs/go-fetcher v1.3.0/go.mod h1:lld7kBIARmpCvhQ/Rob5oOGEKfVjil8L8y4j9jtLdqo=
github.com/ipfs/go-filestore v0.0.3 h1:MhZ1jT5K3NewZwim6rS/akcJLm1xM+r6nz6foeB9EwE=
github.com/ipfs/go-filestore v0.0.3/go.mod h1:dvXRykFzyyXN2CdNlRGzDAkXMDPyI+D7JE066SiKLSE=
github.com/ipfs/go-fs-lock v0.0.6 h1:sn3TWwNVQqSeNjlWy6zQ1uUGAZrV3hPOyEA6y1/N2a0=
Expand Down Expand Up @@ -482,10 +481,9 @@ github.com/ipfs/go-metrics-prometheus v0.0.2/go.mod h1:ELLU99AQQNi+zX6GCGm2lAgnz
github.com/ipfs/go-mfs v0.1.2 h1:DlelNSmH+yz/Riy0RjPKlooPg0KML4lXGdLw7uZkfAg=
github.com/ipfs/go-mfs v0.1.2/go.mod h1:T1QBiZPEpkPLzDqEJLNnbK55BVKVlNi2a+gVm4diFo0=
github.com/ipfs/go-path v0.0.7/go.mod h1:6KTKmeRnBXgqrTvzFrPV3CamxcgvXX/4z79tfAd2Sno=
github.com/ipfs/go-path v0.0.10-0.20210324191207-6a600cd3f256 h1:0Tyb3Na34eTu5GpHDFit6f4296U7qRoG/0j1/XQYHmY=
github.com/ipfs/go-path v0.0.10-0.20210324191207-6a600cd3f256/go.mod h1:QHYtDmpHe3xD5RQSkoco+BM7Vv0sg/tLHWv000t60sw=
github.com/ipfs/go-path v0.0.10-0.20210405201800-40f1060226f7 h1:Sh4fY6j1NxxwbN7ux9gzKC8ve85Jhs5TwxZMJOuLXCo=
github.com/ipfs/go-path v0.0.10-0.20210405201800-40f1060226f7/go.mod h1:g8egwymo/MjfPUu/ojghk4YQKwjn6MGO+UUv4eUw08M=
github.com/ipfs/go-path v0.0.10-0.20210421213242-802a897dfcd8 h1:YT89FH52ynHRCZJIM95z+/4fz0rQ7nFPFLRufOzm9AE=
github.com/ipfs/go-path v0.0.10-0.20210421213242-802a897dfcd8/go.mod h1:YMCaFyukoM1bSaPm1eL9Q6IPr0z4WytNeVKlBSm8a0c=
github.com/ipfs/go-peertaskqueue v0.0.4/go.mod h1:03H8fhyeMfKNFWqzYEVyMbcPUeYrqP1MX6Kd+aN+rMQ=
github.com/ipfs/go-peertaskqueue v0.1.0/go.mod h1:Jmk3IyCcfl1W3jTW3YpghSwSEC6IJ3Vzz/jUmWw8Z0U=
github.com/ipfs/go-peertaskqueue v0.1.1/go.mod h1:Jmk3IyCcfl1W3jTW3YpghSwSEC6IJ3Vzz/jUmWw8Z0U=
Expand All @@ -496,14 +494,11 @@ github.com/ipfs/go-pinning-service-http-client v0.1.0/go.mod h1:tcCKmlkWWH9JUUkK
github.com/ipfs/go-unixfs v0.1.0/go.mod h1:lysk5ELhOso8+Fed9U1QTGey2ocsfaZ18h0NCO2Fj9s=
github.com/ipfs/go-unixfs v0.2.4 h1:6NwppOXefWIyysZ4LR/qUBPvXd5//8J3jiMdvpbw6Lo=
github.com/ipfs/go-unixfs v0.2.4/go.mod h1:SUdisfUjNoSDzzhGVxvCL9QO/nKdwXdr+gbMUdqcbYw=
github.com/ipfs/go-unixfsnode v1.0.0 h1:H9ZFhw3Zl0w0m4RbGdRsJc3I1DXUkiJCDjkVqZSmj3c=
github.com/ipfs/go-unixfsnode v1.0.0/go.mod h1:xmhbd5wPvHYzJAGVJFJ4Oi+U5NGEiax0TatZLOq2imk=
github.com/ipfs/go-unixfsnode v1.0.1-0.20210402214142-de45652f269f/go.mod h1:Xk4qvd3Nb8H31OJapZrLDAE4aBNmKUCvAzypIUZ3ACQ=
github.com/ipfs/go-unixfsnode v1.1.1 h1:YjYe5nQgkloOVlyRUumVGke7ngRFRZmtbI866YkTUgM=
github.com/ipfs/go-unixfsnode v1.1.1/go.mod h1:Xk4qvd3Nb8H31OJapZrLDAE4aBNmKUCvAzypIUZ3ACQ=
github.com/ipfs/go-verifcid v0.0.1 h1:m2HI7zIuR5TFyQ1b79Da5N9dnnCP1vcu2QqawmWlK2E=
github.com/ipfs/go-verifcid v0.0.1/go.mod h1:5Hrva5KBeIog4A+UpqlaIU+DEstipcJYQQZc0g37pY0=
github.com/ipfs/interface-go-ipfs-core v0.4.0 h1:+mUiamyHIwedqP8ZgbCIwpy40oX7QcXUbo4CZOeJVJg=
github.com/ipfs/interface-go-ipfs-core v0.4.0/go.mod h1:UJBcU6iNennuI05amq3FQ7g0JHUkibHFAfhfUIy927o=
github.com/ipfs/interface-go-ipfs-core v0.4.1-0.20210326022702-98763dda3e52 h1:NKvFg6nPuEhMMyoOUswEReIr3PWBNSgwNT6mDPLAcTo=
github.com/ipfs/interface-go-ipfs-core v0.4.1-0.20210326022702-98763dda3e52/go.mod h1:kSN12HNPXg8TrQfdyQOPbPIQVk5j8PcHTn2AAKCaZ6Q=
Expand All @@ -512,17 +507,13 @@ github.com/ipld/go-car v0.2.1-0.20210312021557-7afab98d034f/go.mod h1:kHunAcD305
github.com/ipld/go-codec-dagpb v1.0.2-0.20210308154810-d05d02fa186e/go.mod h1:oYexiw3WkBIVD5UTNkVuOd0iyEcLxqytAQa90F3nH9M=
github.com/ipld/go-codec-dagpb v1.1.0/go.mod h1:6nBN7X7h8EOsEejZGqC7tej5drsdBAXbMHyBT+Fne5s=
github.com/ipld/go-codec-dagpb v1.2.0/go.mod h1:6nBN7X7h8EOsEejZGqC7tej5drsdBAXbMHyBT+Fne5s=
github.com/ipld/go-codec-dagpb v1.2.1-0.20210330082435-8ec6b0fbad18 h1:TpjpdzJdasjzZ2xw7rmoj4+u9WBkWBTKBYGcyyLXX68=
github.com/ipld/go-codec-dagpb v1.2.1-0.20210330082435-8ec6b0fbad18/go.mod h1:GMLfso6KSkYJlIbd2cGKdGMe/hM5/IukeXRQ+u6zTrQ=
github.com/ipld/go-codec-dagpb v1.2.1-0.20210405170603-d0b86f7623c2 h1:m/ZZEoOdswHrrcikTC+fX4x6tnevJs0hoyNzijlT41A=
github.com/ipld/go-codec-dagpb v1.2.1-0.20210405170603-d0b86f7623c2/go.mod h1:GMLfso6KSkYJlIbd2cGKdGMe/hM5/IukeXRQ+u6zTrQ=
github.com/ipld/go-ipld-prime v0.7.1-0.20210225173718-8fef5312eb12/go.mod h1:0xEgdD6MKbZ1vF0GC+YcR/C4SQCAlRuOjIJ2i0HxqzM=
github.com/ipld/go-ipld-prime v0.7.1-0.20210312004928-8a500e6b8a62/go.mod h1:ZwznT3awHhuBoB0SgGxSo8SZEg6XbQtsZ6WahL9r9z0=
github.com/ipld/go-ipld-prime v0.9.0 h1:N2OjJMb+fhyFPwPnVvJcWU/NsumP8etal+d2v3G4eww=
github.com/ipld/go-ipld-prime v0.9.0/go.mod h1:KvBLMr4PX1gWptgkzRjVZCrLmSGcZCb/jioOQwCqZN8=
github.com/ipld/go-ipld-prime v0.9.0/go.mod h1:KvBLMr4PX1gWptgkzRjVZCrLmSGcZCb/jioOQwCqZN8=
github.com/ipld/go-ipld-prime v0.9.1-0.20210324083106-dc342a9917db h1:kFwGn8rXa/Z31ev1OFNQsYeNKNCdifnTPl/NvPy5L38=
github.com/ipld/go-ipld-prime v0.9.1-0.20210324083106-dc342a9917db h1:kFwGn8rXa/Z31ev1OFNQsYeNKNCdifnTPl/NvPy5L38=
github.com/ipld/go-ipld-prime v0.9.1-0.20210324083106-dc342a9917db/go.mod h1:KvBLMr4PX1gWptgkzRjVZCrLmSGcZCb/jioOQwCqZN8=
github.com/ipld/go-ipld-prime v0.9.1-0.20210324083106-dc342a9917db/go.mod h1:KvBLMr4PX1gWptgkzRjVZCrLmSGcZCb/jioOQwCqZN8=
github.com/ipld/go-ipld-prime v0.9.1-0.20210402181957-7406578571d1 h1:dIKSj9r+CCXz9t6p4TfbDx34CfSjLfasRZf37SXlNjg=
Expand Down