Config option for direct retrievals

[aschmahmann]

The latest version enables direct retrieval from peers by default. This code path does a number of things that enable more work and complexity such as:

• Delegated routing for provider and peer records
• Connecting to these peers (and if libp2p peers) managing the connections
• Filtering peers we can't talk to anyway
• Caching so we don't keep trying to find addresses for peers we know we can't dial
• etc.

While this is where we'd like to get to it'd be great to separately be able to debug the recursive-gateway vs direct retrieval cases and only switch on by default the features we're confident are going to work for most people (in this case recursive-gateways). This means this it'd be helpful to have an option in the config file to enable direct-retrievals that's currently off by default

[2color]

Yeah this would be useful.

Relevant code lines for this:

• Default verified fetch config https://github.com/ipfs/helia-verified-fetch/blob/main/packages/verified-fetch/src/index.ts#L792-L806
• Instantiation call https://github.com/ipfs/service-worker-gateway/blob/main/src/sw.ts#L251-L257

• Connecting to these peers (and if libp2p peers) managing the connections

One thing to note here is that only HTTP retrieval is enabled by default. Verified Fetch uses @helia/http by default. So while the delegated routing is called, it only acts on "transport-ipfs-gateway-http" responses.

[SgtPooki]

This was completed with #391 and some subsequent PRs