feat: blake3 support

[laudiacay]

Accepting the relevant blake3 PRs on go-multihash, go-cid, and go-verifcid will make this one ready to go- it's just version bumps that make blake3 work on ipfs. all validation happens in called libraries, the ipfs code more or less doesn't care what hash function multihash is using as far as i can tell.

Closes #8650

[Jorropo]

pls run go mod tidy

[laudiacay]

Closes #8650.

prereqs:
PR 1: multiformats/go-multihash#158
PR 2: ipfs/go-cid#140
PR 3: ipfs/go-verifcid#15

[laudiacay]

pls run go mod tidy

won't run until the other version bump PRs are live- will do before merging.

[Jorropo]

won't run until the other version bump PRs are live- will do before merging.

Yes mb, what we do most of the time is use replace rules or go get repo@PRCommit (PRCommit is literally just the head commit of your PR) so we can test them together and see the green CI passing.

[lidel]

We need to be extra careful here

IIUC this PR introduces blake3 with implicit length of 256 bits (32 bytes), however it is very likely at some point in the future we will be asked to add support for longer versions.

This means the bar here is higher than it would be for existing functions: we not only need to guard the behavior at the implicit default length proposed in this PR, but also have a plan for supporting alternative lengths under the same name.

UX challenges

Custom length can be passed to block put via --mhlen. afaik it was intended for digest truncation in contexts where data comes from legacy third party systems, but in the context of blake3 could be used for producing longer digests.

Other commands like ipfs dag put and ipfs add do not have --hash-len. We need to decide if we want to add them as part of this PR, or leave that for future PR, when a need for longer blake3 is raised by our users. I think leaving it for the future is fine, but we should anchor the default behavior by adding sharness tests for dag put and add.

TODO

Things that needs to happen before we can merge this PR:

• decide if this PR will also add --hash-len to add and dag put commands (only for blake3)
• add sharness tests listed in Allow blake3 #8650 (comment)

[laudiacay]

@lidel some relevant discussion here: ipfs/go-verifcid#15 (comment)

also discussed with @aschmahmann in DM about this.

tl;dr: supporting multiple lengths in any manner except literally having a bunch of different multihash codes for them is going to require a major refactor of the multihash library and interface. i don't have time to implement that right now, it's gonna be really painful... but 32-bytes is standard and gets the work that's needed for my usecase (and probably for many others') done.

right now i have it throwing an error on trying to parse blake3 hashes that are not 32 bytes long, no way to create them without somehow subverting the multihash interface, and tests for that behavior in the verifcid library...

assuming I get tests passing... how do people feel about merging it like this and worrying about the interface whenever someone wants to go through and actually refactor multihash because they need something beyond 32 bytes?

[laudiacay]

i think we're also pending some sharness. adding now...

[Jorropo]

replaced #9187

[Jorropo]

@laudiacay thx for your patches, this has been merged in #9187 🎉