deps!: update multiformats to 10.x.x and all @ipld/* modules (#265)

Also updates uint8arrays and all the it-* deps. BREAKING CHANGE: CIDs returned are instances from `[email protected]` and not `[email protected]`
ipfs · Oct 19, 2022 · 7a12568 · 7a12568
1 parent a5b9cad
commit 7a12568
Show file tree

Hide file tree

Showing 12 changed files with 88 additions and 75 deletions.
diff --git a/README.md b/README.md
@@ -1,8 +1,7 @@
 # js-ipfs-unixfs <!-- omit in toc -->
 
-[![ipfs.io](https://img.shields.io/badge/project-IPFS-blue.svg?style=flat-square)](http://ipfs.io)
-[![IRC](https://img.shields.io/badge/freenode-%23ipfs-blue.svg?style=flat-square)](http://webchat.freenode.net/?channels=%23ipfs)
-[![Discord](https://img.shields.io/discord/806902334369824788?style=flat-square)](https://discord.gg/ipfs)
+[![ipfs.tech](https://img.shields.io/badge/project-IPFS-blue.svg?style=flat-square)](https://ipfs.tech)
+[![Discuss](https://img.shields.io/discourse/https/discuss.ipfs.tech/posts.svg?style=flat-square)](https://discuss.ipfs.tech)
 [![codecov](https://img.shields.io/codecov/c/github/ipfs/js-ipfs-unixfs.svg?style=flat-square)](https://codecov.io/gh/ipfs/js-ipfs-unixfs)
 [![CI](https://img.shields.io/github/workflow/status/ipfs/js-ipfs-unixfs/test%20&%20maybe%20release/master?style=flat-square)](https://github.com/ipfs/js-ipfs-unixfs/actions/workflows/js-test-and-release.yml)
 
@@ -74,8 +73,12 @@ Licensed under either of
 
 ## Contribute
 
-Feel free to join in. All welcome. Open an [issue](https://github.com/ipfs/js-ipfs-unixfs-importer/issues)!
+Contributions welcome! Please check out [the issues](https://github.com/ipfs/js-ipfs-unixfs/issues).
 
-This repository falls under the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md).
+Also see our [contributing document](https://github.com/ipfs/community/blob/master/CONTRIBUTING_JS.md) for more information on how we work, and about contributing in general.
+
+Please be aware that all interactions related to this repo are subject to the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md).
+
+Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.
 
 [![](https://cdn.rawgit.com/jbenet/contribute-ipfs-gif/master/img/contribute.gif)](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md)
diff --git a/lerna.json b/lerna.json
@@ -1,10 +1,13 @@
 {
-  "lerna": "4.0.0",
+  "lerna": "5.4.0",
   "useWorkspaces": true,
   "version": "independent",
   "command": {
     "run": {
       "stream": true
     }
-  }
+  },
+  "packages": [
+    "packages/*"
+  ]
 }
diff --git a/package.json b/package.json
@@ -33,7 +33,7 @@
     "release": "lerna exec --concurrency 1 -- semantic-release -e semantic-release-monorepo"
   },
   "dependencies": {
-    "lerna": "^5.0.0"
+    "lerna": "^6.0.1"
   },
   "workspaces": [
     "packages/*"

diff --git a/packages/ipfs-unixfs-exporter/README.md b/packages/ipfs-unixfs-exporter/README.md
@@ -1,8 +1,7 @@
 # ipfs-unixfs-exporter <!-- omit in toc -->
 
-[![ipfs.io](https://img.shields.io/badge/project-IPFS-blue.svg?style=flat-square)](http://ipfs.io)
-[![IRC](https://img.shields.io/badge/freenode-%23ipfs-blue.svg?style=flat-square)](http://webchat.freenode.net/?channels=%23ipfs)
-[![Discord](https://img.shields.io/discord/806902334369824788?style=flat-square)](https://discord.gg/ipfs)
+[![ipfs.tech](https://img.shields.io/badge/project-IPFS-blue.svg?style=flat-square)](https://ipfs.tech)
+[![Discuss](https://img.shields.io/discourse/https/discuss.ipfs.tech/posts.svg?style=flat-square)](https://discuss.ipfs.tech)
 [![codecov](https://img.shields.io/codecov/c/github/ipfs/js-ipfs-unixfs.svg?style=flat-square)](https://codecov.io/gh/ipfs/js-ipfs-unixfs)
 [![CI](https://img.shields.io/github/workflow/status/ipfs/js-ipfs-unixfs/test%20&%20maybe%20release/master?style=flat-square)](https://github.com/ipfs/js-ipfs-unixfs/actions/workflows/js-test-and-release.yml)
 
@@ -236,12 +235,6 @@ for await (const child of recursive('Qmfoo/foo/bar', blockstore)) {
 // entries contains all children of the `Qmfoo/foo/bar` directory and it's children
 ```
 
-[dag API]: https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/DAG.md
-
-[blockstore]: https://github.com/ipfs/js-ipfs-interfaces/tree/master/packages/interface-blockstore#readme
-
-[UnixFS]: https://github.com/ipfs/specs/tree/master/unixfs
-
 ## Contribute
 
 Feel free to join in. All welcome. Open an [issue](https://github.com/ipfs/js-ipfs-unixfs-exporter/issues)!
@@ -259,8 +252,18 @@ Licensed under either of
 
 ## Contribute
 
-Feel free to join in. All welcome. Open an [issue](https://github.com/ipfs/js-ipfs-unixfs-importer/issues)!
+Contributions welcome! Please check out [the issues](https://github.com/ipfs/js-ipfs-unixfs/issues).
 
-This repository falls under the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md).
+Also see our [contributing document](https://github.com/ipfs/community/blob/master/CONTRIBUTING_JS.md) for more information on how we work, and about contributing in general.
+
+Please be aware that all interactions related to this repo are subject to the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md).
+
+Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.
 
 [![](https://cdn.rawgit.com/jbenet/contribute-ipfs-gif/master/img/contribute.gif)](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md)
+
+[dag API]: https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/DAG.md
+
+[blockstore]: https://github.com/ipfs/js-ipfs-interfaces/tree/master/packages/interface-blockstore#readme
+
+[UnixFS]: https://github.com/ipfs/specs/tree/master/unixfs
diff --git a/packages/ipfs-unixfs-exporter/package.json b/packages/ipfs-unixfs-exporter/package.json
@@ -151,21 +151,21 @@
     "release": "aegir release"
   },
   "dependencies": {
-    "@ipld/dag-cbor": "^7.0.2",
-    "@ipld/dag-pb": "^2.0.2",
-    "@multiformats/murmur3": "^1.0.3",
+    "@ipld/dag-cbor": "^8.0.0",
+    "@ipld/dag-pb": "^3.0.0",
+    "@multiformats/murmur3": "^2.0.0",
     "err-code": "^3.0.1",
     "hamt-sharding": "^3.0.0",
     "interface-blockstore": "^3.0.0",
     "ipfs-unixfs": "^7.0.0",
-    "it-last": "^1.0.5",
-    "it-parallel": "^2.0.1",
+    "it-last": "^2.0.0",
+    "it-map": "^2.0.0",
+    "it-parallel": "^3.0.0",
     "it-pipe": "^2.0.4",
     "it-pushable": "^3.1.0",
-    "it-map": "^1.0.6",
+    "multiformats": "^10.0.0",
     "p-queue": "^7.3.0",
-    "multiformats": "^9.4.2",
-    "uint8arrays": "^3.0.0"
+    "uint8arrays": "^4.0.2"
   },
   "devDependencies": {
     "@types/sinon": "^10.0.0",
@@ -175,9 +175,9 @@
     "crypto-browserify": "^3.12.0",
     "delay": "^5.0.0",
     "ipfs-unixfs-importer": "^10.0.0",
-    "it-all": "^1.0.5",
-    "it-buffer-stream": "^2.0.0",
-    "it-first": "^1.0.6",
+    "it-all": "^2.0.0",
+    "it-buffer-stream": "^3.0.0",
+    "it-first": "^2.0.0",
     "merge-options": "^3.0.4",
     "native-abort-controller": "^1.0.3",
     "sinon": "^14.0.0"

diff --git a/packages/ipfs-unixfs-importer/README.md b/packages/ipfs-unixfs-importer/README.md
@@ -1,8 +1,7 @@
 # ipfs-unixfs-importer <!-- omit in toc -->
 
-[![ipfs.io](https://img.shields.io/badge/project-IPFS-blue.svg?style=flat-square)](http://ipfs.io)
-[![IRC](https://img.shields.io/badge/freenode-%23ipfs-blue.svg?style=flat-square)](http://webchat.freenode.net/?channels=%23ipfs)
-[![Discord](https://img.shields.io/discord/806902334369824788?style=flat-square)](https://discord.gg/ipfs)
+[![ipfs.tech](https://img.shields.io/badge/project-IPFS-blue.svg?style=flat-square)](https://ipfs.tech)
+[![Discuss](https://img.shields.io/discourse/https/discuss.ipfs.tech/posts.svg?style=flat-square)](https://discuss.ipfs.tech)
 [![codecov](https://img.shields.io/codecov/c/github/ipfs/js-ipfs-unixfs.svg?style=flat-square)](https://codecov.io/gh/ipfs/js-ipfs-unixfs)
 [![CI](https://img.shields.io/github/workflow/status/ipfs/js-ipfs-unixfs/test%20&%20maybe%20release/master?style=flat-square)](https://github.com/ipfs/js-ipfs-unixfs/actions/workflows/js-test-and-release.yml)
 
@@ -167,14 +166,6 @@ Several aspects of the importer are overridable by specifying functions as part
   - This function should read `{ cid, path, unixfs, node }` entries from `source` and place them in a directory structure
   - It should yield an object with the properties `{ cid, path, unixfs, size }` where `cid` is a `CID`, `path` is a string, `unixfs` is a UnixFS entry and `size` is a `Number`.
 
-[blockstore]: https://github.com/ipfs/js-ipfs-interfaces/tree/master/packages/interface-blockstore#readme
-
-[UnixFS]: https://github.com/ipfs/specs/tree/master/unixfs
-
-[IPLD]: https://github.com/ipld/js-ipld
-
-[CID]: https://github.com/multiformats/js-cid
-
 ## Contribute
 
 Feel free to join in. All welcome. Open an [issue](https://github.com/ipfs/js-ipfs-unixfs-importer/issues)!
@@ -192,8 +183,20 @@ Licensed under either of
 
 ## Contribute
 
-Feel free to join in. All welcome. Open an [issue](https://github.com/ipfs/js-ipfs-unixfs-importer/issues)!
+Contributions welcome! Please check out [the issues](https://github.com/ipfs/js-ipfs-unixfs/issues).
 
-This repository falls under the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md).
+Also see our [contributing document](https://github.com/ipfs/community/blob/master/CONTRIBUTING_JS.md) for more information on how we work, and about contributing in general.
+
+Please be aware that all interactions related to this repo are subject to the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md).
+
+Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.
 
 [![](https://cdn.rawgit.com/jbenet/contribute-ipfs-gif/master/img/contribute.gif)](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md)
+
+[blockstore]: https://github.com/ipfs/js-ipfs-interfaces/tree/master/packages/interface-blockstore#readme
+
+[UnixFS]: https://github.com/ipfs/specs/tree/master/unixfs
+
+[IPLD]: https://github.com/ipld/js-ipld
+
+[CID]: https://github.com/multiformats/js-cid
diff --git a/packages/ipfs-unixfs-importer/package.json b/packages/ipfs-unixfs-importer/package.json
@@ -151,27 +151,27 @@
     "release": "aegir release"
   },
   "dependencies": {
-    "@ipld/dag-pb": "^2.0.2",
-    "@multiformats/murmur3": "^1.0.3",
-    "bl": "^5.0.0",
+    "@ipld/dag-pb": "^3.0.0",
+    "@multiformats/murmur3": "^2.0.0",
     "err-code": "^3.0.1",
     "hamt-sharding": "^3.0.0",
     "interface-blockstore": "^3.0.0",
     "ipfs-unixfs": "^7.0.0",
-    "it-all": "^1.0.5",
-    "it-batch": "^1.0.8",
-    "it-first": "^1.0.6",
-    "it-parallel-batch": "^1.0.9",
+    "it-all": "^2.0.0",
+    "it-batch": "^2.0.0",
+    "it-first": "^2.0.0",
+    "it-parallel-batch": "^2.0.0",
     "merge-options": "^3.0.4",
-    "multiformats": "^9.4.2",
+    "multiformats": "^10.0.0",
     "rabin-wasm": "^0.1.4",
-    "uint8arrays": "^3.0.0"
+    "uint8arraylist": "^2.3.3",
+    "uint8arrays": "^4.0.2"
   },
   "devDependencies": {
     "aegir": "^37.5.0",
     "assert": "^2.0.0",
     "blockstore-core": "^2.0.1",
-    "it-buffer-stream": "^2.0.0",
+    "it-buffer-stream": "^3.0.0",
     "wherearewe": "^2.0.1"
   },
   "browser": {

diff --git a/packages/ipfs-unixfs-importer/src/chunker/fixed-size.js b/packages/ipfs-unixfs-importer/src/chunker/fixed-size.js
@@ -1,32 +1,31 @@
-// @ts-ignore
-import BufferList from 'bl/BufferList.js'
+import { Uint8ArrayList } from 'uint8arraylist'
 
 /**
  * @type {import('../types').Chunker}
  */
 async function * fixedSizeChunker (source, options) {
-  let bl = new BufferList()
+  let list = new Uint8ArrayList()
   let currentLength = 0
   let emitted = false
   const maxChunkSize = options.maxChunkSize
 
   for await (const buffer of source) {
-    bl.append(buffer)
+    list.append(buffer)
 
     currentLength += buffer.length
 
     while (currentLength >= maxChunkSize) {
-      yield bl.slice(0, maxChunkSize)
+      yield list.slice(0, maxChunkSize)
       emitted = true
 
       // throw away consumed bytes
-      if (maxChunkSize === bl.length) {
-        bl = new BufferList()
+      if (maxChunkSize === list.length) {
+        list = new Uint8ArrayList()
         currentLength = 0
       } else {
-        const newBl = new BufferList()
-        newBl.append(bl.shallowSlice(maxChunkSize))
-        bl = newBl
+        const newBl = new Uint8ArrayList()
+        newBl.append(list.sublist(maxChunkSize))
+        list = newBl
 
         // update our offset
         currentLength -= maxChunkSize
@@ -36,7 +35,7 @@ async function * fixedSizeChunker (source, options) {
 
   if (!emitted || currentLength) {
     // return any remaining bytes or an empty buffer
-    yield bl.slice(0, currentLength)
+    yield list.subarray(0, currentLength)
   }
 }
 

diff --git a/packages/ipfs-unixfs-importer/src/chunker/rabin.js b/packages/ipfs-unixfs-importer/src/chunker/rabin.js
@@ -1,5 +1,4 @@
-// @ts-ignore
-import BufferList from 'bl/BufferList.js'
+import { Uint8ArrayList } from 'uint8arraylist'
 // @ts-ignore
 import { create } from 'rabin-wasm'
 import errcode from 'err-code'
@@ -65,7 +64,7 @@ export default rabinChunker
  */
 async function * rabin (source, options) {
   const r = await create(options.bits, options.min, options.max, options.window)
-  const buffers = new BufferList()
+  const buffers = new Uint8ArrayList()
 
   for await (const chunk of source) {
     buffers.append(chunk)
@@ -82,6 +81,6 @@ async function * rabin (source, options) {
   }
 
   if (buffers.length) {
-    yield buffers.slice(0)
+    yield buffers.subarray(0)
   }
 }
diff --git a/packages/ipfs-unixfs-importer/src/types.ts b/packages/ipfs-unixfs-importer/src/types.ts
@@ -1,5 +1,5 @@
 import type { UnixFS, Mtime } from 'ipfs-unixfs'
-import type { CID, CIDVersion } from 'multiformats/cid'
+import type { CID, Version as CIDVersion } from 'multiformats/cid'
 import type { MultihashHasher } from 'multiformats/hashes/interface'
 import type { BlockCodec } from 'multiformats/codecs/interface'
 import type { Blockstore } from 'interface-blockstore'

diff --git a/packages/ipfs-unixfs/README.md b/packages/ipfs-unixfs/README.md
@@ -1,8 +1,7 @@
 # ipfs-unixfs <!-- omit in toc -->
 
-[![ipfs.io](https://img.shields.io/badge/project-IPFS-blue.svg?style=flat-square)](http://ipfs.io)
-[![IRC](https://img.shields.io/badge/freenode-%23ipfs-blue.svg?style=flat-square)](http://webchat.freenode.net/?channels=%23ipfs)
-[![Discord](https://img.shields.io/discord/806902334369824788?style=flat-square)](https://discord.gg/ipfs)
+[![ipfs.tech](https://img.shields.io/badge/project-IPFS-blue.svg?style=flat-square)](https://ipfs.tech)
+[![Discuss](https://img.shields.io/discourse/https/discuss.ipfs.tech/posts.svg?style=flat-square)](https://discuss.ipfs.tech)
 [![codecov](https://img.shields.io/codecov/c/github/ipfs/js-ipfs-unixfs.svg?style=flat-square)](https://codecov.io/gh/ipfs/js-ipfs-unixfs)
 [![CI](https://img.shields.io/github/workflow/status/ipfs/js-ipfs-unixfs/test%20&%20maybe%20release/master?style=flat-square)](https://github.com/ipfs/js-ipfs-unixfs/actions/workflows/js-test-and-release.yml)
 
@@ -213,8 +212,12 @@ Licensed under either of
 
 ## Contribute
 
-Feel free to join in. All welcome. Open an [issue](https://github.com/ipfs/js-ipfs-unixfs-importer/issues)!
+Contributions welcome! Please check out [the issues](https://github.com/ipfs/js-ipfs-unixfs/issues).
 
-This repository falls under the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md).
+Also see our [contributing document](https://github.com/ipfs/community/blob/master/CONTRIBUTING_JS.md) for more information on how we work, and about contributing in general.
+
+Please be aware that all interactions related to this repo are subject to the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md).
+
+Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.
 
 [![](https://cdn.rawgit.com/jbenet/contribute-ipfs-gif/master/img/contribute.gif)](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md)
diff --git a/packages/ipfs-unixfs/package.json b/packages/ipfs-unixfs/package.json
@@ -163,7 +163,7 @@
   "devDependencies": {
     "aegir": "^37.5.0",
     "protobufjs-cli": "^1.0.0",
-    "uint8arrays": "^3.0.0"
+    "uint8arrays": "^4.0.2"
   },
   "browser": {
     "fs": false