Skip to content
This repository has been archived by the owner on Feb 12, 2024. It is now read-only.

chore: upgrade repo to streaming api #3041

Merged
merged 7 commits into from
May 27, 2020

Conversation

achingbrain
Copy link
Member

The latest interface-datastore contains added streaming apis which will let us use fewer resources when importing large datasets.

This PR upgrades ipfs-repo and all datastore-* deps.

The latest interface-datastore contains added streaming apis which
will let us use fewer resources when importing large datasets.

This PR upgrades ipfs-repo and all datastore-* deps.
@achingbrain achingbrain requested a review from hugomrdias May 18, 2020 14:35
@achingbrain
Copy link
Member Author

Looks like we get a nice little speed increase between this and the new bitswap:

Old:

ipfs:     js -> js2
ipfs:       cat file
ipfs:         ✓ js -> js2: 1.02 kB (486ms)
ipfs:         ✓ js -> js2: 63.5 kB (449ms)
ipfs:         ✓ js -> js2: 65.5 kB (436ms)
ipfs:         ✓ js -> js2: 524 kB (1287ms)
ipfs:         ✓ js -> js2: 786 kB (1676ms)
ipfs:         ✓ js -> js2: 1.05 MB (2066ms)
ipfs:         ✓ js -> js2: 1.05 MB (2123ms)
ipfs:         ✓ js -> js2: 4.19 MB (7091ms)
ipfs:         ✓ js -> js2: 8.39 MB (13711ms)
ipfs:         ✓ js -> js2: 67.1 MB (107704ms)
ipfs:         ✓ js -> js2: 134 MB (214988ms)
ipfs:       get directory
ipfs:         ✓ js -> js2: depth: 5, num: 5 (5885ms)
ipfs:         ✓ js -> js2: depth: 5, num: 10 (13876ms)
ipfs:         ✓ js -> js2: depth: 10, num: 5 (16140ms)
ipfs:         ✓ js -> js2: depth: 10, num: 10 (26152ms)

New:

ipfs:     js -> js2
ipfs:       cat file
ipfs:         ✓ js -> js2: 1.02 kB (378ms) -28%
ipfs:         ✓ js -> js2: 63.5 kB (361ms) -24%
ipfs:         ✓ js -> js2: 65.5 kB (346ms) -26%
ipfs:         ✓ js -> js2: 524 kB (1041ms) -23%
ipfs:         ✓ js -> js2: 786 kB (1297ms) -29%
ipfs:         ✓ js -> js2: 1.05 MB (1626ms) -27%
ipfs:         ✓ js -> js2: 1.05 MB (1655ms) -28%
ipfs:         ✓ js -> js2: 4.19 MB (5499ms) -28%
ipfs:         ✓ js -> js2: 8.39 MB (10866ms) -26%
ipfs:         ✓ js -> js2: 67.1 MB (84231ms) -28%
ipfs:         ✓ js -> js2: 134 MB (169154ms) -27%
ipfs:       get directory
ipfs:         ✓ js -> js2: depth: 5, num: 5 (6913ms) +15%
ipfs:         ✓ js -> js2: depth: 5, num: 10 (9024ms) -53%
ipfs:         ✓ js -> js2: depth: 10, num: 5 (9835ms) -64%
ipfs:         ✓ js -> js2: depth: 10, num: 10 (12646ms) -106%

@vasco-santos
Copy link
Member

Thanks for looking into this @achingbrain

@achingbrain achingbrain merged commit 115948d into master May 27, 2020
@achingbrain achingbrain deleted the chore/upgrade-repo-to-streaming-api branch May 27, 2020 17:04
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants