Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(streams): replace transform streams with minipass #2

Merged
merged 2 commits into from
Sep 9, 2019

Conversation

claudiahdz
Copy link
Contributor

@claudiahdz claudiahdz commented Sep 6, 2019

This change replaces stream.Transform with minipass and refactors the integrityStream fn to call the new IntegrityStream class.

Copy link
Contributor

@isaacs isaacs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good start at a very targeted change that disrupts as little as possible of its surrounding code.

I think we can take the improvement a bit further, but it shouldn't block landing this, assuming tests pass.

I believe that this is a breaking change, since it may subtly change the timing of data processing, but should be low-risk to start incorporating into the CLI's dep tree.

As far as performance benchmarking, that'd be a good addition, but I don't think that anything is set up already. We may see only a tiny improvement in this library, but I suspect we'll see more of an impact in pacote and cacache, when many integrity streams are in use.

index.js Outdated
@@ -24,6 +24,19 @@ const SsriOpts = figgyPudding({
strict: {default: false}
})

class Transform extends MiniPass {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be a bit tighter.

Since we're only using the class here, you can pass in hashes in the options object, and do this.hashes = opts.hashes. Then, in the write method, this.hashes.forEach(h=>h.update(data)); return super.write(data).

Then the call below becomes const stream = new Transform({ hashes }).

Maybe the class name could be IntegrityStream, also.

In fact, while we're at it, the stream.on('end') bit below could be pulled in as well, and made more resilient against stream.removeAllListeners('end'), by overriding the emit method. Something like this:

  emit (ev, data) {
    if (ev !== 'end') {
      return super.emit(ev, data)
    }
    // all the integrity checking stuff
    super.emit('end')
  }

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I guess you'd have to also move most of the start of the function into the IntegrityStream as well, so that it has all the other data that the checking needs.

That's probably also a good idea :)

index.js Outdated
@@ -301,14 +314,11 @@ function integrityStream (opts) {
new Set(opts.algorithms.concat(algorithm ? [algorithm] : []))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the rest of the stuff in this method moves into the stream subclass, maybe this whole function is just:

function integrityStream (opts) {
  return new IntegrityStream(SsriOpts(opts))
}

index.js Outdated
this.emit('integrity', newSri)
match && this.emit('verified', match)
}
super.emit('end')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This raises an eyebrow for me. Won't this mean that end is emitted twice (or more?)

It'd be safer to override the emit function, I think, to ensure that it can't be removed.

emit (ev, data) {
  if (ev === 'end')
    this.onEnd()

  return super.emit(ev, data)
}

index.js Outdated
err.found = this.size
err.expected = this.opts.size
err.sri = this.sri
super.emit('error', err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can just be this.emit('error', err)

@claudiahdz claudiahdz force-pushed the claudiahdz/streams-refactor branch from 81107c8 to f43e8e7 Compare September 9, 2019 21:56
@claudiahdz claudiahdz changed the title chore(refactor-streams): replace transform streams with minipass chore(streams): replace transform streams with minipass Sep 9, 2019
@isaacs
Copy link
Contributor

isaacs commented Sep 9, 2019

💯

@claudiahdz claudiahdz merged commit 875fb3a into latest Sep 9, 2019
@claudiahdz claudiahdz deleted the claudiahdz/streams-refactor branch September 9, 2019 22:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants