Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix soundness issue with stream interruption + translation #2145

Merged
merged 10 commits into from
Nov 22, 2020

Conversation

mpilquist
Copy link
Member

@mpilquist mpilquist commented Nov 21, 2020

Sketch of a fix for #2143, though not sure how to resolve the outstanding issues here.

The main issue is Stream.getScope:

  def getScope[F[x] >: Pure[x]]: Stream[F, Scope[F]] =
    new Stream(Pull.getScope[F].flatMap(Pull.output1(_)))

This operation provides a Scope[F] which provides various operations that return F[X], like interrupt. During stream evaluation, a CompileScope[G] is created, where G is the target effect type. The problem here is that getScope is implemented by returning the current CompileScope[G] and when G != F, we get class cast exceptions (because interrupt returns a G[Unit] but caller thinks they have an F[Unit]).

The sketch of a solution in this PR introduces a new primitive, Interrupt, which invokes the interrupt operation on the current CompileScope. However, we have no mechanism to use this operation in the definition of interruptWhen, as doing so requires concurrent stream evaluation via concurrently or merge or parJoin but those all use interruptWhen internally.

@mpilquist
Copy link
Member Author

One option is adding a private unsafeInterrupt operation to CompileScope, but this would require stream compilation needing an Async instance instead of Concurrent.

@SystemFw
Copy link
Collaborator

but this would require stream compilation needing an Async instance instead of Concurrent.

that means we will be back to every concurrent stream combinator, and therefore pretty much the whole stack, requiring Async again :(

@mpilquist
Copy link
Member Author

I found a better fix, basically interruptWhen(condition: F[Either[Throwable, Unit]]) becomes a new primitive. I need to clean it up which I'll do later today.

@mpilquist
Copy link
Member Author

OK, I pushed a sketch of the better fix. I need to get it compiling on 2.12 and then do something similar for the other operations on Scope, then finally delete getScope entirely.

@mpilquist mpilquist marked this pull request as ready for review November 21, 2020 23:01
@mpilquist
Copy link
Member Author

Alright, ready for review. I made getScope private -- the existing uses are all sound but keeping it public is too error prone.

@mpilquist mpilquist changed the title Sketch of how to fix soundness issue with Scope.interrupt Fix soundness issue with stream interruption + translation Nov 21, 2020
@mpilquist mpilquist merged commit e03233e into develop Nov 22, 2020
mpilquist added a commit that referenced this pull request Nov 22, 2020
* Backport 2145 to main branch

* Scalafmt

* Fix compilation on 2.12
mpilquist added a commit that referenced this pull request Nov 24, 2020
* Backport 2145 to main branch

* Scalafmt

* Fix compilation on 2.12
@mpilquist mpilquist deleted the wip/interrupt-soundness branch February 13, 2021 13:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants