Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Idiomatic Scala Support #336

Closed
benjchristensen opened this issue Sep 2, 2013 · 96 comments
Closed

Idiomatic Scala Support #336

benjchristensen opened this issue Sep 2, 2013 · 96 comments

Comments

@benjchristensen
Copy link
Member

As of version 0.11.0 Scala support is provided through the use of implicits. Conversations on Twitter are bringing up other possible improvements. Let's use this issue to discuss.

@jmhofer
Copy link
Contributor

jmhofer commented Sep 2, 2013

A few comments from my point of view (since I first looked at this project, I always wanted to use it from Scala):

Now that the core is typesafe, imho we have the first workable Scala integration (via implicits). From here on, everthing depends on how much effort we want to put into this vs how convenient it will be for the users.

What we already have is that the most important method names match (map, flatMap, filter). Matt also made Scala for-comprehensions work with observables.

There's a lot of room for improvements, though. It seems that due to Scala-Java interop, there are problems with type inference (I even ran into a Scala compiler crash somewhere), and sometimes asJava collection conversions are necessary. Also, Scala/Akka futures are not yet addressed at all. An Akka-specific scheduler would probably be awesome to have, too.

For full Scala user convenience, we'd probably have to completely wrap rx.Observable into its own rx.scala.Observable. This would mean a lot more maintenance effort for the Scala integration, of course (I don't think we can do this automatically via code generation or macros, but I'm no expert of all that). But still, I think it's well worth it to create a dedicated Scala API.

@xeno-by
Copy link

xeno-by commented Sep 2, 2013

  1. Implicit-based integration with Java-based observables has a serious flaw as it doesn't support parameter type inference for lambdas. The situation isn't going to change until https://issues.scala-lang.org/browse/SI-6221 is fixed. It's definitely fixed in 2.11, and at EPFL we have a compiler plugin that backports the fix to 2.10. Probably Typesafe folks can be convinced to do the backport in one of 2.10.x releases (/cc @adriaanm @gkossakowski @retronym @JamesIry).

  2. Some features that might be desirable in rx.scala.Observable: a) native function types, b) native collections, c) method names familiar to folks that use native collections, d) making parameter of defer by-name, e) creation of Observables via apply rather than via from.

  3. I also don't think that macros can help with rx.scala.Observable, because what you're probably after isn't just a copy/paste of a Java API, but rather a redesign that takes Scala features into account. But if there are some things that can be autogenerated, I'll be happy to answer questions about how macros work and how they can be used.

@aloiscochard
Copy link

IMHO, after looking at the code, using implicits convertions is not a good idea.

I think you have better to start with a complete wrapper of the API, then you'll be able to apply scala idiom (basically writing a DSL) instead of having a Java API with some facilities for converting type.

And that way you don't have to wait for https://issues.scala-lang.org/browse/SI-6221 ... which sounds anyway like a clumsy integration of Java API to me.

@benjchristensen
Copy link
Member Author

Here is some background on why the current design was chosen rather than having each language with a separate version of Observable:


The approach of having language specific packages/classes was pursued but did not work well because Rx is a composable library. It means that every time an Observable is used it needs to be re-wrapped or un-wrapped by whichever language is using it.

For example ...

From Java a library is exposed that has a method like this:

rx.Observable getData()

From Groovy a library is exposed with a method like:

rx.groovy.GroovyObservable getOtherData()

Then from Scala you need to wrap them again:

rx.scala.ScalaObservable.from(getOtherData())

This means we have an rx.Observable wrapped as rx.groovy.GroovyObservable wrapped as rx.scala.ScalaObservable.

To compose the two we would have:

rx.scala.ScalaObservable.zip(rx.scala.ScalaObservable.from(
      getOtherData()), 
      rx.scala.ScalaObservable.from(getData()),
       ... scala closure here ...);

Now what does ScalaObservable return from its operators? ScalaObservable or Observable?

Should the above zip operator return rx.scala.ScalaObservable or rx.Observable? What happens if this library is consumed from another language?

If Observable each step along the way it must be wrapped yet again. If ScalaObservable it has now changed all of the return types of rx.Observable to a subtype.

In short, for interop between languages it very quickly becomes a mess and our primary polyglot goal was that rx.Observable was usable across all libraries as the single type and because the whole point of Rx is chained composition it's not as simple as just a single decoration at the beginning. It affects every single method in an API and step of the chaining.

For this reason we chose the current language-adaptor model so rx.Observable can remain the sole public interface across languages.

@xeno-by
Copy link

xeno-by commented Sep 2, 2013

Thanks for providing the design notes!

I wonder though how often people tend to mix languages in their projects. Usability improvements in switching from a Java-based least common denominator to an idiomatic API might be significant. Does being polyglot overweigh these improvements?

@jmhofer
Copy link
Contributor

jmhofer commented Sep 2, 2013

I'm not sure if wrapping Observable for Scala (not in a subclass though) would really hurt so much. Imho it's worth a try to find out how it feels. There will be a lot of wrapping and unwrapping behind the scene though, I'm afraid.

@benjchristensen
Copy link
Member Author

At Netflix we are using Clojure, Groovy, Java and Scala and I know of apps running code from at least 3 of those 4 in the same JVM instance. I imagine it's not common in most environment for this type of diversity, but it is something we have wanted to support as seamlessly as possible. This is because we have wanted the rx.Observable to act as the interface we can expose across module boundaries as it naturally ends up at the edge of the API in methods such as Observable<T> getDataFromService(args). This is part of what drove us to target the JVM with RxJava and not any specific language.

That said, an idiomatic solution that works best for pure Scala apps is more important. If we can find a solution that can retain the use of rx.Observable without a completely separate wrapper then great, otherwise let's have a ScalaObservable for pure Scala apps and a way of going back and forth across language boundaries when it's needed. Perhaps the implicits that exist right now can solve the immediate interop needs for easy interaction from Scala to Java, but the option to convert to ScalaObservable would also be there.

Another piece of information to guide this ... the Rx.Net version in C# is defined by simple interfaces for Observer and Observable without any of the operator interfaces on them. All of the operator methods (static and instance level) are added via extension methods. It's quite clean and makes the interfaces more flexible (anyone can easily implement them and the extension methods are 'just there' without inheritance involved).

The reason RxJava has Observable as a concrete class is because extension methods don't exist in Java so we don't have a choice but to have them as concrete methods to enable the fluent chaining pattern.

In Scala however we do have extension methods (implicits), macros etc that theoretically can allow the rx.Observable to be made into whatever it needs to be for idiomatic Scala usage, similar to how C# implements Rx.Net using extension methods. I imagine a possible issue is if existing methods on rx.Observable are in the way and cause problems in achieving idiomatic Scala functionality, if that's the case I'd like to understand what those issues are and if they can be resolved and if not if they really are deal breakers.

At this point I become not so helpful as I am not skilled enough in Scala to have a valid opinion or guide the conversation much further. I do have some questions though for the Scala experts here:

  1. What would make rx.Observable idiomatic for Scala?

I'd like to see unit tests or sample code demonstrating expected behavior and usage so that we're all working towards the same goal and know when we've achieved success.

  1. What about the current rx.Observable + Scala implicits is not working well?

  2. What requirements of (1) can not be achieved with implicits and macros and requires a separate concrete class as a wrapper?

Thank you everyone for your involvement, I really do want us to find the ideal solution for Scala and then as a secondary priority make it work well across the JVM for polyglot applications.

@xeno-by
Copy link

xeno-by commented Sep 3, 2013

Thank you for your swift feedback! It's a pleasure to help.

  1. I'm not an Rx expert, and in fact this is the first time I see any Rx code apart from random snippets in web articles, so please don't treat this list as definitive. Comprehensive analysis would require quite some time, and I'm not sure I have it right now.

a) Observable.from could be replaced by Observable.apply, so that one can write Observable(1, 2, 3) instead of Observable.from(1, 2, 3).

b) It shouldn't be necessary to write asJava, e.g. as in https://github.com/Netflix/RxJava/blob/master/language-adaptors/rxjava-scala/src/main/scala/rx/lang/scala/RxImplicits.scala#L349, and asScala (not sure whether it's necessary now).

c) Use of mutable collections in situations like https://github.com/Netflix/RxJava/blob/master/language-adaptors/rxjava-scala/src/main/scala/rx/lang/scala/RxImplicits.scala#L350 is really inconsistent with Scala's pursuit of immutability. I wonder whether it'd be possible to write such code in a functional way.

d) Instead of taking a no-arg function, Observable.defer could take a by-name parameter, which would obviate the need of creating an explicit closure: https://github.com/Netflix/RxJava/blob/master/language-adaptors/rxjava-scala/src/main/scala/rx/lang/scala/RxImplicits.scala#L232.

e) For a former C# developer like me, method names like lastOrDefault sound familiar, but for the Scala croud out there - not so much. How about changing the names and maybe even signatures, so that they mirror method names of native collections? E.g. lastOrDefault could become lastOption (also note the change from default values to options).

f) I'm also not sure whether extension methods are even necessary. From what I would guess, in C# they are forced to use them, because interfaces can't define methods with implementations (at least, that was the story with IEnumerable and Enumerable). In Scala, we can do that, so why not just put all the combinators in Observable?

g) mapMany in https://github.com/Netflix/RxJava/blob/master/language-adaptors/rxjava-scala/src/main/scala/rx/lang/scala/RxImplicits.scala#L351 is essentially flatMap, right?

h) No idea how this name could be stated more succinctly, but toBlockingObservable feels a bit verbose. When I moved from C# to Scala, in people's code I felt an overall tendency to compress everything, including names. This is kind of a vague observation, so feel free to ignore it.

  1. Function literals don't support inference for parameter types unless they are used in a context that requires a Scala native function type. In particular, inference won't work if function literals are supposed to be converted to something implicitly.

E.g. in https://github.com/Netflix/RxJava/blob/master/language-adaptors/rxjava-scala/src/main/scala/rx/lang/scala/RxImplicits.scala#L341 one can't drop type annotations for lambda parameters. I tried removing them and got a compilation error. (Btw why are you using 2.10.1, when 2.10.2 is already available for quite long? Did you have any problems with it?)

That's an instance of https://issues.scala-lang.org/browse/SI-6221, which has been fixed in 2.11 and probably could be backported to 2.10. If you're in need of an immediate solution, we at EPFL have a compiler plugin for 2.10 that provides a backport.

@vjovanov
Copy link

vjovanov commented Sep 3, 2013

Excellent observations about the API @xeno-by. Currently, we are preparing exercises for the Coursera class Principles of Reactive Programming. For this course we would like to make an idiomatic Scala API. @samuelgruetter is working on that.

The cross language compatibility pointed out by @benjchristensen is a reasonable concern. However, I think there is a lot of monolingual projects out there that would greatly benefit from the idiomatic Scala library. It seems to me that adding a Scala wrapper can be only a plus in those cases.

For polyglot projects we could also take the Java rx.Observable as a common ground (being a return type of each operation and in API signatures). Then all of the Scala support can be added by an implicit conversion (rx.Observable => rx.ScalaObservable).

As a long-shot maybe the interface for multi and mono lingual projects can be unified so that we do not repeat our selves. IMHO it is worth a try.

@jmhofer
Copy link
Contributor

jmhofer commented Sep 3, 2013

Thanks for all the observations! I'll try to address some of the ones from @xeno-by.

1a), 1e) and 1g) are just aliasing problems, imho. We already have flatMap as alias for mapMany, and the other aliases could easily be added, either directly in rx.Observable, or in a Scala wrapper, if the namespace gets too big otherwise (not my preferred solution). Methods returning options will have to live in a Scala wrapper, however.

I didn't take a look at 1b) and collection conversions yet. This really shouldn't be necessary often, if at all.

1c) You're looking at test code here which tests the callback "at the end of the world". This is not really normal usage of the API. The whole RxJava codebase is refreshingly (though not completely) free of mutability, if you ask me. Also, the Rx API is actually there to allow you to handle all your events in a functional, immutable way. - In short, I don't see a problem here.

1d) is Scala-specific. By-name parameters are a great feature for methods like defer, I agree. We should make use of them in the Scala wrapper.

I'm not worried about 1f) either.

1h) I like the verbose name in this specific case. Imho, it's the same as asInstanceOf in Scala: Maybe I'm evil, but I like to punish people who do things they probably shouldn't do. :) Ok, there will of course be valid use cases for blocking observables, but there shouldn't be too many of them, hopefully.

  1. is quite a big problem, if you ask me. A backport of the fix to Scala 2.10 would be great. We can probably get around this with a dedicated Scala wrapper, though.

And the reason for 2.10.1 instead of 2.10.2 is currently just a technical problem with the build system. I, too, hope that this will be fixed soon.

@abersnaze
Copy link
Contributor

1c) Even that can be made functional if you replace the subscribe() with .toList().toBlockingObservable().single();

@jmhofer
Copy link
Contributor

jmhofer commented Sep 3, 2013

@abersnaze You're right, good point.

I started a little experiment with a more idiomatic Scala wrapper. It's just humble beginnings currently, though. You can find it here.

Comments and forks are welcome, of course. I started from the super-extends pull request in order to check how this whole thing feels in Scala.

@benjchristensen
Copy link
Member Author

Scala 2.10.2 support was added in version 0.11.2 (https://github.com/Netflix/RxJava/releases/tag/0.11.2).

@adriaanm
Copy link

adriaanm commented Sep 3, 2013

  1. Implicit-based integration with Java-based observables has a serious flaw as it doesn't support parameter type inference for lambdas. The situation isn't going to change until https://issues.scala-lang.org/browse/SI-6221 is fixed. It's definitely fixed in 2.11, and at EPFL we have a compiler plugin that backports the fix to 2.10. Probably Typesafe folks can be convinced to do the backport in one of 2.10.x releases (/cc @adriaanm @gkossakowski @retronym @JamesIry).

Sorry, we don't change type inference in minor versions unless there's a critical bug.
The risk of regression in source compatibility is too high.

@benjchristensen
Copy link
Member Author

The cross language compatibility pointed out by @benjchristensen is a reasonable concern. However, I think there is a lot of monolingual projects out there that would greatly benefit from the idiomatic Scala library. It seems to me that adding a Scala wrapper can be only a plus in those cases.

I completely agree if a wrapper is the only way to achieve idiomatic support. Monolingual projects should take first priority as that is the common case.

@samuelgruetter
Copy link
Contributor

We believe that the following solution would be best:

  • Implicit value class (let's call it ScalaObservable for the moment) wrapping an rx.Observable
  • all return types are rx.Observable
  • Since ScalaObservable is a value class, there is no runtime overhead for the wrapping (some explanations can be found in the second half of this blog post). This is new in Scala 2.10, and actually allows us to have extension methods similar to C#.
  • An object FunctionConversions containing all implicit conversions from Scala functions to Rx functions (the same as in the present Scala adaptor). These conversions are only used by the adapter, users of Scala Rx won't use them.
  • All instance methods of rx.Observable which take a Func or an Action have a wrapper method in ScalaObservable whose signature has Scala functions and whose body uses the conversions from FunctionConversions
  • ScalaObservable has map, flatMap, filter, like in the present adaptor to enable for comprehensions
  • ScalaObservable has lastOption method instead of lastOrDefault etc
  • functions passed by Rx Scala users to Observable needn't be implicitly converted, so we're not affected by the bug that parameter type inference of implicitly converted functions does not work in Scala 2.10
  • An object Observable with apply methods instead of from and create
  • polyglot projects can also use this approach

I've started to implement such an adapter and will post some code soon.

Note that I've not (yet) addressed the following points:

  • compatibility with Scala/Akka futures
  • how to integrate ScalaObservable with Scala collections to get all methods of Scala's Traversable

@jmhofer
Copy link
Contributor

jmhofer commented Sep 4, 2013

@samuelgruetter Sounds awesome! Very much looking forward to your sharing the code.

@samuelgruetter
Copy link
Contributor

You can see what I'm doing here. Note that it's still very incomplete and not yet tested, but work is in progress.

@jmhofer
Copy link
Contributor

jmhofer commented Sep 4, 2013

Great, thanks! I'll play around with it, too.

@mattrjacobs
Copy link
Contributor

@samuelgruetter All of that sounds great. I'll take a look next week and offer feedback.

@samuelgruetter
Copy link
Contributor

There's an interesting problem with map: If I add an explicit conversion like ScalaObservable(numbers).map(...), it works, but without, i.e. numbers.map(...) gives an error... I'll investigate on this. You can see the corresponding test here

@jmhofer
Copy link
Contributor

jmhofer commented Sep 4, 2013

Maybe there's a 2nd implicit somewhere, adding map?

@aloiscochard
Copy link

@samuelgruetter what is the error? if @jmhofer is right you should have an amibgous implicit resolution error.

@benjchristensen
Copy link
Member Author

If anything about the core Java library is causing issues let me know.

@samuelgruetter
Copy link
Contributor

There's no 2nd implicit, but there's a second map: The map in rx.Observable. And scalac wants to use this one and does not convert to ScalaObservable. However, with reduce, it works...

@jmhofer
Copy link
Contributor

jmhofer commented Sep 4, 2013

Just another guess, but then it's maybe due to the type parameter of map (your reduce doesn't have one; you renamed the one with type parameter to fold which luckily avoids the problem).

@benjchristensen
Copy link
Member Author

Covariant support has been merged into master. This also changes the type used with Observable.create.

@benjchristensen
Copy link
Member Author

I believe all of the structural changes are now merged to master. Are there any other changes that should be made before we release 0.12 so as to better support Scala integration?

@samuelgruetter
Copy link
Contributor

I've isolated the implicit conversion problem here. It's a problem with the Scala compiler.
@benjchristensen looking forward to a jar with covariant observables!

@aloiscochard
Copy link

@samuelgruetter I'm pretty sure no one could make the scala compiler be able to solve an ambiguity like that one.

If you where the compiler, which method you think should be choosed?

You guys should seriously consider implementing a full wrapped scala version first, and keep multi-lang support as a cherry on cake.

IMHO it's a mistake to do the opposite, maybe I'm wrong but I think most scala user will want to use it in a full scala project.

And for the course, it would be good to have an idiomatic API.

@samuelgruetter
Copy link
Contributor

We have sufficient workarounds for the compiler bug.

There's nothing fundamental that we need for the Scala adapter right now, but I'm currently trying to translate the dictionary autocomplete example written in C# from this Rx Hands On Lab to Scala to see what we can already do. For this, I miss the distinctUntilChanged and the throttle operator, so these are currently on top of my whishlist ;-)

@benjchristensen
Copy link
Member Author

The 'throttle' operator was just merged to master as debounce``throttleWithTimeout. See #368 for an explanation of the 3 variants ofthrottle that were committed.

We do not yet have distinctUntilChanged but that shouldn't be hard to get added.

@benjchristensen
Copy link
Member Author

That's great that workarounds exist so we can move forward, I look forward to hearing how the autocomplete example works.

If you can get that functioning is that strong enough evidence to move forward with it or are there still pieces of functionality lacking?

How do you envision ongoing maintenance of this class? Do all new operators added to rx.Observable.java required wrapping methods, or does it pass thru if a wrapper isn't there?

@samuelgruetter
Copy link
Contributor

If I can get the autocomplete example functioning then I think that's strong enough evidence to move forward with it. We still lack operations in the wrapper, but it should be possible to add them all.

This adapter will require some maintenance. New operators added to the Java observable will have to be added to the adapter as well, and updates of the documentation too, because the signatures are too different to automate this.
However, if really needed, one can always call .asJava on a Scala observable and use the Java methods directly, and if the implicit function conversions from Scala functions to Java Func/Action are imported, one could even use Scala functions. But that's not the intended way of using it and should only be used in quick hacks.

@benjchristensen
Copy link
Member Author

Good to know regarding maintenance ... we'll need to figure out a reasonable way of handling that as operators get added.

Do you have an idea of when I should expect a pull request with this new Scala Observable?

Is this a breaking change to how the implicits support currently works for Scala, or does this wrapper only take effect when someone imports rx.lang.scala.Observable instead of rx.Observable? Can the existing implicits continue existing alongside?

@samuelgruetter
Copy link
Contributor

It's not a breaking change: The old RxImplicits are still there and usable, but marked as deprecated.

@daveray
Copy link
Contributor

daveray commented Sep 12, 2013

So does it make sense to mark them deprecated since they're still usable and possibly useful for someone accessing operations that haven't made their way to this curated wrapper? Just checking.

@samuelgruetter
Copy link
Contributor

If an operation from rx.Observable has not yet made it to rx.lang.scala.Observable, there are the following solutions (ordered by my preference):

  • report an issue to get the operation into rx.lang.scala.Observable ;-)
  • call .asJava on the Scala observable and use the implicit conversions from rx.lang.scala.internal.ImplicitFunctionConversions
  • use the old RxImplicits

@samuelgruetter
Copy link
Contributor

Btw RxImplicits and ImplicitFunctionConversions are almost the same, except that ImplicitFunctionConversions does not contain an implicit class ScalaObservable.

@benjchristensen
Copy link
Member Author

Pull #376 has been released in 0.13.1

Thank you @samuelgruetter!

@benjchristensen
Copy link
Member Author

Btw RxImplicits and ImplicitFunctionConversions are almost the same, except that ImplicitFunctionConversions does not contain an implicit class ScalaObservable.

With that one difference does it still require the two existing, or can ImplicitFunctionConversions perform the same functionality as RxImplicits and negate it's need? Or does having RxImplicits around still help for any use cases?

@samuelgruetter
Copy link
Contributor

If there is already code out there depending on RxImplicits, we need to keep it to remain compatible with that code. If not, we can remove RxImplicits without any loss of functionality.

@mattrjacobs
Copy link
Contributor

Given that there's no loss of functionality, I think it makes the most sense to remove RxImplicits and not introduce the chance of newcomers to RxJava depending on unsupported functionality. I'll generate a pull request to this effect

@daveray
Copy link
Contributor

daveray commented Sep 13, 2013

Removing duplication makes sense. Since rx.lang.scala.internal.ImplicitFunctionConversions is escape hatch (besides manually using Func1 and friends) for stuff not covered by the Scala wrapper, would it make sense for it not to be in an internal package?

@samuelgruetter
Copy link
Contributor

@mattrjacobs Good idea.

@daveray Well the goal should be that there is no such thing as "stuff not covered by the Scala wrapper"... Having ImplicitFunctionConversions in an internal package would increase the pressure to keep the Scala Observable in sync, which is a desired effect. And if we add a unit test as I outlined at the bottom of this comment, keeping the Scala Observable up to date would become easier.

@benjchristensen
Copy link
Member Author

Removing RxImplicits will be breaking for people until they change their import from rx.Observable to rx.lang.scala.Observable correct? If so I will bump the version to 0.14 when I merge this change.

Can someone also please update the README (https://github.com/Netflix/RxJava/blob/master/language-adaptors/rxjava-scala/README.md) to show usage information to help someone get started?

@daveray
Copy link
Contributor

daveray commented Sep 14, 2013

@samuelgruetter It's up to you guys. I'll just leave with this: open source projects live a lot longer than the enthusiasm of their contributors so I'm skeptical of a wrapper that assumes it will always be kept completely up-to-date with complete coverage forever. If some third party decided to create a project of custom observables beyond what's in RxJava, those would be more difficult to use, assuming users respect the implication of "internal" in the package name. I'll go back to Clojure-land now :)

@martin-g
Copy link

@daveray ScalaObservable has a method to get the wrapped Java Observable, so you should be able to use any method from its API.
And since the project is Open Source you can always send a Pull Request and improve it ;)

@daveray
Copy link
Contributor

daveray commented Sep 14, 2013

@martin-g Right. And if for whatever reason some poor Scala programmer finds herself in yucky Java-land calling these mehods, at least throw her a bone and make the Func implicits available in a non-internal package.

@mattrjacobs
Copy link
Contributor

@martin-g Correct me if I'm wrong, but dropping down to using rx.Observable still mandates passing in Func1/Func2/etc. Being able to use native Scala functions and the implicits in this case seems far better than forcing user code to new up these function types and pass them in.

After thinking about it, +1 to @daveray's idea of making both the value class and implicits available (with the value class preferred). This will allow any temporary mismatch between rx.Observable and rx.lang.scala.Observable to be addressed by implicits while not impeding non-Scala developers from moving core along.

@benjchristensen
Copy link
Member Author

+1 on this. There are other places where Func* are used, such as BlockingObservable not yet wrapped, and as new things are added, if they don't yet have a full wrapper there should still be a way to use them with the implicits.

@benjchristensen
Copy link
Member Author

it might be cool to have a unit test in which we encode the information "which Java method corresponds to which Scala method", and using reflection, we check if there is a Scala method for each Java method. So if a method is added to the Java Observable but not to the Scala Observable, this test would fail.

And if we add a unit test as I outlined at the bottom of this comment, keeping the Scala Observable up to date would become easier.

We can't make the build fail if rx.lang.scala.Observable is out of sync. We can't block the project from building and releasing just because a new operator is added and the developer adding it isn't comfortable deciding what the idiomatic Scala method signature should look like.

It needs to be an async process for Scala developers to add wrapper methods/classes. Since this is open source we can't control people's schedule and require quick turnaround on adding the wrappers. Ideally that will happen when new functionality is added, but we can't make building and releasing dependent on that.

Thus, I think we need to account for the fact that most things will have wrappers but very newly added functionality or fringe functionality may not and should still have a mechanism for being used.

@cer
Copy link

cer commented Sep 16, 2013

If you want rxjava-core to evolve independently then perhaps rxjava-scala
and other language adapters should each be a separate top-level project
with its own release cycle.

On Mon, Sep 16, 2013 at 2:55 PM, Ben Christensen
[email protected]:

it might be cool to have a unit test in which we encode the information
"which Java method corresponds to which Scala method", and using
reflection, we check if there is a Scala method for each Java method. So if
a method is added to the Java Observable but not to the Scala Observable,
this test would fail.

And if we add a unit test as I outlined at the bottom of this comment,
keeping the Scala Observable up to date would become easier.

We can't make the build fail if rx.lang.scala.Observable is out of sync.
We can't block the project from building and releasing just because a new
operator is added and the developer adding it isn't comfortable deciding
what the idiomatic Scala method signature should look like.

It needs to be an async process for Scala developers to add wrapper
methods/classes. Since this is open source we can't control people's
schedule and require quick turnaround on adding the wrappers. Ideally that
will happen when new functionality is added, but we can't make building and
releasing dependent on that.

Thus, I think we need to account for the fact that most things will have
wrappers but very newly added functionality or fringe functionality may not
and should still have a mechanism for being used.


Reply to this email directly or view it on GitHubhttps://github.com//issues/336#issuecomment-24548044
.

@benjchristensen
Copy link
Member Author

Erik Meijer and I have discussed this but it doesn't feel like the right thing to do, as it would hurt the JVM ecosystem more than benefit it if the different languages forked from each other. 

There is no harm in a new version of RxJava being released without the Scala adaptor adding the new functionality, as that is no different than a separate RxScala project not yet supporting it on top of the RxJava core dependency.

As we approach 1.0 the rapid iteration will slow and this will become less of an issue. Around that time as well the project may migrate into a different home than here.
On September 16, 2013 at 3:20:07 PM, Chris Richardson ([email protected]) wrote:

If you want rxjava-core to evolve independently then perhaps rxjava-scala
and other language adapters should each be a separate top-level project
with its own release cycle.

@samuelgruetter
Copy link
Contributor

I like @daveray's point: If some third party decided to create a Java project of custom observables beyond what's in RxJava, and we want to use this project from Scala, then we need the ImplicitFunctionConversions. So I will make this non-internal.

@benjchristensen I agree that it would be kind of crazy to make the build fail just because rx.lang.scala.Observable is out of sync.

@benjchristensen
Copy link
Member Author

Closing this out as completed ... further progress/bugs can use new issues.

Great work and thank you everyone involved on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests