Merge pull request #179 from 47deg/release-notes

Release 1.0.0

CHANGELOG.md

@@ -1,39 +1,39 @@
 # Changelog
 
-## 02/08/2019 - Version 1.0.0-RC2
+## 02/26/2019 - Version 1.0.0
 
-Release changes:
+The 1.0.0 release of Fetch is a redesign of the library in terms of `cats-effect` abstractions. It's a backwards-incompatible release that introduces numerous breaking changes as well as a couple of new features. It now should be easier to use and require less work from the user of the library, specially when you are already using `cats-effect`.
 
-* Fixes micrositePushWith Setting ([#158](https://github.com/47deg/fetch/pull/158))
-* Introduce Fetch#optional for performing optional fetches ([#159](https://github.com/47deg/fetch/pull/159))
-* DataSourceCache parameterised to F[_] ([#160](https://github.com/47deg/fetch/pull/160))
-* Proof-of-concept Redis cache implementation of DataSourceCache ([#161](https://github.com/47deg/fetch/pull/161))
-* Upgrade CI build ([#173](https://github.com/47deg/fetch/pull/173))
-* Don't run 'gem update --system' ([#174](https://github.com/47deg/fetch/pull/174))
-* Update Copyright Notices to 2019. ([#172](https://github.com/47deg/fetch/pull/172))
-* Add F[_] type parameter to DataSource ([#171](https://github.com/47deg/fetch/pull/171))
-* s/Env/Log/ ([#176](https://github.com/47deg/fetch/pull/176))
-* Remove Par typeclass ([#166](https://github.com/47deg/fetch/pull/166))
-* Update to RC2 ([#177](https://github.com/47deg/fetch/pull/177))
+### Breaking changes
 
+- Introduced the `Data` typeclass to identify requests to a data source
+- Redesigned `DataSource` to take an extra `F[_]` type parameter
+  + Renamed `fetchOne` to `fetch` and `fetchMany` to `batch`
+  + Data sources now can be implicitly constructed
+  + Automatic parallel implementation of `DataSource#batch` in terms of `ConcurrentEffect[F]`
+- Removed `FetchMonadError`, a Fetch is now interpreted to a `ConcurrentEffect`
+- Removed `Query`, a data source now returns a `F` that has an implicit `ConcurrentEffect[F]`
+- Renamed `DataSourceCache` to `DataCache`, it now takes a `Data` instance as a parameter for insert and lookup and is parameterised to F
+- Renamed `Env` to `Log`
 
-## Unreleased - Version 1.0.0
+### New features
 
-Release changes:
+- Introduced `Fetch#optional`, an alternative to `Fetch#apply` for optional fetches
+- Different Data Sources now can have the same identity and result types
+
+### API changes
 
-* Library redesign ([#155](https://github.com/47deg/fetch/pull/155))
+- `Fetch#run` now requires a `Timer[F]` and `ContextShift[F]` from `cats-effect`
+- `Fetch#apply` no longer requires an implicit `DataSource` and must be passed explicitly
+- Renamed `Fetch#runEnv` to `Fetch#runLog`
+- `Fetch#traverse`, `Fetch#sequence`, `Fetch#join` & `Fetch#collect` deleted in favor of usign cats typeclass ops
 
-Summary of changes:
+### Documentation
 
-* `Query` and `FetchMonadError` types deleted
-* `Fetch#traverse`, `Fetch#sequence`, `Fetch#join` & `Fetch#collect` deleted in favor of usign cats typeclass ops
-* Introduction of `cats-effect` for the implementation and target types
-  - `DataSource` in terms of `ConcurrentEffect` from `cats-effect`
-  - `DataSourceCache` in terms of `ConcurrentEffect`
-  - `Fetch` is now parameterised to `F[_]` with a `ConcurrentEffect[F]`
-  - `Fetch#apply` now doesn't require an implicit `DataSource` but it must be provided explicitly
-  - `Fetch#run` now requires a `Timer[F]` and `ContextShift[F]` from `cats-effect`
-  - Removed Monix, Future and Twitter Future subprojects, most of them should work with `cats-effect` abstractions already
+- Proof-of-concept Redis cache implementation of `DataCache` ([#161](https://github.com/47deg/fetch/pull/161))
+- Removed Monix, Future and Twitter Future subprojects, most of them should work with `cats-effect` abstractions already
+- GrapQL query interpreter example ([#178](https://github.com/47deg/fetch/pull/178))
+- Example using Monix Scheduler and Task to run fetches ([#178](https://github.com/47deg/fetch/pull/178))
 
 ## 08/21/2018 - Version 0.7.3
 

README.md

@@ -2,7 +2,7 @@
 
 [comment]: # (Start Badges)
 
-[![Join the chat at https://gitter.im/47deg/fetch](https://badges.gitter.im/47deg/fetch.svg)](https://gitter.im/47deg/fetch?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) [![Build Status](https://travis-ci.org/47deg/fetch.svg?branch=master)](https://travis-ci.org/47deg/fetch) [![codecov.io](http://codecov.io/github/47deg/fetch/coverage.svg?branch=master)](http://codecov.io/github/47deg/fetch?branch=master) [![Maven Central](https://img.shields.io/badge/maven%20central-1.0.0-RC2-green.svg)](https://oss.sonatype.org/#nexus-search;gav~com.47deg~fetch*) [![License](https://img.shields.io/badge/license-Apache%202-blue.svg)](https://raw.githubusercontent.com/47deg/fetch/master/LICENSE) [![Latest version](https://img.shields.io/badge/fetch-1.0.0-RC2-green.svg)](https://index.scala-lang.org/47deg/fetch) [![Scala.js](http://scala-js.org/assets/badges/scalajs-0.6.17.svg)](http://scala-js.org) [![GitHub Issues](https://img.shields.io/github/issues/47deg/fetch.svg)](https://github.com/47deg/fetch/issues)
+[![Join the chat at https://gitter.im/47deg/fetch](https://badges.gitter.im/47deg/fetch.svg)](https://gitter.im/47deg/fetch?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) [![Build Status](https://travis-ci.org/47deg/fetch.svg?branch=master)](https://travis-ci.org/47deg/fetch) [![codecov.io](http://codecov.io/github/47deg/fetch/coverage.svg?branch=master)](http://codecov.io/github/47deg/fetch?branch=master) [![Maven Central](https://img.shields.io/badge/maven%20central-0.6.1-green.svg)](https://oss.sonatype.org/#nexus-search;gav~com.47deg~fetch*) [![License](https://img.shields.io/badge/license-Apache%202-blue.svg)](https://raw.githubusercontent.com/47deg/fetch/master/LICENSE) [![Latest version](https://img.shields.io/badge/fetch-0.6.1-green.svg)](https://index.scala-lang.org/47deg/fetch) [![Scala.js](http://scala-js.org/assets/badges/scalajs-0.6.15.svg)](http://scala-js.org) [![GitHub Issues](https://img.shields.io/github/issues/47deg/fetch.svg)](https://github.com/47deg/fetch/issues)
 
 [comment]: # (End Badges)
 
@@ -19,13 +19,13 @@ For Scala 2.11.x and 2.12.x:
 [comment]: # (Start Replace)
 
 ```scala
-"com.47deg" %% "fetch" % "1.0.0-RC2"
+"com.47deg" %% "fetch" % "1.0.0"
 ```
 
 Or, if using Scala.js (0.6.x):
 
 ```scala
-"com.47deg" %%% "fetch" % "1.0.0-RC2"
+"com.47deg" %%% "fetch" % "1.0.0"
 ```
 
 [comment]: # (End Replace)
@@ -137,8 +137,8 @@ import scala.concurrent.duration._
 // import scala.concurrent.duration._
 
 Fetch.run[IO](fetchOne).unsafeRunTimed(5.seconds)
-// --> [101] One ToString 1
-// <-- [101] One ToString 1
+// --> [107] One ToString 1
+// <-- [107] One ToString 1
 // res0: Option[String] = Some(1)
 ```
 
@@ -157,8 +157,8 @@ When executing the above fetch, note how the three identities get batched and th
 
 ```scala
 Fetch.run[IO](fetchThree).unsafeRunTimed(5.seconds)
-// --> [101] Batch ToString NonEmptyList(1, 2, 3)
-// <-- [101] Batch ToString NonEmptyList(1, 2, 3)
+// --> [107] Batch ToString NonEmptyList(1, 2, 3)
+// <-- [107] Batch ToString NonEmptyList(1, 2, 3)
 // res1: Option[(String, String, String)] = Some((1,2,3))
 ```
 
@@ -196,12 +196,12 @@ When executing the above fetch, note how the three identities get requested in p
 
 ```scala
 Fetch.run[IO](fetchUnbatchedThree).unsafeRunTimed(5.seconds)
-// --> [101] One UnbatchedToString 1
-// --> [103] One UnbatchedToString 3
-// --> [104] One UnbatchedToString 2
-// <-- [101] One UnbatchedToString 1
-// <-- [103] One UnbatchedToString 3
-// <-- [104] One UnbatchedToString 2
+// --> [107] One UnbatchedToString 1
+// --> [110] One UnbatchedToString 3
+// --> [109] One UnbatchedToString 2
+// <-- [107] One UnbatchedToString 1
+// <-- [110] One UnbatchedToString 3
+// <-- [109] One UnbatchedToString 2
 // res2: Option[(String, String, String)] = Some((1,2,3))
 ```
 
@@ -247,10 +247,10 @@ Note how the two independent data fetches run in parallel, minimizing the latenc
 
 ```scala
 Fetch.run[IO](fetchMulti).unsafeRunTimed(5.seconds)
-// --> [101] One ToString 1
-// --> [102] One Length one
-// <-- [101] One ToString 1
-// <-- [102] One Length one
+// --> [107] One ToString 1
+// --> [108] One Length one
+// <-- [107] One ToString 1
+// <-- [108] One Length one
 // res3: Option[(String, Int)] = Some((1,3))
 ```
 
@@ -271,8 +271,8 @@ While running it, notice that the data source is only queried once. The next tim
 
 ```scala
 Fetch.run[IO](fetchTwice).unsafeRunTimed(5.seconds)
-// --> [103] One ToString 1
-// <-- [103] One ToString 1
+// --> [110] One ToString 1
+// <-- [110] One ToString 1
 // res4: Option[(String, String)] = Some((1,1))
 ```
 
@@ -288,6 +288,7 @@ For more in-depth information take a look at our [documentation](http://47deg.gi
 If you wish to add your library here please consider a PR to include it in the list below.
 
 [comment]: # (Start Copyright)
+
 # Copyright
 
 Fetch is designed and developed by 47 Degrees

build.sbt

@@ -1,3 +1,5 @@
+import sbtcrossproject.CrossPlugin.autoImport.{crossProject, CrossType}
+
 pgpPassphrase := Some(getEnvVar("PGP_PASSPHRASE").getOrElse("").toCharArray)
 pgpPublicRing := file(s"$gpgFolder/pubring.gpg")
 pgpSecretRing := file(s"$gpgFolder/secring.gpg")
@@ -10,20 +12,21 @@ lazy val root = project
   .settings(moduleName := "root")
   .aggregate(fetchJS, fetchJVM, debugJVM, debugJS)
 
-lazy val fetch = crossProject
+lazy val fetch = crossProject(JSPlatform, JVMPlatform)
   .in(file("."))
   .settings(name := "fetch")
   .jsSettings(sharedJsSettings: _*)
-  .crossDepSettings(commonCrossDependencies: _*)
+  .settings(commonCrossDependencies)
 
 lazy val fetchJVM = fetch.jvm
 lazy val fetchJS  = fetch.js
 
-lazy val debug = (crossProject in file("debug"))
+lazy val debug =  crossProject(JSPlatform, JVMPlatform)
+  .in(file("debug"))
   .settings(name := "fetch-debug")
   .dependsOn(fetch)
   .jsSettings(sharedJsSettings: _*)
-  .crossDepSettings(commonCrossDependencies: _*)
+  .settings(commonCrossDependencies)
 
 lazy val debugJVM = debug.jvm
 lazy val debugJS  = debug.js

docs/src/main/tut/docs.md

@@ -31,13 +31,13 @@ To begin, add the following dependency to your SBT build file:
 [comment]: # (Start Replace)
 
 ```scala
-"com.47deg" %% "fetch" % "1.0.0-RC2"
+"com.47deg" %% "fetch" % "1.0.0"
 ```
 
 Or, if using Scala.js:
 
 ```scala
-"com.47deg" %%% "fetch" % "1.0.0-RC2"
+"com.47deg" %%% "fetch" % "1.0.0"
 ```
 
 [comment]: # (End Replace)
@@ -867,7 +867,7 @@ a fetch execution given an execution log.
 Add the following line to your dependencies for including Fetch's debugging facilities:
 
 ```scala
-"com.47deg" %% "fetch-debug" % "1.0.0-RC2"
+"com.47deg" %% "fetch-debug" % "1.0.0"
 ```
 
 ## Fetch execution

docs/src/main/tut/index.md

@@ -15,13 +15,13 @@ For Scala 2.11.x and 2.12.x:
 [comment]: # (Start Replace)
 
 ```scala
-"com.47deg" %% "fetch" % "1.0.0-RC2"
+"com.47deg" %% "fetch" % "1.0.0"
 ```
 
 Or, if using Scala.js (0.6.x):
 
 ```scala
-"com.47deg" %%% "fetch" % "1.0.0-RC2"
+"com.47deg" %%% "fetch" % "1.0.0"
 ```
 
 [comment]: # (End Replace)

examples/src/test/scala/GithubExample.scala

@@ -218,34 +218,6 @@ class GithubExample extends WordSpec with Matchers {
     log.rounds.size shouldEqual 2
   }
 
-  "We can fetch multiple repos in parallel" in {
-    def fetchRepo[F[_]: ConcurrentEffect](r: (String, String)): Fetch[F, Repo] =
-      Fetch(r, Repos.source)
-
-    def fetch[F[_]: ConcurrentEffect]: Fetch[F, List[Repo]] =
-      List(
-        ("monix", "monix"),
-        ("typelevel", "cats"),
-        ("typelevel", "cats-effect"),
-        ("47deg", "fetch")).traverse(fetchRepo[F])
-
-    val io = Fetch.runLog[IO](fetch)
-
-    val (log, result) = io.unsafeRunSync
-
-    log.rounds.size shouldEqual 1
-  }
-
-  "We can combine everything" in {
-    def fetch[F[_]: ConcurrentEffect](org: String): Fetch[F, (List[Project], Int, Int, Int)] =
-      (fetchOrg(org), fetchOrgStars(org), fetchOrgContributors(org), fetchOrgLanguages(org)).tupled
-
-    val io = Fetch.runLog[IO](fetch("47deg"))
-    val (log, result) = io.unsafeRunSync
-
-    log.rounds.size shouldEqual 2
-  }
-
   // Github HTTP api
 
   val GITHUB: Uri = Uri.unsafeFromString("https://api.github.com")

project/ProjectPlugin.scala

@@ -17,10 +17,11 @@ object ProjectPlugin extends AutoPlugin {
 
   object autoImport {
 
-    lazy val commonCrossDependencies: Seq[ModuleID] =
+    lazy val commonCrossDependencies =
       Seq(
-        "org.typelevel" %% "cats-effect" % "1.0.0",
-          %%("scalatest") % "test")
+		libraryDependencies ++=
+		Seq("org.typelevel" %% "cats-effect" % "1.2.0",
+            %%("scalatest") % "test"))
 
     lazy val micrositeSettings: Seq[Def.Setting[_]] = Seq(
       micrositeName := "Fetch",
@@ -62,15 +63,15 @@ object ProjectPlugin extends AutoPlugin {
       tutNameFilter := """README.md""".r
     )
 
-    lazy val examplesSettings: Seq[Def.Setting[_]] = libraryDependencies ++= Seq(
+    lazy val examplesSettings = Seq(libraryDependencies ++= Seq(
       %%("circe-generic"),
       %%("doobie-core"),
       %%("doobie-h2"),
       "org.tpolecat" %% "atto-core"    % "0.6.5",
       "org.http4s" %% "http4s-blaze-client" % "0.19.0-M2",
       "org.http4s" %% "http4s-circe" % "0.19.0-M2",
       "redis.clients" % "jedis" % "2.9.0"
-    ) ++ commonCrossDependencies
+    )) ++ commonCrossDependencies
   }
 
   lazy val commandAliases: Seq[Def.Setting[_]] =

project/plugins.sbt

@@ -1,2 +1,3 @@
 resolvers += Resolver.sonatypeRepo("releases")
 addSbtPlugin("com.47deg" % "sbt-org-policies" % "0.9.3")
+addSbtPlugin("org.portable-scala" % "sbt-scalajs-crossproject" % "0.6.0")

shared/src/main/scala/fetch.scala

@@ -181,6 +181,25 @@ object `package` {
         }
       } yield result)
 
+    override def map2[A, B, Z](fa: Fetch[F, A], fb: Fetch[F, B])(f: (A, B) => Z): Fetch[F, Z] =
+      Unfetch(for {
+        fab <- (fa.run, fb.run).tupled
+        result = fab match {
+          case (Throw(e), _) =>
+            Throw[F, Z](e)
+          case (Done(a), Done(b)) =>
+            Done[F, Z](f(a, b))
+          case (Done(a), Blocked(br, c)) =>
+            Blocked[F, Z](br, map2(fa, c)(f))
+          case (Blocked(br, c), Done(b)) =>
+            Blocked[F, Z](br, map2(c, fb)(f))
+          case (Blocked(br, c), Blocked(br2, c2)) =>
+            Blocked[F, Z](combineRequestMaps(br, br2), map2(c, c2)(f))
+          case (_, Throw(e)) =>
+            Throw[F, Z](e)
+        }
+      } yield result)
+
     override def product[A, B](fa: Fetch[F, A], fb: Fetch[F, B]): Fetch[F, (A, B)] =
       Unfetch[F, (A, B)](for {
         fab <- (fa.run, fb.run).tupled
@@ -200,6 +219,25 @@ object `package` {
         }
       } yield result)
 
+    override def productR[A, B](fa: Fetch[F, A])(fb: Fetch[F, B]): Fetch[F, B] =
+      Unfetch[F, B](for {
+        fab <- (fa.run, fb.run).tupled
+        result = fab match {
+          case (Throw(e), _) =>
+            Throw[F, B](e)
+          case (Done(a), Done(b)) =>
+            Done[F, B](b)
+          case (Done(a), Blocked(br, c)) =>
+            Blocked[F, B](br, productR(fa)(c))
+          case (Blocked(br, c), Done(b)) =>
+            Blocked[F, B](br, productR(c)(fb))
+          case (Blocked(br, c), Blocked(br2, c2)) =>
+            Blocked[F, B](combineRequestMaps(br, br2), productR(c)(c2))
+          case (_, Throw(e)) =>
+            Throw[F, B](e)
+        }
+      } yield result)
+
     def flatMap[A, B](fa: Fetch[F, A])(f: A => Fetch[F, B]): Fetch[F, B] =
       Unfetch(fa.run.flatMap {
         case Done(v) => f(v).run

shared/src/test/scala/FetchBatchingTests.scala

@@ -103,7 +103,7 @@ class FetchBatchingTests extends FetchSpec {
     }).unsafeToFuture
   }
 
-  "Fetches to datasources with a maximum batch size should be split and executed in parallel and sequentially" in {
+  "Fetches to datasources with a maximum batch size should be split and executed in parallel and sequentially when using productR" in {
     def fetch[F[_] : ConcurrentEffect]: Fetch[F, List[Int]] =
       List.range(1, 6).traverse(fetchBatchedDataPar[F]) *>
         List.range(1, 6).traverse(fetchBatchedDataSeq[F])
@@ -120,6 +120,23 @@ class FetchBatchingTests extends FetchSpec {
     }).unsafeToFuture
   }
 
+  "Fetches to datasources with a maximum batch size should be split and executed in parallel and sequentially when using productL" in {
+    def fetch[F[_] : ConcurrentEffect]: Fetch[F, List[Int]] =
+      List.range(1, 6).traverse(fetchBatchedDataPar[F]) <*
+        List.range(1, 6).traverse(fetchBatchedDataSeq[F])
+
+    val io = Fetch.runLog[IO](fetch)
+
+    io.map({
+      case (log, result) => {
+        result shouldEqual List(1, 2, 3, 4, 5)
+        log.rounds.size shouldEqual 1
+        totalFetched(log.rounds) shouldEqual 5 + 5
+        totalBatches(log.rounds) shouldEqual 3 + 3
+      }
+    }).unsafeToFuture
+  }
+
   "A large (many) fetch to a datasource with a maximum batch size is split and executed in sequence" in {
     def fetch[F[_] : ConcurrentEffect]: Fetch[F, List[Int]] =
       List(fetchBatchedDataSeq[F](1), fetchBatchedDataSeq[F](2), fetchBatchedDataSeq[F](3)).sequence