Applicative do

[ghost]

This is a work in progress, opening the PR for review and design process.

https://ghc.haskell.org/trac/ghc/wiki/ApplicativeDo

Closes #46

[ghost]

Just fixed a couple bugs. The code is still hairy but I'm going to improve the naming of functions and refactor a bit later today.

I've tested it with promissum promises and works too, so excited \o/

I think there may be some bugs still and I'd like to add more test cases, help if welcome in this regard.

[yurrriq]

Sorry for the dumb question, but glancing at the tests, I can't discern the practical difference between alet and mlet. What's the goal here?

[ghost]

Sorry for the dumb question, but glancing at the tests, I can't discern the practical difference between alet and mlet. What's the goal here?

Great question!

One limitation of monadic bind is that all the steps are strictly sequential and happen one at a time. This piece of code:

(bind (just 1)
     (fn [a]
       (bind (just 41)
               (fn [b]
                 (return (+ a b))))))
;; => #<Just 42>

In the first call to bind, (just 1) and the anonymous function will be evaluated. The call of anonymous function will cause the evaluation of the (just 41) and the next anonymous function, which will be also called to create the final result. Note that (just 1) and (just 41) are independent and thus could be evaluated at the same time.

mlet version for reference:

(mlet [a (just 1)
       b (just 41)]
  (return (+ a b)))
;; => #<Just 42>

Now let's see the equivalent alet:

(alet [a (just 1)
       b (just 41)]
  (+ a b))
;; => #<Just 42>

Note that no return is used, this is because alet's body run inside the applicative context with fapply. This is roughly what alet desugars to:

(fapply (fn [a]
           (fn [b]
             (+ a b)))
         (just 1)
         (just 41))
;; => #<Just 42>

Note that now (just 1) and (just 41) are evaluated at the same time. This use of fapply can be called "applicative bind" and in some cases is more efficient than monadic bind. Furthermore, the alet macro splits the bindings into batches that have dependencies only in previous values and evaluates all applicative values in the batch at the same time.

This makes no difference at all for Maybe, but applicatives that have latency in their calculations (for example promises that do an async computation) get a pretty good evaluation strategy, which can minimize overall latency. This is similar to what Manifold's let-flow macro does, but more generic.

I recommend taking a look into Haxl library since it's what motivated the patch to GHC for adding applicative do. I'll be adding documentation for this pull request with more details and may write a blog post disecting what the alet macro does.

[yurrriq]

Aha! I'm familiar with the proposal/patch and Haxl in the Haskell world, and thanks to your description (and rereading the patch description) I think I get it in the cats context now. A blog post sounds great!

---

I've run into a quirk bug, or perhaps a gap in my understanding...

(in-ns 'cats.core)
(require '[cats.monad.maybe :as maybe])

(def ten-million 10000000)

(time
 (mlet [x (maybe/just (nth (iterate inc 6) ten-million))
        y (maybe/just (nth (iterate inc 7) ten-million))
        x (maybe/just (- x ten-million))
        y (maybe/just (- y ten-million))]
   (return (* x y))))
;; "Elapsed time: 773.020927 msecs"
;; => #<Just 42>

If I understand correctly, the same form with alet (and without return) should return the same value, #<Just 42>, and take about half as long... What's going on here?

(time
 (alet [x (maybe/just (nth (iterate inc 6) ten-million))
        y (maybe/just (nth (iterate inc 7) ten-million))
        x (maybe/just (- x ten-million))
        y (maybe/just (- y ten-million))]
   (* x y)))
;; "Elapsed time: 771.776612 msecs"
;; => #<Just 60000042>

[ghost]

Aha! I'm familiar with the proposal/patch and Haxl in the Haskell world, and thanks to your description (and rereading the patch description) I think I get it in the cats context now. A blog post sounds great!

We still have to start the funcool blog but this is definitely worth writing about.

If I understand correctly, the same form with alet (and without return) should return the same value, #<Just 42>

Yes, you're right. This is apparently a bug in the renaming of symbols in the body, nice to have an additional test case, thanks!

and take about half as long... What's going on here?

Take into account that (nth (iterate inc 6) ten-million) and (nth (iterate inc 7) ten-million) are both synchronous operations and thus block the calling thread. This means that, even if they are both evaluated in a batch in alet doesn't matter because for evaluating the second form evaluation of the first must end.

The benefits of alet are better seen with asynchronous operations. A quick session at the REPL with the promissum library, which is a monadic wrapper around Java 8's CompletableFuture. @niwinz please correct if i'm wrong.

(require '[cats.core :as m])
(require '[promissum.core :as p])

(defn sleep-promise [wait]
  (p/promise (fn [deliver]
               (Thread/sleep wait)
               (deliver wait))))

;; note: deref-ing for blocking the current thread waiting for the promise being delivered
(time
 @(m/mlet [x (sleep-promise 42)
           y (sleep-promise 41)]
    (m/return (+ x y))))
;; "Elapsed time: 84.328182 msecs"
;; => 83

(time
 @(m/alet [x (sleep-promise 42)
           y (sleep-promise 41)]
    (+ x y)))
;; "Elapsed time: 44.246427 msecs"
;; => 83

Another example for illustrating dependencies between batches:

(time
 @(m/mlet [x (sleep-promise 42)
           y (sleep-promise 41)
           z (sleep-promise (inc x))
           a (sleep-promise (inc y))]
   (m/return  (+ z a))))
;; "Elapsed time: 194.253182 msecs"
;; => 85

(time
 @(m/alet [x (sleep-promise 42)
           y (sleep-promise 41)
           z (sleep-promise (inc x))
           a (sleep-promise (inc y))]
    (+ z a)))
;; "Elapsed time: 86.20699 msecs"
;; => 85

[yurrriq]

Got it. I hadn't checked out promissum before. Looks cool. Does alet work with core.async too?

[ghost]

Should work with the canal library which implements both Applicative and Monad for core.async channels. It has received little time, we need to port to reader conditionals and make a release with the new name.

[yurrriq]

Cool, thanks! I'll take a look.

[yurrriq]

I gave canal a once-over.

Then, using yurrriq/canal@efbb1cc and 486895b via lein-git-deps:

(require '[clojure.core.async :refer [<!!]])

(defn sleep-chan [wait]
  (m/fmap
   (fn [n] (Thread/sleep n) n)
   (m/pure channel-monad wait)))

(time
 (<!! (m/mlet [x (sleep-chan 42)
               y (sleep-chan 41)]
        (m/return (+ x y)))))
;; "Elapsed time: 86.302538 msecs"
;; => 83

(time
 (<!! (m/alet [x (sleep-chan 42)
               y (sleep-chan 41)]
        (+ x y))))
;; "Elapsed time: 87.723344 msecs"
;; => 83

Thoughts?

[ghost]

It may be an issue with the channel's Applicative implementation, I'll take a close look after work. Thanks for reporting this and your time!

[niwinz]

I'm currently working on it and I have fixed it:

(require '[cats.core :as m]
         '[cats.context :as mc]
         '[cats.monad.either :as either]
         '[cats.builtin])


(require '[clojure.core.async :as a])
(require '[cats.labs.channel])

(defn async-call
  [wait]
  (a/go
    (a/<! (a/timeout wait))
    wait))

(defn check-mlet
  []
  (time
   (a/<!! (m/mlet [x (async-call 100)
                   y (async-call 100)]
            (m/return (+ x y))))))

(defn check-alet
  []
  (time
   (a/<!! (m/alet [x (async-call 100)
                   y (async-call 100)]
            (m/return (+ x y))))))

(check-alet)
;; "Elapsed time: 106.507818 msecs"

(check-mlet)
;; "Elapsed time: 202.745459 msecs"

[ghost]

Awesome!

[yurrriq]

Looks great!

[yurrriq]

It might be good to say in the docs somewhere that in Emacs with clojure-mode, you can add this to your configure for better indentation:

(define-clojure-indent
  (alet 'defun))

The entirety of my cats-related config as of now, for reference:

(define-clojure-indent
  (alet 'defun)
  (bind 'defun)
  (branch 'defun)
  (mlet 'defun)
  (branch-left 'defun))

[ghost]

It might be good to say in the docs somewhere that in Emacs with clojure-mode, you can add this to your configure for better indentation.

+1, going to add it to the docs now.

[ghost]

I think this is ready for merge.

[niwinz]

Ok, just rename please the WIP commit, then I'll merge it :P

[ghost]

I squashed the commit into the previous one, take a look @niwinz.