Example: variational problems using autodiff

[zsteve]

Added a new example notebook examples/variational demonstrating how to use the package to differentiate through the output of sinkhorn2 and approximate solutions to PDEs as Wasserstein gradient flows.

[coveralls]

Pull Request Test Coverage Report for Build 1173827433

• 0 of 0 changed or added relevant lines in 0 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage remained the same at 97.654%

---

Totals
Change from base Build 1165287400:	0.0%
Covered Lines:	666
Relevant Lines:	682

---

💛 - Coveralls

[codecov-commenter]

Codecov Report

Merging #122 (5061289) into master (ce8110c) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #122   +/-   ##
=======================================
  Coverage   97.65%   97.65%           
=======================================
  Files          13       13           
  Lines         682      682           
=======================================
  Hits          666      666           
  Misses         16       16           

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ce8110c...5061289. Read the comment docs.

[davibarreira]

Hey @zsteve , what exactly is this optimization that you used to minimize the $\rho$ (the variational problem)? This is the only thing that it wasn't quite clear to me when reading the example. I mean, since $\rho$ is actually a distribution, how are you using gradient descent on it?

function step(ρ0, τ, ε, C, G)
    opt = optimize(
        u -> G(softmax(u), ρ0, τ, ε, C),
        ones(size(ρ0)),
        LBFGS(),
        Optim.Options(; iterations=50, g_tol=1e-6);
        autodiff=:forward,
    )
    return softmax(Optim.minimizer(opt))
end

[coveralls]

Pull Request Test Coverage Report for Build 1173499718

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 0 of 0 changed or added relevant lines in 0 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage remained the same at 97.654%

---

Totals
Change from base Build 1165287400:	0.0%
Covered Lines:	666
Relevant Lines:	682

---

💛 - Coveralls

[coveralls]

Pull Request Test Coverage Report for Build 1173755911

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 0 of 0 changed or added relevant lines in 0 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage remained the same at 97.654%

---

Totals
Change from base Build 1165287400:	0.0%
Covered Lines:	666
Relevant Lines:	682

---

💛 - Coveralls