Notes on 8 June 2021

Agenda

N/A, nothing planned in advance

Summary of progress on FastAI.jl tabular
- Manikya tentatively to present during next ML/AD call
Nested Params discussion (re. FluxML/Zygote.jl#823)
- Users should not be using params for functionality like L2
  - We should document this and provide some alternative functionality
- Can params just throw in a gradient context?
  - Constructing params in a gradient context is still useful, but has sharp edges
Zeros redux
- Should be a proper array like OneHot
- Would unblock Optimisers.jl and more
FluxBench.jl and speed.fluxml.ai
- Mostly set up, Dhairya will look into moving the bot part over to FluxML
Symbolics.jl discussion
- Is it appropriate for improving performance of semi-static flux models?
- How and who handles memory-related optimizations like buffer reuse and out-of-place -> in-place re-writing?
  - CUDA.jl can do some of the former, unclear on the latter
- What else needs to be done to reduce memory overhead from the framework and AD side?
- Plan to discuss in detail at the next ML/AD call