Merge pull request #159 from FluxML/darsnack/gsoc-2023

FluxML · Feb 17, 2023 · 7e3674b · 7e3674b
2 parents e0ed181 + 07b3ddc
commit 7e3674b
Show file tree

Hide file tree

Showing 3 changed files with 53 additions and 86 deletions.
diff --git a/_layout/navbar.html b/_layout/navbar.html
@@ -22,16 +22,16 @@
         <a class="nav-link" href="https://fluxml.ai/Flux.jl/stable/ecosystem/">Ecosystem</a>
       </li>
       <!--
-      <li class="nav-item">
-        <a class="nav-link" href="/gsoc/">GSoC</a>
-      </li>
       <li class="nav-item">
         <a class="nav-link" href="/gsod/">GSoD</a>
       </li>
       -->
       <li class="nav-item">
         <a class="nav-link" href="/blog/">Blog</a>
       </li>
+      <li class="nav-item">
+        <a class="nav-link" href="/gsoc/">GSoC</a>
+      </li>
       <li>
         <a class="nav-link" href="/governance/">Governance</a>
       </li>

diff --git a/gsoc.md b/gsoc.md
@@ -6,112 +6,79 @@ title = "Google Summer of Code"
 <h1>FluxML Projects - Summer of Code</h1>
 ~~~
 
-Flux usually takes part in [Google Summer of Code](https://summerofcode.withgoogle.com) as a NumFocus organization. We follow the same [rules and application guidelines](https://julialang.org/jsoc/projects/) as Julia, so please check there for more information on applying. Below are a set of ideas for potential projects (though you are welcome to explore anything you are interested in).
+Flux usually takes part in [Google Summer of Code](https://summerofcode.withgoogle.com) as a NumFocus organization. We follow the same [rules and application guidelines](https://julialang.org/jsoc/projects/) as Julia, so please check there for more information on applying. Below are a set of ideas for potential projects (though you are welcome to explore anything you are interested in). **Please note that year for the idea list. Project ideas from a previous year will not always carry over to a new year.**
 
-Flux projects are typically very competitive; we encourage you to get started early, as successful contributors typically have early PRs or working prototypes as part of the application. It is a good idea to simply start contributing via issue discussion and PRs and let a project grow from there; you can take a look at [this list of issues](https://github.com/FluxML/Flux.jl/issues?q=is%3Aopen+is%3Aissue+label%3A%22help+wanted%22) for some starter contributions.
+Flux projects are typically very competitive; we encourage you to get started early, as successful contributors typically have early PRs or working prototypes as part of the application. It is a good idea to simply start contributing via issue discussion and PRs and let a project grow from there; you can take a look at [this list of issues](https://github.com/FluxML/Flux.jl/issues?q=is%3Aopen+is%3Aissue+label%3A%22help+wanted%22) for some starter contributions. Please see the [contributing guide](https://github.com/FluxML/Flux.jl/blob/master/CONTRIBUTING.md) for help first.
 
-## Metalhead.jl Developement
+# FluxML GSoC 2023 Ideas List
 
-**Difficulty:** Medium (175h)
+## Writing Julia-native kernels for common NN operations
 
-**Expected outcomes:** Help us improve [Metalhead.jl](https://github.com/FluxML/Metalhead.jl) by
-@@tight-list
-- adding new models
-- porting pre-trained weights
-- extending the model interfaces to make them more customizable
-@@
+Implement optimized kernels for common neural network operations for which we don't already have Julia-native implementations. This project will require experience with GPU kernel writing and performance optimizations.
 
-**Skills:** Familiarity with vision model architectures and Flux.jl
+**Difficulty.** Hard. **Duration.** 350 hours
 
-**Mentors:** [Kyle Daruwalla](https://github.com/darsnack)
+### Description
 
-## FastAI.jl Time Series Development
+Many ML frameworks are making the move away from vendor-specific libraries (like CUBLAS, CUDNN, etc.) towards more generic, JIT-compiled implementations of ML-related kernels, like BLAS, softmax, ReLU, etc. The reasons for this move are many-fold:
+- Vendor-provided libraries often only work on that vendor's hardware and software
+- These libraries only support certain element types, tensor shapes/sizes, and limited array view/stride/transpose support
+- These libraries often expect to be executed from the host, without a device-side launchable equivalent
+- These libraries have unreliable build systems or are binary blobs
 
-**Difficulty:** Medium (350h)
+Improving this state of affairs for Flux will involve using Julia's existing GPU and compute kernel libraries (e.g KernelAbstractions.jl) to implement various accelerated, cross-vendor routines. These kernels should be both composable and performance competitive with Flux's current generic code paths. Examples of routines specifically useful for implementing Neural Networks include:
 
-In this project, you will assist the [ML community team](https://julialang.zulipchat.com/#narrow/stream/237432-ml-ecosystem-coordination) with building time series methods for FastAI.jl on top of the existing JuliaML + FluxML ecosystem packages. Some familiarity with the following Julia packages is preferred, but it is not required:
+- GEMM and GEMV
+- Softmax
+- Batchnorm and Layernorm
+- ReLU
+- Convolution/correlation
 
-@@tight-list
-* [MLDataPattern.jl](https://github.com/JuliaML/MLDataPattern.jl.git)
-* [FluxTraining.jl](https://github.com/lorenzoh/FluxTraining.jl.git)
-* [DataAugmentation.jl](https://github.com/lorenzoh/DataAugmentation.jl)
-@@
+The ideal candidate should have experience with what operations are used in popular ML models and how they are commonly implemented on GPU. This includes experience writing and benchmarking high performance GPU kernels. Because kernels will be required for both training and inference, an understanding of automatic differentiation (AD) is also highly recommended.
 
-**Expected outcomes:** You will
-@@tight-list
-- load a working time series dataset using the FastAI.jl data registry
-- create new block methods for time series tasks
-- load at least one working time series model into a learner
-- develop an example tutorial that ties all the previous steps together
-@@
+**Mentors.** [Julian Samaroo](https://github.com/jpsamaroo), [Kyle Daruwalla](https://github.com/darsnack), [Brian Chen](https://github.com/ToucheSir)
 
-**Skills:** Familiarity with deep learning pipelines, common practices, Flux.jl, and recurrent neural networks
+### Prerequisites
 
-**Mentors:** [Lorenz Ohly](https://github.com/lorenzoh), [Kyle Daruwalla](https://github.com/darsnack), [Brian Chen](https://github.com/ToucheSir)
+- Julia language fluency is essential.
+- Experience with low-level GPU kernel programming is strongly recommended.
+- Experience with common primitive machine learning ops (forwards and backwards passes) and their interaction is recommended.
+- Familiarity with existing prior art such as [tiny-cuda-nn](https://github.com/NVlabs/tiny-cuda-nn) is preferred.
 
-## FastAI.jl Text Development
+### Your contributions
 
-**Difficulty:** Medium (350h)
+- A new package containing the optimized kernels and any supporting code for integration into Flux/Flux's operation library [NNlib.jl](https://github.com/FluxML/NNlib.jl).
+- Tests on CI and a simple benchmark harness for the new NN kernel library.
+- A proof-of-concept example showing the kernels being used with kernel fusion on device (GPU).
 
-In this project, you will assist the [ML community team](https://julialang.zulipchat.com/#narrow/stream/237432-ml-ecosystem-coordination) with building text methods for FastAI.jl on top of the existing JuliaML + FluxML ecosystem packages. Some familiarity with the following Julia packages is preferred, but it is not required:
 
-@@tight-list
-* [MLDataPattern.jl](https://github.com/JuliaML/MLDataPattern.jl.git)
-* [FluxTraining.jl](https://github.com/lorenzoh/FluxTraining.jl.git)
-* [JuliaText](https://github.com/JuliaText)
-@@
+## Benchmark tooling for common models and operations
 
-**Expected outcomes:** You will
-@@tight-list
-- load a working text dataset using the FastAI.jl data registry
-- create new block methods for textual tasks
-- load at least one working text model into a learner
-- develop an example tutorial that ties all the previous steps together
-@@
+Create a benchmarking tool for the FluxML ecosystem that we can invoke on demand from PRs. This project will require previous experience with training machine learning models at a "low-level" (i.e. without the use of tools like PyTorch Lightning).
 
-**Skills:** Familiarity with deep learning pipelines, common practices, Flux.jl, and JuliaText
+**Difficulty.** Moderate. **Duration.** 350 hours
 
-**Mentors:** [Lorenz Ohly](https://github.com/lorenzoh), [Kyle Daruwalla](https://github.com/darsnack), [Brian Chen](https://github.com/ToucheSir)
+### Description
 
-## Differentiable Computer Vision
+FluxML's machine learning stack is distributed across many packages, and each package is designed to function as independently as possible. This is done to maximize code-reuse across the Julia ML ecosystem. As a result, it is challenging for us to quantitively determine the performance impact of code changes without manual testing. The goal of this project is to develop a FluxML-specific benchmarking package. The package should allow us to install specific commits of various packages across the FluxML stack, then run a benchmarking suite. The test suite will include low-level operations like convolution or simple gradient calls, as well as complete end-to-end examples like a full forward/backward pass of Metalhead.jl models. 
 
-**Difficulty:** Hard (350h)
+The ideal candidate should have experience with multiple ML task setups such as vision, autoregressive language modeling, time series forecasting, etc. Furthermore, some experience with Github Actions and continuous integration (CI) is suggested.
 
-Create a library of utility functions that can consume Julia's imaging libraries to make them differentiable. With Zygote.jl, we have the platform to take a general purpose package and apply automatic differentiation to it.
+**Mentors.** [Brian Chen](https://github.com/ToucheSir), [Kyle Daruwalla](https://github.com/darsnack)
 
-**Expected outcomes:** You will
-@@tight-list
-- write AD rules for functions in existing computer vision libraries
-- demonstrate the use of these newly differentiable libraries for tasks such as homography regression
-@@
+### Prerequisites
 
-**Skills:** Familiarity with automatic differentiation, deep learning, and defining (a lot of) custom adjoints
+- Julia language fluency is essential.
+- Github CI and experience with GH Actions is strongly suggested.
+- Experience with more than one ML task
+  (e.g. image classification, autoregressive language modeling, etc.).
+- Familiarity with prior art is preferred:
+    - [BenchmarkCI.jl](https://github.com/tkf/BenchmarkCI.jl)
+    - [JuliaGPU speed center](https://speed.juliagpu.org)
+    - [TaylorDiff benchmarking site](https://benchmark.tansongchen.com/TaylorDiff.jl)
 
-**Mentors:** [Dhairya Gandhi](https://github.com/DhairyaLGandhi/)
+### Your contributions
 
-## FermiNets: Generative Synthesis for Automating the Choice of Neural Architectures
-
-**Difficulty:** Hard (175h)
-
-The application of machine learning requires a practitioner to understand how to optimize a neural architecture for a given problem, or does it? Recently, techniques in automated machine learning, also known as AutoML, have dropped this requirement by allowing for good architectures to be found automatically. One such method is the [FermiNet](https://arxiv.org/abs/1809.05989), which employs generative synthesis to give a neural architecture which respects certain operational requirements.
-
-**Expected outcomes:** The goal of this project is to implement the FermiNet in Flux to allow for automated synthesis of neural networks.
-
-**Mentors:** [Chris Rackauckas](https://github.com/ChrisRackauckas) and [Dhairya Gandhi](https://github.com/DhairyaLGandhi/)
-
-## Differentiable Rendering
-
-**Difficulty:** Hard (350h+)
-
-We have an existing package, [RayTracer.jl](https://github.com/avik-pal/RayTracer.jl), which is motivated by OpenDR and exists to do differentiable raytracing with Flux.jl and Zygote.jl.
-
-**Expected outcomes:** You will
-@@tight-list
-- implement at least 2 alternative rendering models like NeRF, VolSDF, Neural Raytracing, etc.
-- make improvements to RayTracer.jl to use the latest Flux libraries
-- update RayTracer.jl for ChainRules.jl
-@@
-
-**Skills:** GPU programming, deep learning, familiarity with the literature, familiarity with defining custom adjoints
-
-**Mentors:** [Dhairya Gandhi](https://github.com/DhairyaLGandhi/), [Avik Pal](https://github.com/avik-pal), [Julian Samaroo](https://github.com/jpsamaroo)
+- A new FluxML package, FluxBenchmarks.jl, that will perform configurable benchmarking across our ML stack.
+- Github Actions integration for FluxBenchmarks.jl to invoke the tool from PRs.
+- A benchmarking suite that will build your experience with different types of ML models and operations across the stack.
diff --git a/tutorialposts/2021-02-07-convnet.md b/tutorialposts/2021-02-07-convnet.md
@@ -3,4 +3,4 @@ title = "Simple ConvNet"
 published = "7 February 2021"
 author = "Elliot Saba, Adarsh Kumar, Mike J Innes, Dhairya Gandhi, Sudhanshu Agrawal, Sambit Kumar Dash, fps.io, Carlo Lucibello, Andrew Dinhobl, Liliana Badillo"
 external = "http://fluxml.ai/Flux.jl/stable/tutorials/2021-02-07-convnet/"
-+++
++++