Skip to content

Latest commit

 

History

History
171 lines (116 loc) · 6.81 KB

ROADMAP.md

File metadata and controls

171 lines (116 loc) · 6.81 KB

Road map

February 2020; updated, December 2024

The road map below is more big-picture; to get into the weeds, see also this GH Project.

Please visit contributing guidelines if interested in contributing to MLJ.

Goals

  • Usability, interoperability, extensibility, reproducibility, and code transparency.

  • Offer state-of-art tools for model composition and model optimization (hyper-parameter tuning)

  • Avoid common pain-points of other frameworks with MLJ:

    • identify and list all models that solve a given task

    • easily perform routine operations requiring a lot of code

    • easily transform data, from source to algorithm-specific data format

    • make use of probabilistic predictions: no more inconsistent representations / lack of options for performance evaluation

  • Add some focus to julia machine learning software development more generally

Priorities

Priorities are somewhat fluid, depending on funding offers and available talent. However, we are always keen to review external contributions in any area.

Future enhancements

Adding models

  • Integrate deep learning using Flux.jl deep learning. Done but can improve the experience by:

  • Probabilistic programming: Turing.jl, Gen, Soss.jl #157 discourse thread done but experimental and requires:

    • extension of probabilistic scoring functions to "distributions" that can only be sampled.

  • Feature engineering (python featuretools?, recursive feature elimination ✓ done in FeatureSelection.jl :) #426 MLJModels #314

Enhancing core functionality

  • † Add more tuning strategies. See here for complete wish-list. Particular focus on:

    • Bayesian methods, starting with Gaussian Process methods a la PyMC3. Some preliminary research done.

    • POC for AD-powered gradient descent #74

    • Tuning with adaptive resource allocation, as in Hyperband. This might be implemented elegantly with the help of the recent IterativeModel wrapper, which applies, in particular to TunedModel instances see here.

    • Genetic algorithms #38

    • tuning strategies for non-Cartesian spaces of models MLJTuning #18, architecture search, and other AutoML workflows

  • Systematic benchmarking, probably modeled on MLaut #69

  • Give EnsembleModel a more extendible API and extend beyond bagging (boosting, etc) and migrate to a separate repository? #363

Broadening scope

  • Integrate causal and counterfactual methods for example, applications to FAIRness; see this proposal

  • Explore the possibility of closer integration of Interpretable Machine Learning approaches, such as Shapley values and lime; see Shapley.jl, ShapML.jl, ShapleyValues.jl, Shapley.jl (older) and this proposal

  • Add sparse data support and better support for NLP models; we could use NaiveBayes.jl as a POC (currently wrapped only for dense input) but the API needs to be finalized first {#731](#731). Probably need a new SparseTables.jl package.

  • POC for implementation of time series models classification #303, ScientificTypesBase #14 POC is here

  • POC for time series forecasting, along lines of sktime; probably needs MLJBase #502 first, and someone to finish PR on time series CV. See also this proposal

  • Add tools or a separate repository for visualization in MLJ.

    • Extend visualization of tuning plots beyond two-parameters #85 (closed). #416 Done but might be worth adding alternatives suggested in issue.

    • visualizing decision boundaries? #342

    • provide visualizations that MLR3 provides via mlr3viz

  • Add more pre-processing tools:

    • missing value imputation using Gaussian Mixture Model. Done, via addition of BetaML model, MissingImputator.

    • improve autotype method (from ScientificTypes), perhaps by training on a large collection of datasets with manually labelled scitype schema.

  • Extend integration with OpenML WIP @darenasc

Scalability

  • Roll out data front-ends for all models after MLJBase #501 is merged.

  • Online learning support and distributed data #60

  • DAG scheduling for learning network training #72 (multithreading first?)

  • Automated estimates of cpu/memory requirements #71