XGboost regression model (and probabilistic forecasting faisability)

[dumjax]

• Make sure to support probabilistic forecasts supported by xgboost
• it seems XGBoost also supports multi-outputs natively since version 1.6.2 - we should make sure we leverage this.

[yepher]

Do you have any idea when XGBoost will become available in Darts?

[hrzn]

No estimate yet, although it's fairly high on our backlog. Would you be interested in contributing this model @yepher ?

[yepher]

I am not familiar with the internals of Darts. But could be a fun project. I will take a look to see if it is something I could accomplish.

[hrzn]

Cool, a good place to start could be to try to replicate what has been done for LGBM here, for XGBoost. In particular it would be important to try to provide access to probabilistic estimates of XGBoost, if any (e.g. LGBM provides quantiles estimates, maybe XGBoost too?), and also leverage the multi-output capabilities.

You can have a look and let us know if you want to do it, and/or if you have questions :)

[yepher]

Here in gradient_boosted_model.py#L7 the code says:

...To enable LightGBM support in Darts, follow the detailed install instructions for LightGBM in the README...

But when I look in the README.m I do not see any mention of how to install LightGGM

I think it meant to say, see INSTALL.md

[beckernick]

Would love to see XGBoost in darts! On the topic of multi-output regression, the documentation notes that multi-output has "limited support from objectives and metrics".

If there are certain objective functions that are important for darts but not currently available, we'd love to learn more. cc @trivialfis @dantegd

[trivialfis]

Hi, I help maintain the XGBoost project and developed the initial support for multi-output. Please let me know if I can be useful here. Would love the see XGBoost being part of darts.

[hrzn]

Hi @beckernick @trivialfis, I have started some work to include XGBoost in Darts. I haven't yet played with multi-output regression but this is not super duper important: we are wrapping several models in a MultiOutputRegressor by default so things will work as intended even on multivariate series or for multi-step forecasting.

One thing that I think would be very valuable, though, would be to have a way to do quantile regression, e.g., by having an objective allowing for quantile loss (sometimes called pinball loss). I haven't found anything like it in the documentation. Do you know if xgboost has a feature along those lines that I maybe missed?
Thanks!

[hrzn]

Hi @beckernick @trivialfis, I have started some work to include XGBoost in Darts. I haven't yet played with multi-output regression but this is not super duper important: we are wrapping several models in a MultiOutputRegressor by default so things will work as intended even on multivariate series or for multi-step forecasting.

One thing that I think would be very valuable, though, would be to have a way to do quantile regression, e.g., by having an objective allowing for quantile loss (sometimes called pinball loss). I haven't found anything like it in the documentation. Do you know if xgboost has a feature along those lines that I maybe missed? Thanks!

I think a solution could be to specify a custom objective with a function implementing the quantile loss (such as this one). It'd still be nicer if XGBoost supported quantile regression out of the box of course :)

[trivialfis]

We are planning for quantile regression support. Stay tuned. ;-)