Forecasting Menu [Work in Progress]

[martinb-ai]

Description

• Forecasting menu option
• Forecasting Controller/model/view
• Show dataset
• Plot dataset
• Combine dataset
• Delete dataset column
• Rename dataset column
• Load dataset
• Probabilistic Exponential Smoothing model
• Theta model
• Probabilistic Recurrent Neural Network (RNN, LSTM, GRU)
• Block Recurrent Neural Network (RNN, LSTM, GRU) (feat. Past covariates)
• Neural Bayesian Estimation (feat. Past covariates)
• Temporal Convolutional Neural Network (feat. Past covariates)
• Dataset cleaning
• Dataset descriptive statistics
• Feature engineering menu option
• EMA
• Stochastic Oscillator
• RSI
• ROC
• Momentum
• % Delta change
• Average true range
• simple moving average short/long window
• price signal
• Refresh loaded dataset on menu
• Regression model (@colin99d )
• Transformer model (@colin99d - not temporal)
• Temporal Fusion transformer (performs weirdly)
• Display user compute device (CPU vs. GPU)
• Display user torch and darts version
• Port over Arima (non torch) (@colin99d ) (Hide for now)
• Port over knn (non torch) (@colin99d )
• Port over MonteCarlo (non torch) (@colin99d)
• Linear Regression (@colin99d )
• Plot seasonality of data (@colin99d this technically "works" but graph looks terrible, reached out to @piiq )
  • Going to use our own version of their graphs for now, in the future might work with them to add functionality.
  • Also, submitted a PR to darts: Update coloring for graphs unit8co/darts#1056
• Plot residuals from best_model (flag to set before training) (plot_residuals_analysis()) (@colin99d) --> Theta model does not work - make theta just print out that its not available. Also, theme needs a few tweaks. (@colin99d )
• Plot dataset feature correlation coefficients
• Pytest - Just make it run and successfully train (@colin99d )
• Base class/ refactoring (@colin99d + @martinb-bb ) - 69.420% done for now...
• Get rid of this printout:
  [2022-06-16 14:43:03,449] WARNING | darts.models | Support for AutoARIMA, BATS and TBATS is not available.To enable it, install u8darts[pmdarima] or u8darts[all]. [2022-06-16 14:43:03,450] WARNING | darts.models | Support for Facebook Prophet is not available. To enable it, install "darts", "u8darts[prophet]" or "u8darts[all]" (with pip);or "u8darts-all" (with conda). (fixed with PR Opt-out of darts logging based on environment variable unit8co/darts#1010)
• Remove external_axis from the docstrings or use the argument(@colin99d)
• Glossary (@colin99d) (dict)
• Confirm features work with Polygon and cryptodata (@colin99d)
  • For this PR we support daily, weekly, and monthly
  • [BUG]Business Hourly Reporting unit8co/darts#1057
• Check if ram requirements match data size
• plot only historical forecast (not entire series) (@colin99d)
• Develop API (@colin99d)
• Remove pred menus from the terminal (@colin99d )
• Remove Tensorflow requirements (@colin99d )
• Allow to bring any data from another menu to forecasting directly stocks/load AAPL/forecast/USE MODEL - works for both stocks and crypto menu. - to add economy soon or in V2
• grab exported data from EXPORT folder as well (@colin99d)
• standarize all models/views for ease of use in API
• load mutliple stocks from stocks menu
• Give combine warnings if it needs to use index
• Add --all-covariates flag
• Naive Bayes (for setting baseline)
• If someone doesn't specify combine columns combine all columns
• Rename menu to forecast
• Make the disclaimer when we boot up terminal
• Handle no darts gracefully
• Fix hard-coded export path (this looks like a Darren specific issue)
• Do not format non dollar amounts as dollar amounts
• Standardize names in columns (default string for ticker)
• Have a better way of handling target column not working

Other Bugs Fixed:

• Crypto export command does nothing
• Fixes [Bug] Installer v1.6 - Crypto/chart: Error: the command 'chart' doesn't exist on the /crypto/ menu. #2128
• Fixes [Bug] All Prediction features: When a non-price value is the target column, the table output is incorrectly assumed as a price. #2085
• Fixes [FR] Neural prediction ignores end date (no backtesting) #1088
• Fixes [IMPROVE] Improve error message when openbb.stocks.pred. is used #1778
• Fixes [FR] Prediction Models Accuracy or F-Score metrics? #346

Checklist:

• Update our Hugo documentation following these guidelines.
• Update our tests following these guidelines.
• Make sure you are following our CONTRIBUTING guidelines.
• If a feature was added make sure to add it to the corresponding scripts file.
• One letter parameters for common args
• Handle regular days included with Business days when building timeseries
• make disclaimer for loading menu
• remove default csvs (@colin99d)
• Notebooks with usage examples

Known issues

• rnn residuals with aapl.csv does not work - fixed: Feat/improve zero length datasets unit8co/darts#1066 to be added in 0.21.0
• reset command (@colin99d )
• Make reset work when loading from another menu (im making this out of scope for now, if anyone legitimately wants this feature we can add in V2)
• positive series with MAPE bug ([BUG]Neural Network Models Throwing ValueError: The actual series must be strictly positive to compute the MAPE. unit8co/darts#1063) - fixed
• linregr and regr prints a lot of FutureWarning: pandas.Int64Index is deprecated warnings
• Plot shows numbers instead of dates for X axis
• Big dataframes print in an ugly fashion

V2 Features:

• Bring in Darts Datasets for further user experimentation
• Plot data in different ways (bar chart, box and whisker, 3D???)
• multi input and 2 y axis plotting ( economy menu)
• Static covariates
• stochastic forecasts capturing model uncertainty using Monte Carlo Dropout with deep learning models
• Load previously trained models for inference
• Train OpenBB NN model for off-the-shelf forecasting ( Long term goal))
• past covariates with different delta T
• Create a dashboard
• encoding string variables in columns
• hyperparameter tuning with Raytune: https://docs.ray.io/en/latest/tune/examples/tune-pytorch-lightning.html
• Do a better job at merging dataframes with different times (Polygon data can be 4 hours off)
• Refactor the views to use all the same code
• More descriptive feature engineering names (running the same feat eng on different columns replaces old values)

Others

• I have performed a self-review of my own code.
• I have commented my code, particularly in hard-to-understand areas.
• My code passes all the checks pylint, flake8, black, ... To speed up development you should run pre-commit install.
• New and existing unit tests pass locally with my changes. You can test this locally using pytest tests/....

To use in Jupyter:
from openbb_terminal.api import forecast as fc
from openbb_terminal import api

import matplotlib.pyplot as plt
import matplotlib_inline.backend_inline

%matplotlib inline
matplotlib_inline.backend_inline.set_matplotlib_formats("svg")
api.theme.applyMPLstyle()

[DidierRLopes]

Someone has been avoiding sleep 👀

[jmaslek]

This is looking awesome. Small point I'm noticing. For controllers/argparse choices, y'all are doing --some_thing.

We should stick to doing --some-thing

[martinb-ai]

Gotcha, will update!

[piiq]

We have predict menu leftovers here and there, is it supposed to be like that?

[piiq]

LFGO 🚀

[jmaslek]

I want to join in on the fun