Forecasting Menu [Work in Progress]

.gitignore

@@ -59,3 +59,10 @@ styles/user
 
 # the file containing the API keys for the web version
 openbb_terminal_web/setenv
+
+# dart models
+darts_logs/
+
+# User data
+custom_imports/*.csv
+custom_imports/*/*.csv

build/conda/conda-3-9-env-full.yaml

@@ -0,0 +1,16 @@
+channels:
+  - conda-forge
+  - defaults
+dependencies:
+  - pytorch-lightning=1.6.5
+  - u8darts[torch]=0.21.0 
+  - python=3.9.6
+  - pip
+  - poetry=1.1.13
+  - numpy=1.23.0
+  - cvxpy=1.2.1
+  - pytest
+  - black=22.6.0
+  - sphinx=4.5.0
+  - charset-normalizer=2.1.0
+  - rich=10.16.2

build/pyinstaller/build4mac.sh

@@ -11,7 +11,17 @@ rm -rf build/terminal && rm -rf dist && rm -rf DMG
 # Clean up local logging id
 rm -rf openbb_terminal/logs
 
-pyinstaller build/pyinstaller/terminal.spec
+# Removing Sigtool package which breaks build
+echo "Removing Sigtool Package"
+if conda remove --force sigtool -y &> /dev/null
+    then
+        echo "Succesfully Removed Sigtool Package"
+else
+    echo "Sigtool Already Removed"
+fi
+
+# Running build
+pyinstaller build/pyinstaller/terminal.spec --clean
 
 # Assign icons to the built folder and launcher
 osascript build/pyinstaller/setup_icons.applescript

build/pyinstaller/hooks/hook-rich.py

@@ -0,0 +1,3 @@
+from PyInstaller.utils.hooks import copy_metadata, collect_data_files
+
+datas = copy_metadata("rich") + collect_data_files("rich")

build/pyinstaller/terminal.spec

@@ -91,6 +91,7 @@ hidden_imports = [
     "feedparser",
     "pymongo",
     "bson",
+    "_sysconfigdata__darwin_darwin",
 ]
 
 
@@ -100,7 +101,7 @@ analysis_kwargs = dict(
     binaries=[],
     datas=added_files,
     hiddenimports=hidden_imports,
-    hookspath=[],
+    hookspath=["build/pyinstaller/hooks"],
     hooksconfig={},
     runtime_hooks=[],
     excludes=[],

i18n/en.yml

@@ -78,6 +78,7 @@ en:
   funds: mutual funds search, overview, holdings and sector weights
   alternative: alternative datasets, such as COVID and open source metrics
   econometrics: statistical and quantitative methods for relationships between datasets
+  forecast: timeseries forecasting with machine learning
   portfolio: perform portfolio optimization and look at portfolio performance and attribution
   dashboards: interactive dashboards using voila and jupyter notebooks
   reports: customizable research reports through jupyter notebooks
@@ -122,7 +123,7 @@ en:
   stocks/bt: strategy backtester,          simple ema, ema cross, rsi strategies
   stocks/ta: technical analysis,           ema, macd, rsi, adx, bbands, obv
   stocks/qa: quantitative analysis,        decompose, cusum, residuals analysis
-  stocks/pred: prediction techniques,        regression, arima, rnn, lstm
+  stocks/forecast: forecast techniques,          rnn, nbeats, transformer, block rnn
   stocks/options/unu: show unusual options activity
   stocks/options/calc: basic call/put PnL calculator
   stocks/options/screen: screens tickers based on preset
@@ -473,20 +474,6 @@ en:
   stocks/qa/cusum: detects abrupt changes using cumulative sum algorithm of prices
   stocks/qa/capm: capital asset pricing model
   stocks/qa/beta: display beta w.r.t to any reference ticker
-  stocks/pred/load: load new ticker
-  stocks/pred/pick: pick target column for analysis
-  stocks/pred/_ticker: Ticker
-  stocks/pred/_target: Target Column
-  stocks/pred/_models_: Models
-  stocks/pred/ets: exponential smoothing (e.g. Holt-Winters)
-  stocks/pred/knn: k-Nearest Neighbors
-  stocks/pred/regression: polynomial regression
-  stocks/pred/arima: autoregressive integrated moving average
-  stocks/pred/mlp: MultiLayer Perceptron
-  stocks/pred/rnn: Recurrent Neural Network
-  stocks/pred/lstm: Long-Short Term Memory
-  stocks/pred/conv1d: 1D Convolutional Neural Network
-  stocks/pred/mc: Monte-Carlo simulations
   crypto/load: load a specific cryptocurrency for analysis
   crypto/find: find coins
   crypto/price: real-time price and interval of confidence
@@ -505,7 +492,7 @@ en:
   crypto/nft: non-fungible tokens,                    today drops
   crypto/dd: due-diligence for loaded coin,          coin information, social media, market stats
   crypto/ta: technical analysis for loaded coin,     ema, macd, rsi, adx, bbands, obv
-  crypto/pred: prediction techniques,                  regression, arima, rnn, lstm, conv1d, monte carlo
+  crypto/forecast: forecast techniques,                 rnn, nbeats, transformer, block rnn
   crypto/qa: quantitative analysis                   decompose, cusum, residuals analysis
   crypto/disc/cgtop: top coins (with or without category)
   crypto/disc/cgtrending: trending coins
@@ -670,20 +657,6 @@ en:
   crypto/ta/obv: on balance volume
   crypto/ta/_custom_: Custom
   crypto/ta/fib: fibonacci retracement
-  crypto/pred/load: load new ticker
-  crypto/pred/pick: pick target column for analysis
-  crypto/pred/_ticker: Coin loaded
-  crypto/pred/_target: Target column
-  crypto/pred/_models_: Models
-  crypto/pred/ets: exponential smoothing (e.g. Holt-Winters)
-  crypto/pred/knn: k-Nearest Neighbors
-  crypto/pred/regression: polynomial regression
-  crypto/pred/arima: autoregressive integrated moving average
-  crypto/pred/mlp: MultiLayer Perceptron
-  crypto/pred/rnn: Recurrent Neural Network
-  crypto/pred/lstm: Long-Short Term Memory
-  crypto/pred/conv1d: 1D Convolutional Neural Network
-  crypto/pred/mc: Monte-Carlo simulations
   crypto/qa/load: load new ticker
   crypto/qa/pick: pick target column for analysis
   crypto/qa/_ticker: Ticker
@@ -725,7 +698,6 @@ en:
   etf/pir: create (multiple) passive investor excel report(s)
   etf/compare: compare multiple different ETFs
   etf/ta: technical analysis,           ema, macd, rsi, adx, bbands, obv
-  etf/pred: prediction techniques,        regression, arima, rnn, lstm
   etf/disc/gainers: top gainers
   etf/disc/decliners: top decliners
   etf/disc/active: most active
@@ -762,20 +734,6 @@ en:
   etf/ta/obv: on balance volume
   etf/ta/_custom_: Custom
   etf/ta/fib: fibonacci retracement
-  etf/pred/load: load new ticker
-  etf/pred/pick: pick target column for analysis
-  etf/pred/_ticker: Ticker loaded
-  etf/pred/_target: Target column
-  etf/pred/_models_: Models
-  etf/pred/ets: exponential smoothing (e.g. Holt-Winters)
-  etf/pred/knn: k-Nearest Neighbors
-  etf/pred/regression: polynomial regression
-  etf/pred/arima: autoregressive integrated moving average
-  etf/pred/mlp: MultiLayer Perceptron
-  etf/pred/rnn: Recurrent Neural Network
-  etf/pred/lstm: Long-Short Term Memory
-  etf/pred/conv1d: 1D Convolutional Neural Network
-  etf/pred/mc: Monte-Carlo simulations
   economy/_database_: Databases
   economy/_stored: Stored datasets
   economy/overview: show a market overview of either indices, bonds or currencies
@@ -794,7 +752,7 @@ en:
   economy/valuation: valuation of sectors, industry, country
   economy/performance: performance of sectors, industry, country
   economy/spectrum: spectrum of sectors, industry, country
-  economy/pred: Open the prediction menu to analyse stored data
+  economy/forecast: forecast techniques; rnn, nbeats, transformer, block rnn
   economy/qa: Open quantitative analysis menu with stored data
   economy/qa/pick: pick new series from stored economy data
   economy/qa/_series: Selected series
@@ -819,18 +777,6 @@ en:
   economy/qa/raw: print raw data
   economy/qa/decompose: decomposition in cyclic-trend, season, and residuals of prices
   economy/qa/cusum: detects abrupt changes using cumulative sum algorithm of prices
-  economy/pred/pick: pick new series from stored economy data
-  economy/pred/_series: Selected series
-  economy/pred/_models_: Models
-  economy/pred/ets: exponential smoothing (e.g. Holt-Winters)
-  economy/pred/knn: k-Nearest Neighbors
-  economy/pred/regression: polynomial regression
-  economy/pred/arima: autoregressive integrated moving average
-  economy/pred/mlp: MultiLayer Perceptron
-  economy/pred/rnn: Recurrent Neural Network
-  economy/pred/lstm: Long-Short Term Memory
-  economy/pred/conv1d: 1D Convolutional Neural Network
-  economy/pred/mc: Monte-Carlo simulations
   forex/_ticker: Ticker
   forex/_from: From
   forex/_to: To
@@ -841,7 +787,6 @@ en:
   forex/fwd: get forward rates for loaded pair
   forex/ta: technical analysis,          ema, macd, rsi, adx, bbands, obv
   forex/qa: quantitative analysis,       decompose, cusum, residuals analysis
-  forex/pred: prediction techniques        regression, arima, rnn, lstm, conv1d, monte carlo
   forex/forex: Forex brokerages
   forex/oanda: Oanda menu
   forex/ta/_currency: Currency pair loaded
@@ -889,19 +834,6 @@ en:
   forex/qa/raw: print raw data
   forex/qa/decompose: decomposition in cyclic-trend, season, and residuals of prices
   forex/qa/cusum: detects abrupt changes using cumulative sum algorithm of prices
-  forex/pred/pick: pick new target variable
-  forex/pred/_pair: Pair loaded
-  forex/pred/_target: Target column
-  forex/pred/_models_: Models
-  forex/pred/ets: exponential smoothing (e.g. Holt-Winters)
-  forex/pred/knn: k-Nearest Neighbors
-  forex/pred/regression: polynomial regression
-  forex/pred/arima: autoregressive integrated moving average
-  forex/pred/mlp: MultiLayer Perceptron
-  forex/pred/rnn: Recurrent Neural Network
-  forex/pred/lstm: Long-Short Term Memory
-  forex/pred/conv1d: 1D Convolutional Neural Network
-  forex/pred/mc: Monte-Carlo simulations
   forex/oanda/summary: shows account summary
   forex/oanda/calendar: show calendar
   forex/oanda/list: list order history
@@ -1077,10 +1009,49 @@ en:
   portfolio/po/params/arg: set a different value for an argument
   portfolio/po/params/_parameters_: Parameters
   dashboards/stocks: historic stock information
+  dashboards/forecast: forecasting for timeseries datasets
+  dashboards/forecasting: forecasting for timeseries datasets
   dashboards/correlation: stock correlations
   dashboards/vsurf: options volatility surface
   dashboards/chains: options chain analysis
   dashboards/shortdata: finra shortdata analysis
   dashboards/crypto: cryptocurrency exchange rates against USD
   dashboards/futures: historical futures performance
   reports/_reports_: Select one of the following report
+  forecast/_disclaimer_: DISCLAIMER
+  forecast/_data_loc: Looking for data in
+  forecast/load: load a dataset from csv
+  forecast/_loaded: Loaded files and data columns
+  forecast/_exploration_: Exploration
+  forecast/show: Show a portion of a loaded dataset
+  forecast/plot: Plot a specific column of a loaded dataset
+  forecast/clean: Clean a dataset by filling or dropping NaNs
+  forecast/desc: Show descriptive statistics of a dataset
+  forecast/corr: Plot the correlation coefficients for dataset features
+  forecast/season: Plot the seasonality for a dataset column
+  forecast/combine: Combine columns from different datasets
+  forecast/delete: Delete columns from dataset
+  forecast/rename: Rename columns from dataset
+  forecast/export: Export a processed dataset
+  forecast/_feateng_: Feature Engineering
+  forecast/ema: Add Exponentially Weighted Moving Average
+  forecast/sto: Add Stochastic Oscillator %K and %D
+  forecast/rsi: Add Relative Strength Index
+  forecast/roc: Add Rate of Change
+  forecast/mom: Add Momentum
+  forecast/delta: Add % Change
+  forecast/atr: Add Average True Range
+  forecast/signal: Add Price Signal (short vs. long term)
+  forecast/_tsforecasting_: TimeSeries Forecasting
+  forecast/arima: Arima (Non-darts)
+  forecast/expo: Probabilistic Exponential Smoothing
+  forecast/theta: Theta Method
+  forecast/linregr: Probabilistic Linear Regression
+  forecast/regr: Regression
+  forecast/rnn: Probabilistic Recurrent Neural Network (RNN, LSTM, GRU)
+  forecast/brnn: Block Recurrent Neural Network (RNN, LSTM, GRU) (feat. Past covariates)
+  forecast/nbeats: Neural Bayesian Estimation (feat. Past covariates)
+  forecast/tcn: Temporal Convolutional Neural Network (feat. Past covariates)
+  forecast/trans: Transformer model (feat. Past covariates)
+  forecast/tft: Temporal Fusion Transformer (feat. Past covariates)
+  forecast/_comingsoon_: Coming Soon

openbb_terminal/README.md

@@ -121,10 +121,23 @@ issues.
    or `diamondhands`, we recommend something simple and intuitive like `obb`. This is because this name will be used
    from now onwards.
 
+   Please note, the following setup has been confirmed to work for all OS (including M1) with the standard
+   miniconda distribution.
+
+   `https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.sh`
+
+   If you are using a different distribution, you will need to install it manually before proceeding.
+
    ```bash
    conda env create -n obb --file build/conda/conda-3-9-env.yaml
    ```
 
+   Or, to include machine learning type:
+
+   ```bash
+   conda env create -n obb --file build/conda/conda-3-9-env-full.yaml
+   ```
+
    Note: Using python 3.10 can lead to undesirable functionality for certain commands.
 
 6. Activate the virtual environment
@@ -143,10 +156,10 @@ issues.
    poetry install
    ```
 
-   To enable the `prediction` menu install additional dependencies after installing main dependencies:
+   For machine learning instead type:
 
    ```bash
-   conda install -c conda-forge tensorflow
+   poetry install -E prediction
    ```
 
    If you are having trouble with Poetry (e.g. on a non-conda python), simply install requirements.txt with pip
@@ -189,7 +202,7 @@ before you call `python terminal.py` again.
 ## Advanced User Install - Custom installation procedures
 
 By default we advice using `conda` and `poetry` for environment setup and dependency management.
-Conda ships binaries for packages like `numpy` and `tensorflow` so these dependencies are
+Conda ships binaries for packages like `numpy` so these dependencies are
 not built from source locally by `pip`.
 Poetry solves the dependency tree in a way that the dependencies of dependencies of dependencies
 use versions that are compatible with each other.