Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] ForecastingModel.backtest: Can bypass sanity checks #188

Closed
LeoTafti opened this issue Sep 18, 2020 · 1 comment · Fixed by #189
Closed

[BUG] ForecastingModel.backtest: Can bypass sanity checks #188

LeoTafti opened this issue Sep 18, 2020 · 1 comment · Fixed by #189
Labels
bug Something isn't working

Comments

@LeoTafti
Copy link
Contributor

Describe the bug
Invalid optional arguments to ForecastingModel.backtest can bypass sanity checks when provided as positional arguments

To Reproduce

NaiveDrift().backtest(linear_series, None, start=0.7, forecast_horizon=-1)
NaiveDrift().backtest(linear_series, None, 0.7, -1)

Expected behavior
The first line above is caught by the sanity checks and raise ValueError: 'The provided forecasting horizon must be a positive integer.' as expected

For the second line however the negative forecast_horizon param clears the sanity checks and produces an error further down the line:

NaiveDrift().backtest(linear_series, None, 0.7, -1)
  File ".../darts/utils/utils.py", line 120, in sanitized_method
    return method_to_sanitize(self, *args, **kwargs)
  File ".../darts/models/forecasting_model.py", line 217, in backtest
    pred = self.predict(forecast_horizon, **predict_kwargs)
  File ".../darts/models/baselines.py", line 98, in predict
    forecast = np.linspace(last, last_value, num=n)
  File "<__array_function__ internals>", line 5, in linspace
  File ".../lib/python3.8/site-packages/numpy/core/function_base.py", line 115, in linspace
    raise ValueError("Number of samples, %s, must be non-negative." % num)
ValueError: Number of samples, -1, must be non-negative.

System (please complete the following information):

  • Python version: 3.8.5
  • darts version: develop branch, commit #2977f4f
@LeoTafti LeoTafti added bug Something isn't working triage Issue waiting for triaging and removed triage Issue waiting for triaging labels Sep 18, 2020
@pennfranc
Copy link
Contributor

Good catch! That's definitely a drawback of the current sanity checks implementation. It mostly relies on arguments being passed as keywords right now.. Do you think this could be a possible solution: https://stackoverflow.com/a/831164?

@LeoTafti LeoTafti linked a pull request Sep 22, 2020 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants