Is there a tutorial of using mxnet R LSTM for time series forecasting?

[lichen11]

Can you provide some examples of using mxnet R for (multivariate) time series forecasting? I found this resource https://www.r-bloggers.com/recurrent-models-and-examples-with-mxnetr/. But it is predicting text data. I would like to know how to do it on numeric data. Are you also planning to develop some high-level API for time series forecasting using LSTM?

[jeremiedb]

There's a pull request #7476 meant to facilitate training of various flavours of RNN and support iterators feeding various sequence lengths.

Current design is more adapted to text as it assumes that data feed is of dimension : seq.length X batch.size, to which embedding is applied. If you take a look at the demo of the new functionalities here, you can see that it would be quite simple to adapt the model graph by omitting the embedding operator and instead have an iterator that feed data of shape features.number X batch.size X sequence.length.

By working with the mx.symbol.RNN operator, the modeling approach is essentially the same as for the feedforwardmode: just need to build the symbol graph, for which the rnn.graph is a helper function and then train using mx.rnn.buckets. Caveat: for now only GPU is supported.

Your feedback on whether you think these functionalities would be helpful are most welcome.

[chaitanyabjoshi]

Unfortunately, poor documentation of mxnetR library is causing a lot of trouble to many developers. After spending a lot of time to understand the API it is known only through list of open issues that current LSTM functionality is only for textual data (#7446) . A simple line in documentation description would save days!

[jeremiedb]

The refactor RNN API has now been merged and the demo on its usage updated.

rnn.graph should only be seen as an helper for building the graph and hopefully provides a useful reference on how to assemble the various building blocks to adapt the model to other specific needs. Not sure it's realistic to aim for an universal graph builder for any particular problem. Though if there are common use cases that are not yet covered, please open a request and will try to incorporate it.

Agree for the need on improved documentation of the package. Suggestions welcome!

[alexmosc]

It would still be very helpful to have a tutorial on a simple time-series modelling with MXNET in R.

[jeremiedb]

As a starting point you can take a look at this rough demo for an LSTM on a sin waves: https://github.com/jeremiedb/mxnet_R_bucketing/blob/master/TimeSeries.md.

I think it would be better to switch the assumed data structure to features X seq_length X batch_size despite the RNN symbol format since CSV and arrayiter are all assuming that batches are on the last dimension, I'll try to have this fix rapidly.

[btcpbordeaux]

Looking at time series tutorial and using same code, the following error is being thrown:

Error in mx.varg.symbol.FullyConnected(list(...)) :
Cannot find argument 'flatten', Possible Arguments:

num_hidden : int, required
Number of hidden nodes of the output.
no_bias : boolean, optional, default=False
Whether to disable bias parameter.
, in operator FullyConnected(name="", num_hidden="1", flatten="1")

from rnn.graph.unroll (rnn.graph.R line 336).

[jeremiedb]

The code needs to be run on 0.12 or higher, flatten option wasn't present prior. If not possible to upgrade, you can simply remove the flatten option in the graph and add a flatten operator (if the default FC behavior isn't to flatten).

[btcpbordeaux]

Ahh gotcha, thanks a lot!

[alexmosc]

@jeremiedb
The link you have provided returns 404 error. Could you link to an existing page please?

[jeremiedb]

Sorry about that, did some updates on the examples. These should be cleaner: https://jeremiedb.github.io/mxnet_R_bucketing/TimeSeries_CPU

[alexmosc]

@jeremiedb :
Thank you. This is a master code! However, do you mind if I point to a somewhat strange thing: in your prediction analysis, the predicted timeseries is very much just a lagged version of the real timeseries. It may be due to insufficient training, but in general this is indicative of either the data problem or model problem. So the model learned to copy last observed value as a prediction, which is very a poor quality. What do you say, something wrong with model or not?

[jeremiedb]

Thanks for pointing this out. I took a second look and I believe the code is correct. There hasn't been an extensive parameter tuning of the model and making predictions on a long future horizon can easily result in errors build up (during training, the previous real label is provided).
Not impossible there's a bug though, please advise if you see one.

[alexmosc]

@jeremiedb , I will try to run the code myself. If I will understand where the source of bad modelling is I will post here. Thank you!

[alexmosc]

@jeremiedb, hello. I got some time to try your LSTM code. Could you please advise whuch R mxnet version I must get on my Windows PC to run this code? I cannot locate the rnn.graph.unroll functions in my current version (0.10.1).

[jeremiedb]

Hello, it should be > 1.0. There's unfortunately no such version pre-package right now and OpenCV gave me issues building on Windows.
Simplest would be to run on Linux, or you could run a docker image. Here again the official image hasn't bee nupdate since 0.11 (https://hub.docker.com/r/mxnet/r-lang/), though I've just refresh an image with current master here: https://hub.docker.com/r/jeremiedb/fullkit/, which comes with RStudio, Opencpu, jupyter and couple other packages.

[ankkhedia]

@lichen11
Please find the example added here
https://github.com/apache/incubator-mxnet/blob/master/docs/tutorials/r/MultidimLstm.md

@nswamy @sandeep-krishnamurthy Could you please close this issue as the example has been merged.