The Value of LSTM in Time Series Forecasting

LSTM is an artificial recurrent neural network used in deep learning and can process entire sequences of data. Due to the model’s ability to learn long term sequences of observations, LSTM has become a trending approach to time series forecasting.

The emergence and popularity of LSTM has created a lot of buzz around best practices, processes and more. Below we review LSTM and provide guiding principles that PredictHQ’s data science team has learned.

A review of LSTM and recurrent neural networks

Typically recurrent neural networks (RNN) have short term memory in that they use persistent previous information to be used in the current neural network. Typical recurrent neural networks can experience a loss in information, often referred to as the vanishing gradient problem. This is caused by the repeated use of the recurrent weight matrix in RNN. In an LSTM model, the recurrent weight matrix is replaced by an identify function in the carousel and controlled by a series of gates. The input gate, output gate and forget gate acts like a switch that controls the weights and creates the long term memory function.

Experts discuss LSTM models for time series 

In today’s environment, demand forecasting is complex and the data needed for accurately forecasting at scale isn’t always straightforward. Using LSTM, time series forecasting models can predict future values based on previous, sequential data. This provides greater accuracy for demand forecasters which results in better decision making for the business.

In order to develop demand intelligence and shape it into forecast-grade data that can be used to train prediction models, PredictHQ has a dedicated data science team that has provided the following LSTM learnings.

1. Demand data sets are featured with different seasonalities. The LSTM is capable of capturing the patterns of both long term seasonalities such as a yearly pattern and  short term seasonalities such as weekly patterns.

2. It is natural that events would impact demand on the day when it is happening as well as the days before and after the event is happening. For example, people would book more days of accommodation in order to attend a sports event. The LSTM has the ability to triage the impact patterns from different categories of events.

3. The LSTM could take inputs with different lengths. This feature is especially useful when LSTM is used to build general forecasting models for specific customers or industries. 

4. The different gates inside LSTM boost its capability for capturing non linear relationships for forecasting. Causal factors generally have non-linear impact on demand. When these factors are used as part of the input variable, the LSTM could learn the nonlinear relationship for forecasting.

5. The LSTM requires more computation than other recurrent neural networks. The main reason is that it has more parameters, which are used for demand forecasts. The computation issue is alleviated by cuDNN from NVIDIA. From our experience, cuDNN could introduce 10+ times the speed than the default setting of using CUDA directly.

Other models that can be used to forecast time series data

Aside from LSTM, Autoregressive Integrated Moving Average (ARIMA) and Facebook Prophet are two other popular models that are used for time series forecasting.

ARIMA

ARIMA is a popular statistical method used in time series forecasting to predict future trends for time series data. It is a class of models that explains time series data based on its past values. Adopting ARIMA for time series assumes information in the past can alone be used to predict future values. 

Facebook Prophet 

Prophet is a procedure for forecasting time series data based on an additive model where non-linear trends are used. It works best with time series data that has strong seasonal effects.