The Effects of the Memory Capacity of a Recurrent Neural Network for Time Series Prediction

FULL TEXT in - | page 19 - 25

Abstract

The memory capacity of a recurrent network is the ability of a recurrent neural network to go back into the past and how far into the past the network can go to obtain input history to use to predict future output when performing a task. This study investigates the empirical and theoretical memory capacity of a fully connected and parametrized recurrent neural network. The rationale for doing this is to determine if the memory capacity of a recurrent neural network plays any significant role in the performance of the network. The data used in this study is the Santa Fe Laser dataset and the error function used is the Mean Square Error (MSE) function. The MSE at each epoch was calculated for the network before and after it was trained. A statistical analysis method known as the T-test was used to determine if there was a significant difference between the mean memory capacity of the network before and after it was trained. The main result of the study is that the theoretical memory capacity of a fully connected recurrent neural network does not in any way influence the performance of the network.

Keywords: Artificial neural network, Memory capacity, Recurrent neural network, Time series predict,

References