Anton Maximilian Schäfer

Recurrent neural networks are universal approximators

Recurrent Neural Networks (RNN) have been developed for a better understanding and analysis of open dynamical systems. Still the question often arises if RNN are able to map every open dynamical system, which would be desirable for a broad spectrum of applications. In this article we give a proof for the universal approximation ability of RNN in state space model form and even extend it to Error Correction and Normalized Recurrent Neural Networks.

Recurrent Neural Networks are universal approximators.

Recurrent Neural Networks (RNN) have been developed for a better understanding and analysis of open dynamical systems. Still the question often arises if RNN are able to map every open dynamical system, which would be desirable for a broad spectrum of applications. In this article we give a proof for the universal approximation ability of RNN in state space model form and even extend it to Error Correction and Normalized Recurrent Neural Networks.

Learning long term dependencies with recurrent neural networks

Recurrent neural networks (RNNs) unfolded in time are in theory able to map any open dynamical system. Still they are often blamed to be unable to identify long-term dependencies in the data. Especially when they are trained with backpropagation through time (BPTT) it is claimed that RNNs unfolded in time fail to learn inter-temporal influences more than ten time steps apart. This paper provides a disproof of this often cited statement. We show that RNNs and especially normalised recurrent neural networks (NRNNs) unfolded in time are indeed very capable of learning time lags of at least a hundred time steps. We further demonstrate that the problem of a vanishing gradient does not apply to these networks.

A technical trading indicator based on dynamical consistent neural networks

In econometrics, the behaviour of financial markets is described by quantitative variables. Mathematical and statistical methods are used to explore economic relationships and to forecast the future market development. However, econometric modeling is often limited to a single financial market. In the age of globalisation financial markets are highly interrelated and thus, single market analyses are misleading. In this paper we present a new way to model the dynamics of coherent financial markets. Our approach is based on so-called dynamical consistent neural networks (DCNN), which are able to map multiple scales and different sub-dynamics of the coherent market movement. Unlikely to standard econometric methods, small market movements are not treated as noise but as valuable market information. We apply the DCNN to forecast monthly movements of major foreign exchange (FX) rates. Based on the DCNN forecasts we develop a technical trading indicator to support investment decisions.