Juan-Pablo Ortega Lahuerta

The universal approximation properties with respect to Lp-type criteria of three important families of reservoir computers with stochastic discrete-time semi-infinite inputs are shown. First, it is proved that linear reservoir systems with either polynomial or neural network readout maps are universal. More importantly, it is proved that the same property holds for two families with linear readouts, namely, trigonometric state-affine systems and echo state networks, which are the most widely used reservoir systems in applications. The linearity in the readouts is a key feature in supervised machine learning applications. It guarantees that these systems can be used in high-dimensional situations and in the presence of large datasets. The Lp criteria used in this paper allow the formulation of universality results that do not necessarily impose almost sure uniform boundedness in the inputs or the fading memory property in the filter that needs to be approximated.

This paper shows that echo state networks are universal uniform approximants in the context of discrete- time fading memory filters with uniformly bounded inputs defined on negative infinite times. This result guarantees that any fading memory input/output system in discrete time can be realized as a simple finite- dimensional neural network-type state-space model with a static linear readout map. This approximation is valid for infinite time intervals. The proof of this statement is based on fundamental results, also presented in this work, about the topological nature of the fading memory property and about reservoir computing systems generated by continuous reservoir maps.

A new class of non-homogeneous state-affine systems is introduced for use in reservoir computing. Sufficient conditions are identified that guarantee first, that the associated reservoir computers with linear readouts are causal, time-invariant, and satisfy the fading memory property and second, that a subset of this class is universal in the category of fading memory filters with stochastic almost surely uniformly bounded inputs. This means that any discrete-time filter that satisfies the fading memory property with random inputs of that type can be uniformly approximated by elements in the non-homogeneous state-affine family.

This paper investigates the weak convergence of general non-Gaussian GARCH models together with an application to the pricing of European style options determined using an extended Girsanov principle and a conditional Esscher transform as the pricing kernel candidates. Applying these changes of measure to asymmetric GARCH models sampled at increasing frequencies, we obtain two risk neutral families of processes which converge to different bivariate diffusions, which are no longer standard Hull–White stochastic volatility models. Regardless of the innovations used, the GARCH implied diffusion limit based on the Esscher transform can be obtained by applying the minimal martingale measure under the physical measure. However, we further show that for skewed GARCH driving noise, the risk neutral diffusion limit of the extended Girsanov principle exhibits a non-zero market price of volatility risk which is proportional to the market price of the equity risk, where the constant of proportionality depends on the skewness and kurtosis of the underlying distribution. Our theoretical results are further supported by numerical simulations and a calibration exercise to observed market quotes.

Reservoir computing is a recently introduced brain-inspired machine learning paradigm capable of excellent performances in the processing of empirical data. We focus in a particular kind of time-delay based reservoir computers that have been physically implemented using optical and electronic systems and have shown unprecedented data processing rates. Reservoir computing is well-known for the ease of the associated training scheme but also for the problematic sensitivity of its performance to architecture parameters. This article addresses the reservoir design problem, which remains the biggest challenge in the applicability of this information processing scheme. More specifically, we use the information available regarding the optimal reservoir working regimes to construct a functional link between the reservoir parameters and its performance. This function is used to explore various properties of the device and to choose the optimal reservoir architecture, thus replacing the tedious and time consuming parameter scannings used so far in the literature.