Discrete-time signatures and randomness in reservoir computing
Type
journal article
Date Issued
2021
Author(s)
Cuchiero, Christa
;
;
;
;
Teichmann, Josef
Abstract
A new explanation of geometric nature of the reservoir computing phenomenon is presented. Reservoir computing is understood in the literature as the possibility of approximating input/output systems with randomly chosen recurrent neural systems and a trained linear readout layer. Light is shed on this phenomenon by constructing what is called strongly universal reservoir systems as random projections of a family of state-space systems that generate Volterra series expansions. This procedure yields a state-affine reservoir system with randomly generated coefficients in a dimension that is logarithmically reduced with respect to the original system. This reservoir system is able to approximate any element in the fading memory filters class just by training a different linear readout for each different filter. Explicit expressions for the probability distributions needed in the generation of the projected reservoir system are stated and bounds for the committed approximation error are provided.
Language
English
Keywords
Reservoir computing
recurrent neural network
state-affine system
SAS
signature state-affine system
SigSAS
echo state network
ESN
Johnson-Lindenstrauss Lemma
Volterra series
machine learning.
HSG Classification
contribution to scientific community
HSG Profile Area
SEPS - Quantitative Economic Methods
Refereed
No
Subject(s)
Eprints ID
261549
File(s)![Thumbnail Image]()
open.access
Name
RC13_IEEE.pdf
Size
4 MB
Format
Adobe PDF
Checksum (MD5)
55ef37809ae86552ebe69085c102f488