Representational Capacity of Deep Neural Networks – A Computing Study

Bernhard Bermeitinger; Tomas Hrycej; Siegfried Handschuh

doi:10.48550/arxiv.1907.08475

Representational Capacity of Deep Neural Networks – A Computing Study

Journal

arXiv

Type

forthcoming

Date Issued

2019-07-19

Author(s)

Bernhard Bermeitinger

;

Tomas Hrycej

;

Siegfried Handschuh

DOI

10.48550/arxiv.1907.08475

Abstract

There is some theoretical evidence that deep neural networks with multiple hidden layers have a potential for more efficient representation of multidimensional mappings than shallow networks with a single hidden layer. The question is whether it is possible to exploit this theoretical advantage for finding such representations with help of numerical training methods. Tests using prototypical problems with a known mean square minimum did not confirm this hypothesis. Minima found with the help of deep networks have always been worse than those found using shallow networks. This does not directly contradict the theoretical findings---it is possible that the superior representational capacity of deep networks is genuine while finding the mean square minimum of such deep networks is a substantially harder problem than with shallow ones.

Language

English

HSG Classification

contribution to scientific community

Refereed

No

Pages

8

Official URL

https://arxiv.org/abs/1907.08475

URL

https://www.alexandria.unisg.ch/handle/20.500.14171/123206

File(s)

Name

1907.08475v1.pdf

Size

117.25 KB

Format

Adobe PDF

Checksum (MD5)

c1b8b11042c37c4e23bf76e3a7fb18e7