Representational Capacity of Deep Neural Networks: A Computing Study

Item Type Journal paper
Abstract There is some theoretical evidence that deep neural networks with multiple hidden layers have a potential for more efficient representation of multidimensional mappings than shallow networks with a single hidden layer. The question is whether it is possible to exploit this theoretical advantage for finding such representations with help of numerical training methods. Tests using prototypical problems with a known mean square minimum did not confirm this hypothesis. Minima found with the help of deep networks have always been worse than those found using shallow networks. This does not directly contradict the theoretical findings—it is possible that the superior representational capacity of deep networks is genuine while finding the mean square minimum of such deep networks is a substantially harder problem than with shallow ones.
Authors Bermeitinger, Bernhard; Hrycej, Tomas & Handschuh, Siegfried
Journal or Publication Title Proceedings of the 11th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management
Language English
Subjects computer science
HSG Classification contribution to scientific community
HSG Profile Area None
Refereed Yes
Date September 2019
Publisher SCITEPRESS - Science and Technology Publications
Page Range 532-538
ISSN 978-989-758-382-7
Publisher DOI https://doi.org/10.5220/0008364305320538
Official URL http://www.scitepress.org/DigitalLibrary/Link.aspx...
Contact Email Address bernhard.bermeitinger@unisg.ch
Depositing User Bernhard Bermeitinger
Date Deposited 02 Oct 2019 10:01
Last Modified 20 Jul 2022 17:39
URI: https://www.alexandria.unisg.ch/publications/257996

Download

[img] Text
Bermeitinger et al_2019_Representational Capacity of Deep Neural Networks_PREPRINT.pdf - Submitted Version
Download (120kB)
[img] Slideshow
2019-09-18 - KDIR2019, Wien.pdf - Presentation
Download (189kB)
[img] Text
Bermeitinger et al_2019_Representational Capacity of Deep Neural Networks.pdf - Published Version
Restricted to Registered users only
Available under License Creative Commons Attribution Non-commercial No Derivatives.
Download (335kB) | Request a copy

Citation

Bermeitinger, Bernhard; Hrycej, Tomas & Handschuh, Siegfried (2019) Representational Capacity of Deep Neural Networks: A Computing Study. Proceedings of the 11th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management, 532-538. ISSN 978-989-758-382-7

Statistics

https://www.alexandria.unisg.ch/id/eprint/257996
Edit item Edit item
Feedback?