Options
Representational Capacity of Deep Neural Networks: A Computing Study
Journal
Proceedings of the 11th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management
ISSN
978-989-758-382-7
Type
conference paper
Date Issued
2019-09
Author(s)
Abstract
There is some theoretical evidence that deep neural networks with multiple hidden layers have a potential for more efficient representation of multidimensional mappings than shallow networks with a single hidden layer. The question is whether it is possible to exploit this theoretical advantage for finding such representations with help of numerical training methods. Tests using prototypical problems with a known mean square minimum did not confirm this hypothesis. Minima found with the help of deep networks have always been worse than those found using shallow networks. This does not directly contradict the theoretical findings—it is possible that the superior representational capacity of deep networks is genuine while finding the mean square minimum of such deep networks is a substantially harder problem than with shallow ones.
Language
English
HSG Classification
contribution to scientific community
HSG Profile Area
None
Refereed
Yes
Publisher
SCITEPRESS - Science and Technology Publications
Start page
532
End page
538
Subject(s)
Division(s)
Contact Email Address
bernhard.bermeitinger@unisg.ch
Eprints ID
257996