<abstract xmlns="http://www.w3.org/1999/xhtml"><p>The learning of neural networks is becoming more and more important. Researchers have constructed dozens of learning algorithms, but it is still necessary to develop faster, more flexible, or more accurate learning algorithms. With fast learning we can examine more learning scenarios for a given problem, especially in the case of meta-learning. In this article we focus on the construction of a much faster learning algorithm and its modifications, especially for nonlinear versions of neural networks. The main idea of this algorithm lies in the usage of fast approximation of the Moore–Penrose pseudo-inverse matrix. The complexity of the original singular value decomposition algorithm is <italic>O</italic>(<italic>mn</italic>2). We consider algorithms with a complexity of <italic>O</italic>(<italic>mnl</italic>),where <italic>l&lt;n</italic> and <italic>l</italic> is often significantly smaller than <italic>n</italic>. Such learning algorithms can be applied to the learning of radial basis function networks, extreme learning machines or deep ELMs, principal component analysis or even missing data imputation.</p></abstract>

The learning of neural networks is becoming more and more important. Researchers have constructed dozens of learning algorithms, but it is still necessary to develop faster, more flexible, or more accurate learning algorithms. With fast learning we can examine more learning scenarios for a given problem, especially in the case of meta-learning. In this article we focus on the construction of a much faster learning algorithm and its modifications, especially for nonlinear versions of neural networks. The main idea of this algorithm lies in the usage of fast approximation of the Moore–Penrose pseudo-inverse matrix. The complexity of the original singular value decomposition algorithm is O(mn2). We consider algorithms with a complexity of O(mnl),where l<n and l is often significantly smaller than n. Such learning algorithms can be applied to the learning of radial basis function networks, extreme learning machines or deep ELMs, principal component analysis or even missing data imputation.

A Fast Neural Network Learning Algorithm with Approximate Singular Value Decomposition

Department of Informatics, Faculty of Physics, Astronomy and Informatics

International Journal of Applied Mathematics and Computer Science

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

The learning of neural networks is becoming more and more important. Researchers have constructed dozens of learning algorithms, but it is still necessary to...

一种近似奇异值分解的快速神经网络学习算法

在线发布:2019年9月28日

页面范围： 581 - 594

收到:2018年9月6日

认可的:2019年2月22日

内政部： https://doi.org/10.2478/amcs-2019-0043

关键词
Moore–Penrose伪逆学习,径向基函数网络,极端学习机器,内核方法,机器学习,奇异值分解,深度极限学习,主成分分析

©2019 Norbert Jankowski等人，由Sciendo出版

本作品是根据Creative Commons Attribution-NonCommercial-NoDerivatives 4.0许可证授权的。