Adaptive Orthogonal Characteristics of Bio-Inspired Neural Networks.

In recent years, neural networks have attracted much attention in the machine learning and the deep learning technologies. Bio-inspired functions and intelligence are also expected to process efficiently and improve existing technologies. In the visual pathway, the prominent features consist of nonlinear characteristics of squaring and rectification functions observed in the retinal and visual cortex networks, respectively. Further, adaptation is an important feature to activate the biological systems, efficiently. Recently, to overcome short-comings of the deep learning techniques, orthogonality for the weights in the networks has been developed for the signal propagation and the efficient optimization of the learning. In this paper, bio-inspired asymmetric networks with nonlinear characteristics are proposed, which are derived from the retinal networks in the biological visual pathway. The asymmetric network proposed here was verified to detect the movement of the object, efficiently in our previous studies. This paper shows a new characteristic of the adaptive orthogonality in the asymmetric networks. First, it is shown that the asymmetric network with nonlinear characteristics is effective for generating orthogonality. Second, the proposed asymmetric network with Gabor filters is compared with the conventional energy model from the point of the orthogonality characteristics. Finally, the asymmetric networks with nonlinear characteristics can generate the extended orthogonal bases in independent subspaces, which are useful for classification and efficient learning. [ABSTRACT FROM AUTHOR]

Copyright of Logic Journal of the IGPL is the property of Oxford University Press / USA and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)