Perception Coordination Network: A Neuro Framework for Multimodal Concept Acquisition and Binding.

SENSORY perception; ARTIFICIAL neural networks; MACHINE learning; IMAGE color analysis; BIOLOGICAL neural networks

To simulate the concept acquisition and binding of different senses in the brain, a biologically inspired neural network model named perception coordination network (PCN) is proposed. It is a hierarchical structure, which is functionally divided into the primary sensory area (PSA), the primary sensory association area (SAA), and the higher order association area (HAA). The PSA contains feature neurons which respond to many elementary features, e.g., colors, shapes, syllables, and basic flavors. The SAA contains primary concept neurons which combine the elementary features in the PSA to represent unimodal concept of objects, e.g., the image of an apple, the Chinese word “[píng guǒ]” which names the apple, and the taste of the apple. The HAA contains associated neurons which connect the primary concept neurons of several PSA, e.g., connects the image, the taste, and the name of an apple. It means that the associated neurons have a multimodal response mode. Therefore, this area executes multisensory integration. PCN is an online incremental learning system, it is able to continuously acquire and bind multimodality concepts in an online way. The experimental results suggest that PCN is able to handle the multimodal concept acquisition and binding effectively. [ABSTRACT FROM AUTHOR]

Copyright of IEEE Transactions on Neural Networks & Learning Systems is the property of IEEE and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)