Abstract: Previous studies have shown that the cross-modal pre-preparation effect is an important factor for audiovisual integration. However, the facilitating influence of the pre-preparation effect on the integration of emotional cues remains unclear. Therefore, this study examined the emotional pre-preparation effect during the multistage process of audiovisual integration. Event-related potentials (ERPs) were recorded while participants performed a synchronous or asynchronous integration task with fearful or neutral stimuli. The results indicated that, compared with the sum of the unisensory presentation of visual (V) and auditory (A) stimuli (A+V), only fearful audiovisual stimuli induced a decreased N1 and an enhanced P2; this was not found for the neutral stimuli. Moreover, the fearful stimuli triggered a larger P2 than the neutral stimuli in the audiovisual condition, but not in the sum of the combined (A+V) waveforms. Our findings imply that, in the early perceptual processing stage and perceptual fine processing stage, fear improves the processing efficiency of the emotional audiovisual integration. In the last cognitively assessing stage, the fearful audiovisual induced a larger late positive component (LPC) than the neutral audiovisual. Moreover, the asynchronous-audiovisual induced a greater LPC than the synchronous-audiovisual during the 400-550 ms period. The different integration effects between the fearful and neutral stimuli may reflect the existence of distinct mechanisms of the pre-preparation in terms of the emotional dimension. In light of these results, we present a cross-modal emotional pre-preparation effect involving a three-phase emotional audiovisual integration.
References: Parise CV, Ernst MO. Correlation detection as a general mechanism for multisensory integration. Nat Commun. 2016; 7:11543.
Zhang H, Chen X, Chen S, Li Y, Chen C, Long Q, et al. Facial expression enhances emotion perception compared to vocal prosody: behavioral and fMRI studies. Neurosci Bull. 2018; 34:801–815.
Ho HT, Schröger E, Kotz SA. Selective attention modulates early human evoked potentials during emotional face-voice processing. J Cogn Neurosci. 2015; 27:798–818.
Cecere R, Gross J, Willis A, Thut G. Being first matters: topographical representational similarity analysis of ERP signals reveals separate networks for audiovisual temporal binding depending on the leading sense. J Neurosci. 2017; 37:5274–5287.
Stekelenburg JJ, Vroomen J. Neural correlates of multisensory integration of ecologically valid audiovisual events. J Cogn Neurosci. 2007; 19:1964–1973.
Simon DM, Wallace MT. Integration and temporal processing of asynchronous audiovisual speech. J Cogn Neurosci. 2018; 30:319–337.
Kokinous J, Kotz SA, Tavano A, Schröger E. The role of emotion in dynamic audiovisual integration of faces and voices. Soc Cogn Affect Neurosci. 2015; 10:713–720.
Liu T, Pinheiro A, Zhao Z, Nestor PG, McCarley RW, Niznikiewicz MA. Emotional cues during simultaneous face and voice processing: electrophysiological insights. PLoS One. 2012; 7:e31001.
Pan Z, Liu X, Luo Y, Chen X. Emotional intensity modulates the integration of bimodal angry expressions: ERP evidence. Front Neurosci. 2017; 11:349.
Paulmann S, Jessen S, Kotz SA. Investigating the multimodal nature of human communication: Insights from ERPs. J Psychophysiol. 2009; 23:63–76.
Doi H, Shinohara K. Unconscious presentation of fearful face modulates electrophysiological responses to emotional prosody. Cereb Cortex. 2015; 25:817–832.
Chen X, Zhao L, Jiang A, Yang Y. Event-related potential correlates of the expectancy violation effect during emotional prosody processing. Biol Psychol. 2011; 86:158–167.
Klemen J, Chambers CD. Current perspectives and methods in studying neural mechanisms of multisensory interactions. Neurosci Biobehav Rev. 2012; 36:111–133.
Cecere R, Rees G, Romei V. Individual differences in alpha frequency drive crossmodal illusory perception. Curr Biol. 2015; 25:231–235.
Tünnermann J, Petersen A, Scharlau I. Does attention speed up processing? Decreases and increases of processing rates in visual prior entry. J Vis. 2015; 15:1–1.
Yeh PW, Geangu E, Reid V. Coherent emotional perception from body expressions and the voice. Neuropsychologia. 2016; 91:99–108.
Neuberg SL, Kenrick DT, Schaller M. Human threat management systems: self-protection and disease avoidance. Neurosci Biobehav Rev. 2011; 35:1042–1051.
van Wassenhove V, Grant KW, Poeppel D. Temporal window of integration in auditory-visual speech perception. Neuropsychologia. 2007; 45:598–607.
Ethofer T, Van De Ville D, Scherer K, Vuilleumier P. Decoding of emotional information in voice-sensitive cortices. Curr Biol. 2009; 19:1028–1033.
Petrides M, Pandya DN. Association fiber pathways to the frontal cortex from the superior temporal region in the rhesus monkey. J Comp Neurol. 1988; 273:52–66.
Baart M, Stekelenburg JJ, Vroomen J. Electrophysiological evidence for speech-specific audiovisual integration. Neuropsychologia. 2014; 53:115–121.
Brück C, Kreifelts B, Wildgruber D. Emotional voices in context: a neurobiological model of multimodal affective information processing. Phys Life Rev. 2011; 8:383–403.
Frey S, Kostopoulos P, Petrides M. Orbitofrontal involvement in the processing of unpleasant auditory information. Eur J Neurosci. 2000; 12:3709–3712.
Frey S, Kostopoulos P, Petrides M. Orbitofrontal contribution to auditory encoding. Neuroimage. 2004; 22:1384–1389.
Calvert GA, Campbell R, Brammer MJ. Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr Biol. 2000; 10:649–657.
Processing Request
No Comments.