Your Paper's Title Starts Here: Please Center Measurement of Brain Activity Related to Unpleasant Emotion in the Prefrontal Cortex Region Using NIRS Haruki Kawanaka, Atsushi Nagase, Koji Oguri Graduate School of Information Science and Technology, Aichi Prefectural University, Japan, kawanaka@ist.aichi-pu.ac.jp Abstract. We measured noninvasively a response of blood flow (hemoglobin concentration) in the prefrontal cortex area caused by emotional unpleasant arousal using the near-infrared spectroscopy (NIRS). NIRS can be easily measured brain activity in comparison with other measurement methods. And it has the advantage of low load for subjects. In this study, auditory stimuli applied to subjects in order to arouse unpleasant emotions, changes in blood flow were measured at that time. As a result, we have confirmed a significant increase of the oxygenated hemoglobin concentration in the peripheral cortex (Brodmann area 9) caused by the emotional unpleasant associated. And, the effectiveness of NIRS in measuring the brain activity related to emotional was demonstrated. Keywords: near-infrared spectroscopy, unpleasant emotion, prefrontal cortex, auditory stimuli, brain activity 1. Introduction An emotion is one of the important functions of the brain. The emotion has various types of fear, anger, joy and so on. In this study we have differentiated in pleasant and unpleasant emotion. It have been known that humans and other animals take approach behavior to the preferred things (ex. foods) for themselves, while escape behavior and aggressive behavior to angered, feared or unpleasant things (ex. predators and dangers). The former is known as Positive emotional behavior, and the latter is Negative emotional behavior. The brain activities which cause these behaviors are called Positive (pleasant, comfort) and Negative (unpleasant, discomfort) emotion. It is becoming clear that the limbic system structures including the amygdala have an important role in the expression of emotions. And the relationship between emotion and the prefrontal cortex which is located in front of the frontal lobe have been reported. In recent years, non-invasive measurement of brain activity has elucidated many aspects of brain function. Near- infrared spectroscopy (NIRS) is a non-invasive measurement technique using near-infrared light to measure changes of brain activity, especially the blood flow (hemoglobin concentration) in cortical areas. NIRS is low load for the subjects and has the advantage of being easy to measure in comparison with Functional magnetic resonance imaging (fMRI) and Positron emission tomography (PET). For example, fMRI and PET measurements must continue to maintain a supine posture with a sense of stagnation within the device. Also, fMRI has the noise of the device and PET has the possibility of radiation exposure due to radioactive solution injected into the body. So, if you measure emotion-related brain activity using PET and fMRI, emotions of the subject could be affected by the measurement itself. NIRS of low load compared with those methods are more suitable method to measure brain activities related to emotions. In this study, we focus on unpleasant emotions, and analyze blood flow changes measured by NIRS in the prefrontal cortex area related to emotional unpleasant evoked by auditory stimuli. 2. Unpleasant Emotion A. Emotion and Prefrontal Cortex. Amygdala in the limbic system is one of the most important structure on emotion, which is considered that is involved in the expression of pleasure or unpleasant emotion based on valuation and meaning to stimuli. In previous research on emotion-related brain activity measurements, brain activities when sensory stimulus was applied have been measured by PET or fMRI. Zald et al. have performed to measure a brain activity using PET and reported the activity in the amygdala which is associated with unpleasant emotions evoked by auditory stimulus. In addition to the limbic system, it is believed that prefrontal cortex relates to emotion. It is thought that prefrontal cortex has close fibrous contacts with the structure, such as the amygdala and plays a key role in emotion and motivation. And it has been reported that prefrontal cortex works to control or cooperate with the limbic system. Thus, the prefrontal cortex is closely related to the area involved in the expression of emotion, such as the limbic system. NIRS can capture the reaction of blood flow associated with emotion by measuring at the prefrontal cortex. B. Auditory Stimuli to Evoke Unpleasant Emotion. Visual stimuli, auditory stimuli or olfactory stimuli have been generally used as stimulus to evoke emotional. Auditory stimuli can be applied easily and continuously, and elicit an emotional arousal whose individual differences are small. Therefore, in this study we use auditory stimulus as sensory stimulation to evoke emotion, in particular unpleasant emotions. A preliminary experiment was conducted to determine the auditory stimuli which can effectively evoke unpleasant emotion. A white noise and seven sine waves with different frequencies were used as auditory stimuli. The sine waves frequency are 100 [Hz], 250 [Hz], 500 [Hz], 1 [kHz], 2 [kHz], 5 [kHz], 10 [kHz]. The 12 healthy subjects listened to auditory stimuli for 10 seconds, judged pleasant/unpleasant on a scale of -5 to 5 ('-5' corresponds to the most unpleasant, '5' the most pleasant). Fig. 1. is the result which shows the average and standard deviation of the judgments of all subjects. When a sine wave was present, the average grade has been generally negative. Subjects responded to the unpleasant at almost sine waves. The more the frequency increases, the more the unpleasant degree increases. In particular, 5 [kHz] has the lowest average grade -4.58 and the standard deviation is small compared to other. On the other hand, the average grade of 250 [Hz] sine wave is -0.75, and it is close to zero. In other words, 250 [Hz] sine wave is an auditory stimulus which there is very little effect to evoke unpleasant emotion. White noise has average grade -3.92, it shows white noise tends to evoke unpleasant emotion. White noise has been used as a control stimulus which can't evoke both pleasant and unpleasant emotions in a previous study [2]. However, using white noise as a control stimulus is not reasonable. In this study, we use 5 [kHz] sine wave as an auditory stimuli to evoke unpleasant emotion (unpleasant stimulus), and 250 [Hz] sine wave as control stimulus. Fig.1 Ratings of pleasant vs. Unpleasant 3. Measurement and Analysis Methods in NIRS A. NIRS (Near-infrared Spectroscopy). Near-infrared spectroscopy is one of the brain activity measurement techniques which quantify the blood flow changes (hemoglobin concentration) in cortical areas using near-infrared light which is high permeability to the body. Near-infrared light irradiated from the optical transmission fiber reaches to the cerebral cortex while being diffused and absorbed at the scalp, skull and cerebrospinal fluid. The amount of the transmitted light came back again on the scalp from the cortex is measured by receiving fiber. In a bloodstream, ther e are two kinds of hemoglobin i.e. oxygenated hemoglobin (oxy-Hb) and deoxygenated hemoglobin (deoxy-Hb). oxy-Hb and deoxy-Hb absorption spectrum is different, so it is possible to calculate the concentration of each hemoglobin based on the amount of transmitted light. When the blood vessels expand in association with the neural activity, arterial blood which includes rich glucose and oxygen as an energy source is adjusted. Such adjustment mechanism is called neuro-vascular coupling and known as a regional cerebral blood flow (rCBF) increases by this mechanism. Thus, the increase of rCBF means an existence of neural activity in the region [1]. And if rCBF has been increased, oxy-Hb increases and deoxy-Hb decreases in the venous capillary. We can obtain the concentration of hemoglobin in the capillaries of the cortical areas from the NIRS signal. NIRS signal where the concentration of oxy-Hb increases and the concentration of deoxy-Hb decreases show that neural activity has occurred around the area. B. Measuring method. Four healthy male subjects which were 18-22 years old (average 20.0) were conducted with informed consent and participated in experiments. As shown in Fig. 2, one trial was set as a series of a rest for 10 seconds (former rest), an infliction of auditory stimuli for 10-second (task) and a rest for 20 seconds (latter rest). Each subject was instructed to gaze onto '+' sign on a screen in front of him in the sitting posture during the trial, to empty his mind at both rests, and to focus on auditory stimuli at the task. Also, the experimental laboratory was kept dark to prevent subjects from distraction. Fig.2 Time sequence of a trial Fig.3 Optical fibers and channels The unpleasant emotional stimulus and the control stimulus were inflicted through earphones attached to the both ears of the subject. Optical brain function imaging was FOIRE-3000 (Shimadzu). The sampling frequency of oxy-Hb and deoxy-Hb concentrations was 10 [Hz]. In order to measure prefrontal cortex region, optical transmit and receive fiber were attached to the forehead of the subject. As shown in Fig. 3, 32 channels were measured in all. The channel 29 and channel 30 are corresponding to Fp1 and Fp2 in International 10-20 method. The data signals of 50 trials for each stimulus were obtained by using NIRS one by one. The existence of the evocation of emotional discomfort of the subject was confirmed per 5 trials. C. Analysis Method. 1) Step 1: Pretreatment. The raw signal has been measured by NIRS is shown in Fig. 4(a). It is the hemoglobin concentration change of channel 15 during 5 trails for the unpleasant emotional stimulus. The solid line corresponds to oxy-Hb concentration, dashed line to deoxy-Hb concentration in Fig. 4. Because it contains artifacts such as noise and pulse rate variability in the original signal, a removal processing performed using 4th-order Butterworth low pass filter of 0.5 [Hz]. In addition, the NIRS signal has been known to change the baseline [3]. To reduce the variation of this baseline, the baseline correction was given on the basis of the former rest of each trial. The baseline correction was performed by taking the difference between xn(t) (raw data at time t in n -th trial) and xn (average at the former rest in n- th trial). bn (t )= xn(t )− x̄n (1) where, let bn(t) be the value after baseline correction. Fig. 4(b) shows NIRS signal after the preprocessing. (a) (b) Fig.4 Orignal signals of NIRS and denoise signals after preprocessing 2) Step 2: Averaging. Averaging is often used to detect the changes in hemoglobin concentration by the type of task. Because averaging improves the S/N ratio, we can efficiently get the components related to the task. In addition, we performed grand averaging from data of all subjects by the type of task. However, if the grand averaging waveform is directly obtained from all subjects' data then it is difficult to analyze because of differences in trends for each subject. Thus, the grand averaging was calculated for each subject after normalizing preprocessed signals by Z-score such that the average value of the former rest of all the trials for each subject is 0 and the standard deviation is 1 (Eq.2). Z ( t )= { x (t )− x̄ f }/σ f (2) where, let xf be the average hemoglobin concentration in the former rest, σf be the standard deviation of the former rest. The change in hemoglobin concentration of the former rest is reflected in neural activity at rest, we can be compared between subjects by normalizing based on changes in former rest. 3) Step 3: Statistical Analysis. In general, measurements of brain activity using fMRI and PET have been judged the presence and intensity of brain activity by statistical analysis. In this study, Student's t-test for changes of hemoglobin concentration was performed using Welch's t-test in accordance with the previous study [5]. Also, the visualization image was created by mapping based on t-value and two-dimensional interpolation using the inverse distance weighted. 4. Results and Discussion Fig. 5 shows the result measured at channel 15 in the case of the control stimulus, Fig. 6 shows in the case of the unpleasant emotional stimulus. Regardless of whether the stimulus was unpleasant, oxy-Hb concentration began to increase shortly after start of task, to decline before the end of the task. On the other hand, deoxy-Hb concentration has been decreasing from the start of the task. Generally, since rCBF increases in association with the neural activity by neuro-vascular coupling, NIRS signal at the area has a tendency to increase in oxy-Hb concentration and to decrease in deoxy-Hb concentration. Averaged waveforms show changes of hemoglobin concentration related in the neural activity. The oxygen consumption and glucose metabolism increases at neuron cell by the neural activity, accordingly cerebral blood flow begins to increase after an interval of about three seconds [4]. The average waveform of oxy-Hb concentration which was obtained in this study shows increase from after a few seconds of stimulus in the same way. In the control stimulus versus the unpleasant emotional stimulus, the degree of increase in the concentration of oxy- Hb at the case of the unpleasant emotional stimulus is larger. This is presumed that the control stimulus evokes neural activity related to only the perception of auditory stimuli, and the unpleasant emotional stimulus caused neural activity related to not only the perception of stimuli but also the evoked emotion. Fig. 7 shows the grand averaging waveform of oxy-Hb concentration at every channel. The oxy-Hb concentration waveform for the unpleasant emotional stimulus significantly increased at almost channels. In addition, oxy-Hb concentration for the control stimulus is increased at latter rest section of several channels. In order to compare the change of oxy-Hb concentration when auditory stimuli were applied, we evaluated the mean values for the former rest and the task in each trial by t-test. Fig. 8 shows the comparison between the former rest and the task of the control stimulus, and Fig. 9 shows the former rest and the task of the unpleasant emotional stimulus. The mark '*' in the figures indicates p < 0.05 significance level, the mark '**' indicates p < 0.01 significance level that the channel confirmed a significant increase in oxy-Hb concentration. Fig.5 Averaging waveforms unpleasant emotional stimulus presentation at ch 15 Fig.6 Averaging waveforms for control stimulus presentation at ch 15 Fig.7 Averaging waveforms of oxy-Hb concentration in all channels Fig.8 Visualization of t-values obtained by comparison of oxy-Hb conc. during control stimulus and rest Fig.9 Visualization of t-values obtained by comparison of oxy-Hb conc. during unpleasant emotional stimulus and rest In the case of the control stimulus, the concentration of oxy-Hb significantly increased with p < 0.01 significance level at channel 23 and p < 0.05 significance level at channel 15,18 and 30. The oxy-Hb concentration had positive t- value on the whole and tended to increase. Especially, markedly increase was observed in the lower part of measurement area. In the case of the unpleasant emotional stimulus, a significant increase in concentration of oxy-Hb was observed with p < 0.01 significance level at the overall measured area. As described above, the neural activity associated with both the evocation of unpleasant emotion and the perception of auditory stimuli is thought to be the cause of that. In order to compare the change of oxy-Hb concentration when the unpleasant emotional stimuli were inflicted, we evaluated the mean values for the task of the unpleasant emotional stimulus and the task of the control stimulus in each trial by t-test. Fig. 10 shows the comparison of oxy- Hb concentration between the unpleasant emotional stimulus and the control stimulus. The concentration of oxy-Hb significantly increased with p < 0.01 significance level at channel 2and 3 and p < 0.05 significance level at channel 1,5,15 and 22. In addition, oxy-Hb concentration significantly decreased with p < 0.05 significance level at channel 20. Especially, markedly increase was observed in the upper part of measurement area. Fig.10 Visualization of t-values obtained by comparison of oxy-Hb conc. during unpleasant emotional stimulus and control stimulus 5. Conclusions In this study, we non-invasively measured blood flows in prefrontal cortex by using NIRS while the auditory stimulus was inflicted. 5 [kHz] sine wave was inflicted to the subjects as an unpleasant emotional stimulus. Similarly 250 [Hz] sine wave was as a control stimulus. As a result of comparison between the control stimulus and the unpleasant emotional stimulus, significantly increase of oxy-Hb concentration was found in the periphery of Brodmann's area 9. The blood flow response associated with unpleasant emotion has been obtained using NIRS. Therefore, it was suggested that NIRS has the effectiveness of measuring a brain activity related emotion with low load for subjects. Future work is the NIRS measurements in a variety of emotions to be applied to brain machine interface. 6. Acknowledgment A part of this research is supported by a grant from the Artificial Intelligence Research Promotion Foundation (No. 175-11). References [1] M. Jueptner and C. Weiller, “Does measurement of regional cerebral blood flow reflect synaptic activity? - implications for PET and fMRI”, NeuroImage, vol.2, no.2, pp.148-156, 1995 [2] D.H. Zald and J.V. Pardo, “The neural correlates of aversive auditory stimulation”, NeuroImage, vol.16, pp.746- 753, 2002 [3] B. Chance, Z. Zhuang, C. UnAh, C. Alter and L. Lipton, “Cognition-activated low-frequency modulation of light absorption in human brain”, Proc.Natl.Acad.Sci.USA, vol.90, no.8, pp.3770-3774, 1993 [4] P.A. Bandettini, E.C. Wong, R.S. Hinks, R.S. Tikofsky and J.S. Hyde, “Time course EPI of human brain function during task activation”, Magnetic Resonance in Medicine, vol.25, issue 2, pp.390-397, 1992 [5] W. Niide, T. Tsubone, Y. Wada, “Discrimination of Moving Limb by Near Infrared Spectroscopic Measurement of Brain Activation”, Transaction of IEICE, vol.J92-D, no.9, pp.1632-1642, 2009