A Wavelet Tool to Discriminate Imagery Versus Actual Finger Movements Towards a Brain–Computer Interface Maria L. Stavrinou1, Liviu Moraru1, Polyxeni Pelekouda2, Vasileios Kokkinos2, and Anastasios Bezerianos1 1

2

Dept. of Medical Physics, Department of Physiology, School of Medicine University of Patras, 26500 University Campus, Rio, Greece [email protected]

Abstract. The present work explores the spatiotemporal aspects of the eventrelated desynchronization (ERD) and synchronization (ERS) during rhythmic finger tapping execution and imagery task. High resolution event related brain potentials were recorded to capture the brain activation underlying the motor execution and motor imagery. ERS and ERD were studied using a complex morlet wavelet decomposition of EEG responses. The results show similar patterns of beta ERD/ERS after the stimulus onset, for both the actual and imagery finger tapping task. This approach and results can be regarded as indicative evidences of a new strategy for recognizing imagined movements in EEG-based brain computer interface research. The long-term objective of this study is to create a multiposition brain controlled switch that is activated by signals that are measured directly from a human’s brain. Keywords: EEG, Brain-Computer Interface, finger-tapping, imagery, beta rhythm, wavelet, Event Related Synchronization (ERS) -Desynchronization (ERD).

1 Introduction The electroencephalogram (EEG) based Brain-Computer Interface (BCI) is a communication system which represents a direct connection between the human brain and the computer. The general idea behind any BCI research is to establish patterns of activation that can be used to help the disabled people perform the desired action [1]. The BCI research revolves around the design of effective experimental protocols, the development of efficient methods for feature extraction of brain activation and the evaluation of algorithms for translating these features into commands. Nowadays, a great variety of EEG-based BCI systems are in use [2, 3]. The methodology used for these machines make use of slow potentials as P300 or beta (14-30 Hz) and mu (8-12 Hz) rhythm detection. The detection of such signals is then transformed into commands that operate a computer display or other device [4]. Motor imagery has become the newest trend in BCI research [5, 6, 7]. This is due to the fact that imagination of movements appears to recruit similar -or the same- neural networks in the brain, to those used to perform actually the same movements [8]. N. Maglaveras et al. (Eds.): ISBMDA 2006, LNBI 4345, pp. 323 – 333, 2006. © Springer-Verlag Berlin Heidelberg 2006

324

M.L. Stavrinou et al.

In our work, we investigated the spatio-temporal EEG brain activity during a real and an imaginary rhythmic finger tapping task. Previous studies using relative longer inter-stimulus interval protocols (of 4 to 10 seconds) have reported characteristic synchronization and desynchronization timecourses, following the onset of the action (from 4 to 7 sec). We investigated whether we could see similar activation patterns for an imagery task using an 1.5 sec inter-stimulus interval protocol. Time-frequency decomposition of brain electrical signals by means of wavelet transform has been widely employed in the study of brain rhythms. The major advantage of the wavelet transform is that it allows the decomposition and manipulation of time-varying non-stationary signals, being particularly suited to the analysis of ERPs. In our study, we first applied the continuous wavelet transform (CWT) in order to calculate the power spectra of various frequency bands for the each single trial, using a complex Morlet mother wavelet. Next, task related neural responses have been uncovered by averaging single trial time-frequency representations. It is widely accepted that while brain processes certain events the ongoing brain rhythmical activity can be blocked or desynchronized. These types of changes are better detected by frequency analysis because they represent frequency specific changes of the ongoing EEG activity. They consist, in general of an amplitude attenuation or power decrease and/or of an amplitude/power enhancement in certain frequency bands. This is considered to be due to a decrease or an increase in synchrony of the underlying neuronal populations. The former case is called event-related desynchronization (ERD), and the latter event-related synchronization (ERS) [9]. In this paper we focus on the detection of beta rhythms and associated ERD/ERS patterns, previously described to occur with initiation and execution of motor actions [10] as well as with motor imagination [11]. Discrimination of short time activations during motor imagery based on the frequency content may improve decision making and enhance performance of a BCI system.

2 Methods Subjects and Experimental Paradigm Two healthy volunteers participated in this study (2 males) with age range 26-28 years. Subjects were strongly right-handed according to the Edinburgh Inventory [12]. None of them had a previous history of neurological disease and took no medication at the time of the experiment. All subjects gave their written informed consent. The protocol and experimental procedures were approved by the local ethics committee and were in compliance with the declaration of Helsinki. The experiments have been conducted in the EEG Laboratory of Neurophysiology Unit, Department of Physiology, Medical School, University of Patras, Greece. Subjects sat on a comfortable armchair in an electrically isolated room, dimly illuminated. A small led light was adjusted on the wall in front of the subjects, in order to fixate their sight on it, to avoid ocular movements. The experimental session consisted of four parts. In the first part, median nerve stimulation of subject’s right wrist, above the motor threshold where a definite twitch of the thumb was visible, was performed. The ISI of the electric stimulation was 1500 msec and stimulus duration

A Wavelet Tool to Discriminate Imagery Versus Actual Finger Movements

325

200 microseconds, with a total number of trials 250. For the second part the subject was instructed to make a right index finger task, tapping a key of a keyboard, externally paced by an auditory signal (1000 Hz, 64dB max arranged to be heard but not annoying to the subject, 50microseconds duration and ISI 1500 msec). The next session consisted of a sub-session were the subjects practiced right index finger tapping for another 250 trials before the right index finger imagery task begun. Subjects were instructed to imagine the right index finger tapping task with the auditory stimulation providing the pace. Subjects were instructed to imagine the kinesthetic of the movement and not the visual image of the movement itself. After training, approximately 130 trials of right index finger imagery were recorded. The last session consisted of a control auditory stimulation, were the subject was instructed to hear passively the auditory stimulation without executing any movement, while being relaxed. Data Acquisition EEG signals were recorded from 60-electrodes mounted on an elastic cap (Electrocap International, Ohio, USA), and acquired with a SynAmps amplifier (Neuroscan, USA). The Neuroscan software was used for recording. Impedances were kept bellow 5 KΩ. Linked earlobes were used as a reference and AFZ electrode as ground. The signals, were filtered between 0.1 and 200 Hz, with a sampling frequency 1000 Hz. The positions of all the electrodes as well as of four anatomical landmarks (nasion, inion and the two preauricular points) and points on the head were digitized with a 3d Digitizer (Pohlemus 3Dspace Fastrack, Colchester Vt, USA). Data Analysis The datasets were visually checked for noisy epochs which were excluded from further analysis. Approximately 220 artifact-free trials were selected for each session and each subject for the actual finger tapping task and about 120 for the imagery task. Each epoch consisted of a time window of 300 ms pre- and 1200 ms post- stimulus, while a (-100 -80) interval was used for baseline correction. Because we were interested in the activity in the sensory and motor cortex we selected for timefrequency analysis only the corresponding electrodes. The selection of electrodes was based also on the detection of maximum activity during the actual finger tapping task and the control median nerve recording which served to provide a landmark for the sensory cortex. Therefore the cluster of electrodes selected for further analysis was FC3, FC1, FCZ, C5, C3, C1, CZ, CP3, CP1, CPZ, for the left (contralateral) hemisphere and for the right hemisphere FCZ, FC2, FC4, CZ, C2, C4, C6, CPZ, CP2, CP4. Laplacian Filtering The electric potential recorded on the brain cortex results from the generation of sources of current. However, it depends on the reference electrode chosen and devices used. Laplacian transformation of scalp surface potentials is commonly used as a reference-free method to attenuate low spatial frequencies (‘smearing’) introduced into

326

M.L. Stavrinou et al.

the scalp potential distribution due to volume conduction, thus sharpen ERP topographies in a physiologically meaningful way [13, 14]. 2-D Laplacian has been computed by using the electrode locations from the digitization procedure. In this way we reduced the contribution of distant sources at each scalp location. This transformation has been applied before further signal processing and feature extraction. Time – Frequency Representations Single trials were further detrended in order to remove DC-offset and slow drifts (< 1 Hz). The time-frequency maps were constructed by the calculation of the power spectra of the detrended single trials, by squaring the convolution of them to the Morlet complex mother wavelet, which in our case is defined as follows:

u (t , f 0 ) = A exp(−t 2 / 2σ 2f ) exp(2 fπf 0 t )

(1)

where σ f = 1 / 2πσ t and σ t are the frequency and time resolution respectively, around the central frequency f 0 , is a frequency inside the frequency band of our interest for which we calculate the power spectrum. A = (σ t π ) −1 2 is a normalization factor which ensures that the wavelet has unit energy [15, 16, 17]. The energy for each frequency analyzed, is the absolute value of the convolution of this mother wavelet with the signal (in our case, each single trial).

E (t , f 0 ) = u (t , f 0 ) * s(t )

2

(2)

The time-frequency maps are the result of the average of this energy averaged over time for all the single trials. Quantification of ERD/ERS The time-frequency maps were calculated for the frequencies from 4 to 44 Hz. Based on the time-frequency plots we selected the most reactive individual frequency band in the beta range with 2 Hz span [9]. The ERD/ERS is defined as percentage power decrease (ERD) or power increase (ERS) in relation to a baseline time interval, in our case from -120 to -20 ms before stimulus onset (which is assumed as time zero). Due to inter-individual differences in the peak-frequency activity, ERD\ERS is calculated within individually determined frequency bands. A wavelet-based estimator is therefore calculated in order to determine the most reactive frequency, based on the average of the power of the wavelet coefficients for each frequency inside the entire frequency band under analysis. The estimators in this study were derived from the average of wavelets coefficients for the four following time intervals: from -200 to 50 ms, the pre-stimulus interval, and post-stimulus 10 to 200, 205 to 400 and 400 to 900 ms. An algorithm providing the quantitative outcome regarding the central frequency of the frequency band under consideration, measures the occurrence of a maximum of frequency in the 4 pre-selected time periods for the electrode selected, and provide us with the value having the most occurrences. The ERD/ERS then is calculated on 2 Hz interval around this frequency (most reactive frequency band).

A Wavelet Tool to Discriminate Imagery Versus Actual Finger Movements

327

Fig. 1. Time-frequency representations of signals recorded for actual movement at electrodes of the contralateral hemisphere (C1, CP1) top row and ipsilateral hemisphere (C2, CP2)

Fig. 2. Time-frequency representations of signals recorded at electrodes of the contralateral hemisphere (C1, CP3) for actual movement (top row), and during imagery (bottom row)

328

M.L. Stavrinou et al.

3 Results Examples of time-frequency ERD/ERS maps during the actual execution of the movement, from one subject are displayed in Figure 1. As it was expected, we found in all channels prominent alpha (mu) and beta band rhythmical activity. We focused our analysis of the beta band, as a dominant beta rhythm has been found in most of the selected EEG electrodes and it displayed for our subjects a more distinct ERD pattern than the alpha (or mu) rhythm. In Figure 1, on top plot, we can see the pattern of activation for electrodes C1 and CP1, for the execution of movement. The maximum average activation analysis revealed maximum activation at the C1 electrode. A clear decrease in the signal energy can be see after the presentation of the stimulus at C1 and CP1 (contralateral hemisphere), ending at about 200 ms poststimulus. After, the 200 ms energy power rebounds and an event related synchronization (ERS) occurs again. However, the ipsilateral hemisphere (e.g., at C2 and CP2) displays beta synchronization as well, but no similar to contralateral hemisphere beta power decrease is observed. During imagination of finger tapping, time-frequency energy maps reveals similar energy patterns in the contralateral hemisphere, at the same electrode sites as during the

Fig. 3. Wavelet-based estimators for C1 (a, c) and CP3 (b, d) electrodes, for movement and imagination of movement

A Wavelet Tool to Discriminate Imagery Versus Actual Finger Movements

329

actual execution of the movement (see Figure 2, for C1 and CP3). Moreover, at the majority of electrodes over the contralateral sensorimotor cortex, the desynchronization seems to have a longer duration comparing to the actual movement. For the auditory control experiment, a constant beta ERS was observed, with no ERD pattern either for contralateral or ipsilateral hemisphere (results not shown here). In Fig. 3, we present the wavelet-based estimators for C1 and CP3 electrodes, for movement and imagination of movement. The results, as shown and in the above figures, indicate that the most reactive frequencies for these time intervals are in the range of 18-20 Hz. Moreover, they also show that the decrease in this beta sub-band occurs between 10-200 ms post-stimulus, as can be seen from the time-frequency maps (Fig. 2).

Fig. 4. Percentage ERD/ERS time courses of the activity recorded in C1 and CP3, for the actual (top) and imagination (bottom) of finger tapping movement

In Fig. 4 are shown the percentage ERD/ERS time courses of the activity recorded in C1 and CP3, for the actual and imagery of movement. In this case, both ERD and ERS were located inside the 18-20 Hz (beta) sub-band. The ERD, present in the first 200 msec, reaches a percentage decrease in the order of -10% (-12.5 % for C1 and 8.7 % for CP3), for this subject, while the percentage increase for the rebound of ERS is 14% for C1 and 32.6% for CP3. For the imagination of movement, the percentage

330

M.L. Stavrinou et al.

of ERD is in the order of -10% (-9.21% for C1 and -16.39% for CP3) and the increase of the ERS rebound +25% (26.42% for C1 and 24.14% for CP3).

4 Discussion and Conclusions As it established from previous studies, the beta rhythm represents activity of the motor cortices including planning and execution of movements [8, 19, 9]. Thus, the desynchronization/synchronization properties of beta rhythm are expected to play a major role, in the detection of movement intention of disabled people and imagination of movement. Therefore, for the BCI research, detection of ERD/ERS patterns and especially quantification of ERD can serve as an effective and indicative parameter. Desynchronization of rhythmical activity dominantly on the contralateral hemisphere has been also reported in other studies for the mu rhythm [5, 7] after execution of movement. It has been reported however also in the beta band during imagery tasks [11] and during actual motor execution [10, 18]. As has been reported, these beta oscillations are highly somatotopically localized in the sensorimotor hand representational areas [19, 20]. On the other hand the alpha rhythms demonstrate a more widepsread desynchronization after various cognitive and motor tasks. In our study we investigated the prominent beta band (18-20 Hz) desynchronization over the sensorimotor cortex that occurred shortly after the stimulus onset. This ERD lasted for about 200 ms for the actual finger movement. A similar ERD at the same frequency band was observed during the imagery of the same task. Functional Brain Imaging studies have indicated almost the same areas where activated during imagination or actual execution of hand movement [21]. A prominent ERS rebound occurred after the end of the ERD (Fig. 2, 3) for both cases. Similar results for a beta rebound were reported after 200 ms post stimulus [17], as in our results. These results indicate a similarity in the processing of the brain between imagery and actual movement that exists both anatomically and functionally. Through quantitative analysis of ERD and ERS, features efficient to detect an intention of movement could be extracted. Moreover, another criterion is the frequency for which less or no ERD occurs in the ipsilateral hemisphere. The ERD/ERS quantification is based on the subtraction of a pre-selected baseline period from the energy of the wavelet coefficients. One could argue that this could not be valid due to a different level of activation in the preparation and execution of a task. However, it has been found that at time scales similar to ours, this activation is similar in both actual and imagery tasks [22], and is in agreement with the result of the wavelet-based estimator for the pre and post –stimulus period. The pattern of activation between subjects can be different, as inter-individual differences exist for the most reactive ERD/ERS frequency band [9]. Thus the wavelet-based estimator is a first step to quantitatively detect exactly this parameter (the most reactive frequency band) for every subject. However, the percentage of ERD/ERS for the individualized most reactive frequency band, must exhibit a similar form. And this is what one should detect and extract as a feature for the BCI system. Here results are shown from a representative subject therefore, necessity of further investigation including a larger population study will provide us with statistically

A Wavelet Tool to Discriminate Imagery Versus Actual Finger Movements

331

valuable results. However, in studies with higher ISIs stability in the ERD/ERS patterns was observed [23]. The same methodology can be performed for another frequency band like alpha, or both alpha and beta, depending on the response of the subject. The ability to perform an imagery task relies on the ability to suppress and enhance the hand cortical rhythms, in order for them to be detected from the BCI feature extraction application. Thus concentration, alertness and good performance are critical parameters for the efficacy of the method. In our study, the similarity in the patterns of imagery and actual movement shows that subjects could actually efficiently direct and maintain their attention in order to mentally realize the movement. It is important to note here that stability of intra-subject beta ERD/ERS during imagery has been previously reported [23]. Moreover, indications exist that higher attentional activation is achieved for self-paced than externally paced responses to a stimulus [24]. In addition with short interstimulus intervals, around 1 Hz, attention seems to be not necessary to recruit or re-activate the neuronal circuits that were previously –not so long ago activated [25, 26, 27]. Thus we could assume that the task itself activates attention. It has been reported that not only attention, but also age and intelligence level, contribute to the enhancement of the beta ERD [9]. In addition as we have expressed before, alertness and attention have been related to spatiotemporal changes for the alpha band; lately they have been found to affect the gamma band while for the beta band, results are still controversial [28]. It should be kept in mind that for the imagined movement we cannot exclude some missed trials. Then the more smooth less power of the wavelet coefficients could be due to this fact or to a lesser degree of excitability from the underlying neuronal networks responsible for the preparation but not execution of the movement. Possible effects of the auditory stimulation on the beta oscillations and their ERD/ERS properties can be excluded. Even though beta oscillations have been detected during the auditory control experiment they did not exhibit any desynchronization. Moreover, as this experiment was performed last, these beta oscillations during the auditory control stimulation could reflect evaluation of auditory information in order to prepare motor responses in posterior parietal cortex [29]. The promising results we have obtained in this pilot study suggest that ERD/ERS phenomena detected in this short interstimulus interval could help design a BCI application by recognizing imagination of a movement. High-quality features from wavelet coefficients can be used as input to a machine learning technique for classification of motor imagery EEG signals. Clearly, the present offline analysis results have to be further investigated during online settings, which consists the topic of our long term BCI research. Acknowledgments. We thank the European Social Fund (ESF), Operational Program for Educational and Vocational Training II (EPEAEK II), and particularly the Program PYTHAGORAS II, for funding the above work. Authors would like to thank Dr. Stefania Della Penna and Dr. Laura Cimponeriu.

332

M.L. Stavrinou et al.

References 1. Nicolelis, M.A.L.: Actions from thoughts. Nature. 409(2001) 403-407 2. Birbaumer, N., Kubler, A., Ghanayim, N., Hinterberger, T., Perelmouter, J., Kaiser, J., Iversen, I., Kotchoubey, B., Neumann, N., Flor, H.: The thought translation device (TTD) for completely paralyzed patients. IEEE Trans Rehabil Eng. 2000 8(2)190-193 3. Pfurtscheller, G., Neuper, C., Muller, G.R., Obermaier, B., Krausz, G., Schlogl, A., Scherer, R., Graimann, B., Keinrath, C., Skliris, D., Wortz, M., Supp, G., Schrank, C.: Graz-BCI: state of the art and clinical applications. IEEE Trans Neural Syst Rehabil Eng. 11 (2003) 177-180 4. Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtsceller, G., Vaughan, T.M.: BrainComputer Interfaces for communication and control. Clin Neurophysiol. (2002) 113: 767791 5. Pfurtscheller, G., Brunner, C., Schlogl, A., Lopes da Silva, F.H.: Mu rhythm (de)synchronization and EEG single-trial classification of different motor imagery tasks. Neuroimage. 31 (2006) 153-159 6. Hung, C.I., Lee, P.L., Chen, L.F., Yeh, T.C., Hsieh, H.C.: Recognition of Motor Imagery Electroencephalography Using Independent Component Analysis and Machine Classifiers Ann Biomed Engineer. 33 (2005) 1053-1070 7. Qin, L., Ding, L., He, B.: Motor Imagery classification by means of source analysis for brain-computer interface applications. J Neural Eng.. 1 (2004) 135:141 8. Neuper, C., Pfurtscheller, G.: Motor imagery and ERD. In: Pfurtscheller G., Lopes da Silva F. (eds.): Event-Related Desynchronization. Handbook of Electroencephalography and Clinical Neurophysiology. 6th Revised edition, Elsevier, Amsterdam (1999) 303-325 9. Pfurtscheller, G., Lopes da Silva, F.H.: Event-related EEG/MEG synchronization and desynchronization: basic principles. Clinical Neurophysiology. 110 (1999) 1842-1857. 10. Della Penna, S., Torquati, K., Pizzella, V., Babiloni, C., Franciotti, R., Rossini, P.M., Romani G.L.: Temporal dynamics of alpha and beta rhythms in human SI and SII after galvanic median nerve stimulation A MEG study. NeuroImage. 22 (2004) 1438-1446 11. Pfurtscheller, G., Neuper, C., Brunner, C., Lopes da Silva, F.: Beta rebound after different types of motor imagery in man. Neurosci Lett 378 (2005) 156-159 12. Oldfield ,R.C.: The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia. 9 (1971) 97-113 13. Kayser, J., Tenke, C.E.: Principal components analysis of Laplacian waveforms as a generic method for identifying ERP generator patterns: I. Evaluation with auditory oddball tasks. Clin Neurophysiol 117 (2006) 348–368 14. Nunez, P.L, Westdorp, A.F.: The surface Laplacian, high resolution EEG and controversies. Brain Topogr. 6 (1994) 221–226 15. Tallon-Baudry, C., Bertrand, O., Delpuech, C., Pernier, J.: Oscillatory γ-Band (30-70 Hz) activity Induced by a Visual Search Task in Humans. J Neurosci. 17(1997) 722-734 16. Jensen, O., Tesche, C.D.: Frontal theta activity in humans increases with memory load in a working memory task. Eur J Neurosci. 15 (2002) 1395-1399 17. Jensen, O. 4-D toolbox, version 1.1, A Matlab toolbox for the analysis of Neuromag Data. 18. Jurkiewicz, M.T., Gaetz, W.C., Bostan, A.C., Cheyne, D.: Post-movement beta rebound is generated in the motor cortex: Evidence from neuromagnetic recordings. NeuroImage 32 (2006) 1281-1289 19. Salmelin, R., Hamalainen, M., Kajola, M., Hari, R.: Functional segregation of movementrelated rhythmic activity in the human brain. Neuroimage. 2(1995) 237-243

A Wavelet Tool to Discriminate Imagery Versus Actual Finger Movements

333

20. Neuper, C., Pfurtscheller, G.: Post-movement synchronization of beta rhythms in the EEG over the cortical foot area in man. Neurosci Lett 216(1996) 17-20 21. Michelon, P., Vettel, J.M,, Zacks, J.M.: Lateral somatotopic organization during imagined and prepared movements. J Neurophysiol. 95 (2006) 811-822 22. Caldara, R., Deiber, M.P., Andrey C., Michel, M.C., Thut, G., Hauert, C.A.: Actual and mental motor preparation and execution: a spatiotemporal ERP study. Exp Brain Res. 159 (2004) 389-399 23. Pfurtscheller, G., Neuper, C., Flotzinger, D., Pregenzer, M.: EEG-based discrimination between imagination of right and left hand movement. Electroencephalogr Clin Neurophysiol. 103(1997) 642-651 24. Kincade, J.M., Abrams, R.A., Astafiev, S.V., Shulman, G.L., Corbetta, M.,:An eventrelated functional magnetic resonance imaging study of voluntary and stimulus-driven orienting of attention. J Neurosci. 25(2005) 4593-4604 25. Miyake, Y., Onishi, Y., Poppel, E.: Two types of anticipation in synchronization tapping, Acta Neurobiol Exp. (Wars) 64 (2004) 415-426 26. Heuer, H., Spijkers, W., Kleinsorge, T., van der Loo, H.: Period Duration of Physical and Imagery Movement Sequences Affects Contralateral Amplitude Modulation. Q J Exp Psychol A. 51 (1998) 755-779 27. Woodrow, H.: The effect of rate of sequence upon the accuracy of synchronization. J Exp Psychol 15(1932) 357-379 28. Bauer, M., Oostenveld, R., Peeters, M., Fries, M.: Tactile Spatial Attention Enhances Gamma-Band Activity in Somatosensory Cortex and Reduces Low-Frequency Activity in Parieto-Occipital Areas. J Neurosci. 26 (2006) 490-501 29. Pesonen M., Bjornberg CH., Hamalainen H., Krause CM.: Brain Oscillatory 1-30 Hz EEG ERD/ERS responses during the different stages of an auditory memory search task, Neurosci Lett. 399 (2006) 45-50