The use of EEG-based inverse models for both BCI design and brain activity vizualization in VR F. Lotte, M. Congedo, A. Lécuyer, C. Arrouet, F. Lamarche, J.-E. Marvie, B. Arnaldi
[email protected] June 13th, Cybertherapy 2006, symposium on prosthetics and orthotics training
1
Open-ViBE
Open-ViBE : An open-source software for Brain-Computer Interfaces (BCI) and Virtual Reality (VR) French national Project Duration : 3 years Beginning : December 15th 2005 4 Partners : INRIA/IRISA (VR, computer science), INSERM (medical/neurophysiology), France Télécom (signal processing, HCI), AFM (disabled people) Objective : develop an open-source software with innovative techniques for more efficient BCI Coordinator: Anatole Lécuyer (
[email protected]) June 13th, Cybertherapy 2006, symposium on prosthetics and orthotics training
2
Brain-Computer Interfaces (BCI)
feedback
Brain activity processing Measurement of ElectroEncephaloGrams (EEG)
Translation into a command
June 13th, Cybertherapy 2006, symposium on prosthetics and orthotics training
3
Contents
Introduction Inverse model and sources localization Brain activity visualization in Virtual Reality BCI design using an inverse model Conclusion
June 13th, Cybertherapy 2006, symposium on prosthetics and orthotics training
4
Contents
Introduction Inverse model and sources localization Brain activity visualization in Virtual Reality BCI design using an inverse model Conclusion
June 13th, Cybertherapy 2006, symposium on prosthetics and orthotics training
5
Inverse models and sources localization
The problem
EEG are the scalp measurents m resulting from the mixing A of several unknown sources s
m=As
Inverse models
Reconstruct the sources s from the scalp measurements m
s=Tm
Advantage
Enable the localization of active sources in the brain, using EEG June 13th, Cybertherapy 2006, symposium on prosthetics and orthotics training
6
Contents
Introduction Inverse model and sources localization Brain activity visualization in Virtual Reality BCI design using an inverse model Conclusion
June 13th, Cybertherapy 2006, symposium on prosthetics and orthotics training
7
3D visualization in VR
Representation of the current density reconstructed in the brain volume (2394 points)
Use of the LORETA inverse model [Pascual-Marqui94] Real-time representation
(Arrouet et al., Journal of Neurotherapy, 2005) June 13th, Cybertherapy 2006, symposium on prosthetics and orthotics training
8
Visualization of the global brain activity
June 13th, Cybertherapy 2006, symposium on prosthetics and orthotics training
9
Visualization of a local brain activity
June 13th, Cybertherapy 2006, symposium on prosthetics and orthotics training
10
Contents
Introduction Inverse model and sources localization Brain activity visualization in Virtual Reality BCI design using an inverse model Conclusion
June 13th, Cybertherapy 2006, symposium on prosthetics and orthotics training
11
BCI and 3D localization of sources
Context: most EEG-based BCI only use surface information Idea: reconstruct, from EEG, the volume information How to: use a source localization method Advantage: obtention of a spatial information
Identification with a physiological meaning June 13th, Cybertherapy 2006, symposium on prosthetics and orthotics training
12
General principal
How identifying the “brain pattern” ?
Preprocessing
Feature Extraction
Classification
EEG
Frequency and spatial filtering
Activity in specific brain regions
Linear classifier
(using source localization)
June 13th, Cybertherapy 2006, symposium on prosthetics and orthotics training
13
Experimental Validation
Data set IV of the « BCI competition 2003 » [Blankertz et al, 04]
Protocol : self paced finger tapping experiment
Two data sets
2 Classes: left/right Training set Test set
Objective of the competition : obtain the highest recognition rate on the test set Winner score : 84% [Wang et al, 04]
June 13th, Cybertherapy 2006, symposium on prosthetics and orthotics training
14
Preprocessing: filtering
Frequency filtering
Objective: Enhance relevent information Method: filtering in rhythms µ (10-15 Hz) and β (> 15 Hz) [Dornhege et al, 04] [Müller et al, 04] [Wang et al, 04]
Spatial filtering
Objective: Reduce the noise and increase the discriminative power Method: Common Spatial Pattern [Ramoser et al, 99]
June 13th, Cybertherapy 2006, symposium on prosthetics and orthotics training
15
Sources localization
Method : sLORETA [Pascual-Marqui02] Principle : computation of the current density γλ in the region or voxel λ, using measurements vt at instant t
γ λ = v Rλ vt T t
Advantage : perfect localization
June 13th, Cybertherapy 2006, symposium on prosthetics and orthotics training
16
Identifying Region Of Interest (ROI)
Method
Statistical analysis [Holmes et al, 96]
Principle
Determine the voxels whose activity makes it possible to discriminate the two classes
Obtention of two ROI (left and right motor area)
June 13th, Cybertherapy 2006, symposium on prosthetics and orthotics training
17
identification
Feature extraction
Two features (one per ROI): Activity in the left ROI, averaged over time Activity in the right ROI, averaged over time
Classification
Linear classifier [Duda et al, 01]
June 13th, Cybertherapy 2006, symposium on prosthetics and orthotics training
18
Results
Test set classification :
Using a linear classifier : 84 % (same score as the winners)
Shows the interest of the method
(Congedo, Lotte and Lecuyer, Physics in Medicine and Biology, 2006) June 13th, Cybertherapy 2006, symposium on prosthetics and orthotics training
19
Conclusion
Two approaches using inverse models
Brain activity visualization in VR Brain-Computer Interface design
Novelty and efficiency of the methods Several improvements possible… …research is going on
the Open-ViBE project
June 13th, Cybertherapy 2006, symposium on prosthetics and orthotics training
20
Questions ?
fabien lotte
[email protected] June 13th, Cybertherapy 2006, symposium on prosthetics and orthotics training
21
