Intelligent tutors exploiting novel sensing modalities for decoding students’ attention Alvaro Soto1 , Felipe Orihuela-Espina2 , Diego Cosmelli1 , Cristian Alcholado1 , Patrick Heyer2 and L. Enrique Sucar2 1

2

Pontificia Universidad Cat´ olica de Chile, Santaigo, Chile ´ Instituto Nacional de Astrof´ısica, Optica y Electr´ onica, Puebla, Mexico

Abstract. To afford personalized instruction, Intelligent Tutoring Systems (ITS) require appropriate technologies to effectively access the internal state of each student. This includes attentional disposition, emotional attitude, and cognitive state in general. This work presents our initial steps towards building an ITS exploiting electroencephalography (EEG) and body posture to deduce relevant aspects of the attentional state of the student. Binarized attentional state of a student based on posture alone can be successfully discriminated with F-measure 76.47 ± 4.58. Emerging patterns in preliminary exploration of the EEG, still underway, suggest non-cued identification of attention is a feasible undertaking. Keywords: Cognitive Tutors, Attention Detection, EEG.

1

Introduction

Exploitation of underused sensing modalities combined with existing psychophysiological indexes of attentional state, emotion disposition and exploratory attitude can boost the possibilities of ITS. Electroencephalography (EEG) for attention [3] and computer vision technologies for posture [2] are among the amenable technologies to decode internal cognitive states of an ITS user.

2

Methodology

We focus on topics related to mathematical properties of fractions [1]. Activity is organised according to 5 levels of increasing difficulty. Progress across levels is achieved based on number of correct answers. Five volunteers in 5th grade plus other 3 in 7th grade of the chilean basic educational system participated (average age: 10.75 yrs) following parent or legal guardian prior consent. None had any history of neurological or psychiatric disorder, and IQs where in the average normal range. The experimental session was ended when the participant failed to advance any further in the task or reported boredom and/or fatigue to the experimenter using a small bell, ranging between 15 to 40 minutes approximately. Figure 1 illustrates the ITS interface and the experimental set-up.

a) Interface

b) Set-up

c) Examples of attentional episodes

Fig. 1. (a) User interface of the educational activity indicating the 4 main areas of the interface; (1) problem statement zone, (2) answer selection zone operable by scroll controls, (3) decision zone to confirm answer, and (4) feedback zone. (b) Experimental set-up showing a participant wearing an EEG to capture their brain activity during a session with our educational software. (c) Two examples of attentional episodes extracted during preliminary visual exploratory analysis of the EEG.

Digital EEG was obtained at 32 channels of the 10/20 system at 2048 Hz. Mastoid electrodes where recorded and used for off-line referencing. Simultaneous video stream accompanied by depth perception maps was recorded using a Kinect system (Microsoft, USA) at 15 fps for monitoring posture. A log-file of relevant actions on the educational software was also automatically recorded. Two coders labelled the data obtained during the task with specific behaviors that we expect to recover from the EEG data analysis.

3

Preliminary Results

Analysis of the multi-modal data is still in progress. Preliminary, visual exploratory analysis highlight a recurrent pattern following episodes of attention in the EEG (see Figure 1).

4

Conclusions

Incorporating, underused sensing modalities as EEG and Kinect to access the internal state of the student is a feasible endeavour that can potentially boost the capabilities of ITS to provide personalized education. Acknowledgements. Funded by Microsoft LACCIR project R1211LAC0001.

References 1. Alcoholado, C., Nussbaum, M., et al: One Mouse per Child: Interpersonal Computer for Individual Arithmetic Practice Journal of Computer Assisted Learning, 28, 295–309 (2012) 2. Heyer, P., Herrera-Vega, J., et al: Posture Based Detection of Attention in Human Computer Interaction 6th Pacific-Rim Symposium, PSIVT 2013, Guanajuato, Mexico, October 28-November 1, 2013, pp 220-229 3. Wang, Q., Sourina, O.: Real-time mental arithmetic task recognition from EEG signals. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 21, 225-232 (2013)