Tactile exploration of virtual objects for blind and sighted people: the role of beta 1 EEG band in sensory substitution and supramodal mental mapping

C. Campus, L. Brayda, F. De Carli, R. Chellali, F. Famà, C. Bruzzo, L. Lucagrossi and G. Rodriguez J Neurophysiol 107:2713-2729, 2012. First published 15 February 2012; doi:10.1152/jn.00624.2011 You might find this additional info useful... This article cites 93 articles, 26 of which can be accessed free at: http://jn.physiology.org/content/107/10/2713.full.html#ref-list-1 Updated information and services including high resolution figures, can be found at: http://jn.physiology.org/content/107/10/2713.full.html Additional material and information about Journal of Neurophysiology can be found at: http://www.the-aps.org/publications/jn

Journal of Neurophysiology publishes original articles on the function of the nervous system. It is published 12 times a year (monthly) by the American Physiological Society, 9650 Rockville Pike, Bethesda MD 20814-3991. Copyright © 2012 by the American Physiological Society. ISSN: 0022-3077, ESSN: 1522-1598. Visit our website at http://www.the-aps.org/.

Downloaded from jn.physiology.org on July 10, 2012

This information is current as of July 10, 2012.

J Neurophysiol 107: 2713–2729, 2012. First published February 15, 2012; doi:10.1152/jn.00624.2011.

Tactile exploration of virtual objects for blind and sighted people: the role of beta 1 EEG band in sensory substitution and supramodal mental mapping C. Campus,1,3 L. Brayda,1 F. De Carli,2 R. Chellali,1 F. Famà,3 C. Bruzzo,1,3 L. Lucagrossi,4 and G. Rodriguez3 1 3

Istituto Italiano di Tecnologia; 2Institute of Molecular Bioimaging and Physiology, Consiglio Nazionale delle Ricerche; Clinical Neurophysiology Unit, University of Genoa; and 4Istituto David Chiossone onlus, Genoa, Italy

Submitted 3 July 2011; accepted in final form 11 February 2012

spatial processing; visuo-spatial pathways; cognitive mapping; virtual reality

gathering, encoding, storing, and retrieving spatial information, is fundamental in everyday tasks. This is true when mapping spaces near and far from the body, for example, in manipulation and navigation, respectively. Both sighted and nonsighted people have shown such abilities using various sensory modalities (Kupers et al. 2010; Seemungal et al. 2007). It has been found that specific brain COGNITIVE MAPPING, THAT IS,

Address for reprint requests and other correspondence: C. Campus, Istituto Italiano di Tecnologia, Via Morego, 30, I 16163, Genoa, Italy (e-mail: [email protected]). www.jn.org

areas, including the hippocampus, posterior mesial lobe, and posterior parietal, occipital, and infero-temporal cortices, play an important role in spatial processing (Bird and Burgess 2008; Ekstrom et al. 2003; Kravitz et al. 2011; Latini-Corazzini et al. 2010; Maguire et al. 1998; Wascher et al. 2009). Spatial cues are sometimes unavailable, unfamiliar, or confusing when a real environment is explored. In such cases, useful supplementary, prior information can be obtained from tactile maps. The sense of touch has been shown to be effective for exploring maps of real (Picard et al. 2009) and virtual (Lahav and Mioduser 2008) environments. The representation of space through maps is particularly important for and preserved in visually impaired and blind individuals (Fortin et al. 2008; Loomis et al. 1993; Thinus-Blanc and Gaunet 1997). However, the neural correlates of mental mapping in blindness and their dependence on the sensory modality remain elusive. The sense of touch is able to carry information normally acquired through vision. In fact, during tactile cognitive tasks, activations have been observed in the visual cortex of blind people (Bavelier and Neville 2002; Burton et al. 2002a, b; Cohen et al. 1997; Kriegseis et al. 2006; Kujala et al. 1997; Lewis et al. 2010; Merabet and Pascual-Leone 2010), as well as in sighted subjects (Amedi et al. 2010; Deibert et al. 1999; Deshpande et al. 2008; Hartmann et al. 2008; Kägi et al. 2010; Reed et al. 2004, 2005; Sathian 2005; Stoeckel et al. 2003). This transfer of sensory information from one sense to another (e.g., touch to vision) is called sensory substitution (Bach-yRita and Kercel 2003; Visell 2009). This phenomenon can link sensory information to cognitive representations. In the case of touch-to-vision sensory substitution, tactile cues can potentially convey the content of images or more abstract representations, such as maps, possibly eliciting visuo-spatial processing. However, the mechanisms that could elicit a visuo-spatial representation from a tactile stimulation are necessarily complex, because discovering an object with touch implies the stimulation of several types of mechanoreceptors (Dahiya et al. 2010), continuous motor programming and proprioception (Fiehler and Rösler 2010), and hypothesis generation and testing in the working memory (Kupers et al. 2010). Classical visual neuroscience (Goodale et al. 1991) hypothesizes that there are two distinct streams— one dorsal and one ventral— for cortical visual processing. The more recent “three streams” hypothesis (Kravitz et al. 2011) considers projections from the occipital cortex to the prefrontal and premotor cortices and a projection to the medial temporal lobe. These three pathways seem to support different types of visuo-spatial processing and

0022-3077/12 Copyright © 2012 the American Physiological Society

2713

Downloaded from jn.physiology.org on July 10, 2012

Campus C, Brayda L, De Carli F, Chellali R, Famà F, Bruzzo C, Lucagrossi L, Rodriguez G. Tactile exploration of virtual objects for blind and sighted people: the role of beta 1 EEG band in sensory substitution and supramodal mental mapping. J Neurophysiol 107: 2713–2729, 2012. First published February 15, 2012; doi:10.1152/jn.00624.2011.—The neural correlates of exploration and cognitive mapping in blindness remain elusive. The role of visuo-spatial pathways in blind vs. sighted subjects is still under debate. In this preliminary study, we investigate, as a possible estimation of the activity in the visuo-spatial pathways, the EEG patterns of blind and blindfolded-sighted subjects during the active tactile construction of cognitive maps from virtual objects compared with rest and passive tactile stimulation. Ten blind and ten matched, blindfolded-sighted subjects participated in the study. Events were defined as moments when the finger was only stimulated (passive stimulation) or the contour of a virtual object was touched (during active exploration). Event-related spectral power and coherence perturbations were evaluated within the beta 1 band (14 –18 Hz). They were then related to a subjective cognitive-load estimation required by the explorations [namely, perceived levels of difficulty (PLD)]. We found complementary cues for sensory substitution and spatial processing in both groups: both blind and sighted subjects showed, while exploring, late power decreases and early power increases, potentially associated with motor programming and touch, respectively. The latter involved occipital areas only for blind subjects (long-term plasticity) and only during active exploration, thus supporting tactileto-visual sensory substitution. In both groups, coherences emerged among the fronto-central, centro-parietal, and occipito-temporal derivations associated with visuo-spatial processing. This seems in accordance with mental map construction involving spatial processing, sensory-motor processing, and working memory. The observed involvement of the occipital regions suggests that a substitution process also occurs in sighted subjects. Only during explorations did coherence correlate positively with PLD for both groups and in derivations, which can be related to visuo-spatial processing, supporting the existence of supramodal spatial processing independently of vision capabilities.

2714

SENSORY SUBSTITUTION/MENTAL MAPPING IN BLIND/SIGHTED PEOPLE

Hz), due to its role in maintaining (Kopell et al. 2011) cell assemblies, i.e., in short-term memory (Tallon-Baudry et al. 2001, 2004) and complex associative functions (Weiss and Rappelsberger 1996). The aim of this study is to make a preliminary analysis of EEG activity in blind and sighted subjects while they build mental maps from virtual objects, sensed with a tactile device (Brayda et al. 2011). We focused on the beta 1 band, considering previous indications concerning its role in information processing, visuo-motor activities, and working memory. In this context, we compared active tactile exploration both with a rest and with a passive tactile stimulation condition. Moreover, we evaluated the success and the subjective cognitive load of this exploration. We investigated the existence of possible EEG sensory substitution and spatial-processing cues that link tactile perception to mental mapping. Specifically, we hypothesized that the nature of these cues and their link with mental mapping could be better clarified by their possible relationships with the subjective cognitive load. We also hypothesized that the spatial cues could be independent of vision, which would result in similar results for both sighted and blind subjects. The following questions were addressed in this study. 1) Is it possible to elicit visuo-tactile sensory substitution during nonvisual exploration? This hypothesis predicts EEG patterns, suggesting an involvement of the occipital area during tactile exploration tasks. The hypothesis is further supported if the suggested involvement is exclusively present or at least more pronounced in active tactile cognitive mapping tasks, whereas absent or weaker in pure, passive tactile stimulation. The sensory substitution process could be reflected in the occipital derivations by local beta 1 power or coherence increases with visuo-spatial-related derivations. 2) Do EEG observations support the involvement—possibly vision independent— of visuo-spatial pathways during mental mapping? If this hypothesis is true, we would expect to find high coherences, suggesting functional coupling, between derivations usually considered as related to these pathways in both blind and sighted subjects. Moreover, these coherences would be present during an active tactile mental mapping task and absent during passive tactile stimulation and rest. 3) Is there a relationship between EEG and cognitive load in successful mental mapping using minimal tactile information? If correlations exist between a subset of EEG parameters and a measure of perceived difficulty and if these correlations are present in active explorations, while absent in passive stimulations and rest, then correlated EEG parameters could represent an indirect estimation of cognitive load, at least during a mental mapping task. 4) Are there cues allowing us to consider mental mapping as supramodal in this task? If so, both blind and sighted should exhibit: a) a similar capability to construct the mental map of the explored object; b) similar subjective cognitive loads; c) EEG patterns suggesting an involvement, in a nonvisual task, of scalp derivations related to visuo-spatial processing; and d) a similar relationship between subjective and objective parameters. We found some preliminary evidence from two complementary parameters [event-related spectral perturbations (ERSPs) and coherence] for sensory substitution and mental mapping. The involvement of these processes is only suggested during active explorations, whereas not during rest and passive stim-

J Neurophysiol • doi:10.1152/jn.00624.2011 • www.jn.org

Downloaded from jn.physiology.org on July 10, 2012

are thought to be, respectively, related to spatial working memory (“Where”), visually guided action (“How”), and object recognition (“What”). However, whether this scheme applies equally to blind and sighted people remains a matter of debate. If that were the case, the streams would reflect cognitive processes less necessarily linked with vision only. Then, these processes could also be triggered by other modalities, possibly through an underlying sensory substitution. To verify this hypothesis, one expects to find neurophysiological measures also in a tactile spatial processing task, possibly reflecting activations in the visual cortex. This should occur only when tactile stimulation is linked to a meaningful spatial representation. One also expects that visuospatial pathways would be involved when elicited by touch, similarly as reported for visual information processing. Additional, nonphysiological parameters, e.g., reflecting subjective cognitive load during spatial exploration, may be coupled with physiological patterns related to the pathways, thus reinforcing the hypothesis. To better clarify whether the streams rely on existing visual-learned patterns, i.e., whether vision is involved in the tactile-stimulus processing aimed to the construction of cognitive maps, one should observe similar phenomena in sighted as well as in early blind subjects. If these expectations are met, one might argue that tactile stimulation can, under certain circumstances, activate supramodal brain circuits (Amedi et al. 2010). In this vein, functional MRI data showed overlapping activation patterns in the dorsal stream (Fiehler et al. 2010) for kinesthetically guided hand movements in congenitally blind and sighted people. Activations in the parahippocampus and visual cortex during tactile navigation were reported in blind but not in blindfolded-sighted subjects and were restored for sighted subjects when the blindfold was removed (Kravitz et al. 2011). The uncertainty concerning the neural correlates of spatial processing is partly due to the difficulty in testing the spatial exploration skills of blind subjects within the constraints of a functional imaging study. EEG is a noninvasive measure with a high temporal resolution. EEG spectral power and coherence can be used to investigate cognitive processes. Coherence (Nunez et al. 1999) measures the correlation between two signals in the frequency domain, possibly reflecting functional couplings between cortical areas (Fries 2005) at rest (Chorlian et al. 2009; Tucker et al. 1986) or in active tasks. Scalp EEG coherence between central parietal and posterior areas was used to investigate visual (Goto et al. 2004), visuo-motor/ visuo-spatial (Babiloni et al. 2006; Classen et al. 1998; Del Percio et al. 2011), and audio-spatial (Leclerc et al. 2005) processing. The frequencies showing high coherence can suggest possible underlying processes: theta (4 – 8 Hz) coherence can reflect executive functions (Sauseng et al. 2005), whereas gamma (30 –90 Hz) has been related to information integration (Lachaux et al. 2005; Meador et al. 2002). The beta (14 –30 Hz) band is influenced by somatosensory perception (Lalo et al. 2007; Tecchio et al. 2003) and motor planning/execution (Perfetti et al. 2011; Ritter et al. 2009). It is also involved in top-down processing (Buschman and Miller 2007, 2009; Kukleta et al. 2009) and multimodal model matching (Brovelli et al. 2004; Engel and Fries 2010; Kopell et al. 2000; Okuhata et al. 2009; Spitzer and Blankenburg 2011). This involvement is even stronger in the beta 1 band (14 –18

SENSORY SUBSTITUTION/MENTAL MAPPING IN BLIND/SIGHTED PEOPLE

Table 1. Characteristics of the blind participants Blind Participant

Age (Years)

Gender

1 2 3 4 5 6 7 8 9 10

19 22 23 29 30 33 38 42 54 61

M M M F F M M M M M

Aetiology of Blindness retinitis pigmentosa glaucoma congenital glaucoma congenital glaucoma accident retinitis pigmentosa glaucoma congenital cataract infectious disease retinitis pigmentosa

Age at Onset of Blindness (Years) 3 5 birth birth 7 birth 6 birth 8 birth

MATERIALS AND METHODS

Subjects and Experimental Procedure Ten blind subjects, whose characteristics are summarized in Table 1, and a control group of ten age- and sex-matched, blindfolded volunteers with normal or corrected-to-normal vision participated in the study. Approval by the local ethics research committee in Genoa and

a written, informed consent according to the Declaration of Helsinki were obtained. Classification of visual impairment was based on the criteria suggested by Burton (2003) and integrated by the International Federation of Clinical Neurophysiology (Holder et al. 2010). All participants had no medical factors that could influence experimental results, and all were naive to the procedure. All subjects were right handed. Each subject was seated comfortably at a desk while EEG was recorded continuously, both at rest (baseline) and while exploring one virtual environment (VE) with a tactile device (Brayda et al. 2011) as part of our Digital Eye (DIGEYE) system (Chellali et al. 2009), as represented in APPENDIX A (see Fig. A1). Experimental Protocol 1) Baseline. During the first part (baseline) of the experiment, EEG was recorded during a pure resting condition for 2 min with subjects in a relaxed, awake state with closed eyes. 2) Passive stimulation. In a second part (passive stimulation), subjects kept their right hand on the tactile mouse-shaped device (TAMO; see APPENDIX A) at rest, while passively receiving tactile stimulations. The lever of the device randomly raised (time intervals between 800 and 1,200 ms), always reaching the same height. The passive stimulation period lasted 10 s and was repeated 10 times. Each period of stimulation was started and stopped by two distinct sounds (two clearly distinguishable sawtooth waveforms— one with a fundamental frequency of 220 Hz and the other with one of 880 Hz, lasting 100 ms) and was followed by 10 s of rest. 3) Active exploration. Then, the subjects used the TAMO device to actively explore one unknown VE, represented in this study by only one virtual object (a one-level, square parallelepiped), placed at the center of an empty, flat surface (Fig. 1). The height of the parallelepiped was the same as the one used in the passive condition. Subjects had neither prior information about the number of objects nor about their shape, size, or number of height levels. They were requested to mentally construct the object’s shape while exploring it with touch by

Fig. 1. The experimental protocol. 1) Baseline: subjects stay at rest. 2) Passive tactile stimulation: subject’s finger is passively stimulated by the randomly raising lever. Subjects are stimulated (Sti) 10 times for 10 s. Rest (10 s) is inserted in between subsequent stimulations. Passive events are defined as transitions between ground level and level 1. 3) Active exploration: subjects explore a unique, unknown object. The object is a 1-level, virtual square parallelepiped, explored (Exp) 30 times for 10 s. Rest (10 s) is inserted in between subsequent explorations. Active events are defined as transitions between ground level and level 1. EEG is continuously recorded and then analyzed in rest and separately, in passive/active time frames, [⫺200 400] ms around corresponding (passive/active) events (see the white arrows). 4) Cognitive load estimation: subjects vote perceived levels of difficulty (PLD) at the end of the experiment. 5) Evaluation of object construction: see APPENDIX A for additional details regarding setup and the tactile device. J Neurophysiol • doi:10.1152/jn.00624.2011 • www.jn.org

Downloaded from jn.physiology.org on July 10, 2012

ulation conditions. This interpretation seems to be confirmed by the positive correlation found between coherences and subjective cognitive load: the correlation emerged only during active explorations and involved derivations related in literature to visual and visuo-spatial processing. Moreover, this relationship was observed for both blind and sighted subjects. These results support the possibility of supramodal spatial processing abilities existing independently of vision capabilities.

2715

2716

SENSORY SUBSTITUTION/MENTAL MAPPING IN BLIND/SIGHTED PEOPLE

EEG Recording and Processing In this study, we focused on EEG parameters at rest, as well as before/after the transitions between ground level (no tactile stimulation) and level 1 (tactile stimulation). During the passive stimulation, blind and sighted subjects received a total of 1,123 (mean 112; SD 25) and 1,094 (109 ⫾ 18) tactile transitions (passive events), respectively. Only in the active exploration did these transitions correspond to the moments in which the borders of the virtual object emerged from the empty space and were felt by the subject’s fingertip, due to the upward movement of the lever. During the active exploration, subjects were free to move the device on the tablet, and the number of these transitions (active events) was determined by the exploration strategy chosen by each subject: blind subjects showed a total of 1,004 active events (for each subject and across all 30 explorations: mean 100; SD 62), whereas sighted subjects showed 744 (74 ⫾ 31). The total number of active events was 1,748. No significant differences (N.S.) between groups arose from the paired t-test {t ⫽ 1.1; df ⫽ 9; 95% confidence interval (CI) on the difference of population means [⫺26.3, 78.3]; N.S.}. We did not include the opposite transitions (i.e., from level 1 to ground level) in our elaborations. EEG was recorded using Ag/AgCl cup electrodes attached to the scalp (previously prepared with abrasive cream) with conductive cream at 14 active sites (F3, C3, P3, O1, T3, F4, C4, P4, O2, T4, Fz, Cz, Pz, Oz; see Fig. A3), according to the international 10/20 system. Electrooculogram (EOG), ECG, and electromyogram (EMG) were also acquired by means of cup electrodes applied with adhesive tape, respectively, around the orbits to the precordial region and to the submental region, that is, under the chin, according to the standard electrode placement in polysomnography (Iber et al. 2007). A Mizar 40 amplifier (EBNeuro, Florence, Italy) was used. After being passed through a low-pass antialiasing filter, set at 256 Hz, data were digitally sampled and stored at 1,024 Hz. The experimental video and the track of the finger movements, as drawn from the device coordinates [X, Y, and height (H) of the VE], were recorded as a behavioral check. The following analyses concerned bipolar derivations (see Fig. A3), which were supposed to reflect cerebral activities along the pathways involved in spatial exploration (F3-C3, C3-P3, P3-T3, and O1-T3 on the left side and the homologous derivations on the right side). EEG activities within each hemisphere and interhemispheric differences were analyzed, whereas electrodes at midline scalp loca-

tions were not analyzed in this study but were used in data preprocessing (to remove the EEG artifact). All signals were digitally filtered (1–50 Hz pass band), and ocular, cardiac, and muscular artifacts were removed from the EEG data using visually inspected, independent component analysis. The intermediate NNNM-filtering and maximal EMG correction were used in this study, according to procedures described recently by McMenamin et al. (2010). The selected bipolar derivations were then postprocessed using wavelet transform and interevents averaging to analyze the spatial distribution of ERSPs (i.e., powers) and increases/decreases in coherences. EEGLAB software toolbox (Delorme and Makeig 2004) was used for signal processing. The following analyses were focused on the spatiotemporal evolution of the beta 1 band. Statistical Analysis Log-transformed power and hyperbolic arctangent-transformed coherence (Tecchio et al. 2003) within the beta 1 band were computed for over four time intervals (epochs around the passive and the active events separately). Those epochs were the initial rest period, and the three intervals defined within the (passive or active) time frame around the (passive or active) tactile stimulus ([⫺200 ⫹400] ms) include, namely: 1) prestimulus (Pre) [⫺200 0] ms; 2) early poststimulus (E-post) [0 ⫹300] ms; and 3) late poststimulus (L-post) [⫹300 ⫹400] ms. Here, 0 ms refers to stimulus onset. In a first analysis, data were averaged within subjects, and experimental effects were evaluated for passive and active events separately by two distinct group ⫻ epoch ⫻ derivation ⫻ hemisphere repeated measures ANOVA, followed by post hoc tests with Bonferroni correction, using group (blind or sighted) as a between-subject factor and epoch, hemisphere, and derivation as within-subject factors. P values were corrected according to the Huynh-Feldt method (Leclerc et al. 2005). In a second analysis, two conditions were considered: passive Pre and active Pre. Data were evaluated by a group ⫻ derivation ⫻ hemisphere ⫻ condition (passive or active) repeated measures ANOVA post hoc. In a third analysis, the poststimulus epochs (E-post and L-post) were compared between active and passive condition. Data were normalized for each condition independently by removing, for each subject, derivation and hemisphere, the average of the corresponding Pre. Then, an evaluation was performed by a group ⫻ epoch (E-post or L-post) ⫻ derivation ⫻ hemisphere ⫻ condition (passive or active) repeated measures ANOVA post hoc. In a fourth analysis, we considered only those derivations where ANOVA and post hoc (of the first analysis) revealed, at least in one condition, significant power or coherence variations with respect to the Pre. For these derivations, the temporal evolution of spectral parameters was analyzed with a higher time resolution. The time interval characterized by event-related variations (increase or decrease) was assessed (Allena et al. 2009) as follows: the average pattern for each group and derivation was evaluated with a temporal resolution of 10 ms. Pre epoch was considered as background: mean and SD of its average pattern were computed. A CI was then set for each group and derivation, including values falling within three SDs from the background mean value (corresponding to a nominal significance level of 0.01). An increase/decrease (e.g., peak/valley) was detected when parameter (ERSP or coherence) values continuously exceeded the 3-SD limit for more than 30 ms: this time constraint was empirically chosen to detect only periods of meaningful variation (e.g., peaks and valleys) from average Pre values and to reduce the probability of false detections. The difference in PLDs between blind and blindfolded-sighted subjects was evaluated by performing a paired t-test between the two groups of age- and sex-matched subjects. Possible associations among ERSP and coherence, independently considered, and PLD were evaluated using linear regression. Specif-

J Neurophysiol • doi:10.1152/jn.00624.2011 • www.jn.org

Downloaded from jn.physiology.org on July 10, 2012

freely moving the TAMO device on a sensing tablet. Upward and downward movements of a lever on the top of the TAMO indicated height changes in the VE (see APPENDIX A for details). Subjects explored the environment 30 times, each time for 10 s. The object presented did not change over subsequent trials. Each exploration was started and stopped by the same distinct sounds used in the passive tactile stimulation and was followed by 10 s of rest. 4) Cognitive load estimation. As a subjective estimation of cognitive load, each subject was requested to rate the level of difficulty encountered in the mental construction of the VE. These perceived levels of difficulty (PLDs) were ranked on a scale from 1 to 10 (higher grades corresponded to higher perceived difficulty). The ranking was made by all of the participants upon completion of the experiment. 5) Evaluation of object construction. As a last step, subjects provided a free description of the explored environment. The description was verbal (for all subjects) or graphical (for sighted subjects only). Then, they answered four questions: a) “Apart from the ground level, how many other different levels were present in the environment?” b) “How many objects did you identify in the environment?” c) “What was the contour of each object?” d) “Where were objects located in the tablet?” The mapping was classified as “successful” if and only if subjects answered, “One level”, “One object”, “Squared shape”, and “In the center of the tablet”, respectively. These four answers matched the object in question.

SENSORY SUBSTITUTION/MENTAL MAPPING IN BLIND/SIGHTED PEOPLE

ically, for each condition (passive or active), average ERSP and coherence were calculated for each subject and derivation, for each epoch (rest, Pre, E-post, or L-post), and over the entire (passive/ active) time frame. The relationship with PLDs was then investigated. Moreover, possible correlations between PLD and the averages of significant (Allena et al. 2009) positive or negative deflections (peaks or valleys, respectively) were only investigated for derivations in which significant variations compared with Pre were revealed by the ANOVA post hoc in the previous analysis. R software was used for all statistical analyses (R Foundation for Statistical Computing R Development Core Team 2010). RESULTS

Success of Object Construction Since all 20 subjects provided correct descriptions and answers about the explored object, the exploration and the consequent construction process can be considered successful for the global sample and for both groups.

As for EEG beta 1 power, the distribution of ERSP mean values as a function of the four factors (group ⫻ epoch ⫻ derivation ⫻ hemisphere) considered in ANOVA is shown in Fig. 2 for the passive and active condition. Considering the passive stimulation, main effects of epoch (F3,54 ⫽ 6.1; P ⬍ 0.01) and derivation (F3,54 ⫽ 5.8; P ⬍ 0.01) were found, whereas the effects of hemisphere

(F1,18 ⫽ 0.98) and group (F1,18 ⫽ 0.77) were nonsignificant. A significant derivation ⫻ epoch interaction (F6,108 ⫽ 3.3; P ⬍ 0.01) was found. Post hoc comparisons are illustrated in Fig. 2. Only the E-post was different with respect to the other epochs: it showed higher powers (P ⬍ 0.001), both compared with rest and to the other epochs in the passive time frame. The significant (P ⬍ 0.01), early increase was found in all but the occipito-temporal (OT) derivations of both groups. We also analyzed, with a higher time resolution, the derivations with significant (ANOVA and post hoc) spectral power changes. We marked the time segments in which beta 1 power continuously exceeded the 3-SD limit for more than 30 ms (as described in Statistical analysis). Power increases were detected (Fig. 3) within a time interval ranging from 120 to 180 ms after the passive tactile stimulus. Considering the active exploration, the effect of epoch was highly significant (F3,54 ⫽ 12.9; P ⬍ 0.00001). A main effect for derivation was found (F3,54 ⫽ 4.0; P ⬍ 0.05). No significant effects were found for hemisphere (F1,18 ⫽ 0.98), whereas the group just failed to reach the standard significance level (F1,18 ⫽ 4.0; P ⫽ 0.06). A significant interaction was observed between group and epoch (F3,54 ⫽ 5.6; P ⬍ 0.01). A weaker, though significant, interaction was observed between group and derivation (F3,54 ⫽ 2.8; P ⬍ 0.05) and a stronger threefactor group ⫻ epoch ⫻ derivation interaction (F9,162 ⫽ 2.9; P ⬍ 0.01) was also seen.

Fig. 2. Event-related spectral perturbations (ERSP) in the beta 1 band at rest and in 3 epochs—prestimulus (Pre), early-poststimulus (E-post), and late-poststimulus (L-post). Mean and SD of log-transformed powers for each derivation and epoch. Asterisks indicate, where present, a significant (P ⬍ 0.05 at post hoc test) variation, separately for active/passive time frame, each compared with its corresponding Pre epoch ([⫺200 0] ms). Left: passive time frame. In the E-post epoch ([0 300] ms), both groups showed a power increase in all but occipito-temporal (OT) derivations. Right: active time frame. All epochs in the active time frame ([⫺200 400] ms) were lower in all derivations compared with rest and with the passive time frame. In the E-post epoch ([0 300] ms), only blind subjects (E and F) showed power increases, also in OT derivations (not involved for sighted), whereas elsewhere, both groups feature power increases. Everywhere, power variations in the active poststimulus were greater than in the passive poststimulus. J Neurophysiol • doi:10.1152/jn.00624.2011 • www.jn.org

Downloaded from jn.physiology.org on July 10, 2012

ERSPs

2717

2718

SENSORY SUBSTITUTION/MENTAL MAPPING IN BLIND/SIGHTED PEOPLE

Downloaded from jn.physiology.org on July 10, 2012

Fig. 3. Mean ERSP patterns for blind (black circles) and sighted (gray triangles) subjects. Each circle/triangle corresponds to an interval of 10 ms. For each group, significant variations (larger than 3⫻ SD) with respect to average Pre background are indicated for blind (horizontal black lines) and sighted (horizontal gray lines) people below curves. Left: passive time frame. Both groups showed similar patterns with specular power increases in all derivations except OT. Right: active time frame. Both groups showed power increases in all derivations except OT. For both groups, these increases were generally longer and higher in the active time frame. The 2 groups showed some differences in patterns. Only blind subjects, generally having higher increases, also evidenced a peculiar OT involvement, which was absent in the passive time frame.

Post hoc comparisons are illustrated in Fig. 2. All epochs in the active time frame proved significantly lower with respect to rest (P ⬍ 0.0001 for Pre and L-post; P ⬍ 0.01 for E-post). Compared with rest, all derivations showed a significant de-

crease (always P ⬍ 0.01) in power for each epoch in the active time frame, whereas no significant differences emerged in basal conditions between the two groups. Within the active time frame, compared with the Pre epoch, higher (P ⬍ 0.001)

J Neurophysiol • doi:10.1152/jn.00624.2011 • www.jn.org

SENSORY SUBSTITUTION/MENTAL MAPPING IN BLIND/SIGHTED PEOPLE

Pre), the effect of condition was weakly significant (F1,18 ⫽ 5.6; P ⬍ 0.05). The effect of epoch was significant (F1,18 ⫽ 15.5; P ⬍ 0.01), as well as the effect of derivation (F3,54 ⫽ 4.4; P ⬍ 0.01). No significant effects were found for hemisphere (F1,18 ⫽ 0.68) and group (F1,18 ⫽ 0.55). Highly significant epoch ⫻ condition (F1,18 ⫽ 24.9; P ⬍ 0.001), epoch ⫻ condition ⫻ derivation (F3,54 ⫽ 8.6; P ⬍ 0.0001), and epoch ⫻ condition ⫻ derivation ⫻ group (F3,54 ⫽ 9.2; P ⬍ 0.0001) interactions were found. In the post hoc analysis, powers of active E-post resulted significantly higher (P ⬍ 0.001) compared with passive E-post for all derivations in blind subjects and in all but OT derivations for sighted subjects. Powers in the active L-post resulted significantly (P ⬍ 0.0001) lower than those of the passive L-post in all derivations. Coherences The distribution of mean spectral coherence between pairs of derivations as a function of ANOVA factors (group, epoch, location, hemisphere) for the passive and active conditions is presented in Fig. 4. Considering the passive time frame, a hemisphere effect emerged (F1,18 ⫽ 8.3; P ⬍ 0.01) with higher values on the right, whereas the derivation effect was highly significant (F3,54 ⫽ 10.1; P ⬍ 0.00001), with coherences increasing from anterior to posterior derivations. The coherences were not different comparing the two groups (F1,18 ⫽ 0.7; N.S.), as well as comparing the epochs (F3,54 ⫽ 1.2; N.S.). Post hoc comparisons are illustrated in Fig. 4. No significant variations were

Fig. 4. Coherences in the beta 1 band at rest and in 3 epochs—Pre, E-post, and L-post. Identical time frames are used for the 3 epochs, as shown in Fig. 2. Mean and SD are represented for each derivation pair and epoch. Asterisks indicate, where present, a significant (P ⬍ 0.05 at post hoc test) variation, separately for the active/passive time frame, each compared with its corresponding Pre epoch. Left: passive time frame. No variations were observed compared with rest and Pre. The right hemisphere presented higher coherences. Right: active time frame. All derivation pairs presented, for both groups, higher coherences during the entire active time frame compared with rest and with the passive time frame. In the E-post epoch, only blind subjects showed (E and F) coherence increases in fronto-central (FC)-centro-parietal (CP) and CP-OT pairs. J Neurophysiol • doi:10.1152/jn.00624.2011 • www.jn.org

Downloaded from jn.physiology.org on July 10, 2012

and lower (P ⬍ 0.001) beta 1 power occurred in the E-post and in the L-post epochs, respectively. The early increase was found in all derivations of blind subjects (P ⬍ 0.01) and in all but the OT derivations of sighted subjects (P ⬍ 0.05); moreover, in all derivations, the increase was stronger in blind than in sighted subjects (P ⬍ 0.05), with the maximum difference being found in the OT derivations (P ⬍ 0.01). As for comparison between derivations in the E-post epoch, the only significant difference was found in sighted people for the OT derivation, which showed a lower ERSP (P ⬍ 0.05) with respect to other derivations. As for the L-post epoch, a beta 1 power decrease (P ⬍ 0.01) was detected for both groups in all derivations, except the fronto-central (FC) locations (N.S.). The mean variation of beta 1 power in the FC derivation was significantly lower than the decrease found in all of the other derivations. We also analyzed, with a higher time resolution, the derivations with significant (ANOVA and post hoc) spectral power changes. As for the passive condition, we marked the time segments in which beta 1 power continuously exceeded the 3-SD limit for more than 30 ms. Power increases were detected (Fig. 3) within a time interval ranging from 120 to 260 ms after the tactile stimulus, whereas power decreases occurred in a time interval ranging from 270 to 400 ms after the tactile stimulus. Comparing passive and active Pre, only the effect of condition was significant (F1,18 ⫽ 37.1; P ⬍ 0.0001), reflecting lower powers in the active condition. Comparing the E-post and L-post epochs of passive and active conditions (normalized to the respective condition’s

2719

2720

SENSORY SUBSTITUTION/MENTAL MAPPING IN BLIND/SIGHTED PEOPLE

Correlation Between EEG Parameters and PLDs PLDs, as ranked by blind and sighted subjects, were compared by paired t-test and revealed no significant differences

between groups (t ⫽ ⫺0.42; df ⫽ 9; 95% CI on the difference of population means [⫺1.62, 1.09]; N.S.). For each derivation, the correlation with PLD was tested for both coherence and ERSP in the active and passive time frames independently. As for coherence, both groups showed positive correlations (Fig. 6) with PLD in both hemispheres, only in the active time frame and in a subset of derivation pairs: FC-CP and CP-OT. Blind subjects only showed a correlation for values within the above-mentioned peaks (Fig. 5, I, J, O, and P), whereas sighted subjects showed a correlation across the entire active time frame. No correlation was found for the rest condition, as well as considering the passive time frame and its subepochs (always N.S.). Details concerning the significant correlations can be found in APPENDIX B (see Table A1), organized by group, derivation, and epoch. No association (always N.S.) was found between PLDs and mean ERSP in any derivations. This held true when considering the rest condition, the entire passive and active time frames, their three subepochs separately, or when present, the significant power increase/decrease periods. DISCUSSION

The main goal of this preliminary study was to investigate tactile-to-visual sensory substitution and mental mapping in blind subjects and matched, blindfolded controls. This was achieved by analyzing EEG power and coherence patterns in the beta 1 band. The task was a tactile exploration of a virtual object displayed by the DIGEYE system (Brayda et al. 2011) compared with rest and passive stimulation. This is one of the first studies addressing sensory substitution and mental mapping in the context of virtual reality, a technique that facilitates unconstrained perception of abstract models. Unlike previous literature (Bach-y-Rita and Kercel 2003), this study directly investigates the construction of cognitive maps during its learning phase, using high temporal resolution EEG parameters. In this paper, we focus on the simplest event in an active exploration, namely, the emergence of an unknown virtual object from an empty space. We observed peculiar EEG patterns in association with the acquisition of spatial information, involving, in particular, spectral coherence. We then showed that a VE can elicit sensory substitution and mental mapping, both in blind and sighted subjects. This result will be the basis for further studies concerning more complex geometrical features and other EEG characteristics. Summary of Results The main findings of this study are shown in Table 2. Compared with rest and the passive time frame, the active time frame showed that beta 1 decreased power and increased coherence in all derivations. Within the active time frame, an early power increase (higher in blind people) and a late power decrease were detected after the active event. The early increase was weaker, and the late decrease was absent in the passive time frame. In the passive time frame, the two groups showed similar patterns, whereas in the active time frame, only blind subjects had a power increase in OT derivations, which instead, was absent in sighted people. Only in the active time frame did blind subjects generally show lower coherences compared with sighted controls. However, in FC-CP and CP-OT derivation pairs, blind subjects exhibited early coher-

J Neurophysiol • doi:10.1152/jn.00624.2011 • www.jn.org

Downloaded from jn.physiology.org on July 10, 2012

observed, both compared with rest and compared with Pre, in all of the derivations and epochs. Considering the active time frame, no hemisphere effect emerged (F1,18 ⫽ 0.8; N.S.). The derivation effect was highly significant (F3,54 ⫽ 14.0; P ⬍ 0.000001), generally showing an increasing trend from anterior to posterior derivations. The coherences of the two groups were significantly different (F1,18 ⫽ 30.8; P ⬍ 0.0001), with sighted subjects showing higher coherences with respect to blind subjects. A main epoch effect (F3,54 ⫽ 5.2; P ⬍ 0.001) and a weaker interaction between group and epoch (F3,54 ⫽ 3.4; P ⬍ 0.05) emerged, whereas a significant derivation ⫻ epoch interaction (F6,108 ⫽ 2.7; P ⬍ 0.05) and a stronger group ⫻ derivation ⫻ epoch interaction (F9,162 ⫽ 5.5; P ⬍ 0.001) were found. Post hoc comparisons are illustrated in Fig. 4. Compared with the initial rest basal condition, the epochs in the active time frame were significantly higher (always at least P ⬍ 0.0001) in all derivations (always at least P ⬍ 0.001), whereas no significant difference emerged in rest between the two groups. Within the active time frame and compared with the Pre epoch, only for blind subjects and only in the E-post interval, a significant increase of coherence was found. This increase only involved FC vs. centro-parietal (CP) and OT vs. CP derivations (P ⬍ 0.01) and is visible in Fig. 5, I, J, O, and P, as a “peak”. On the contrary, all other locations in blind people (Fig. 5, K–N) and all derivations in sighted people (Fig. 5, I–P) did not show any time-dependent variation (N.S.), with high stationary values across the entire active time frame (i.e., not including rest). For each epoch, coherence was compared between the two groups. Considering FC vs. CP and CP vs. OT derivations, coherences were lower for blind subjects (P ⬍ 0.01) in all epochs, except the E-post, where they were similar in the two groups (N.S.): here, the peak values found in blind subjects (in a range between 130 and 250 ms after the tactile stimulation; see Fig. 5, I, J, O, and P) reached levels similar to those of sighted subjects. Comparing passive and active Pre, the effect of condition was significant (F1,18 ⫽ 38.1; P ⬍ 0.0001), reflecting an increased coherence in the active condition. A significant effect of derivation (F3,54 ⫽ 12.2; P ⬍ 0.00001) was found, corresponding to an increasing trend from anterior to posterior derivations. Comparing the E-post and L-post epochs of passive and active conditions (normalized to the respective Pre condition), the effect of condition was significant (F1,18 ⫽ 4.7; P ⬍ 0.05), as well as the effect of epoch (F1,18 ⫽ 4.8; P ⬍ 0.05), derivation (F3,54 ⫽ 2.9; P ⬍ 0.05), and group (F1,18 ⫽ 4.7; P ⬍ 0.05). No significant effects were found for hemisphere (F1,18 ⫽ 0.54; N.S.). Significant epoch ⫻ condition (F1,18 ⫽ 8.6; P ⬍ 0.01), epoch ⫻ condition ⫻ derivation (F3,54 ⫽ 5.3; P ⬍ 0.01), and epoch ⫻ condition ⫻ derivation ⫻ group (F3,54 ⫽ 6.5; P ⬍ 0.001) interactions were found. In the post hoc analysis, coherences of active E-post resulted significantly higher (P ⬍ 0.001) compared with passive E-post for only in FC-CP and CP-OT derivation pairs of blind subjects (P ⬍ 0.001).

SENSORY SUBSTITUTION/MENTAL MAPPING IN BLIND/SIGHTED PEOPLE

2721

Downloaded from jn.physiology.org on July 10, 2012

Fig. 5. Mean coherence patterns for blind (black circles) and sighted (gray triangles) subjects. Each circle/triangle corresponds to an interval of 10 ms. For each group, significant variations (larger than 3⫻ SD) with respect to average Pre are indicated for blind (horizontal black lines) people below curves. Unlike with ERSP (see Fig. 3), coherence shows no significant variations for sighted people. Left: passive time frame. Similar pattern were observed for both groups, without variations compared with Pre. Right: active time frame. Everywhere, coherences were higher than in the passive time frame. The 2 groups showed some differences in patterns. Compared with Pre, only blind subjects showed (I, J, O, and P) significant coherence peaks during the E-post epoch, only in FC-CP and CP-OT pairs and for both hemispheres. In the same derivations, sighted subjects showed high, stable coherences at the same peak levels of the blind subjects.

ence peaks only after the active event. Sighted subjects, instead, in the same derivations and only in the active time frame, showed a high, stable coherence at the same peak levels of the blind subjects. No differences emerged between the average

PLD across the two groups. For both sighted and blind subjects, a positive correlation was found, only in the active time frame, between the PLD and beta 1 coherence in FC-CP and CP-OT derivation pairs. For sighted people, this correlation

J Neurophysiol • doi:10.1152/jn.00624.2011 • www.jn.org

2722

SENSORY SUBSTITUTION/MENTAL MAPPING IN BLIND/SIGHTED PEOPLE

was found over the entire active time frame and all its subepochs; for blind subjects, instead, a similar relationship only occurred during the above-mentioned coherence peaks.

ERSPs

Sample Characteristics According to the classification observable in Burton (2003), our sample did not comprise late-blind subjects, since all of our subjects lost their sight before the age of 12, and they did not learn to read print. Additionally, the five impaired subjects, who were not congenitally blind, indicated that they had no ability to distinguish spatial patterns using sight. We can therefore assume that none of them used visual imagery to create the representation of the virtual object. The recruitment of the sample, considered homogeneous for the purpose of our study, was performed in collaboration with the Rehabilitation Institute for the Blind, Istituto David Chiossone onlus (Genoa, Italy). Motivation of Experimental Choices By actively exploring the environment and mentally interpolating (see APPENDIX A) the perceived heights in each point of the tablet, subjects were asked to construct a bas-relief-like representation. Our results on single objects may be extended to complex VEs, as qualitatively shown elsewhere (Chellali et al. 2009). Moreover, during postexperiment interviews, several of our blind volunteers reported that they had intuitively encoded the ground level as a surrounding void space. This

ERSP was considered as a parameter, possibly reflecting holistic, cognitive modifications induced by, but not phaselocked to, the onset of a stimulus (Makeig 1993). Compared with rest and with the passive time frame, both blind and sighted subjects showed a significant power decrease, in all epochs of the active time frame and in all derivations, mainly associated with motor programming (Ritter et al. 2009). This interpretation seems to be confirmed by the total absence of power decrease, with respect to rest, not only after but also before the passive stimulation. The power decrease in the active time frame became even stronger in the L-post epoch, suggesting increased sensory-motor processing after tactile stimulation (Neuper et al. 2006; Perfetti et al. 2011; Ritter et al. 2009). Both groups exhibited, in the passive as well as in the active time frame, a power increase in the E-post epoch compared with the corresponding Pre. However, the increase was greater in the active time frame. This can be associated with tactile sensation, attention, and perception, which are likely to be enhanced during active exploration, in accordance with previous studies (Forster et al. 2007; Röder et al. 1996; Trenner et al. 2008). Considering OT derivations, both groups showed a similar absence of power variations in the passive time frame, whereas in the active time frame, only blind and

J Neurophysiol • doi:10.1152/jn.00624.2011 • www.jn.org

Downloaded from jn.physiology.org on July 10, 2012

Fig. 6. PLDs and coherence. FC-CP (A and B) and CP-OT pairs (C and D). These are the only derivation pairs to have significant correlations with PLD. Only significant associations are represented. Mean coherences in the significant peaks for blind (circles) subjects and mean coherence in the entire active time frame for sighted (triangles) subjects are independently represented with the associated best linear fit (for blind subjects, longer, dashed lines; for sighted subjects, shorter, dotted lines).

may help to understand their mental self-projection inside the proposed virtual scene. In accordance with other studies (Ortiz et al. 2011; Picard et al. 2009), we proposed to all subjects a simple abstract geometrical object, in this case, a one-level, square parallelepiped. This reduced possible impairments of blind subjects in mental construction, due to their poor knowledge of visual conventions. In our experiments, we alternated short explorations with periods of rest. We also provided a single stimulation on the fingertip for each taxel (see APPENDIX A) of the tablet, thus giving a minimal amount of information. This challenging setup is justified as follows: first, the experiment was presented to all participants as a mental mapping game, and the shortness of the explorations encouraged smart strategies. Second, the forced pauses slowed down the learning process, keeping mentation at the cortical level and limiting the involvement of implicit memory (Raichle et al. 1994). Finally, exploration breaking/resetting reduced the effects of habituation and adaptation (Anguera et al. 2009; Bastian 2008; Hamada et al. 2003). Importantly, our setup allowed observing mental mapping without previous learning, since EEG was recorded during the learning/exploration phase. The validation of the TAMO device as a sensory substitution interface is presented elsewhere (Chellali et al. 2009). However, our methodology could be applied to other interfaces (Wall et al. 2006) without loss of generality. Furthermore, we only studied void-to-object transitions. The opposite, object-to-void transition, was outside the scope of this study, since it elicited a reported different sensation in several blind subjects. Further work is required to clarify this aspect. Both ERSP and coherence were considered in this study as revealing partially common, although still complementary, elements of the tactile mental mapping (Amedi et al. 2010), which we attempted to elicit with our device.

SENSORY SUBSTITUTION/MENTAL MAPPING IN BLIND/SIGHTED PEOPLE

2723

Table 2. Main findings of the study Phenomena Tactile-to-Visual Sensory Substitution ERSP BLIND

SIGHTED

COHERENCE BLIND

PRO: power increase in OT only during active exploration

CON: no correlation with PLD PROs: •power decrease in the active exploration (especially in the L-post) compared with rest and the passive stimulation •stronger increase in the E-post of active exploration compared with passive stimulation CON: no correlation with PLD PROs: •power decrease in the active exploration (especially in the L-post) compared with rest and the passive stimulation •stronger increase in the E-post of active exploration compared with passive stimulation

CON: no power increase in OT

PRO: OT involved by high coherence peaks and only in the active exploration

PRO: OT involved by high, stable coherence in the entire active exploration

PROs: •higher values than rest and than passive stimulation within active exploration •high coherence peaks in FC-CP and CP-OT derivation pairs and only during active exploration •no correlation with PLD in rest and passive stimulation •positive correlation of coherence with PLD in peaks •spatial processing mainly linked to tactile stimuli PROs: •higher values than rest and than passive stimulation within active exploration •high stable values in FC-CP and CP-OT derivation pairs and only during active exploration •no correlation with PLD in rest and passive stimulation •positive correlation of coherence with PLD in the entire active time frame •mainly ongoing spatial processing

This study found sensory substitution and supramodal mental mapping cues deriving from the involvement, in a tactile exploration task, of scalp derivations considered as related to visuo-spatial pathways. PROs/CONs are cues supporting/not supporting the presence of the phenomena (columns). The 3 measures involved are event-related spectral perturbations (ERSP), coherence, and perceived levels of difficulty (PLD). OT, occipito-temporal; L-post, late-poststimulus; E-post, early-poststimulus; FC, fronto-central; CP, centro-parietal.

not sighted people showed a power increase. This supports the hypothesis (Cohen et al. 1997; Kujala et al. 1997; Röder et al. 1996) of stronger sensory substitution in blind subjects, due to long-term plasticity, which activates visual areas in the presence of a tactile stimulation linked to a cognitive representation. In the other derivations and only in the active time frame, blind subjects presented stronger E-post power increases with respect to sighted controls, in accordance with enhanced tactile spatial acuity/attention, reported with visual deprivation (Forster et al. 2007; Goldreich and Kanics 2003; Wong et al. 2011). Coherences Coherence was evaluated, because it is known to be related to cognitive processes involved in mental mapping (Leclerc et al. 2005). Bipolar derivations were considered to avoid possible spurious effects due to a common reference (Nunez et al. 1999). Admittedly, high coherence between scalp EEG derivations cannot provide precise information about underlying signal flow. However, it can be associated with a functional neural connection or a common source, likely formed by an extended cortical neural network. Nunez and Srinivasan (2006) showed that a coherence increase could be a spurious effect of a parallel increase in spectral power, mainly due to a higher signal-to-noise ratio (the

normalized coherence increases with the power of the common component). However, the opposite pattern was observed in the comparison among rest, passive stimulation, and active exploration, as during active exploration, spectral coherence increased, whereas spectral power decreased with respect to the other two conditions. High spectral power characterizing the rest condition could indicate the presence of extended cortical areas with synchronous and in-phase oscillations, whereas the lower spectral power, characterizing active exploration, suggests a loss of such an extended phase synchrony: a desynchronization, which is often associated to cognitive activities and attentive processes (Pfurtscheller and Lopes da Silva 1999). However, the increase of spectral coherence during the active exploration indicates that neuronal-assemblies oscillations within beta band, even if weaker or out of phase, tended to keep steady-phase differences, supporting the hypothesis of a stronger functional coupling (Srinivasan et al. 2007). No differences were observed between rest and the passive time frame in coherences. Higher coherences were detected, both in rest and in passive stimulation conditions in the right hemisphere, in agreement with a more diffuse receptotopic organization, evidenced in previous studies (Tucker et al. 1986). Instead, compared with rest and with the passive time

J Neurophysiol • doi:10.1152/jn.00624.2011 • www.jn.org

Downloaded from jn.physiology.org on July 10, 2012

SIGHTED

Supramodal Mental Mapping

2724

SENSORY SUBSTITUTION/MENTAL MAPPING IN BLIND/SIGHTED PEOPLE

Objective and Subjective Parameters We conducted a further investigation about possible relations between objective ERSP/coherence patterns and subjective PLDs. Intriguingly, only CP-OT and FC-CP pairs, and only during the active time frame, showed a positive correlation between coherence and PLD. The role of these peculiar derivations and visuo-spatial pathways in multimodal (Burton and Sinclair 2000) and memory-related spatial processing (Curtis 2006) is still debated. On the one hand, CP-OT coherence seems to be related to mental mapping through the ventral stream as a What pathway (Sakata and Kusunoki 1992). On the

other hand, FC-CP coherence seems to be related to the dorsal stream, which is thought (Kravitz et al. 2011) to mediate both spatial perception (Where) and nonconscious-guided action (How). These pathways are also involved in tactile real object recognition/manipulation and haptic perception (Deibert et al. 1999; Deshpande et al. 2008; Hartmann et al. 2008; Kägi et al. 2010; Reed et al. 2004, 2005; Stoeckel et al. 2003). Importantly, there was no correlation in these derivations between PLD and coherences in rest and the passive time frame, where the cognitive processes are strongly reduced or absent. Moreover, no correlations emerged between PLD and ERSP, probably mainly reflecting tactile-evoked reactions (Forster et al. 2007; Röder et al. 1996; Trenner et al. 2008), also affected by the strength of the stimulus provided by the lever. The greater E-post increase of beta 1 ERSP during the active time frame with respect to the passive time frame suggested a contribution due to enhanced attention. However, the lack of the ERSP-PLD correlation, as opposed to the significant coherence-PLD correlation, suggests that for what concerns beta 1 band, cognitive load could be reflected more directly by coherence than by ERSP. Moreover, the beta 1 coherence pattern and its positive correlation with PLD, only during active exploration, suggest an information flow (Brovelli et al. 2004; Del Percio et al. 2011; Fries 2005; Halgren et al. 2002; Kopell et al. 2000), reflecting cognitive activity (Kukleta et al. 2009). This can be interpreted in terms of top-down processes involving an information search (Buschman and Miller 2007), short-term memory in cognitive status quo keeping (Engel and Fries 2010), and stimuli localization (Leclerc et al. 2005). This information flow is possibly related to a global source, functionally implied in spatial processing. When it is highest, it may correspond to a proportionally high cognitive load. Its link with performance quality is not trivial and will be investigated in future works: in this study, all subjects correctly built the mental map of the proposed simple object. These results suggest a speculative interpretation: during active exploration, blind people may only need to exchange additional information between different brain areas every time they mentally map the contour of virtual objects. This could

Fig. A1. Block scheme of the Digital Eye (DIGEYE) system: the subject actively explores a virtual environment through a tactile user interface (TUI), connected to a personal computer (PC), where the height function (H) is mapped on X,Y coordinates and wirelessly sent back to the lever of the tactile mouse-shaped device (TAMO). Start/stop sounds pace the rest/exploration epochs and are synchronized with electrophysiological data [EEG; ECG; electrooculogram (EOG); electromyogram (EMG)], captured by a polygraph (Poly). Metrics are obtained offline.

J Neurophysiol • doi:10.1152/jn.00624.2011 • www.jn.org

Downloaded from jn.physiology.org on July 10, 2012

frame, both blind and sighted subjects showed higher coherence in all epochs of the active time frame and in all derivations, as can be expected due to a global cognitive recruitment. This seems to agree with the link emerging in stereo-EEG studies between beta coherence and visual cognitive activity (Kukleta et al. 2009), possibly related to a continuous cognitive status quo-keeping in short-term memory, aimed at gradually building the mental map. Moreover, increases in beta coherence were found to be associated (Perfetti et al. 2011) with a decrease in beta power during motor planning or execution. All of these processes are expected to occur in the active exploration and to be absent in the passive stimulation, which, in fact, did not show coherence variations compared with rest. Moreover, during the active time frame, compared with Pre, only blind subjects exhibited peaks in E-post epochs and only in FC-CP and CP-OT derivation pairs. Sighted subjects, instead, showed during the active time frame continuously high coherence between the same derivation pairs, at values similar to the peak levels of blind subjects. These results suggest the existence, during the active explorations, of a specific information flow within the FC-CP and CP-OT derivation pairs, which are considered to reflect visuo-spatial processing (Goodale et al. 1991; Kravitz et al. 2011; Pollmann et al. 2000). Moreover, the involvement of the occipital cortex, expressed by coherence, in purely tactile exploration (whereas absent in passive tactile stimulation), suggests that sensory substitution also occurs in sighted subjects.

SENSORY SUBSTITUTION/MENTAL MAPPING IN BLIND/SIGHTED PEOPLE

2725

Fig. A2. Subject interacting with the single “taxel” of the TAMO device (left image): while the subject moves the device across the X,Y plane (center image), the lever rises/lowers at transitions of H values (right image).

Conclusions Our findings build on previous research efforts regarding supramodal spatial processing. Beyond the existing literature, which is mainly focused on perception of real objects, we extend the knowledge of tactile stimulation and perception and active exploration to the context of virtual objects. Our results positively support all four of our research hypotheses. Concerning the first question—“Is it possible to elicit visuotactile sensory substitution during nonvisual exploration?”—a coherence increase compared with rest, involving occipital (visual) derivations, suggests the existence, during active tactile explorations, of an underlying sensory substitution for both groups. Additionally, this increase was not found during passive tactile stimulation, meaning that occipital regions are elicited only during a mental mapping task. Moreover, significant changes of ERSP only involved occipital areas in blind people and only during active explorations, suggesting a deeper sensory substitution in a context of neural plasticity. As for the second question—“Do EEG observations support the involvement—possibly vision independent— of visuo-spatial pathways during mental mapping?”— both groups exhibited a global coherence increase, compared with rest, along the active explorations. This held not only in visual areas but also for all analyzed derivations, which are supposed to be related to visuo-spatial processing. Moreover, this global coherence increase was absent in the passive tactile stimulation. Thus our results suggest an involvement, for both blind and sighted people and during tactile mental mapping tasks, of the visuospatial pathways. However, further studies are required to confirm specific activation of these pathways and to explore possible sources of coherent EEG signals. Concerning the third question—“Is there a relationship between EEG and cognitive load in successful mental mapping using minimal tactile information?”—in both groups, coherences positively correlated with subjective PLD. This correlation emerged only during active exploration and was absent during rest and the passive stimulation, which supports an association of beta 1 coherence with cognitive load.

Lastly, the fourth question—“Are there cues allowing us to consider mental mapping as supramodal in this task?”—received a positive answer, because both groups 1) were able to correctly construct the mental map of the explored object; 2) showed similar cognitive loads (PLDs) while executing the task; 3) exhibited EEG patterns, suggesting an involvement— but only during a tactile exploration task and not during rest and passive stimulation— of scalp derivations thought to reflect visuo-spatial processing; and 4) showed similar positive correlations (exclusively during active exploration) between subjective PLD and objective coherences in visuo-spatialrelated derivations. Our results confirmed, on the one hand, the sensitivity of beta 1 ERSP and coherence in elucidating sensory substitution and mental mapping. On the other hand, they encourage further investigations involving: performance and behavioral aspects, complex objects, longitudinal training effects (Ortiz et al. 2011), as well as other complementary EEG features (Chiu et al. 2011). APPENDIX A: EXPERIMENTAL SETUP

The DIGEYE system is shown in Fig. A1. It comprises three parts: 1) a tactile user interface (TUI), which includes a tactile device, coupled with a sensing tablet; 2) a polygraph, which monitors the neurophysiological status of subjects while they use the interface; and

Fig. A3. Acquired and analyzed derivations. Bipolar analyzed derivations are indicated by continuous lines: F3-C3, C3-P3, P3-T3, and T3-O1 on the left side and the homologous derivations on the right side.

J Neurophysiol • doi:10.1152/jn.00624.2011 • www.jn.org

Downloaded from jn.physiology.org on July 10, 2012

also be explained by their enhanced haptic-tactile acuity (Postma et al. 2007; Wolbers et al. 2005). The stable, high coherence (also before the active tactile stimulus) in sighted subjects, instead, suggests a continuous, high information flow, possibly related to memory and mental mapping (Engel and Fries 2010). In other words, similar levels of resources seem to be recruited by both groups, although with different temporal patterns.

R2 ⴝ 0.67, F1,8 ⴝ 19.4, P ⴝ 0.002* R2 ⴝ 0.69, F1,8 ⴝ 21.4, P ⴝ 0.002*

R2 ⴝ 0.61, F1,8 ⴝ 15.2, P ⴝ 0.004* R2 ⴝ 0.70, F1,8 ⴝ 21.6, P ⴝ 0.001*

Active Time Frame

R2 ⫽ 0.19, F1,8 ⫽ 3.1, P ⫽ 0.12 R2 ⫽ 0.14, F1,8 ⫽ 2.4, P ⫽ 0.16

R2 ⫽ 0.17, F1,8 ⫽ 2.9, P ⫽ 0.13 R2 ⫽ 0.14, F1,8 ⫽ 2.4, P ⫽ 0.16

Active Time Frame

BLIND

R2 ⴝ 0.64, F1,8 ⴝ 16.8, P ⴝ 0.003* R2 ⴝ 0.66, F1,8 ⴝ 18.2, P ⴝ 0.003*

Peak

Late Poststimulus

R2 ⴝ 0.59, F1,8 ⴝ 14.0, P ⴝ 0.006* R2 ⴝ 0.68, F1,8 ⴝ 20.2, P ⴝ 0.002* R2 ⴝ 0.66, F1,8 ⴝ 19.6, P ⴝ 0.002* R2 ⴝ 0.67, F1,8 ⴝ 19.4, P ⴝ 0.002*

R2 ⴝ 0.63, F1,8 ⴝ 16.5, P ⴝ 0.004* R2 ⴝ 0.72, F1,8 ⴝ 24.6, P ⴝ 0.001* R2 ⴝ 0.69, F1,8 ⴝ 20.8, P ⴝ 0.002* R2 ⴝ 0.70, F1,8 ⴝ 22.5, P ⴝ 0.002*

absent

absent

absent

absent

Peak

R2 ⴝ 0.71, F1,8 ⴝ 23.4, P ⴝ 0.001* R2 ⴝ 0.72, F1,8 ⴝ 23.8, P ⴝ 0.001*

R2 ⴝ 0.59, F1,8 ⴝ 14.0, P ⴝ 0.005* R2 ⴝ 0.67, F1,8 ⴝ 19.4, P ⴝ 0.002*

Early Poststimulus

R2 ⫽ 0.20, F1,8 ⫽ 3.3, P ⫽ 0.11 R2 ⫽ 0.10, F1,8 ⫽ 2.1, P ⫽ 0.19

R2 ⫽ 0.27, F1,8 ⫽ 4.3, P ⫽ 0.07* R2 ⫽ 0.29, F1,8 ⫽ 4.6, P ⫽ 0.06*

Late Poststimulus

R2 ⫽ 0.17, F1,8 ⫽ 2.9, P ⫽ 0.13 R2 ⫽ 0.15, F1,8 ⫽ 2.6, P ⫽ 0.15

Early Poststimulus

R2 ⫽ 0.30, F1,8 ⫽ 4.8, P ⫽ 0.06* R2 ⫽ 0.30, F1,8 ⫽ 4.9, P ⫽ 0.06*

R2 ⴝ 0.58, F1,8 ⴝ 13.5, P ⴝ 0.006* R2 ⴝ 0.66, F1,8 ⴝ 18.2, P ⴝ 0.003*

Prestimulus

SIGHTED

R2 ⫽ 0.18, F1,8 ⫽ 2.2, P ⫽ 0.18 R2 ⫽ 0.20, F1,8 ⫽ 3.3, P ⫽ 0.11

R2 ⫽ 0.14, F1,8 ⫽ 2.5, P ⫽ 0.15 R2 ⫽ 0.19, F1,8 ⫽ 3.1, P ⫽ 0.12

Prestimulus

Summary of the correlations between coherence values and PLDs: fit results and level for significance arranged by group, derivation, and hemisphere. Significant results (P ⬍ 0.05) are marked in bold and with an asterisk (*). All correlations are positive.

R2 ⫽ 0.14, F1,8 ⫽ 2.5, P ⫽ 0.15 R2 ⫽ 0.20, F1,8 ⫽ 3.3, P ⫽ 0.11

R2 ⫽ 0.12, F1,8 ⫽ 2.2, P ⫽ 0.18 R2 ⫽ 0.15, F1,8 ⫽ 2.6, P ⫽ 0.15

Rest

R2 ⫽ 0.11, F1,8 ⫽ 2.1, P ⫽ 0.18 R2 ⫽ 0.06, F1,8 ⫽ 0.9, P ⫽ 0.37

R2 ⫽ 0.07, F1,8 ⫽ 1.1, P ⫽ 0.32 R2 ⫽ 0.10, F1,8 ⫽ 2.1, P ⫽ 0.19

Rest

Downloaded from jn.physiology.org on July 10, 2012

J Neurophysiol • doi:10.1152/jn.00624.2011 • www.jn.org

left

CP vs. OT right

left

FC vs. CP right

Derivations

left

CP vs. OT right

left

FC vs. CP right

Derivations

Table A1. Correlations between coherence and PLD

2726 SENSORY SUBSTITUTION/MENTAL MAPPING IN BLIND/SIGHTED PEOPLE

SENSORY SUBSTITUTION/MENTAL MAPPING IN BLIND/SIGHTED PEOPLE

APPENDIX B: CORRELATIONS BETWEEN PLD AND EEG PARAMETERS

Detailed results of the regressions between PLDs and EEG coherence are shown in Table A1. ACKNOWLEDGMENTS We thank J. Jenvrin, S. Saliceti, J. Ortiz, T. Hauptman, and S. Hennig for their technical support, the personnel of the Clinical Neurophysiology Unit of the University of Genoa for the organization of experiments, as well as C. Martinoli and the personnel of the Istituto David Chiossone onlus and of the Italian Union of the Blind (UIC) for their precious feedback. GRANTS Support for this research project was provided by the Italian Institute of Technology (Protocollo No. 0017619/10). DISCLOSURES No conflicts of interest, financial or otherwise, are declared by the author(s). AUTHOR CONTRIBUTIONS Author contributions: C.C., L.B., and G.R. conception and design of research; C.C., L.B., F.F., C.B., and L.L. performed experiments; C.C. and L.B. analyzed data; C.C., L.B., F.D.C., and G.R. interpreted results of experiments; C.C. and L.B. prepared figures; C.C., L.B., F.D.C., R.C., F.F., and G.R. drafted manuscript; C.C., L.B., F.D.C., R.C., and G.R. edited and revised manuscript; C.C., L.B., F.D.C., R.C., and G.R. approved final version of manuscript. REFERENCES Allena M, Campus C, Morrone E, De Carli F, Garbarino S, Manfredi C, Rossi Sebastiano D, Ferrillo F. Periodic limb movements both in nonREM and REM sleep: relationships between cerebral and autonomic activities. Clin Neurophysiol 120: 1282–1290, 2009. Amedi A, Raz N, Azulay H, Malach R, Zohary E. Cortical activity during tactile exploration of objects in blind and sighted humans. Restor Neurol Neurosci 28: 143–156, 2010. Anguera JA, Seidler RD, Gehring WJ. Changes in performance monitoring during sensorimotor adaptation. J Neurophysiol 102: 1868 –1879, 2009.

Babiloni C, Brancucci A, Vecchio F, Arendt-Nielsen L, Chen AC, Rossini PM. Anticipation of somatosensory and motor events increases centroparietal functional coupling: an EEG coherence study. Clin Neurophysiol 117: 1000 –1008, 2006. Bach-y-Rita P, Kercel S. Sensory substitution and the human-machine interface. Trends Cogn Sci 7: 541–546, 2003. Bastian AJ. Understanding sensorimotor adaptation and learning for rehabilitation. Curr Opin Neurol 21: 628 – 633, 2008. Bavelier D, Neville HJ. Cross-modal plasticity: where and how? Nat Rev Neurosci 3: 443– 452, 2002. Benson BL, Anguera JA, Seidler RD. A spatial explicit strategy reduces error but interferes with sensorimotor adaptation. J Neurophysiol 105: 2843– 2851, 2011. Bird CM, Burgess N. The hippocampus and memory: insights from spatial processing. Nat Rev Neurosci 9: 182–194, 2008. Brayda L, Campus C, Chellali R, Rodriguez G, Martinoli C. An investigation of search behaviour in a tactile exploration task for sighted and non-sighted adults. ACM International Conference on Human Factors in Computing Systems (ACM CHI 2011), Vancouver, Canada, May 7–12, 2011. Brovelli A, Ding M, Ledberg A, Chen Y, Nakamura R, Bressler SL. Beta oscillations in a large-scale sensorimotor cortical network: directional influences revealed by Granger causality. Proc Natl Acad Sci USA 101: 9849 – 9854, 2004. Burton H. Visual cortex activity in early and late blind people. J Neurosci 23: 4005– 4011, 2003. Burton H, Sinclair RJ. Attending to and remembering tactile stimuli: a review of brain imaging data and single-neuron responses. J Clin Neurophysiol 17: 575–591, 2000. Burton H, Snyder AZ, Conturo TE, Akbudak E, Ollinger JM, Raichle ME. Adaptive changes in early and late blind: a fMRI study of Braille reading. J Neurophysiol 87: 589 – 607, 2002a. Burton H, Snyder AZ, Diamond JB, Raichle ME. Adaptive changes in early and late blind: a fMRI study of verb generation to heard nouns. J Neurophysiol 88: 3359 –3371, 2002b. Buschman TJ, Miller EK. Serial, covert shifts of attention during visual search are reflected by the frontal eye fields and correlated with population oscillations. Neuron 63: 386 –396, 2009. Buschman TJ, Miller EK. Top-down versus bottom-up control of attention in the prefrontal and posterior parietal cortices. Science 315: 1860 –1862, 2007. Chellali R, Brayda L, Martinoli C, Fontaine E. How taxel-based displaying devices can help visually impaired people to navigate safely. In: Autonomous Robots and Agents, 2009. ICARA 2009. 4th International Conference, p. 470 – 475. Chiu TC, Gramann K, Ko LW, Duann JR, Jung TP, Lin CT. Alpha modulation in parietal and retrosplenial cortex correlates with navigation performance. Psychophysiology 49: 43–55, 2012. Chorlian DB, Rangaswamy M, Porjesz B. EEG coherence: topography and frequency structure. Exp Brain Res 198: 59 – 83, 2009. Classen J, Gerloff C, Honda M, Hallett M. Integrative visuomotor behavior is associated with interregionally coherent oscillations in the human brain. J Neurophysiol 79: 1567–1573, 1998. Cohen LG, Celnik P, Pascual-Leone A, Corwell B, Falz L, Dambrosia J, Honda M. Functional relevance of cross-modal plasticity in blind humans. Nature 389: 180 –183, 1997. Curtis CE. Prefrontal and parietal contributions to spatial working memory. Neuroscience 139: 173–180, 2006. Dahiya RS, Metta G, Valle M, Sandini G. Tactile sensing—from humans to humanoids. IEEE Trans Robot 26: 1–20, 2010. Deibert E, Kraut M, Kremen S, Hart J Jr. Neural pathways in tactile object recognition. Neurology 52: 1413–1417, 1999. Del Percio C, Iacoboni M, Lizio R, Marzano N, Infarinato F, Vecchio F, Bertollo M, Robazza C, Comani S, Limatola C, Babiloni C. Functional coupling of parietal alpha rhythms is enhanced in athletes before visuomotor performance: a coherence electroencephalographic study. Neuroscience 175: 198 –211, 2011. Delorme A, Makeig S. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods 134: 9 –21, 2004. Deshpande G, Hu X, Randall S, Sathian K. Effective connectivity during haptic perception: a study using Granger causality analysis of functional magnetic resonance imaging data. Neuroimage 40: 1807–1814, 2008.

J Neurophysiol • doi:10.1152/jn.00624.2011 • www.jn.org

Downloaded from jn.physiology.org on July 10, 2012

3) a personal computer, where a software application presents a VE synchronized with the TUI, whereas another software module processes offline both user behavior and neurophysiological status. Specifically, given some height function, H ⫽ f(X,Y), a TAMO-shaped device (Fig. A2) is able to display height information, H, at any absolute position (X,Y) of the 210 ⫻ 297-mm tablet. While users freely move the TAMO (Chellali et al. 2009) on the tablet, the VE receives the current position and wirelessly sends back the value of H to the controller inside the TAMO. The motor controller generates tactile feedback by activating a stepper motor, which raises a small lever proportionally to H. TAMO thereby generates a tactile counterpart (a taxel) for each pixel of the tablet, much like a tactile bas-relief representation. When the subjects explore the ground level, no tactile stimulation is provided by the lever. When the subjects cross a level other than ground, the lever rises to signal a virtual object edge. When the height of the object is constant, the height of the lever is also constant. When the height of the object increases/decreases, the height of the lever varies proportionally. Subjects actively seek spatial information by freely moving the TAMO device (Fig. A2) without any prior cue (Benson et al. 2011) and attempting to build a mental map of the VE. More details are available in Brayda et al. (2011). While using the tactile device, EEG (Fig. A3), ECG, EMG, and EOG biodata are recorded and synchronized with behavioral data (i.e., X, Y, and H values).

2727

2728

SENSORY SUBSTITUTION/MENTAL MAPPING IN BLIND/SIGHTED PEOPLE Latini-Corazzini L, Pascale Nesa M, Ceccaldi M, Guedj E, Thinus-Blanc C, Cauda F, Dagata F, Péruch P. Route and survey processing of topographical memory during navigation. Psychol Res 74: 545–559, 2010. Leclerc C, Segalowitz SJ, Desjardins J, Lassonde M, Lepore F. EEG coherence in early-blind humans during sound localization. Neurosci Lett 376: 154 –159, 2005. Lewis LB, Saenz M, Fine I. Mechanisms of cross-modal plasticity in early-blind subjects. J Neurophysiol 104: 2995–3008, 2010. Loomis JM, Klatzky RL, Golledge RG, Cicinelli JG, Pellegrino JW, Fry PA. Nonvisual navigation by blind and sighted: assessment of path integration ability. J Exp Psychol Gen 122: 73–91, 1993. Maguire EA, Burgess N, Donnett JG, Frackowiak RS, Frith CD, O’Keefe J. Knowing where and getting there: a human navigation network. Science 280: 921–924, 1998. Makeig, S. Auditory event-related dynamics of the EEG spectrum and effects of exposure to tones. Electroencephalogr Clin Neurophysiol 86: 283–293, 1993. McMenamin BW, Shackman AJ, Maxwell JS, Bachhuber DR, Koppenhaver AM, Greischar LL, Davidson RJ. Validation of ICA-based myogenic artifact correction for scalp and source-localized EEG. Neuroimage 49: 2416 –2432, 2010. Meador KJ, Ray PG, Echauz JR, Loring DW, Vachtsevanos GJ. Gamma coherence and conscious perception. Neurology 59: 847– 854, 2002. Merabet LB, Pascual-Leone A. Neural reorganization following sensory loss: the opportunity of change. Nat Rev Neurosci 11: 44 –52, 2010. Neuper C, Wörtz M, Pfurtscheller G. ERD/ERS patterns reflecting sensorimotor activation and deactivation. Prog Brain Res 159: 211–222, 2006. Nunez PL, Silberstein RB, Shi Z, Carpenter MR, Srinivasan R, Tucker DM, Doran SM, Cadusch PJ, Wijesinghe RS. EEG coherency II: experimental comparisons of multiple measures. Clin Neurophysiol 110: 469 – 486, 1999. Nunez PL, Srinivasan R. Electric Fields of the Brain: The Neurophysics of EEG. Oxford, UK: Oxford University Press, 2006. Okuhata ST, Okazaki S, Maekawa H. EEG coherence pattern during simultaneous and successive processing tasks. Int J Psychophysiol 72: 89 –96, 2009. Ortiz T, Poch J, Santos JM, Requena C, Martínez AM, Ortiz-Terán L, Turrero A, Barcia J, Nogales R, Calvo A, Martínez JM, Córdoba JL, Pascual-Leone A. Recruitment of occipital cortex during sensory substitution training linked to subjective experience of seeing in people with blindness. PLoS ONE 6: e23264, 2011. Perfetti B, Moisello C, Landsness C, Kvint S, Pruski A, Onofrj M, Tononi G, Ghilardi MF. Temporal evolution of oscillatory activity predicts performance in a choice-reaction time reaching task. J Neurophysiol 105: 18 –27, 2011. Pfurtscheller G, Lopes da Silva FH. Event-related EEG/MEG synchronization and desynchronization: basic principles. Clin Neurophysiol 110: 1842– 1857, 1999. Picard D, Pry R. Does knowledge of spatial configuration in adults with visual impairments improve with tactile exposure to a small-scale model of their urban environment? J Vis Impair Blind 103: 199 –209, 2009. Pollmann S, von Cramon DY. Object working memory and visuospatial processing: functional neuroanatomy analyzed by event-related fMRI. Exp Brain Res 133: 12–22, 2000. Postma A, Zuidhoek S, Noordzij ML, Kappers AML. Differences between early-blind, late-blind, and blindfolded-sighted people in haptic spatialconfiguration learning and resulting memory traces. Perception 36: 1253– 1265, 2007. R Foundation for Statistical Computing R Development Core Team. R: A language and environment for statistical computing. Vienna Austria R Foundation for Statistical Computing, 1: 2673, 2008, http://www.r-project.org/ [2010]. Raichle ME, Fiez JA, Videen TO, MacLeod AMK, Pardo JV, Fox PT, Petersen SE. Practice-related changes in human brain functional anatomy during nonmotor learning. Cereb Cortex 4: 8 –26, 1994. Reed CL, Klatzky RL, Halgren E. What vs. where in touch: an fMRI study. Neuroimage 25: 718 –726, 2005. Reed CL, Shoham S, Halgren E. Neural substrates of tactile object recognition: an fMRI study. Hum Brain Mapp 21: 236 –246, 2004. Ritter P, Moosmann M, Villringer A. Rolandic alpha and beta EEG rhythms’ strengths are inversely related to fMRI-BOLD signal in primary somatosensory and motor cortex. Hum Brain Mapp 30: 1168 –1187, 2009. Röder B, Rösler F, Hennighausen E, Näcker F. Event-related potentials during auditory and somatosensory discrimination in sighted and blind human subjects. Brain Res 4: 77–93, 1996.

J Neurophysiol • doi:10.1152/jn.00624.2011 • www.jn.org

Downloaded from jn.physiology.org on July 10, 2012

Ekstrom AD, Kahana MJ, Caplan JB, Fields TA, Isham EA, Newman EL, Fried I. Cellular networks underlying human spatial navigation. Nature 425: 184 –188, 2003. Engel AK, Fries P. Beta-band oscillations—signalling the status quo? Curr Opin Neurobiol 20: 156 –165, 2010. Fiehler K, Rösler F. Plasticity of multisensory dorsal stream functions: evidence from congenitally blind and sighted adults. Restor Neurol Neurosci 28: 193–205, 2010. Forster B, Eardley AF, Eimer M. Altered tactile spatial attention in the early blind. Brain Res 1131: 149 –154, 2007. Fortin M, Voss P, Lord C, Lassonde M, Pruessner J, Saint-Amour D, Rainville C, Lepore F. Wayfinding in the blind: larger hippocampal volume and supranormal spatial navigation. Brain 131: 2995–3005, 2008. Fries P. A mechanism for cognitive dynamics: neuronal communication through neuronal coherence. Trends Cogn Sci 9: 474 – 480, 2005. Goldreich D, Kanics IM. Tactile acuity is enhanced in blindness. J Neurosci 23: 3439 –3445, 2003. Goodale MA, Milner AD, Jakobson LS, Carey DP. A neurological dissociation between perceiving objects and grasping them. Nature 349: 154 – 156, 1991. Goto Y, Taniwaki T, Kinukawa N, Tobimatsu S. Interhemispheric functional synchronization at the first step of visual information processing in humans. Clin Neurophysiol 115: 1409 –1416, 2004. Halgren E, Boujon C, Clarke J, Wang C, Chauvel P. Rapid distributed fronto-parieto-occipital processing stages during working memory in humans. Cereb Cortex 12: 710 –728, 2002. Hamada Y, Okita H, Suzuki R. Effect of interstimulus interval on attentional modulation of cortical activities in human somatosensory areas. Clin Neurophysiol 114: 548 –555, 2003. Hartmann S, Missimer JH, Stoeckel C, Abela E, Shah J, Seitz RJ, Weder BJ. Functional connectivity in tactile object discrimination: a principal component analysis of an event related fMRI-Study. PloS ONE 3: e3831, 2008. Holder GE, Celesia GG, Miyake Y, Tobimatsu S, Weleber RG. International Federation of Clinical Neurophysiology: recommendations for visual system testing. Clin Neurophysiol 121: 1393–1409, 2010. Iber C, Ancoli-Israel S, Chesson A, Quan SF. American Academy of Sleep Medicine. The AASM Manual for the Scoring of Sleep and Associated Events: Rules, Terminology and Technical Specifications (1st ed.), Westchester, IL: American Academy of Sleep Medicine, 2007. Kägi G, Missimer JH, Abela E, Seitz RJ, Weder BJ. Neural networks engaged in tactile object manipulation: patterns of expression among healthy individuals. Behav Brain Funct 6: 71, 2010. Kopell N, Ermentrout GB, Whittington MA, Traub RD. Gamma rhythms and beta rhythms have different synchronization properties. Proc Natl Acad Sci USA 97: 1867–1872, 2000. Kopell N, Whittington MA, Kramer MA. Neuronal assembly dynamics in the beta1 frequency range permits short-term memory. Proc Natl Acad Sci USA 108: 3779 –3784, 2011. Kravitz DJ, Saleem KS, Baker CI, Mishkin M. A new neural framework for visuospatial processing. Nat Rev Neurosci 12: 217–230, 2011. Kriegseis A, Hennighausen E, Rösler F, Röder B. Reduced EEG alpha activity over parieto-occipital brain areas in congenitally blind adults. Clin Neurophysiol 117: 1560 –1573, 2006. Kujala T, Alho K, Huotilainen M, Ilmoniemi RJ, Lehtokoski A, Leinonen A, Rinne T. Electrophysiological evidence for cross-modal plasticity in humans with early- and late-onset blindness. Psychophysiology 34: 213– 216, 1997. Kukleta M, Bob P, Brázdil M, Roman R, Rektor I. Beta 2-band synchronization during a visual oddball task. Physiol Res 58: 725–732, 2009. Kupers R, Chebat DR, Madsen KH, Paulson OB, Ptito M. Neural correlates of virtual route recognition in congenital blindness. Proc Natl Acad Sci USA 107: 12716 –12721, 2010. Lachaux JP, George N, Tallon-Baudry C, Martinerie J, Hugueville L, Minotti L, Kahane P, Renault B. The many faces of the gamma band response to complex visual stimuli. Neuroimage 25: 491–501, 2005. Lahav O, Mioduser D. Construction of cognitive maps of unknown spaces using a multi-sensory virtual environment for people who are blind. Comput Hum Behav 24, 1139 –1155, 2008. Lalo E, Gilbertson T, Doyle L, Di Lazzaro V, Cioni B, Brown P. Phasic increases in cortical beta activity are associated with alterations in sensory processing in the human. Exp Brain Res 177: 137–145, 2007.

SENSORY SUBSTITUTION/MENTAL MAPPING IN BLIND/SIGHTED PEOPLE

Tecchio F, Babiloni C, Zappasodi F, Vecchio F, Pizzella V, Romani GL, Rossini PM. Gamma synchronization in human primary somatosensory cortex as revealed by somatosensory evoked neuromagnetic fields. Brain Res 986: 63–70, 2003. Thinus-Blanc C, Gaunet F. Representation of space in blind persons: vision as a spatial sense? Psychol Bull 121: 20 – 42, 1997. Trenner MU, Heekeren HR, Bauer M, Rössner K, Wenzel R, Villringer A, Fahle M. What happens in between? Human oscillatory brain activity related to crossmodal spatial cueing. PloS ONE 3: e1467, 2008. Tucker DM, Roth DL, Bair TB. Functional connections among cortical regions: topography of EEG coherence. Electroencephalogr Clin Neurophysiol 63: 242–250, 1986. Visell Y. Tactile sensory substitution: models for enaction in HCI. Interact Comput 21: 38 –53, 2009. Wall S, Brewster S. Sensory substitution using tactile pin arrays: human factors, technology, and applications. Signal Process 86: 3674 –3695, 2006. Wascher E, Hoffmann S, Sänger J, Grosjean M. Visuo-spatial processing and the N1 component of the ERP. Psychophysiology 46: 1270 –1277, 2009. Weiss S, Rappelsberger P. EEG coherence within the 13–18 Hz band as a correlate of a distinct lexical organisation of concrete and abstract nouns in humans. Neurosci Lett 209: 17–20, 1996. Wolbers T, Büchel C. Dissociable retrosplenial and hippocampal contributions to successful formation of survey representations. J Neurosci 25: 3333–3340, 2005. Wong M, Gnanakumaran V, Goldreich D. Tactile spatial acuity enhancement in blindness: evidence for experience-dependent mechanisms. J Neurosci 31: 7028 –7037, 2011.

J Neurophysiol • doi:10.1152/jn.00624.2011 • www.jn.org

Downloaded from jn.physiology.org on July 10, 2012

Sakata H, Kusunoki M. Organization of space perception: neural representation of three-dimensional space in the posterior parietal cortex. Curr Opin Neurobiol 2: 170 –174, 1992. Sathian K. Visual cortical activity during tactile perception in the sighted and the visually deprived. Dev Psychobiol 46: 279 –286, 2005. Sauseng P, Klimesch W, Schabus M, Doppelmayr M. Fronto-parietal EEG coherence in theta and upper alpha reflect central executive functions of working memory. Int J Psychophysiol 57: 97–103, 2005. Seemungal BM, Glasauer S, Gresty MA, Bronstein AM. Vestibular perception and navigation in the congenitally blind. J Neurophysiol 97: 4341– 4356, 2007. Spitzer B, Blankenburg F. Stimulus-dependent EEG activity reflects internal updating of tactile working memory in humans. Proc Natl Acad Sci USA 108: 8444 – 8449, 2011. Srinivasan R, Winter WR, Ding J, Nunez PL. EEG and MEG coherence: measures of functional connectivity at distinct spatial scales of neocortical dynamics. J Neurosci Methods 166: 41–52, 2007. Stoeckel MC, Weder B, Binkofski F, Buccino G, Shah NJ, Seitz RJ. A fronto-parietal circuit for tactile object discrimination: an event-related fMRI study. Neuroimage 19: 1103–1114, 2003. Tallon-Baudry C, Bertrand O, Fischer C. Oscillatory synchrony between human extrastriate areas during visual short-term memory maintenance. J Neurosci 21: RC177, 2001. Tallon-Baudry C, Mandon S, Freiwald WA, Kreiter AK. Oscillatory synchrony in the monkey temporal lobe correlates with performance in a visual short-term memory task. Cereb Cortex 14: 713–720, 2004.

2729