ED: gowri Op: savitha/doss WNR: lww_wnr_3355 NEUROREPORT

VISION, CENTRAL

Figure^ground segregation requires two distinct periods of activity in V1: a transcranial magnetic stimulation study Klaartje Heinen,1,CA* Jacob Jolij2,* and Victor A.F. Lamme2 1 Anatomy Department, Wellcome Department of Imaging and Neuroscience, University College London,Gower Street, London WC1E6BT, UK; 2Cognitive Neuroscience Group, Department of Psychology,The Netherlands Ophthalmic Research Institute, University of Amsterdam, Amsterdam,The Netherlands CA,

Corresponding Author: [email protected]

*

These authors contributed equally to this manuscript. Received 8 June 2005; accepted12 June 2005

Discriminating objects from their surroundings by the visual system is known as ¢gure^ ground segregation. This process entails two di¡erent subprocesses: boundary detection and subsequent surface segregation or ‘¢lling in’. In this study, we used transcranial magnetic stimulation to test the hypothesis that temporally distinct processes in V1 and related early visual areas such as V2 or V3 are causally related to the process of ¢gure^ ground

segregation. Our results indicate that correct discrimination between two visual stimuli, which relies on ¢gure^ ground segregation, requires two separate periods of information processing in the early visual cortex: one around 130^160 ms and the other c 2005 Lippincott around 250^280 ms. NeuroReport 00:000^ 000
Williams & Wilkins.

Key words: Figure^ ground segregation; Primary visual cortex; Transcranial magnetic stimulation

INTRODUCTION While discriminating objects from their surroundings, the visual system groups elements belonging to the objects and segregates them from elements belonging to the background. This is illustrated in Figure 1a, which shows a square composed of line segments oriented in one direction, overlaying a background with line segments oriented in the orthogonal direction. At the location where the two orientations meet, a boundary is formed. However, in the final percept, the boundary is interpreted by the visual system as belonging to the figure, while the background seems to continue behind it [1]. Therefore, perception of the figure seems to entail two different subprocesses: boundary detection and subsequent surface segregation or ‘filling in’. Figure–ground segregation is the end result. Psychophysical studies in humans, addressing the question in what order events take place, indicate that boundary detection precedes filling in of the figure [2–4]. This was confirmed in more detail with physiological studies in monkeys. Neurons responding to a figure–ground display exhibit a temporal sequence of processing. Elementary feature detection (i.e. detecting the different orientations of the line segments that make up a textured scene like Fig. 1a) occurs as early as 60 ms after stimulus onset. This is followed by texture boundary detection around 90 ms. A possible correlate of filling in or figure–ground segregation shows a latency of about 120 ms [5]. This relative late neuronal response to figure detection has been suggested to derive from recurrent activity either from horizontal

c Lippincott Williams & Wilkins 0959- 4965


connections within V1 or from higher visual areas back to V1 [6,7]. In contrast to the classical view on visual perception, which states that information processing occurs hierarchically, starting at V1 and proceeding in a feedforward manner to higher cortical areas [8,9], more recent research has indicated that equally important for visual perception might be the occurrence of recurrent processing, where high-level areas interact with early areas such as V1. Anatomical studies revealed numerous recurrent connections between different brain areas [10,11]. Physiological studies in monkeys have suggested that these recurrent connections are functionally involved in more complex visual processing, comprising not only contextual modulation during figure–ground segregation (as mentioned above) but also modulation involved in perceived brightness [2], colour constancy [12], perceptual grouping [13] and attentional modulation [14]. To summarize, it has been shown by several studies that temporally distributed neuronal processes in V1, including a late period of activity that presumably derives from feedback connections, correlate with different aspects of figure–ground segregation. In this study, we aim to investigate whether those temporally distinct processes in V1 and other early visual areas are really causally related to the process of figure–ground segregation. Also, we aim to clarify the timing of these processes in humans. In order to isolate the process of figure–ground segregation from more low-level visual processing such as

Vol 00 No 00 & & 2005

1

NEUROREPORT

K. HEINEN, J. JOLIJ AND V. A. F. LAMME

(a) (b)

(c)

(d)

−500

0

70

100

130

160

190

Stimulus onset

220

250

280

350

500 Timing TMS pulse, in relation to stimulus onset (ms)

Fig. 1. (a) Square overlaying a background, de¢ned by di¡erences in orientation between the line elements belonging to either ¢gure or ground. (b) Frame. (c) Stack. (d) Experimental set-up. At t¼0 the visual stimulus was presented for 30 ms followed (or preceded) by a transcranial magnetic stimulation (TMS) pulse at di¡erent time points, ranging from ^500 to 500 ms, randomly distributed among trials.

surround inhibition [15] or other forms of contextual modulation, we use texture-defined visual stimuli, containing figures that arise because of differences in line orientation. The stimuli were designed in such a way that they contain an equal amount of texture boundaries; however, they differ in the amount of figure surfaces. Whereas one stimulus only contains a frame through which the background is visible (frame, shown in Fig. 1b), the other stimulus contains an additional figure within the frame, so that the percept of a stack of three surfaces emanates (stack, shown in Fig. 1c). Note that in both stimuli, a 451 jump is observed in the orientation of the texture going from either region to the next. The amount of texture boundary ‘signal’ thus is equal for stack and frame stimuli. Also, we made sure that, on average, all regions (background, frame and interior) were composed of the same line elements. Therefore, discrimination between these two types of stimuli requires figure–ground segregation purely on the basis of grouping of line elements. Because of an equal amount of texture boundaries in both frame and stack cases, a clear separation can be made between lower level processes like orientation and boundary detection on the one hand and surface (or figure–ground) segregation on the other. We hypothesize that for correct discrimination between these two stimuli, two temporally separate processes are required following the initial local feature detection, boundary detection followed by figure–ground segregation. Moreover, we hypothesize that both processes require activity of early visual cortex, including V1. This hypothesis was tested by means of transcranial magnetic stimulation (TMS) [16], which was applied over V1 and possibly neighbouring early visual areas, at different time points starting at 70 ms following stimulus presentation while participants were performing a discrimination task. According to our hypothesis we expect two dips in performance.

MATERIALS AND METHODS Study participants: Eight participants took part in the TMS experiments. They were between 23 and 31 years of

2

Vol 00 No 00 & & 2005

age, and had normal or corrected-to-normal vision. Informed consent was obtained in agreement with the standards demanded by the Ethics Committee of the Psychology department of the University of Amsterdam and participants were paid 7 euros per hour. Transcranial magnetic stimulation: TMS was conducted using a Magstim 200 with a 90 mm circular coil. Stimulation intensity was set on 90% of the maximal output, which elicited a 1.5 T magnetic field. The lower rim of the coil was centred approximately 1.5 cm above the inion. This position was selected on the basis of earlier published literature, which indicated that stimulation over this region principally excites V1 [16–19]. The current direction in the coil was directed clockwise. We did not use traditional sham TMS, but instead compared the effects at different TMS latencies, to account for all possible alternative effects of the TMS pulse, such as sound or startling of the participant by the cutaneous stimulation. Discrimination task: Participants had to discriminate between two types of visual stimuli: the so-called stacks and frames (illustrated in Fig. 1). These stimuli were composed of textures, with different orientations of line segments (671, 1121, 1571, 221). Depending on the orientations of the line segments of figure and background, one could either see a frame through which the background was visible (Fig. 1b) or perceive an extra figure within the frame (stack, Fig. 1c). The size of the relevant stimuli was ca. 11  11, presented on a full-screen textured background measuring 8.51  6.51. Stimuli were presented on a 1024  768 resolution screen with a red fixation dot (d¼5 pixels) in the middle. The line elements were 12 4 pixels. The luminance of the background was 145.5 cd/m2, of the line elements 127.4 cd/m2 and the contrast 7%. Presentation of the stimuli was intervened by homogeneous grey screens. Both stimuli contain an equal amount of texture boundary, consisting of 451 orientation discontinuities. The different orientations of the line segments were matched among the different

NEUROREPORT

FIGURE^ GROUND SEGREGATION IN V1: ATMS STUDY

displays so that all orientations were used in equal amounts for the two types of displays and at each location. Previous functional magnetic resonance imaging and electroencephalography measurements revealed that subtracting frame from stack activity isolates late activity that is more strongly present in low than in high visual areas and was therefore suggested to reflect recurrent activity [20]. Stack and frame stimuli covering an area of 11 11, on an 8.51  6.51 background, were presented in random order for 30 ms. Participants had to press the left response button when perceiving a frame and the right response button when perceiving a stack. Participants received TMS pulses at randomly selected latencies, either 70, 100, 130, 160, 190, 220, 265, 290, 350 or 500 ms after or 500 ms before stimulus presentation (as illustrated in Fig. 1d). Session blocks lasted for ca. 12 min, each containing all TMS latencies and stimulus conditions. Each participant did three blocks, yielding ca. 30 data points per stimulus–TMS latency per participant. Sessions were selected on the basis of the average performance obtained with TMS pulses at 500 ms before and after stimulus, which was taken as a control for overall behaviour of the participants, and general compliance with the TMS procedure, and had to be above an 80% score. Independent of this, baseline level was calculated on the basis of the average performance at the first (70 ms) and the last (350 ms) data point of the time course.

Analysis: For each data point, relative performance was calculated (i.e. proportion of correct responses). Performance was compared with baseline level and tested for significant difference using a w2-test, with n equal to the total number of trials for that data point. SEMs were calculated using a bootstrap approach.

RESULTS TMS pulses were applied over V1 at different time points while participants had to discriminate between stacks and frames (see Fig. 1d for the experimental set-up). These stimuli differ only in the sense that stacks contain an extra figure inside the frame, while the amount of texture boundary is identical. Therefore, we conjecture that discrimination between the two types involves figure–ground segregation. Figure 2a shows the average, over eight participants, of the proportion correct discrimination between stacks and frames for each stimulus–TMS latency. The dashed lines represent 7standard error of the mean (SEM). TMS induces two separate significant dips in performance: one at 130–160 ms after stimulus onset [mean¼0.7570.002 (SEM), po0.05 compared with baseline (mean¼0.8)] and one at 250 (0.7570.002, po0.05) to 280 ms (0.7270.002, po0.05) after stimulus onset. Figure 2b shows the proportion of correct discrimination between stacks and frames for each participant. Five out of eight participants show two significant dips, however with different latencies, which explains the relatively small effect on the average values. The first dip is narrow and occurs at 160 ms for participants 3, 6, 5 and 7, at 130 ms for participants 4 and 1, and at 100 ms for participant 2. The second dip is mostly broader and occurs between 290 and 350 ms for participants

2 and 4, between 250 and 280 for participants 1 and 5, and around 250 ms for participant 3. In summary, although not visible in all participants, two significant dips in discrimination performance were detected in most participants, reflected in the grand average with a first significant dip around 130–160 ms and a second dip around 250–280 ms after stimulus presentation.

DISCUSSION We find, in most participants, two separate significant dips in discrimination performance, which are reflected in the grand average of all participants around 130–160 and 250– 280 ms. This indicates that correct discrimination between two visual stimuli, which relies on figure–ground segregation, requires two separate periods of information processing in the primary visual cortex. The effect we found was not a full suppression of discrimination between stack and frame to chance level (50%), but rather a significant dip to a lower level of performance (B70%). In part, this is because we averaged results from participants who had their largest dips in performance at different latencies. Regardless, our results indicate that unperturbed V1 activity during these two epochs is necessary for optimal performance, that is, that the delayed activity in V1 is used in discriminating between scenes with a different figure–ground layout. Also, we cannot state with certainty that we only perturbed the ongoing activity of V1 with the occipital TMS. Adjacent areas like V2 or V3 might have been targeted as well. The strongest effect would have been expected on V1 activity, however. Also, if adjacent visual areas were affected this does not alter our conclusion that the activity of the earliest visual areas, such as V1, V2 or V3, is necessary for discriminating between stack and frame stimuli. Other areas, which in humans have been shown to be involved in analysing texture-defined shape, such as the lateral–occipital complex [21], were certainly out of reach of the TMS coil positioned at the inion. The finding that disrupting activity in V1 during a period of 130–160 ms following stimulus onset impairs visual discrimination seems to correlate with late dips in figure detection performance reported in earlier studies applying TMS over V1 [17,18,22]. These studies report a very early dip before 50 ms following stimulus onset and a later dip after 50 ms following stimulus onset, being maximal around 100 ms [18,23]. Our data show that a second period of information processing in V1 is required as well for correct discrimination between stacks and frames. The stimuli were designed in such a way that local features and the amount of texture boundaries were the same in both conditions. Only extra surface segregation is needed to discriminate between them. The second dip is therefore likely to reflect interference with processes involved in figure–ground segregation. Roelfsema et al. [24] propose a recurrent model that explains how the combination of early feedforward and late feedback processes can account for figure–ground segregation. During feedforward processing, lateral inhibition of cells responding to the same texture elements results in enhanced response to the boundaries, where differently orientated line elements meet. Following this first feedforward processing, higher visual areas like V2, V4 and TE, with larger receptive fields, will send back excitatory input

Vol 00 No 00 & & 2005

3

NEUROREPORT

Proportion correct discrimination

(a)

1.0

0.9

0.8

0.7

0.6

(b)

∗

0

50

∗

∗

∗

150 250 Time post stimulus onset (ms)

1

1

0.9

0.9

0.8

0.8

0.7

0.7

0.6 0.5

Proportion correct discrimination

K. HEINEN, J. JOLIJ AND V. A. F. LAMME

∗

∗ 0 50

150

250

350

0.5

1

1

0.9

0.9

0.8 0.6

0.6

∗ 0 50

150

250

350

0.5 1

0.9

0.9

0.8

0.8

0.7

∗

0.6

150

250

350

0 50

150

250

350

0.5

0.9

0.9

0.8

0.8

0 50

150

150

∗

250

∗

350

∗

0 50

150

250

350

0 50

150

250

350

0.7

∗

0.6

0 50

0.6

1

0.7

∗

∗

0.7 ∗ ∗

1

0.5

0 50

0.7

1

0.5

∗

∗

∗

0.8 ∗

0.7 0.5

∗

0.6

∗

350

0.6 250

350

0.5

Time post stimulus onset (ms) Fig. 2. (a) Grand average of relative performance on stack^ frame discrimination task, based on eight participants. Y-axis re£ects relative performance; X-axis re£ects timing of transcranial magnetic stimulation pulses following stimulus presentation (vertical dashed line indicates end of stimulus presentation). Asterisks indicate signi¢cant dips in performance compared with baseline level (indicated by the horizontal line). Two separate signi¢cant dips in performance are observed. (b) Relative performance for each participant individually. The dashed lines indicate 7SEMs of the average performance (see Materials and Methods section).

to cells responding to elements belonging to the figure, enhancing the response to the figure compared with the background. According to this model, one would expect two periods during which the process of figure–ground

4

Vol 00 No 00 & & 2005

segregation could be disrupted: one when feedforward processing and lateral inhibition occur, and the other when the feedback signal reaches V1. The two dips we find may correspond to these periods. However, the two time windows we find, between 130 and 160 ms and between 250 and 280 ms, do not fit this model exactly. The model mimicks the results found in monkeys, where feedforward processing and lateral inhibition occur at about 90 ms, while feedback processing starts at 120 ms. Our second dip in performance seems too late to fit these latencies. An alternative explanation might be that this dip in performance reflects interference with perceptual decision-making following figure–ground segregation, or interference with the final perceptual interpretation of the scene. This would be in agreement with the hypothesis that frontal brain areas form a ‘preliminary’ interpretation of a visual scene on the basis of the information transferred by the first feedforward sweep of activity. Recurrent interactions back to V1 would serve as a comparison of this interpretation with the complete retinotopic information in V1 [25]. On the other hand, latencies in the human brain may be delayed compared with processing speed in the monkey visual system, for example, because of differences in size. In addition, the stimuli we used in this experiment were of considerably lower contrast (7%) compared with the experiments performed in monkeys, which were typically 100% contrast, and it is known that low contrast stimuli have much longer response latencies [26]. This is probably also the reason why the early dip that we find is somewhat later (130–160 ms) than the dips found in other studies [18,23], which have their maximum typically at around 100 ms. The fact that in three out of eight participants no second dip was observed, does not necessarily mean that no feedback processing has taken place. In fact, it might mean that in these cases participants were able to give a (correct) response on the basis of the information extracted from the first period of information processing in V1 and feedforward sweep to higher cognitive areas. Possibly, enough information was already transferred during the first period of iterations. Alternatively, because in both cases the single dip occurred rather late, it may reflect both the first and the second processing period, lumped together. In conclusion, our study supports the idea that two separate processing periods in the primary visual cortex are required for figure–ground segregation. Whether the second phase purely reflects the process of figure–ground segregation or might be involved in perceptual interpretation, decision-making or other cognitive processes demands further study.

CONCLUSION These results indicate that two temporally distinct periods of activity within the early visual cortex are required to enable the process of figure–ground segregation: one around 130–160 ms and the other around 250–280 ms following stimulus presentation.

REFERENCES 1. Nakayama K, Shimojo S, Silverman GH. Stereoscopic depth: its relation to image segmentation, grouping, and the recognition of occluded objects. Perception 1989; 18:55–68.

FIGURE^ GROUND SEGREGATION IN V1: ATMS STUDY

2. Rossi AF, Paradiso MA. Temporal limits of brightness induction and mechanisms of brightness perception. Vision Res 1996; 36:1391–1398. 3. Caputo G. Texture brightness filling-in. Vision Res 1998; 38:841–851. 4. Julesz B. Texton gradients: the texton theory revisited. Biol Cybern 1986; 54:245–251. 5. Lamme VA, Rodriguez-Rodriguez V, Spekreijse H. Separate processing dynamics for texture elements, boundaries and surfaces in primary visual cortex of the macaque monkey. Cereb Cortex 1999; 9:406–413. 6. Lamme VA, Super H, Spekreijse H. Feedforward, horizontal, and feedback processing in the visual cortex. Curr Opin Neurobiol 1998; 8:529–535. 7. Hupe JM, James AC, Payne BR, Lomber SG, Girard P, Bullier J. Cortical feedback improves discrimination between figure and background by V1, V2 and V3 neurons. Nature 1998; 394:784–787. 8. Rolls ET, Tovee MJ, Panzeri S. The neurophysiology of backward visual masking: information analysis. J Cogn Neurosci 1999; 11:300–311. 9. Tovee MJ. Neuronal processing. How fast is the speed of thought? Curr Biol 1994; 4:1125–1127. 10. Budd JM. Extrastriate feedback to primary visual cortex in primates: a quantitative analysis of connectivity. Proc R Soc Lond B Biol Sci 1998; 265:1037–1044. 11. Bullier J, Hupe JM, James AC, Girard P. The role of feedback connections in shaping the responses of visual cortical neurons. Prog Brain Res 2001; 134:193–204. 12. Wachtler T, Albright TD, Sejnowski TJ. Nonlocal interactions in color perception: nonlinear processing of chromatic signals from remote inducers. Vision Res 2001; 41:1535–1546. 13. Kapadia MK, Ito M, Gilbert CD, Westheimer G. Improvement in visual sensitivity by changes in local context: parallel studies in human observers and in V1 of alert monkeys. Neuron 1995; 15:843–856. 14. Lamme VA, Spekreijse H. Modulations of primary visual cortex activity representing attentive and conscious scene perception. Front Biosci 2000; 5:D232–D243.

NEUROREPORT 15. Sillito AM, Grieve KL, Jones HE, Cudeiro J, Davis J. Visual cortical mechanisms detecting focal orientation discontinuities. Nature 1995; 378:492–496. 16. Cowey A, Walsh V. Tickling the brain: studying visual sensation, perception and cognition by transcranial magnetic stimulation. Prog Brain Res 2001; 134:411–425. 17. Amassian VE, Maccabee PJ, Cracco RQ, Cracco JB, Rudell AP, Eberle L. Measurement of information processing delays in human visual cortex with repetitive magnetic coil stimulation. Brain Res 1993; 605:317–321. 18. Corthout E, Uttl B, Walsh V, Hallett M, Cowey A. Timing of activity in early visual cortex as revealed by transcranial magnetic stimulation. Neuroreport 1999; 10:2631–2634. 19. Pascual-Leone A, Walsh V. Fast backprojections from the motion to the primary visual area necessary for visual awareness. Science 2001; 292:510– 512. 20. Scholte HS. Scene Segmentation. Ph.D. Thesis. Amsterdam: Faculty of Medicine, University of Amsterdam; 2003. Online available at: http:// dare.uva.nl/record/141824 21. Kourtzi Z, Kanwisher N. Representation of perceived object shape by the human lateral occipital complex. Science 2001; 293:1506–1509. 22. Juan CH, Walsh V. Feedback to V1: a reverse hierarchy in vision. Exp Brain Res 2003; 150:259–263 (E-pub. 2003 Apr. 2008). 23. Paulus W, Korinth S, Wischer S, Tergau F. Differential inhibition of chromatic and achromatic perception by transcranial magnetic stimulation of the human visual cortex. Neuroreport 1999; 10:1245–1248. 24. Roelfsema PR, Lamme VA, Spekreijse H, Bosch H. Figure–ground segregation in a recurrent network architecture. J Cogn Neurosci 2002; 14:525–537. 25. Hochstein S, Ahissar M. View from the top: hierarchies and reverse hierarchies in the visual system. Neuron 2002; 36:791–804. 26. Parker DM, Salzen EA, Lishman JR. Visual-evoked responses elicited by the onset and offset of sinusoidal gratings: latency, waveform, and topographic characteristics. Invest Ophthalmol Vis Sci 1982; 22:675–680.

Acknowledgements: K.H. and V.A.F.L. were supported by a grant from the McDonnel Foundation for Cognitive Science.

Vol 00 No 00 & & 2005

5

AUTHOR QUERY FORM

LIPPINCOTT WILLIAMS AND WILKINS JOURNAL NAME ARTICLE NO:

WNR 3355

7/5/05

QUERIES AND / OR REMARKS Query No

AQ1

Details Required

Authors Response

Please confirm whether the change of ‘Roelfsema and Lamme’ to ‘Roelfsema et al.’ is ok.

1