Current Biology Vol 24 No 13 R600

Dispatches

Visual Perception: Early Visual Cortex Fills in the Gaps A new human fMRI study shows how early visual cortex makes sense of complex visual scenes by segregating foreground and background, and by highlighting outlier objects. The findings are consistent with two attractive theories: biased competition and predictive coding. Andreas Bartels Our visual system achieves remarkable feats: we can easily recognize an elephant, even when its image in our eye actually appears as a set of stripes when it is located behind a fence. This mechanism also works for objects less familiar to us, and also when we remove surface-identifying cues such as color: our visual system interpolates missing elements, infers what is in the foreground and what in the background, disentangles occluded parts from occluder, and allows us to perceive surfaces and objects rather than the clutter of unconnected lines and patches that fall onto the retina. In technical terms, our visual system tidies up the incoming information by grouping, scene segmentation, and selection. Psychologists have come up with ways to expose this mechanism in extreme forms, a classic example being the Kanizsa triangle: the only physical cues in this stimulus are pacmen, which form the corners and give rise to the perception of illusory contours forming a triangle (Figure 1). A number of past human functional magnetic resonance imaging (fMRI) studies used such illusory contourinducing stimuli and compared the evoked activity to that of control stimuli, such as rotated pacmen that did not give rise to the illusion. Several researchers have reported an enhancement of activity in early visual cortex associated with perception of the illusion [1], whereas others reported an overall reduction of activity [2,3]. In a human fMRI study reported in this issue of Current Biology, Kok and de Lange [4] have now mapped with unprecedented spatial resolution and clarity the activity in the early visual cortex (areas V1 and V2) when participants viewed Kanizsa displays. The results are remarkable: compared to viewing a display with rotated inducers that does not give rise to the illusion, when the illusion was

perceived, activity in cortical sites representing regions inside the illusory triangle was enhanced, and activity of sites representing the inducers suppressed. In addition, activity increased in the cortical site representing a pacman that was not part of the illusion. It appears that, depending on the precise cortical representation of the Kanizsa triangle, opposite neural effects occur that were overseen in prior studies as a result of averaging across neural regions containing both effects. Attention (illusory contours may draw more attention) cannot easily account for the present results or for their divergence with earlier results: distractor tasks had been used in either set of prior studies, and the present study [4] shows that the results remain qualitatively the same when participants either actively attend the illusory shape or perform a highly demanding central distractor task. The real news, however, is not the resolution of a prior conundrum, but rather that some of the results have not been shown with such clarity in fMRI or in invasive electrophysiology before. So far, the much more detailed research using electrophysiology on scene segmentation distinguished four distinct processes, signaling of: real or illusory contours; border ownership; figure–ground surface; and attentional selection. Most neurons in visual cortex respond more to edges within their receptive field than to a uniform surface. The processing of contours therefore accounts for most of the neural activity in early visual cortex. In addition to coding for real contours, about a third of orientationselective V2 neurons also respond to illusory contours that span across uniform space, as seen in Kanizsa-type stimuli. Many fewer V1 neurons also signal illusory borders, and they do so only when these are formed by closely spaced patterns on either side of the contour [5,6]. Contour signaling is

comparably unsophisticated, as it does not distinguish what a given contour belongs to: every contour has only one ‘owner’, and this ownership determines which side of the contour contains foreand background. In the example of the elephant behind the fence, all vertical contours ‘belong’ exclusively to the fence, even though each contour has one side facing the elephant. If there were neurons in the brain that signal for each contour which side of it contains the foreground, a major problem of scene-segmentation would be resolved. It turns out that more than 50% of edge-responsive neurons in V2, and about 20% of those in V1, show this so-called border-ownership selectivity [7]. Computational modeling has shown that long-range connections between neurons within a single area (V1 or V2) can in principle account for illusory contour responses and for border-ownership selectivity, but, as so often, experimental evidence has shown that the brain goes beyond model predictions: feedback from higher-level areas plays a decisive role in scene segmentation. Research has shown that the time delays of border-ownership-selectivity responses are so short that they must reflect feedback from higher-level areas, and thus cannot be due to the much slower intra-areal spread [8]. Also, the removal of extrastriate regions reduces perception of foreground figures and of illusory contours, suggesting that higher-level regions play an important role in scene segmentation (see, for example, discussion in [9]). Finally, a human study suggests that the perception of illusory figures is influenced by activity in parietal cortex [10]. Scene segmentation signals in V1 and V2 are therefore at least partially relayed to them via higher-level regions. In visual perception, foreground surfaces stand out relative to the background (see illusory triangle in Figure 1). In theory, the brain could save energy by merely encoding edge-related information, which is interpreted perceptually as filled

Dispatch R601

surfaces. But this is not necessarily the case: neurons in V1 and V2 with receptive fields inside the foreground tend to fire with higher rates than those in the background, thus negating the above edge-only coding scheme [11,12]. The filling-in response could be mediated by lateral inhibition between neurons of similar feature-selectivity. For example, imagine a small foreground object with a unique feature (for example, a red surface on green background). Lateral inhibition within V1 or V2 will spread and eventually lead to a divergence of firing between foreground-encoding neurons (the minority, with less inhibition) and background (the majority, with more inhibition). This could cause rapid filling-in responses (within 10 ms) also in the absence of attention in V1 and V2 [11]. Alternatively, there is good evidence suggesting that feedback from figure-encoding neurons, for example in area V4, could also enhance responses [9,12]. A central question is how much of the above occurs automatically, and how much only when we pay attention to a particular figure. Patients with parietal lesions provided evidence for ‘preattentive’ scene segmentation early on: they were blind to rotated pacmen in their left, deficient, visual hemifield, but could perceive them when illusory Kanizsa-edges connected them to pacmen in their right, intact, hemifield [13]. In accord with this, border-ownership selectivity responses and surface-filling occur in parallel across a scene, and independent of attention [11,12,14]. Interestingly, the subpopulation of neurons showing border-ownership selectivity largely overlaps with that receiving attentional modulation. Also, attentional modulation is largely asymmetric, in that it primarily enhances border-ownership selectivity and surface responses of the attended foreground rather than suppressing unattended background [12,14]. The visual unit of attentional selection therefore occurs at the object-level in early cortex, leading to a competition between objects and surfaces for attentional resources. With this in mind, the enhancement of the Kanizsa-surface in the new study [4] makes sense — it reflects automatic enhancement of a foreground figure. What is puzzling, new, and likely to inspire more research is the selective reduction of the activity relating to the

A Experiment Kanizsa stimulus

Control stimulus

Early visual activity

-

= Contrast: (Kanizsa – Control)

B Scene segmentation

-

=

-

=

C Competition

2 objects

4 objects

D Predictive coding

-

= Current Biology

Figure 1. fMRI modulation and theoretical predictions related to scene segmentation. (A) Kanisza triangle and outlier pacman (left) and control stimulus (middle). On the right is illustrated the activity difference (Kanisza minus control) in early visual cortex as observed in the current study [4] and projected into the stimulus display. (B) Predicted activity based on electrophysiology related to scene segmentation. Reduction of background has only rarely been observed. (C) Predicted modulation based on ‘biased competition’ theory: distributing limited resources over two versus over four objects can account for the results in (A). (D) Predicted modulation based on ‘predictive coding’ theory: illusory contours should receive massive up-regulation, fate of triangle-foreground is uncertain.

inducers, along with enhancement of the outlier pacman. Two potential mechanisms related to currently popular theories come to mind. In the first of these, ‘biased competition’, visual objects compete for a limited set of neural resources. Processes such as attention can bias the competition

[15]. In the control-condition, four pacmen compete for resources. In the Kanizsa-condition, it is down to two: Kanizsa-triangle and outlier-pacman; compared to the control, both enjoy enhanced top-down resources, while the Kanizsa-inducers receive less, as they are no longer competing objects.

Current Biology Vol 24 No 13 R602

A second possible interpretation invokes ‘predictive coding’: early cortex receives predictive feedback from higher-level regions, and signal mismatches with respect to the sensory input [16]. There is a mismatch between prediction and input with regards to the (top-down predicted) illusory Kanizsa-contours, and arguably also with the surface: they have no sensory correlate, leading to enhanced activity. The sensory input of the three inducing pacmen matches the prediction: they form the corners of the predicted triangle, thus generating less error-signal, accounting for their reduced activity. This account does not easily explain why the outlier pacman’s activity is enhanced, as it was equally un-predictable in the control condition that served as baseline for the comparison. The strength and weakness of predictive coding is its simplicity — it tempts us to snub detailed mechanistic accounts. For example, it is unclear whether or how predictive coding combines with the concrete mechanisms of scene segmentation and competition described above that provide an equally good account for the results (Figure 1). A number of questions arise from the results [4] that have implications for past and future studies. First, the elephant in the room, the target of this and of prior studies, that still remains hidden behind the fence — what is the signal of illusory Gestalt contours? Can it eventually be unmasked from

fore- and background modulation? Also, why was there no interaction of scene segmentation with attention, whereas physiology suggests otherwise? Why did Gestalt-encoding in so many prior fMRI studies lead to negative net-signal [2,3,10], while physiology almost invariably reported positive modulations? What is the functional difference between V1 and V2? What are the origins of the observed modulations — intra-areal, V4, object-coding regions, parietal cortex? And, perhaps most importantly: do the answers to some of the above questions lie in fMRI, once again, picking up signals that physiology missed out on, and vice versa [17,18]? References 1. Seghier, M.L., and Vuilleumier, P. (2006). Functional neuroimaging findings on the human perception of illusory contours. Neurosci. Biobehav. Rev. 30, 595–612. 2. Fang, F., Kersten, D., and Murray, S.O. (2008). Perceptual grouping and inverse fMRI activity patterns in human visual cortex. J. Vis. 8, 2, 1–9. 3. de-Wit, L.H., Kubilius, J., Wagemans, J., and Op de Beeck, H.P. (2012). Bistable Gestalts reduce activity in the whole of V1, not just the retinotopically predicted parts. J. Vis. 12, 1–14. 4. Kok, P., and de Lange, F.P. (2014). Shape perception simultaneously up- and down-regulates neural activity in the primary visual cortex. Curr. Biol. 24, 1531–1535. 5. Peterhans, E., and von der Heydt, R. (1991). Subjective contours–bridging the gap between psychophysics and physiology. Trends Neurosci. 14, 112–119. 6. Grosof, D.H., Shapley, R.M., and Hawken, M.J. (1993). Macaque V1 neurons can signal ‘illusory’ contours. Nature 365, 550–552. 7. Zhou, H., Friedman, H.S., and von der Heydt, R. (2000). Coding of border ownership in monkey visual cortex. J. Neurosci. 20, 6594–6611. 8. Zhang, N.R., and von der Heydt, R. (2010). Analysis of the context integration mechanisms

Seasonal Timing: How Does a Hibernator Know When to Stop Hibernating? Deep hibernators that spend winter in a hypothermic coma below ground can still emerge and reproduce in spring at the right moment. A recent study shows that specific cells of the pituitary may harbor the internal calendar responsible for this. Roelof A. Hut1,*, Hugues Dardente2, and Sjaak J. Riede1 Many species in seasonal environments enter a state of dormancy in winter to avoid unfavorable conditions, such as low

temperatures and reduced energy resources. This behaviour can be found in micro-organisms, plants, invertebrates and vertebrates. An example of a unicellular organism that enters winter dormancy is the dinoflagellate Alexandrium

9.

10.

11.

12.

13.

14.

15.

16.

17.

18.

underlying figure-ground organization in the visual cortex. J. Neurosci. 30, 6482–6496. Cox, M.A., Schmid, M.C., Peters, A.J., Saunders, R.C., Leopold, D.A., and Maier, A. (2013). Receptive field focus of visual area V4 neurons determines responses to illusory surfaces. Proc. Natl. Acad. Sci. USA 110, 17095–17100. Zaretskaya, N., Anstis, S., and Bartels, A. (2013). Parietal cortex mediates conscious perception of illusory gestalt. J. Neurosci. 33, 523–531. Roelfsema, P.R., Tolboom, M., and Khayat, P.S. (2007). Different processing phases for features, figures, and selective attention in the primary visual cortex. Neuron 56, 785–792. Poort, J., Raudies, F., Wannig, A., Lamme, V.A., Neumann, H., and Roelfsema, P.R. (2012). The role of attention in figure-ground segregation in areas V1 and V4 of the visual cortex. Neuron 75, 143–156. Mattingley, J.B., Davis, G., and Driver, J. (1997). Preattentive filling-in of visual surfaces in parietal extinction. Science 275, 671–674. Qiu, F.T., Sugihara, T., and von der Heydt, R. (2007). Figure-ground mechanisms provide structure for selective attention. Nat. Neurosci. 10, 1492–1499. Desimone, R., and Duncan, J. (1995). Neural mechanisms of selective visual attention. Annu. Rev. Neurosci. 18, 193–222. Rao, R.P., and Ballard, D.H. (1999). Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat. Neurosci. 2, 79–87. Bartels, A., Logothetis, N.K., and Moutoussis, K. (2008). fMRI and its interpretations: an illustration on directional selectivity in area V5/MT. Trends Neurosci. 31, 444–453. Logothetis, N.K. (2008). What we can do and what we cannot do with fMRI. Nature 453, 869–878.

Vision and Cognition Lab, Centre for Integrative Neuroscience, University of Tu¨bingen, Otfried-Mu¨ller-strasse 25, 72076 Tu¨bingen, Germany. E-mail: [email protected] http://dx.doi.org/10.1016/j.cub.2014.05.055

(Gonyaulaceae). It drops to the sea floor to enter a state of winter quiescence when days get short and light availability is low. A similar behaviour occurs in deep hibernators like ground squirrels and several hamster species: they retreat in their burrows, seal the entrance and stay below ground for 6–8 months in a state of deep hibernation with body temperatures slightly above ambient temperature (w5–10 C) [1]. Although winter dormancy may increase survival, it also introduces a problem: both the algae in the mud of the sea floor and the hibernators in their burrows are overwintering under stable conditions in the absence of light. Since perception of day