NEUROCOMPUTINC Neurocomputing 12 (1996) 289-310

How the brain can discover the existence of external egocentric space T P. Morass0 *, V. Sanguineti Departmentof Informtics, Systems and Tekmmmunications (DIST), University of Geneva, Vu OperaRa 13, I-16145, Genova, Italy

Received 25 November 1994; accepted 2 October 1995

Abstract The neurobiological problem is addressed of how the brain might build an internal representation of external egocentric space by fusing the information from different sensory modalities. The proposed model is based on self-organizing topology representing networks, activated by multi-modal sensory vectors. The learnt representation is invariant to coordinate transformations and can support an active planning function by exploiting the lateral connectivity of the network. Keywords:

sentations;

Neurobiology; Self-organization; Multi-sensory fusion

Topology-representing

networks; Space repre-

1. Introduction

In spite of the doubts of philosophers like Berkeley about the existence of an external space in which objects and individuals are located, human beings seem to take it for granted and the richness of linguistic elements to denote space, spatial relations and actions in space is a clear indication of this common wisdom. This implies a number of underlying assumptions which are shared by a large majority of people, although obviously are not necessarily true: l Space has an objective existence, independent of the fact that we have any direct perception of or focus our attention to it.

tork was partly supported by the Esprit Basic Research Action SPEECH-MAPS and MIAMI. Corresponding author. Email: [email protected] l

0925~2312/96/$15.00 0 1996 Elsevier Science B.V. All rights reserved SSDI 0925-2312(95)00115-8

290

P. Morasso, K Sanguineti /Neurocomputing 12 (1996) 289-310

We know about space through our sensory channels and although they have markedly different characteristics, the spatial content of perceptual experiences is substantially modality-independent: we cannot compare a color with a whistle but we do compare (and fuse) the up/down /right/left cues which come from vision, audition, and touch. l We can think about space even without any direct perception of it and the corresponding mental experiences are somehow equivalent to the ones based on direct perception; moreover, any metal/physical activity which has to do with space can be carried out so effortlessly, spontaneously, and with remarkable efficiency that we take it for granted the explicit representation of it inside our brain. Apart from the philosophical debate among idealistic and empiricistic interpretations of these facts, which has a very long history, we are interested here on the neurobiological aspects and their implications for a neurocomputing theory of brain function. In particular, the intuitive/informal introspection about space sketched above has led some researchers to the neurobiological hypothesis that space might be mapped somehow topographically in the brain [26,10,13]; the hypothesis is supported by a number of points: l The existence, primarily in the cortex, of somatotopically organized topographic maps of different types. l The typical geometrical distortion of the sensory channels and the corresponding primary cortical representations which contrasts with the more uniform representation of space suggested by the observation of natural movements. l The perceptual stability which contrasts with the complex time-varying patterns related to our dynamic interaction with the outside world. l The multiplicity of coordinate systems associated with the different sensory and motor channels and the corresponding non-linear mappings. l The kinematic invariances of simple common movements like reaching, expressed by the Bernsteinian principle of motor equivalence. On the other hand, the possible existence of some kind of topographic map of external space, as a common substrate for integrating the different sensory and motor channels, is dismissed by other researchers (see, for example [28]) mainly on the basis of negative evidence, from single-cell recordings and lesions. In particular, it has been observed that (i) maps of neurons with a clear ego/geotopic organization (similar to the clear retino-topic organization of the neurons in the primary visual cortex) have not been found and (ii) small lesions in the posterior parietal cortex do not cause ‘space scotomata’ [ll] whereas large lesions do cause a massive impairment of space perception (the phenomenon of negfect). Such negative evidence, in our opinion, is not conclusive and in any case is not sufficient to completely offset the circumstantial positive evidence about the plausibility of the existence of some form of topographical representation of space. In support of this hypothesis, we review the available neurophysiological evidence (Section 2) and we describe a feasible computational model based on self-organized, topology-representing topographic maps of egocentric space (Section 3), further developing a previous line of research [18,19,21,201. Section 4 is a simulation study l

291

P. Morasso, K Sanguineti/ Neurocomputing12 (19%) 289-310

and Section 5 discusses an example of distributed computation topographic maps, with concluding remarks in Section 6.

on the learnt

2. Neurobiological basis If a topographic map of space exists, it is necessarily much more difficult to detect than the topographic maps of specific sensory channels, like vision. In this case, indeed, the sensory organ is already organized in a topographic, 2-D way and this organization is clearly maintained up to the primary cortical areas. On the contrary, a ‘space sensory organ’ for measuring spatial positions does not exist and space measurements can only be derived by some kind of correlation among different sensory channels: visual, auditory, somaesthetic, kinesthetic. Therefore, spatial cells will be inevitably multi-modal and the organization of their receptive fields will be very difficult to characterize experimentally. Experimental difficulties are also responsible for the clear paucity of neural data about multisensory problems which would require to seek the integration of cues across, rather than within, modalities. Thus, the well-known convergence of different sensory inputs at most levels of the neuroaxis is the introduction of a book which still awaits to be written. A frequently overlooked source of difficulty is that external space has more than 2 dimensions (3 if we limit ourselves to pointing or 6 if we take into account orientation as well as position) whereas the cortex looks like a 2-D sheet of computational units (microcolumns) and many subcortical areas are similarly organized in sheets. However, what really matters is not the physical macroscopic bi-dimensionality of the cortex but the computational dimensionality which depends on the pattern of lateral connections. For example, Fig. 1 shows how a 2-D layer of units can support a l-D, 2-D or 3-D topology: It is not the physical vicinity of units which is computationally relevant but the topological vicinity, which depends on the connectivity pattern. (The distance between the same two units A and B in the figure is 8, 1 and 3 in the three cases, respectively.) In principle, it is possible to represent space lattices of any dimension on the same 2-D sheet of units and the only requirement is to setup the appropriate pattern of lateral connectivity.

Fig. 1. Different topologies implemented on the same sheet of neural elements.

292

P. Morasso, V. Sanguheti / Neurocomputing 12 (1996) 289-310

It is interesting to note that there is a one-to-one relation between dimensionality and numerosity of the lateral connections, as it is better explained in the next section. Therefore, counting the lateral connections in a section of the cortex should be a robust indicator of the hidden dimensionality of the patterns which are processed there. Moreover, since connectivity grows very quickly with dimensionality, there could be a hard-limit to the greatest dimensionality representable in the cortex, due to the limited size of layers I and II which supposedly support the connections. To our knowledge, this is a point still open to experimental investigation, particularly as regards the topology of cortico-cortical connections. In any case, the possible computational role of lateral connections in the cortex is currently a hot topic and different functions have been suggested, such as (1) the modulation of receptive field properties in a context dependent manner and/or a mechanism of adult cortical plasticity, (2) the storage of associative information such as Gestalt rules, (3) the stimulus-dependent synchronization and dynamic feature binding [2]. The paper is a contribution to this debate in relation to the integration of multi-sensory data and the training of topographic maps for the representation of egocentric space: it requires, in order to yield a stable picture of the world, comparisons over long distances rather than local ones and this can be provided by a topographical map of space with a topology-preserving lateral connectivity.

Fig. 2. Block-diagram of the cortico/subcortical areas and the cortical/basal ganglionic loop: PMC (primary motor cortex), PMA (pre-motor area), SMA (supplementarymotor area), CG (cingulate gyrus),PsC (primary somatosensory cortex), PPC (posterior parietal cortex), VC (visual cortex), PAC (primary auditory cortex), BG (basal ganglia/striatum), PU (putamen n.), CA (caudate n.), VN (ventral n.), GP (globus pallidus), SN (substantia nigra), TH (thalamus), VL (ventro-lateral group of nuclei), VA (ventro-anterior group), PG (posterior group), IL (inferior lamina), CB (cerebellum). Open circles: excitatory synapses; filled circles: inhibitory synapses.

P. Morasso, K Sanguineti/ Neurocotnputing12 (1996) 289-310

293

The most likely site for the cortical representation of egocentric space is the lower part of the PPC (posterior parietal cortex, area 7) (Fig. 2) [281. It has cortical inputs from somaesthetic, auditory, and visual areas as well from the limbic system (posterior cingulate motor cortex: motivation/attention); subcortical inputs from the superior culliculus via the pulvinar (retinal, oculomotor, and neck position information); cortical outputs to the supplementary motor cortex and premotor cortex; subcortical outputs to basal ganglia, pulvinar, superior culliculus, and cerebellum. Thus, as suggested by Stein, ‘on anatomical grounds we can consider the PPC as a multimodal sensorimotor association area’. Another site of multi-sensory integration can be found in the deep layers of the superior culliculus (SC): they receive inputs from different sensory modalities and motor-related structures and send their outputs to areas of the brain stem and spinal cord directly involved in orienting sensory organs, but do not appear to play any role in initiating voluntary movements [27]. Interestingly, however, auditory projections to PPC & SC mainly consist of binaural neurons, rather insensitive to pure tones, with spatially restricted receptive fields, which are sensitive to interaural time differences and intensity differences. The upper part of the PPC (area 5) has no exteroceptive inputs (visual & auditory) but strong somatosensory and limbic inputs as well as inputs from motor and premotor areas (4 & 6) and from area 7. It projects to thalamus (pulvinar) and basal ganglia (putamen) thus interacting with the cortical/basal ganglionic circuitry [l]. This is a loop in which motor cortical areas (primary motor area 4 and supplementary motor area 6) project to an input nucleus of the basal ganglia (the putamen) with excitatory connections; the putamen sends inhibitory projections to two output nuclei of the basal ganglia (globus pallidus and substantia nigra) which in turn inhibit a nucleus of the thalamus (ventralis lateralis) sending back its projections to the motor cortical areas with excitatory synapses. The loop is characterized by both serial and parallel connectivity; by substantial layer-to-layer convergence and divergence at the level of individual neurons; by topographic organization at a gross level combined with a high degree of functional tuning at the neuronal level; by concurrent, distributed processing at the various analytically defined sensorimotor transformations; by widespread plasticity with respect to connection strengths and the response properties of individual neurons. Thus, it has the potential to reinforce the activation of any of the cortical motor areas, and this could implement different interesting types of global map computations in terms of force fields. In the loop it is possible to find both target-dependent preparatory cells, independent of movement’s kinematics or dynamics, and limbdependent movement-related cells. Although the neural onset times of the different neurons of the cortical/basal ganglionic loop as well as the neurons in the PPC are not coincident, there is so much temporal overlap among the activities of these areas that the global processing appears to proceed concurrently, without any clear relation of causality. PPC lesions cause a variety of symptoms. In particular, lesions in the superior part (area 5) give rise to disorders in active touch (a-stereognosti: inability to

294

P. Morasso, V. Sanguineti/Neurocomputing 12 (19%) 289-310

recognize objects by touching them), stability of the body-image (a-morphosynthesis: inability to establish spatial relations of one’s own limb), or even awareness of it (a-somatognosia). Lesions in the inferior part (area 7) cause the inability to pay attention to the outside world (visual inattention and neglect), with a mixture of body-centered and head-centered symptoms [3]. Moreover, asymmetries in symptoms of left vs right emispheric lesions seem to support a specialization of attention to spatial order in the right PPC and serial or timing order in the left PPC. On the other hand, small lesions do not cause any detectable spuce scotomu. The representation of space and its link with motor preparation is also at the basis of the notion of motor imagery, which has been recently investigated by Jeannerod [131 for the study of motor actions. The experimental difficulty of identifying a topographic representation of egocentric space in the PPC and related areas of the cortical/subcortical loop has two aspects. First of all, the unknown patterns of lateral connections which would support the representation will make it difficult to interpret the firing patterns of neurons which are nearby in the cortical area but are faraway in the map or vice versa. Perhaps even more important is the fact that since a spatial code only can arise from some kind of conjunction of multimodal patterns, then nodes of an hypothetical topographical map might not correspond to a single neuron but to the behavior of a whole ensemble, as a cortical column or part of a column: this would make the identification of spatial receptive fields almost impossible or, at least, quite hidden to classical electrophysiological techniques. Nevertheless, some evidence of spatial organization has been found in cortical and subcortical areas. Three brain regions, which are monosynaptically connected, contain a somatotopic map of the egocentric space: two cortical areas (premotor area 6 and associative area 7) and a subcortical area (putamen). Cells in the putamen respond to tactile and visual stimuli with matched receptive fields, invariant to arm movements [lo]. Bimodal cells with matched receptive cells were found in area 6 1241and in area 7 [12]. Spatial elements are also present in the series of experiments of Georgopoulos and colleagues, who found cortical neurons which code in a collective way (population coding) the direction of movement in space, thus hypothesizing the organization of receptive fields in terms of spatial coordinates rather than joint or muscle coordinates [8]. In particular, comparing loaded and unloaded movements, it was possible to find neurons in area 4 and 5 with a similar organization of receptive fields but differing as regards dynamics [14]: while the latter ones are load-insensitive and thus have to do with kinematics, the former ones are load-sensitive and thus are related to dynamics. The interpretation of these data is not unique and, as observed by Fetz [6], descriptive correlations of this kind do not provide a causal framework for dealing with the underlying neural computation. Moreover, Mussa-Ivaldi [22] showed that the experimental data cannot distinguish between population of cells which code muscle shortening and populations which code hand trajectories, re-opening the question about the frame of reference which is most appropriate from the epistemological point of view (best for

P. Momso, V. Sanguineti /Neurocomputing 12 (19%) 289-310

295

describing the data) or from the ontologicai point of view (actual implementation in the brain). The notion of population coding -emphasizes the collective properties of cell assemblies rather than the motor features or movement parameters which determine the tuning curve of individual neurons and this has lead several people to understand that movement parameters need not be explicitly coded in the activity of single neurons. On the contrary, the pattern which is most appropriate for a given cell is mainly determined by its connections with the rest of the network and, from the modeling point of view, this suggest to simulate motor activity without explicitly representing movement parameters in the activity of particular units. Unfortunately, the possibility that such connections could in fact represent a topology-preserving map has gone unnoticed so far. Along the same lines, we can observe that a topographical representation of egocentric space does not require an explicit coding of spatial coordinates, in a specific coordinate frame. We might as well have neurons or neural assemblies which respond to the conjunction of visual, auditory, somatosensory afferences and represent space implicitly, via topology preserving lateral connections. Moreover, this kind of representation would be invariant to coordinate transformations.

3. Egocentric space as a topology-representing network Consider the numerous systems of coordinates which can be used for describing goal-directed actions: from the system in which the spatial location of the goal is coded to the systems where sensing occurs, and then to the systems in which the actual movement is executed. In order to solve problems which require to handle descriptions in the different systems, as in perception and motor control, it is possible to adopt two different approaches which, shortly, we may label cybernetic and ropoZogic. The former one, popularized by David Marr, is based on the explicit computation of coordinate transformations and thus is computation intensive and intrinsically serial. The alternative view, which is supported in this paper, is based on the explicit representation of topological invariants which underly the transformations; in particular, we propose a computational scheme which represents space as a topology-preserving, self-organized network of high-dimensional vector prototypes, which bring together heterogeneous and redundant sets of measurements, each one expressed in its own natural coordinate frame. This scheme is storage-intensive (the large number of vector prototypes) and intrinsically parallel because computation can be carried out at each node, simultaneously and globally, i.e. operating on the different heterogeneous data in their own reference frames. The computations are local but the topological structure allows global interactions to emerge, similarly to the propagation of waveforms in systems of interacting particles. The price is in terms of memory but the gain is robustness, as regards both of invariance to coordinates transformation and graceful degradation to small local lesions. The geometric theory which underlies the concepts above is outlined first, by taking into account the topic of tessellation of n-D space; then we analyze

296

P. Morasso, K Sanguineti/Neurocomputing 12 (1996) 289-310

its relevance for representing multi-sensory mentation by means of topology-representing

space and the computational networks (TRN) [17].

imple-

3.1 Tessellationof n-dimensionalfeature space If we sample a limited compact region of n-D space according to a uniform distribution, it is possible to derive from the set of samples a unique subdivision into small polyhedral cells which is called Voronoi tessellation of the space. If {q; i = 1,. . . N} is the set of n-D samples, then the corresponding Voronoi (hyperjpolyhedra are defined as follows:

p’: lip’-sll

I IIP’-~llVj+i

>

p’, +R”

For example, in 2-D most of the polyhedra are exagons, in 3-D are dodecahedra, in 4-D are 24-hedra and so on. The associated Delaunay triangulation, which consists of linking together the nodes whose Voronoi cells have an (hyper)face in common, is characterized by a number n, of connections per node whose stochastic distribution p(n,) is a function of n. If we limit our attention to the mean value of n, as a function of n, which is a much simpler problem than estimating the full probability density, we can take advantage of the results from the theory of the sphere-packing-problem (SPP) which, in qualitative terms, can be described as the problem of determining the densest packing of equal spheres in n-D space [5]. The exact answer is only known up to two dimensions, but if we limit our consideration to lattice packings which have regular periodic spacing of the spheres, there are exact results up to eight dimensions. Moreover, the densest packing presently known up to 29 dimensions are of the lattice type. An important parameter of a packing is the kbsing number K, which can be defined as the number of spheres in the packing which touch (or ‘kiss’) a generic sphere. This parameter is space-invariant for lattice packings, in the sense that has only one value, and is characteristic of the dimensionality of the space. For embedding spaces from 1 to 8 dimensions, the values of K are: 2, 6, 12, 24, 40, 72, 126, 240. The link between SPP and TRN’s comes from the fact that the best packings presently known are dual with respect to the best known quantizers, in the sense that if the input data are uniformly distributed, the centers of a packing ‘perform well’ as a codebook [5]. In the case of regular lattices extended indefinitely, which are characterized by a periodic spacing of sphere centers in different directions, it is immediate to observe that the K is equal to the value n, of the corresponding Delaunay triangulation: it is sufficient indeed to construct the Voronoi cells by tracing the (hyper)planes through each of the contact points and intersect them, thus yielding a number of faces equal to K. In others words, the Voronoi cells are the circumscribed (hyperjpolyhedra of the given (hyperbpheres. With non-periodic lattices which results from some regular but stochastic process, such as sampling a uniform (or also sufficiently smooth) distribution, we cannot apply the theory of dense sphere packings in a direct way. However, we may think that as a result of the sampling process a thermal noise is superimposed on the regular lattice, thus

P. Morasso, V: Sanguineti/Neurocomputing 12 (1996) 289-310

1D

291

2D 3D _______-___-___--___-~~~-~~~-~~~--~~~-~~~-~~~~~~ ,~______________________‘* 0 ,‘: l l ; .,.*’ I II . . y* .i /. 1 I8 #’ d l 0 l l /’ ,’ ..I a’* .i l l I, I /’ .’ ;__________:n___~____ /f’ l * I I, l : . 0: 1 j.. . . i t .i . 0 0 I I . II i. I. l 0 . l *I Ie *I l . II I I l t . l ;_______ . .I ________; ,I i. . ,N’ 0,’ I I. . l I’ # I * #’ : II 0 . /’ a’ 1 i. ,1’. l ’ .!0 l/o’ . l I* ,,’ I r,r . l . . I ,’ I #’ ‘~_~~~_~~~_____~~~__~__________________________~ U____~______________?_~~ Fig. 3. Dimensionality and Delaunay triangulation in l-D, 2-D, and 3-D space.

deforming the spheres and the corresponding Voronoi cells with respect to the regular configuration, the more the larger the noise. Correspondingly, we may expect that local topological changes (merging or splitting of faces), which would locally affect the connectivity, have a probability which is a growing function of noise. We are not aware of formal results for general n-D lattices but simulation experiments with embedding spaces up to 5-D indicate that at least in these cases the probability of local topological fluctuations of the lattice is very small [7]. For higher dimensions the probability of fluctuations is likely to grow, but we do not know how steeply. In any case, we may expect a computational limit to arise from this sort of noise as regards the maximum dimensionality compatible with given performance criteria of an architecture which exploits the topological structure described above. Summing up, we propose the conjecture that the average connectivity of the Delaunay triangulation acquired via a process of sampling/learning is a good indicator of the dimensionality of the underlying manifold, as shown in Fig. 3 (the 1-D lattice has 2 links per node, the 2-D lattice has 6, and the 3-D lattice has 12). The topological approach emphasizes three main aspects of the curse of dimension&y with architectures based on topographical maps: (1) the exploding mass of lateral connections, (2) the topological instability of the lattices, and (3) the size of the training set, which is able to sufficiently jU the workspace. Theoretical and experimental investigations are needed for understanding, for a given problem, which limit is reached first, thus bounding achievable complejrity and performance. 3.2 Topological invariances in representing multi-sensory space Next, we consider a continuous non-linear mapping from a given object space ?E %“cW” to a multi-modal measurement or sensory space ZECC cW” with m > n, which might result from a redundant set of measurements of the location of points in the object space. The mapping transforms a region in the input space into

298

P. Morasso, K Sanguineti / Neurocomputing I2 (1996) 289-310

a corresponding non-linear manifold in the output embedding space and a cloud of samples in one space into the corresponding cloud of samples, constrained on the same manifold. The continuity implies that relations of adjacency are preserved: if 2~9 is an adherent point of an arbitrary subset 9 ~9, then the image f(a of a’ is an adherent point of the imape f(S) of 9. However, this is not sufficient for our purpose of representing space by means of a topologic scheme. We also need that in addition to preserve the old adjacencies, the mapping does not induce new ones. This is the concept of topological or homeomorphic mapping which implies continuity, one-to-one correspondence, and continuity of the inverse mappings. Let us now consider a set of samples in 9, its image in S’, and the corresponding Delaunay triangulations. In the limit, i.e. for a sufftciently dense set of samples, the two triangulations must coincide as a consequence of the topological continuity of the mapping, i.e. Delaunay triangulations are, in the small, topological invariant in topological mappings. For less dense sets of samples, we might have variations of the connectivity, another kind of topological fluctuation which depends on the shape of the manifold, in addition to its dimensional&y. If the manifold is linear, Delaunay triangulations are invariant in the large as well as in the small but for curved manifolds we may expect the probability of ‘topological inconsistencies’ (points which merge and/or links which are split) to increase with the curvature. The curvature of the manifold is related to the the Jacobian J of the transformation s’+ x”: u!?= Jds: In particular, the metric on the manifold is given by: di2 = @a!3= aT’( JTJ)ds’

(2) and topological continuity requires that the determinant of the matrix d =.lTJ does not got to zero; this means that there must be no singular points in the S-volume of interest. We may call B the observabilitymatrix (because it allows to recover s’ from ~3 and its determinant the obseruabiktyindex, in analogy with the manipulability index defined for the inverse kinematics of redundant kinematic chains. The observability index is useful for estimating the goodness of the design of a given multi-sensory measurement system and comparing different arrangements of sensors. If it is sufficiently far from zero in the volume of interest, then it is also possible to recover the dimensionality and the topology of the space by constructing the triangulation in the measuring space and analyzing its topological features (see Fig. 4). In these circumstances, the topology-preserving topological map in the measuring space is an effective representation of the objective space, without any need of explicit coordinate transformations and any explicit representation of the objective coordinates. Building the map is a learning problem which requires the following computational elements: (1) A physical process driven by an appropriate behavioral strategy which samples the object space in a sufficiently uniform way, thus generating a suitably large set of input vectors $ (2) A measurement or sensory system, which collects the multi-sensory vectors x’ causally dependent on the corresponding s” vectors and has been designed in such a way to guarantee a sufficient level of the observability index all over 9; (3) A codebook of multimodal reference vectors l-lx = I?;‘, i ~9);

P. Morasso, E Sanguineti/Neurocomputing 12 (1996) 289-310

299

k4easurement space

Object space

xln

Fig. 4. Topology preserving transformation from object space to multi-sensory measurement space: ?=f($J. F,, Fz,... F,,,: reference frames of the different sensory modalities. x1, x2, . . . x, corresponding sensory measurements. The transformation is one-to-one and preserves, in particular, the Delaunay triangulation, provided that the observability index de@J) is sufficiently > 0.

(4) A leuming rule for optimally allocating the reference vectors ?i” and building the lateral connections which embody the topology of the manifold. 3.3 Topology representingnetworks The class of network models which are more naturally associated with the concept of representing space as a topology-preserving topographical map is the so called neural gas and its generalizations (Topology RepresentingNetworks: TRN’s), which are based on neurons with radially symmetric receptive fields: these are centered around prototype vectors in the embedding multi-dimensional space and learning operates as a competition process among both the prototype vectors and the lateral connections [16,17]. The network can be treated as a graph 9 = W, 81, where JY = {vi ~9) is a set of neurons and B = (cij cf x_/)is the corresponding set of connnection vectors. The neurons in the map, which are activated by a common input vector 3~ PC Rm, are characterized by two quantities: the center of the receptive field ?;i” E R” and an activation level ui E [0, l] which is determined by an activation function ui = Ui(X3 such as the softmax function [4]. The protype vectors are learnt with the Hebbian rule

(3) and the lateral connections

Cij

with a similar competition

among connections [16].

300

P. Momso, I/ Sanguineti/Neurocomputing

I2 (1996) 289-310

At the end of training, the database of protype vectors or codebook lJX tessellates the input manifold SY into Voronoi hyper-polygons which cover the manifold and the connectivity pattern perfectly preserves topology: cij = 1 if and only if the corresponding Voronoi polygons are neighbors in 2. An interpolation function is also available

which represents a given vector x’ by a cluster of activity or population code, centered around the winning neuron - the neuron whose receptive field center is closest to x - and restricted to the subset of its neigbors in the network 9 ~3. Thus, the trained map is not merely a quantized table but can operate as a smooth interpolation mechanism as well: this allows to smoothly navigate in the TRN by shifting the cluster of activity. An importatn property of TRN’s is that the identification of the winner is a local, not a global computation [21]. It is sufficient indeed that each neuron computes the distance from the input vector hi = IIx’- ?;i” I1 and transmits this information to all its neighbors: the winner is then the one for which this value is the smallest in the set of its immediate neighbors. In a standard self-organized map [15] this is not true, in general. 3.4 Egocentric space We can now address the problem of multi-sensory integration into an internal representation of egocentric space, i.e. a computationaal structure which is not simply a passive topographical map but an active topology-preserving planning mechanism. The multi-modal sensory vector x’ is causally related to the spatial location of an external stimulus, which can be an object, a part of the body, or the coincidence of both, when an object is touched/bitted/grasped or manipulated in some way. In any case, we assume that what matters for the hypothesized multi-modal sensory process is just the spatial location P of the stimulus as if it were collapsed to a single point. This implies, on one hand, the exclusion of non-spatial sensory features such as color or tone and, on the other, the extraction/abstraction by the primary sensory processes of spatial estimates of P, each in its own natural system of coordinates. The different components of x’ can be considered as projections of P on any particular sensory subsytem, such as the following ones: Retinal coordinates of the image of P, both in the monocular or binocular case; Oculomotor coordinates, i.e. position of one or both eyes in their respective orbits when P is foveated; Aural coordinates, i.e. binaural time and/or intensity differences determined by the position P of an object which emits sound either autonomously (e.g. a singing bird) or as a consequence of physical interaction (e.g. impact); Huptic coordinates, which derive from the conjunction of tactile information,

P. Morasso, K Sanguineti/Neurocomputing 12 (1996) 289-310

301

due to a physical contact, with the articulatory coordinates of the limb/body part stimulated by a point P ‘. Moreover, if P corresponds to the contact of a body part, say a fingertip, not with an object with another body part, then the haptic vector is in fact doubled into a touching and touched component. Audio-visual coordinates are intrinsically head-centered whereas haptic coordinates are intrinsically body-centered. The articular coordinates of the head with respect to the body are the bridge between the two, thus allowing the audiovisual-neck-haptic vector x’ to be uniquely related to P, arbitrarily chosen in the egocentric space. This implies that x” is the internal projection of a body-centered egocentric space, without any need to hypothesize an explicit coordinate transformation between craniocentric and bodycentric reference frames. The transformation is implicit in the simultaneous activation of the different components of x’: in a sense, it is an affordance [91 of the multi-sensory array, which still contains all the retinocentric and craniocentric information in an explicit form. However, picking up the egocentric affordance hidden in x’ is not an easy job at all. According to the assumption of causality, P uniquely determines x’: its coordinates are all measurable and are redundant, highly non-linear functions of P. This means that in the high-dimensional measurement space the consistent multi-modal sensory vectors are constrained to lay on a lower-dimensional (in fact, 3-dimensional) manifold 2: 2~ X CL%?. We propose that the egocentric space is represented by means of 2, i.e. a curved manifold in the multi-modal sensory space, and the representation is both implicit and distributed, thus avoiding the need of cells that fire proportionally to the coordinates of the space. In a sense, we are proposing a spatial representation of space.

In fact, this corresponds to a rather general alternative in the theory of neural networks, as regards the representation of (scalar/vectorial) variables: intensity representation vs spatial representation. The former one is a direct representation in which the time-variations of a variable are associated with similar time-variations of the activity level of a neuron or small group of neurons. The latter one encodes the time-variations with the motion in a field of neurons of a spatially limited cluster of acitivity, the poprdution code of the variable. Both types of codes are probably used by the brain: In early stages of sensory processing, for example, the intensity representation is more plausible, whereas the spatial coding is strongly suggested in large areas of the cerebral cortex by the well known somatotopic organization and the electrophysiological evidence of population coding. However, spatial representations are more expensive, in the sense that need many more neurons and are probably slower than intensity representations in carrying out computational tasks. Therefore, the question naturally arises: from which point of view is the price worth or, in other terms, which kind of evolution-

‘Tactile coordinates per se are spatially irrelevant because any point P could stimulate any point of the skin in the same way as the color of a target does not tell us anything about its position in space.

302

P. Morasso, V. Sanguineti / Neurocomputing 12 (19%) 289-310

ary pressure can justify the emergence of this mechanism? We wish to offer two lines of explanation at two rather different levels: a cognitive and a computational one. From the cognitive point of view, we can observe that the spatial representation of sensory-motor variables is in fact a map and cortical maps can be deemed to play a crucial role in task adaptation on the basis of the following conjecture: Spatial representations of sensory-motor variables are important for capturing the variability of purposive behavior, novel tasks or novel combinations of old skills because they are cognitively penetrable. From the computational point of view, we can say that the need for a spatial representation may come from the purely technical problem of stability of the representation. In fact, stable tonic discharge patterns are rather unusual in the brain, phasic behaviors being the dominant mode of operation; from this point of view, expressing the values of a variable by means of a spatially coded cluster of activity on a population of cells is certainly a much more reliable implementation than maintaining a stable firing rate. This is particularly true when the variable is itself related to a notion of space and fluctuations in the representation would be unacceptable for the stability of the planning process. In other cases, on the contrary, such fluctuations are intrinsically filtered out, as in muscle contraction, and thus are functionally irrelevant.

4. Simulation study In a simple simulation study, we considered a simplified visual acoustic paradigm, in a binocular/binaural situation which is sketched in Fig. 5. The ‘eyes’ and the ‘head’ are fixed, for simplicity, and the directional capabilities of the auditory systems, due to binaural time and/or intensity differences, are simply represented by using, for each ‘ear’, a pair of measuring points which pickup the distance/propagation delay form the stimulus P. Thus, the visual-acoustic vector x’ is 8-dimensional, with 4 retinal components and 4 cochlear components, the former ones expressed as angles and the latter ones as distances. The system was stimulated with points uniformly sampled from different supports: 1-D (a circumference, Fig. 61, 2-D (a circle, Fig. 71, 3-D (a cube, Fig. 8). Three different topology-preserving networks were trained with corresponding sets of data: a 20-neurons net in the 1-D case, a lO+neurons net in the 2-D case, and a lOOneurons net in the 3-D case. The dataset (400 points in the 1-D case, 1000 points in the 2-D case, and 2000 points in the 3-D case) was presented 20 times to the networks and the figures show the state of the network after 5, 10, 15, and 20 epochs, respectively 2. Each neuron in the networks learned a prototype vector Gf in the visuoacoustical 8-dimensional space and developed a set of lateral connections. For the purpose of visualization, each 8-D prototype was back-projected in 3-D space by

2Thellowing learning parameters, A = 30 -+ 0.01; age = 20 -9 200.

typical of the neural gas model, were used: q = 0.3 + 0.05;

P. Morasso, VI Sanguineti / Neurocomputing 12 (19%) 289310

Right ear

303

1

Fig. 5. Sketch of the setup in the simulation study. Coordinatess of the stimulus point: s,, s2, s,; Visual coordinates: x1, x2, x3, x4; Auditory coordinates: x5, x6, x7, xs.

estimating the point s” that minimizes the norm IIx7?? - ?;i” II; these points are plotted as small circles in the figures, linking by means of segments the circles corresponding to neurons which are neighbors in the map. The figures show that the visuo-acoustic TRN’s are in fact able to learn the topology of the underlying manifold and thus a feasible mechanism for representing the geometry of egocentric space is a TRN of multi-sensory neurons, possibly localized in the upper part of the PPC.

5. Distributed computation on the topographical .map We now briefly outline a possible computational mechanism which exploits the lateral connectivity of the multi-sensory TRN, thus behaving as a Cellular Automation (CIA) or, more precisely, as a coupled map lattices because space and time are discrete but the state variables vary in a continuous way. In other words, we can view the topology-preserving cortical map not merely as a mechanism for a ‘passive’ topographic representation of space but as dynamic. computational engine which operates on the population code 9, shifting it on the field as a soliton, according to local interaction rules. In particular, let us address the problem of how the cortical spatial map might plausibly solve the problem of generating the internal representation of a virtual trajectory, as a response to the presentation of a target Zr. This stimulus point, perceived through the different sensory channels,

304

P. Morasso, V Sanguineti / Neurocomputing 12 (1996) 289-310

Fig. 6. Visuo-acoustic map trained with a l-dimensional training set Top: training set (400 points) and geometrical arrangement of the visuo-acoustic sensory sytem. Bottom: State of the network after 5, 10, 15, and 20 epochs.

is reflected in the map as a cluster of activity P’,, centered around a neuron of receptive field center n’;. Moreover, we may think that the initial position of the end-effector is represented by a similar cluster 9,, centered around another neuron of receptive field center 3;. The problem, then, is to smoothly shift 9,, until it reaches 9, and this can be achieved by means of the attraction dynamics determined by a convergent force field, such as the gradient of a &unce field from the target 6 = 6(% ?r>, which measures the Euclidean distance of the end-effector from the target and operates as a Liapunov function for the map of neurons. This distance is only indirectly accessible, via the connectivity of the map, which is a topological or gestaltian knowledge. Therefore, we need a mechanism

P. hforasso, V Sanguineti/Neurocomputing 12 (19!%) 289-310

305

Fig. 7. Visuo-acoustic map trained with a 2-dimensional training set Top: training set (1000 points) and geometrical arrangement of the v&w-acoustic sensory system. Bottom: State of the network after 5, 10, 15, and 20 epochs.

for translating gestaltian knowledge, relevant at the population level, into local knowledge, exploitable at the neuron level. The natural solution is to establish an analogy between Euclidean distances in the external space and distances in the graph of neurons which represents that space. Given the target x’r (represented by its population code), the distance field can be erqxessed by the map interpolation Eq. 4, using a codebook of distance

306

P. Morasso, V. Sanguineti / Neurocomputing 12 (1996) 289-310

Fig. 8. Visuo-acoustic map trained with a 3-dimensional training set Top: training set (2000 points) and geometrical arrangement of the visuo*acoustic sensory system. Bottom: State of the network after $10, 15, and 20 epochs.

values: l-l’ = ($, i EX}, but how can we assign each neuron vj in the map such a distance estimate? This function can carried out in a parallel and distributed way, which only involves local computations, by exploiting the topographical connections of .the network, i.e. as a CA which carries out a diffusion process of local distance information. In particular, in a preliminary simulation we used a parallel version of the well known Dijkstra algorithm, which propagates to the nodes of a

P. Morasso, K Sanguineti / Neurocomputing 12 (1996) 289-310

307

Fig. 9. A: circular reaction strategy (motor command * external stimulus Q + multimodal vector x’c _H E R").B: Self-organization of a topology representing map. C: Diffusion of a distance field from the target T. D: Gradient-descent of the population code 9.

graph of neurons local estimates of distance. Let us call Sjk the local distance between two generic neurons of the graph, which is determined by some outside process. (In the simplest case, it can be chosen as a constant, thus yielding a

roughly isotropic field with approximately straight flow-lines; local anisotropies could allow other cortical processes to influence the shape of the flow-lines.) The diffusion process can be implemented by the following local interaction rule: $(t+

1) =min

(

7$(t),

tsjj(n,6(f)

+ 6jk)}

g(j) is the set of neurons which are neighbors of neuron ui and the initial state is characterized by the fact that T! is set to 0 and the distance values of all the others are set to a very large number 3. After diffusion, the shortest path from any neuron to the target, similarly to the Dijkstra algorithm, is simply found by jumping from neuron to neuron in order of decreasing ?;i” values. This kind of where

3Pmof. The dynamics has a point attractor because the rule can only decrease the distances. Moreover, since distances are positive and the variation steps are finite, then equilibrium is reached in a finite number of steps. At equilibrium, we have ?;i”< i;i + &Vk E S’(j) and this means that a local minimum (?;j”< ii:Vk E fi j)) is stable. The target neuron is a local minimum from the beginning and it remains the only one also during the diffusion process because every time the rule (5) triggers the decrease of a distance value ?;i”,the updated value must be larger than at least one of its neighbors, provided that the local distances are non negative.

308

P. Morasso, K Sanguineti /Neurocotnputing

12 (1996) 289-310

discrete solution is not immediately useful for trajectory formation, which requires a smooth, time-controlled path. However, it is the starting point for performing gradient descent on the continuous distance field, interpolated according to the equation 4 [21]. In summary, the proposed mechanism for the cortical representation of external space and trajectory formation is characterized as follows: 0 A suitable behavioral strategy, such as the circular reaction proposed by Piaget [23], guides the acquisition of correlated sensory-motor patterns coming from the workspace SV (Fig. 9(A)); l The learning equations build a lattice g of neurons which preserves the topology of the input manifold A, isomorphic with Y (Fig. 9(B)); l The activation of a target neuron triggers the diffusion of a distance field according to equation (5) (Fig. 9(C)); l The population code 9 of the current state vector starts moving in the direction of the target, tracking the flow-lines of the field.

6. Conclusions Finally, let us consider in which sense the architecture described above is consistent with neurobiological evidence. First of all, the apparent paradox that small lesions in the PPC do not cause space scotomata (whereas major ablations do cause spatial neglect) can be easily explained in our computational framework by interpreting small lesions as local destructions of small clusters of nodes in the map. After the local destruction, in fact, the topology of the remaining net is still the same and the interpolation function provided by the population code together with the CL4 dynamics for trajectory formation is still operant, although it might be less precise and somehow distorted. In other words, the topographic representation of space we are proposing is not a static picture, for which local scotoma cannot be filled, but an active mechanism where the topography is the result of topology-preserving connectivity and scotoma can be ‘filled’ precisely in the same way in which the discrete lattice is made to look like a continuous manifold, in the first place. Moreover, the same Hebbian learning which is instrumental for organizing the map, can be recalled into operation for fine tuning weights and links in the damaged network. The training mechanism has an intrinsic demagnification effect, allowing highresolution information to influence the self-organization of the topology-preserving maps in different parts of the workspace. Thus, the map can handle functional deprivation in a graceful way, given its redundancy. Learning - as a process of integration over time - and the law of large numbers do exploit the full redundancy in the multi-modal sensory array x’, thus obtaining the full available precision in the space-map correspondence, compatible with the physiological noise levels. In any specific usage of the map for motor planning, on the contrary, only a subset of the array may be available or relevant. In this case the input vector n’ will have a reduced number of dimensions and the remaining ones will be

P. Morasso, K Sanguineti/Neurocomputing 12 (1996) 289-310

309

treated as ‘don’t care’ input lines. This is also consistent with the behavior of non-seeing or non-hearing people who miss part of the array but supposedly not the cortical machinery which builds and maintains the cortical spatial map and indeed are able of normal reaching movements. Moreover, this distributed machinery which supports movement planning is obviously suitable for mental simulation and mental practice. Finally, a promising contribution of the proposed model is in the interpretation of experimental neuroanathomical data, particularly as regards the modern mapping techniques based on the retrograde tracing of fluorescent markers, injected by means of micro-syringes or micro-pipettes. In particular, if the technique were applied to co&o-cortical connections in the PPC, we predict that the resulting diffusion patterns of the markers should be characteristically determined by the underlying dimensional&y. The geomtry of TRN’s should be useful in order to interpret such neuroanathomical data and help to prove or disprove the existence of the proposed topology representing organization in the cortex.

References [l] G.E. Alexander, M.R. DeLong and M.D. Crutcher, Do cortical and basal ganglionic motor areas use motor programs to control movements?, Behavioraland Brain Science, 1.5(1992) 656-665. [2] H. Asamnna, l%e Motor Carter (Raven Press, New York, 1989). [3] E. Bisiach, E. Capitani and E. Porta, Two basic properties of space representations in the brain, J. ~eurok~gy, Neurosurgery and psychiatry48 (1985) 141-144. [4] J.S. Bridle, Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition, in: F. Fogelman Soulie and J. Herault, eds., Neurocomputing, vol. NATO ASI F-68 (Springer Verlag, Berlin, 1989) 227-236. [5] J.H. Conway and N.J.A. Sloane, Sphere Packings, Lattice and Groups (Springer-Verlag, New York, NY, 2 ed., 1993). [6] E.E. Fetz, Are movement parameters recognizably coded in the activity of single neurons? Behavioraland Brain Sciences 15 (1992) 679-690. [7] F. Frisone, Analisi della dimensionaliti intrinseca di una distribuzione di dati mediante reti neurali auto-organizzanti, Technical report, Master Doctoral Thesis in Electronic Engineering, University of Genova, Italy, 1995. IS] A.P. Georgopoulos, J.T. Lurito, M. Petrides, A.B. Schwartz and J.T. Massey, Mental rotation of the neuronal population vector, Science 243 (1989) 234-236. [9] J.J. Gibson, IfIre Perceprionof the Visual World (Houghton Mifflin, Boston, 1950). [lo] M.S. Graziano and C.G. Gross, Somatotopically organized maps of near $sual space exist, Behaubral and Brain Sciences 15 (1992) 750. [ll] KM. Heihnan, R.T. Watson and E. Valenstein, Neglect and related disorders, in KM. Heihnan and E. Valenstein, eds., ClinicalNeuropsychology(Oxford University Press, Oxford, UK, 1985). [12] J. Hyvarinen, 7%e Par&al Cortex of Monkey and Man (Springer-Verlag, Berlin, 1982). (131 M. Jeannerod, The representing brain: neural correlates of motor intention and imagery, Behaoioral and Brain Sciences 17 (1994) 187-201. [14I J.F. Kalaska, D.A.D. Cohen, M. Prud’homme and M.L. Hyde, Parietal area 5 neuronal activity encodes movement kinematics, not movement dynamics, Eqwimental Brain Research 80 (1990) 351-364. I151 T. Kohonen, Self organization and AssociativeMemory (Springer-Verlag, Berlin, 3 ed., 1989). [16] T. Martinetz and K. Schulten, A ‘Neural-Gas’ network learns topologies, in T. Kohonen, K.

310

P. Morasso, K Sanguineti /Neurocomputing

12 (19%) 289-310

Makisara, 0. Simula and J. Kangas, eds, Artificial Neural Networks @rnsterdam, 1991. North-Holland) 397-402. [17] T. Martinetz and K. Schulten, Topology representing networks, Neural Networks 7 (1994) 507-522. [18] P. Morasso, Spatial control of arm movements, fin. Brain. Res. 42 (1981) 223-227. [W P. Morass0 and V. Sanguineti, Neurocomputing concepts in motor control, in J. Paillard, ed., Brain and Space (University Press, oxford, UK, 1991) 409-432. LB1P. Morass0 and V. Sanguineti, Self-organizing body-schema for motor planning, J. Motor Behavior, 26 (1994) 131-148. ml P. Morasso and V. Sanguineti, Self-organizing topographic maps and motor planning, (In D. Cliff, P. Husbands, J.A. Meyer and S.W. Wilson, eds, From Anima2.s to Animats 3 (MIT Press, Cambridge, MA, 1994) 214-220. D21F.A. kussa-Ivaldi, Do neurons in the motor cortex encode movement direction? an alternative hypothesis, Neuroscience Letters 91 (1988) 106-l 11. [231J. Piaget, The Origin of Intelligencein Children (Norton, New York, 1963). [241G. Rizzolatti, M. Matelli and G. Pavesi, Deficits in attention and movement following the removal of postarcuate and prearc uate cortex in macaque monkeys, Brain 106 (1983) 655-673. tw V. Sanguineti, T. Tsuji and P. Morasso, A dynamical model for the generation of curved trajectories, in S. Gielen and B. Kappen, eds., Int. Cant on Artificial Neural Networks, London, (1993) (Springer-Verlag) 115-118. [W R.N. Shepard and L.A. Cooper, Mental Images and Their Transformations (MIT Press, Cambridge, MA, 1982). WI B.A. Stein and M.A. Meredith, The Merging of the Senses (MIT Press, Cambridge, MA, 1993). t281J.F. Stein, The representation of egocentric space in the posterior parietal cortex, Behavioral and Brain Sciences 1.5(1992) 691-700.

pietru G. Morass0 obtained an Electronic Engineering degre (with honors) from the University of Genova in 1968. After working as a post-doctoral fellow with Emilio Bizzi at M.I.T. (1970-72) in the field of motor control, he joined the Electrotechnic Institute and then the Department of Informatics, Systems, and Telecommunications at the University of Genova where he is currently Professor of Anthropomorphic Robotics. His research interests include: coordination of multi-joint, multi-muscle systems; robotic planning and control; robot navigation; self-organizing neural nets; analysis and recognition of cursive handwriting. He is member of several scientific societies in bioengineering, robotics, and neural networks. He is currently secretary of the ENNS.

VitturIo Sanguineti was born in Genoa, Italy in 1964. He got a degree in Electronic Enzineerinn in 1989. and a Ph.D. in Robotics in 1994, both at the University of Genoa. He is currently a post-doctoral fellow at the.Department of Informatics, Systems and Telecommunications of the University of Genoa, where he is working on issues related to motor control in humans and robots. He is currently investigating the computational role of cortical maps in performing sensorimotor transformations and control at muscle level. His research interests also include neural network models, the role of biomechanical complexity in theories of motor control and trajectory formation in handwriting and speech.