Distal place recognition based navigation control inspired by Hippocampus - Amygdala interaction Ansgar Koene∗

Gianluca Baldassarre∗∗ Francesco Mannella∗∗ Tony J. Prescott∗ ∗ Department of Psychology, University of Sheffield, Sheffield S10 2TP, UK ∗∗ Instituto di Scienze e Technologie della Cognizione, Consiglio Nazionale delle Ricerche I-00185 Roma, Italy Abstract We present a novel robot navigation system based on distal place and value recognition. The navigation control system is inspired by the hippocampus - amygdala circuit that is involved in place learning/recognition and stimulus value association.

1.

Introduction

A computational model of the hippocampus - amygdala circuit was developed focusing on the ability to recognize not only the current location where the robot is but also surrounding locations that are currently visible to the robot. This distal location recognition relies on the property that, in the absence of occlusions, place defining stimulus configurations change in a gradual manner as the robot moves from one location to another. The difference between the current sensory inputs and the templates associated with known locations therefore increases gradually as a function of distance to the perceived locations. Distal recognition of value associated places allows our system to navigate towards goals without exploration of the intermediate space. Further more, navigation behavior naturally becomes contingent upon the stimulus state of the target location (stimulus configuration changes when target light is ON or OFF) providing our controller with added flexibility for dealing with state changes in the environment. This model was integrated into a robot control system that was previously published in (1) without the new hippocampus and amygdala implementations.

nition cells where the magnitude of activation indicates the recognition confidence. Winner-takes-all competition between the distal recognition cell outputs yields the current location estimate in the place cells (3). Place cells and distal recognition cells are associated with spatial locations by their connectivity to output grid cells (4) that use a population code representation of spatial coordinates. By subtracting the current location estimate from the distal recognition cell associated location the output encodes the required displacement for reaching distally recognized location. In order to do this with a single set of grid cells however it is necessary to process the distal recognition cells sequentially.

2.

3.

Distal place recognition & localization

Figure 1 shows a diagram of the hippocampus model for place recognition. The perceived stimulus configuration forms the sensory input that is matched to heading direction specific templates (i.e. view cells (2)). The activity of all view cells associated with a particular location is summed in the distal recog-

Figure 1: Hippocampus network. Solid: excitatory connections; Dashed lines: inhibitory connections

Place value association

The amygdala provides association of values with basic sensory stimuli (e.g. target lights) or stimulus configurations encoded via hippocampal distal recognition cells. Whenever an innately rewarding/punishing input is received any stimulus (configuration) that predicted the reward becomes associated with the rewarding input. In order to trace

tag strength

1 0.5

6

0 P1 arrival

P1−>P2 time

Figure 2: Solid line: tag related to place P1; Dotted line: tag related to place P2

4

2

0

which stimulus configuration (i.e. place) predicted reward delivery we tag place cell activations with a signal that gradually increases to a saturation level as long as the place cell is active and gradually decreases when place cell activation is removed (see figure 2). For basic stimuli tagging is triggered when the perceived stimulus strongly changes (e.g. light goes on or off). For further details, see (5).

4.

Experiment

To test our rat brain inspired navigation system we used a differentially rewarded plus-maze task (1). First the robot explored the maze guided simply by attraction to un-mapped visible locations. When the first reward location was encountered the corresponding place and sensory stimulus (target light) were associated with the reward value. Subsequently, exploration behavior was overruled by target light approach behavior whenever the robot was able to see target lights. Once all maze arms were visited the robot recognized the valued locations and visited them in order of learned reward magnitude. The robot produced a sparse map of the environment with a majority of place cells in maze corner areas where small movements dramatically changed the visual inputs. Analysis of the visual pattern templates (view cells) revealed that the values were successfully associated with stimulus configurations where the target object light is on even though each maze arm end is also mapped to a visual configuration where the light is off. The robot successfully recognized not only its current location in the maze but also produced gradually reducing recognitions for distal locations in the current field of view (figure 3).

5.

Conclusion

Distal recognition of value associated places produces flexible navigate without requiring full exploration of the movement space. The resulting navigation behavior is intrinsically contingent upon the stimulus state of the target location enabling the controller to cope with state changes in the environment. Based on the combination of distal place recognition and value association the hippocampusamygdala network successfully guided the robot to-

−2

−4

−6 −6

−4

−2

0

2

4

6

Figure 3: Solid blue line: estimated movement by robot; Dashed red line: actual movement; Blue disks: place cell locations; Green disks: activity of distal recognition cells (bigger = more active); Magenta disks: values associated with place cells (bigger = higher value)

wards visible locations that were learned to be most rewarding.

Acknowledgements This work was supported by the European Union Framework 6 IST project 027819 (ICEA project: www.iceaproject.eu).

References Koene A., Prescott T.J.: Hippocampus, Amygdala and Basal Ganglia based navigation control. In: ICANN 2009 proceedings. LNCS, in press. Springer, Heidelberg (2009) Rolls, E.R., Stringer, S.M.: Spatial view cells in the hippocampus, and their idiothetic update based on place and head direction. Neural Networks 18(9), 1229-1241 (2005) OKeefe, J., Dostrovsky, J.: The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely moving rat. Brain Research 34, 171-175 (1971) Fyhn, M., Molden, S., Witter, M.P., Moser, E.I., Moser, M.B.: Spatial representation in the entorhinal corex. Science 305, 1258-1264 (2004) Mannella F., Koene A., Baldassarre G.: Navigation via Pavlovian Conditioning: A Robotic BioConstrained Model of Autoshaping in Rats. Proceedings of the Ninth International Conference on Epigenetic Robotics (2009)