Home

Search

Collections

Journals

About

Contact us

My IOPscience

Autonomous spacecraft landing through human pre-attentive vision

This article has been downloaded from IOPscience. Please scroll down to see the full text article. 2012 Bioinspir. Biomim. 7 025007 (http://iopscience.iop.org/1748-3190/7/2/025007) View the table of contents for this issue, or go to the journal homepage for more

Download details: IP Address: 217.77.157.57 The article was downloaded on 25/05/2012 at 08:34

Please note that terms and conditions apply.

IOP PUBLISHING

BIOINSPIRATION & BIOMIMETICS

doi:10.1088/1748-3182/7/2/025007

Bioinspir. Biomim. 7 (2012) 025007 (11pp)

Autonomous spacecraft landing through human pre-attentive vision Giuseppina Schiavone, Dario Izzo, Lu´ıs F Sim˜oes and Guido C H E de Croon Advanced Concepts Team, European Space Agency, Noordwijk, the Netherlands E-mail: [email protected]

Received 18 July 2011 Accepted for publication 11 January 2012 Published 22 May 2012 Online at stacks.iop.org/BB/7/025007 Abstract In this work, we exploit a computational model of human pre-attentive vision to guide the descent of a spacecraft on extraterrestrial bodies. Providing the spacecraft with high degrees of autonomy is a challenge for future space missions. Up to present, major effort in this research field has been concentrated in hazard avoidance algorithms and landmark detection, often by reference to a priori maps, ranked by scientists according to specific scientific criteria. Here, we present a bio-inspired approach based on the human ability to quickly select intrinsically salient targets in the visual scene; this ability is fundamental for fast decision-making processes in unpredictable and unknown circumstances. The proposed system integrates a simple model of the spacecraft and optimality principles which guarantee minimum fuel consumption during the landing procedure; detected salient sites are used for retargeting the spacecraft trajectory, under safety and reachability conditions. We compare the decisions taken by the proposed algorithm with that of a number of human subjects tested under the same conditions. Our results show how the developed algorithm is indistinguishable from the human subjects with respect to areas, occurrence and timing of the retargeting. (Some figures may appear in colour only in the online journal)

1. Introduction: overview of unmanned spacecraft landing

the selected landing area. On-line remote control by human operators is an improbable and risky solution because of limited and unstable communication bandwidth and significant delays. On-board reliable hazard and avoidance capabilities, able to reduce risks and constraints imposed by pre-planned landing manoeuvres, are a necessary challenge for the next generation of autonomous unmanned spacecrafts. Up to now, considerable effort has been focused on the development of autonomous systems able to analyse the planetary surface, generate safety maps and identify acceptable landing sites. Autonomous safe site selection is mainly based on landmark detection algorithms, which exploit measurements from single [6] or multiple [11] on-board sensors (i.e. camera, LIDAR or RADAR) for characterizing terrain features [5, 4], such as slope, shadow, roughness and the presence of craters and boulders. In [4] and [23], the authors use a fuzzy logic framework to solve the landing site classification. In the first case, a terrain

The landing phase of a planetary mission is the most complex and risky procedure to design. At the same time, it represents the first step for successful exploration of a celestial body. Safety, reachability and scientific return are the main issues to face during mission design. The identification of safe sites for landing, possibly close to scientifically interesting locations, is, so far, based on offline procedures: scientists examine and select aerial images gathered by previous orbiting probes and design sequences of automatic operations for the spacecraft, accordingly with fuel availability. Once the landing site is selected, the spacecraft performs a ‘blind’ landing operation with little provision of corrective manoeuvres prior to touchdown [23]. Dangerous circumstances leading to mission failure can still occur, due to the relative inaccuracy of the landing path followed by the spacecraft, and the relative uncertainty on the safeness of 1748-3182/12/025007+11$33.00

1

© 2012 IOP Publishing Ltd

Printed in the UK & the USA

Bioinspir. Biomim. 7 (2012) 025007

G Schiavone et al

2. Model of human pre-attentive vision for saliency detection

hazard map is obtained as a result of heterogeneous sensor information fusion; in the latter, safeness and distance of the proposed site from a nominal target are used as input variables for the fuzzy interference system. In [25] and [2], a fuzzy logic approach is taken for the processing of the multiple values characterizing landing site safety and navigational constraints, but the identification of the best landing sites is tackled through a meta-heuristics approach, so as to meet the real-time processing requirements with available on-board hardware. In [24], a probabilistic model (Bayesian Networks), which combines engineering and science factors with terrain safety issues, is proposed for landing site selection. In all these works, methods for detecting interesting sites take into account some form of a priori knowledge of the landing region, such as maps or absolute coordinates ranked by scientists according to specific scientific criteria. Here, we introduce a novel approach for selecting landing sites, which does not rely on a priori knowledge of the planetary surface and is specifically oriented to space exploration of unknown environments. In these scenarios, higher selectivity in detecting scientifically appealing locations through matching of a priori maps can be a limiting procedure that prevents the discovery of surprising and unpredictable scientific findings. We adopt a bio-inspired approach, introducing in the control loop of the landing process a computational model of human pre-attentive vision [14], which provides the spacecraft with fast saliency detection abilities. To the best of our knowledge, this is the first time that a saliency-based visual attention model is applied to the successful landing problem on outer space planetary surfaces (in [20], a neuromorphic vision circuit has been proposed as the spacecraft’s motion sensing unit). During the descent, saliency maps are combined with safety maps and reachability estimates to guarantee that the final landing point is interesting, distant enough from dangerous sites and reachable with the fuel available onboard. Because continuous retargeting requires autonomous replanning of the guidance trajectory, we use optimality principles for the feedback control, considering a simplified spacecraft model and a coarse numerical grid and maximizing the mass of the spacecraft at the touchdown point. This approach has also a bio-inspired foundation [21] and agrees with recent scientific findings which state that sensori-motor control strategies of biological systems, including humans, are, indeed, governed by optimality principles [26, 1]. The reminder of the paper is organized as follows. In section 2, we present the saliency-based model of pre-attentive vision. In section 3, we describe the physical model of the spacecraft and the feedback control law for the replanning of the trajectory. We show the complete control architecture in section 4, and the landing simulation results in section 5. In section 6, we present the experimental platform developed for gathering human judgements about saliency and safety issues while landing on simulated planetary surfaces. Here, we also discuss results from the comparison between our model and the humans’ performance. In section 7, we draw conclusions and future directions of our research.

In humans and animals, saliency detection is an automatic process involved in selective visual attention that allows them to quickly shift the gaze towards specific attractive targets in the visual environment. This ability has evolutionary explanation: it enables organisms to rapidly detect their prey, escape from predators and recognize mates. Two different dynamical forms of attention can be identified [17]: a preattentive form and a more complex form of attention. In the first form, also referred to as ‘bottom-up’, simple features (such as intensity, colour opponency and orientation) are processed rapidly and in parallel over the entire visual field and intrinsic saliency of objects is automatically detected. The second form, referred to as ‘top-down’, is tuned by voluntary control; it is slower, conjugate features are processed serially and the saliency of objects is dependent on specific task (or interest). In this work, we focused on the ‘bottom-up’, saliencybased, attentional mechanism. Several algorithms for saliency detection are available within the computer vision literature. The identification of fixation points that a human viewer would focus within an image is, indeed, a fundamental issue for implementing more complex tasks, such as object recognition, segmentation and image compression. Spectral residual [10] and phase spectrum of quaternion [8] approaches, based on suppression of frequently occurring features and retention of features that deviate from the norm, have been shown to be fast and reliable although relatively sensitive to local saliency. In [18], salient object detection is considered as a binary labelling problem that separates a salient object from the background, and supervised learning through the conditional random field is used to combine competitive features. A context-aware saliency approach has recently been proposed in [7], where local and global features are considered together with Gestalt psychology principles, stating that areas that are close to the foci of attention should be explored significantly more than faraway regions; thus, visual forms may possess one or several centres of gravity about which the form is organized. More biologically inspired algorithms are presented in [3] where saliency is determined by quantifying the self-information of local image patches and in [9] where graph theory has been used to form activation maps from raw features and to concentrate weights on activation maps. This latter method results in a naturally parallelizable algorithm, which is reliable at predicting human fixations. In our study, we use the computational model proposed first by Koch and Ullman (1985) [17], and later extended by Itti et al (1998) [14, 13]. It is based on neuro-physiological findings; it has been proven to qualitatively reproduce human performances [14] and it is considered a benchmark for saliency detection algorithms. The pre-attentive vision algorithm can be synthesized in the following steps: (i) the input image is filtered using dyadic Gaussian pyramids, creating a set of different spatial scales; (ii) early visual features (intensity, colour opponency and orientation) are computed by linear ‘centre-surround’ 2

Bioinspir. Biomim. 7 (2012) 025007

(iii)

(iv)

(v)

(vi)

G Schiavone et al

operations across the scales (difference between fine and coarse scales), mimicking the centre-surround contrast activation in visual receptive fields [13]; the activity from all feature maps is combined across scale into three ‘conspicuity’ maps for intensity colour and orientation [14]; the sum of the three conspicuity maps gives rise to a topographic saliency map which codes for how different and how salient a particular stimulus is relative to its neighbourhood [14]; the most salient location is selected by modelling the saliency map as a ‘winner-take-all’ neural network, where neurons compete with each other and synaptic interactions among units ensure that only the most active location remains, while all other locations are suppressed; since more salient points can be identified within an image, after the most active neuron is selected an inhibition mechanism (‘inhibition of return’) allows us to shift to the next most salient location [14].

gravity and Isp g0 represents the effective exhaust velocity; and the following optimal control problem (OCP): find: to maximize: subject to:

x˙ = f(x, u) x(t f ) = x f , y(t f ) = y f , z(t f ) = 0 vx (t f ) = 0, vy (t f ) = 0, vz (t f ) = 0 x(t0 ) = x0 ,

3. Optimality principles for trajectory replanning During the descent, whenever a new landing site is chosen, the spacecraft needs to replan its descent accordingly. Ideally one would want this process to occur continuously, controlling the thrust vector, u = [ux , uy , uz ], at a high frequency as to be able to react quickly to unmodelled dynamics or changes of higher level goals. In order to meet these requirements, we calculate an optimal state feedback u∗ (x) using a simplified discrete model of the lander dynamics and controls; subsequently, we relate it to the “real” state feedback u˜ ∗ (˜x ) used in the full spacecraft dynamic and control. In other words, the lander plans its activities relying on a simplified model of itself and of its environment to then map its decisions to the real world. Consider the lander dynamics, x˙ = f(x, u), defined as

m˙ = −

(2)

where U is the space of the admissible controls, J(x(t f )) is the cost function, which in our simulation is the mass of the spacecraft at touchdown at time t f , x(t f ), y(t f ), z(t f ), vx (t f ), vy (t f ), vz (t f ) are the final conditions on the position and velocity of the spacecraft and x0 = [x, y, z, vx , vy , vz , m] contains the position, velocity and mass values of the spacecraft at the current time instant t0 . We transform this OCP into a nonlinear programming (NLP) problem using the impulsive transcription developed for a similar landing problem in [15]. By setting the number of segments in the numerical transcription to the minimal n = 3,1 we obtain an m = 32 dimensional NLP problem on the variables [t f , xi , u j ], where i = 1, . . . , N (N = 5 collocation nodes for the state) and j = 1, . . . , M, (M = 3 collocation nodes for the impulsive control). We define ceq = 28, equality constraints, and c = n, inequality constraints (refer to [15] for more details of these parameters). Such a NLP problem can be efficiently solved applying a sequential quadratic programming method. To give an estimate of the method efficiency, we report the average computational time for solving the NLP problem, imposing maximum final mass high gate to touchdown descent as the optimal condition, being 30 ms (we use Intel Xeon processor at 2.3 GHz). A real onboard implementation would result in a different performance depending on the hardware and implementation used; for this reason, in our simulation, we impose a pessimistic time of 500 ms for the completion of the algorithm. The NLP solution returns the optimal thrust u∗ (x0 ) = 1 1 1 [ux , uy , uz ] and we set it to be the real state feedback, u˜ ∗ (x0 ) = u∗ (x0 ), where the value x0 is given by the navigation algorithms on-board the lander. The proposed numerical scheme is able to provide a realtime feedback accounting for optimality principles that can actually be used as a feedback to much more complicated dynamics than those expressed in (1). In this latter case, the discrepancy between reality and the model used by the lander to find a state feedback would result in a suboptimal trajectory with a minimal penalty on the consumed propellant [15]. In figure 1, the simplified plan estimated via the NLP at the beginning of the trajectory is compared with the actual descent profile of the spacecraft. In this example, the trajectory is recalculated every second, no final landing site is preassigned and the descent is driven only by optimality criteria that maximize the mass of the spacecraft at the instant of touchdown.

Although compared to the state of the art this model is computationally expensive, it represents a neurally plausible model and because of its parallel neuro-biological architecture is suitable for implementation on VLSI systems [12]. Neuromorphic devices [16] arouse, indeed, great interest in the autonomous spacecraft design, offering small, cheap and robust technology able to reproduce neural systems and biological processes with real-time capabilities [20].

v˙ x = ux /m v˙ y = uy /m v˙ z = uz /m − g pl x˙ = vx y˙ = vy z˙ = vz √

u ∈ U, t f J(x(t f ))

(1)

(u2x +u2y +u2z ) , Isp g0

where x, y, z denote the spacecraft position, vx , vy , vz denote its velocity, m is its mass, ux , uy , uz are the components of the thrust vector, gpl is the gravity of the planet or celestial body where the spacecraft should land, Isp is the engine specific impulse expressed in seconds, g0 = 9.8065 m s−2 is the Earth’s

1 n = 3 defines a three-impulse trajectory; n = 2 would correspond to a ballistic descent model, such as that described in [23].

3

z [m]

Bioinspir. Biomim. 7 (2012) 025007

G Schiavone et al

2000

2000

1500

1500

z [m] 1000

1000

500

500

0 0

500

1000

-50

1500

2000

0

50

0 0

2500

500

1000

-50

1500

2000

0

2500 y [m]

y [m]

(a)

50

x [m]

(b)

x [m]

Figure 1. Comparison between the simplified plan designed at the beginning of the trajectory (a) and the actual descent profile (b). (a) Optimal simplified plan at the beginning of the simulation: since the first second of the simulation, the spacecraft is able to build an internal model of the reality by efficiently designing the entire trajectory (blue line) and estimating the thrust vectors (black arrows). (b) Actual descent profile: external representation of the entire trajectory after 54 steps, close to the touchdown instant.

Figure 2. Schematic of the control architecture.

4. Complete system architecture

variety of digital elevation maps by setting parameters such as fractal number, shadows, densities and characteristics of craters and boulders and allows us to gather images similar to that acquired by the camera on-board the spacecraft (we consider a camera field of view of 60 ◦ ). At each step of the simulation, the acquired image, of dimension 512 × 512 pixels, is processed through the visual attention module and the variance detector module. The visual attention module uses the functions of the saliency toolbox, developed by Dirk B Walther (available at http://www.saliencytoolbox.net/), for building the saliency map. Within the saliency map, we select the first five detectable salient points (see the red dots in figure 3(c)). The variance detector module estimates the homogeneity of the observed surface. The variance of the image, V (P), is calculated with the following equation, considering a squared

The integrated landing control system has been implemented in the Matlab/Simulink environment and it is composed of the following elements: • • • • • •

a sensing device (camera), a visual attention module, a detector of the surface’s variance, a module for estimating the reachable zone, a decision-making unit, a module integrating spacecraft dynamics and control.

A schematic of the control architecture is shown in figure 2. The visual environment for the simulation is generated with PANGU 3.10 [22], which allows us to create a wide 4

Bioinspir. Biomim. 7 (2012) 025007

G Schiavone et al

(a)

(b)

(c)

-1500

-1500

-1500

-1000

-1000

-1000

-500

-500

-500

0

0

0

500

500

500

1000

1000

1000

-1000

0

-1000

1000

0

1000

-1000

0

1000

Figure 3. (a) Image generated by PANGU at a determined position of the spacecraft; (b) safety map; (c) saliency map. The red dots represent the first five salient points detected within the image; the blue ellipse circumscribes the reachable terrain and the blue dot indicates the final point according to the ballistic model; the red diamond represents the selected landing point; and the green cross, which coincides with the red diamond, represents the final landing point estimated by the optimal control law. The axes express distances in metres.

3000

2500

2000

1500

1000 -1000 500 0 0 -1500

1000 -1000

-500

0

500

1000

Figure 4. Example of a trajectory profile during the spacecraft descent: blue stars, discrete positions of the spacecraft in a three-dimensional space; yellow squares, points of retargeting; green crosses, the final landing points estimated by the optimal control law at each retargeting; red ellipses, landing footprints estimated at each step of the trajectory. The axes express distances in metres.

window of size 11 × 11 pixels centred in each pixel, P, of the image: N (pi − μ)2 , (3) V (P) = i=1 N−1 where N is the number of pixels in the window, pi is the ith-pixel within the window and μ is the mean of the pixels inside the window.

In order to avoid craters and regions in shadow, we add to the variance map a filtered version of the original image in which we cut off darker areas, considered possible dangerous sites. The final result produces the safety map, see figure 3(b) where white stands for safe and black for danger. The reachable terrain is estimated using the ballistic model described in [23], according to which the landing footprint is 5

Bioinspir. Biomim. 7 (2012) 025007

G Schiavone et al

to error, we shrink the landing ellipse by 20%, as suggested in [24]. As the spacecraft approaches the surface, the reachable area shrinks (see figure 4) and collapses in the final point at the instant of touchdown. The salient points, safety map and reachable footprint are inputs of the decision-making module which is responsible for selecting the best retargeting point, so that it is (i) as close as possible to salient sites, (ii) distant enough from dangerous sites and (iii) reachable with the fuel available on-board. Hereafter, the reasoning adopted by the decision process is presented. We distinguish two reasoning processes, one for the beginning of the trajectory and another for trajectory replanning in successive time steps.

100

90

80

70

T

60

50

40

(1) Since in the majority of the cases salient points correspond to regions where the surface is highly heterogeneous, we consider as the best site for landing the one which is close to as many salient points as possible. First, we exclude the salient points outside the reachable area, and then we calculate the point, inside the reachable area, which has the minimum average distance from all the other salient points. (2) For the first step of the trajectory, we check if the selected landing point is also safe, by referring to the safety map. If this condition holds, the selected point will be considered for retargeting, and if not, the next point with the minimum average distance is selected and this procedure is iterated until the condition is verified. (3) For successive replanning, item (1) is repeated; again, if the selected landing site is also safe, it will be considered

30

20

10

20

30

40

50

60

70

80

90

steps

Figure 5. Profile of the synthetic temperature over a trajectory discretized in 90 steps. T (i + 1) = T (i)/0.975, where i = 1, . . . ,90, and T (1) = 10.

bounded by an ellipse, the axes of which depend on the mass of the spacecraft, the horizontal velocity, the energy, the time to impact and the allowable change in velocity based on fuel allocation. Since the estimate of the landing footprint is subject (a)

(b)

−1500 3000

(32) (1)

−1000

(84)

2500

2000

−500

1500 0 1000

500

500

0 1000

(1)

(84) (32)

1500

1000

1000

500

500

0

0

−500 1500

−1000 −1500

−1000

−500

0

500

1000

−1500

1500

−500 −1000

Figure 6. Successful landing simulation. (a) Top view of surface at the initial point of the trajectory: the red ellipse delimits the reachable area; the yellow squares indicate the points of retargeting during the whole descent and the numbers refer to the specific steps in which the instructions for retargeting have been imposed. (b) Three-dimensional view of the landing path: the red ellipses represent the reachable terrain that shrinks around the touchdown point; the blue stars represent the positions of the spacecraft at each step of the trajectory and the blue dots their projections on the planar surface. The axes express distances in metres. 6

Bioinspir. Biomim. 7 (2012) 025007

G Schiavone et al

(a) surface 1

(b) surface 2

(c) surface 3

(d) surface 4

(e) surface 5

(f) surface 6

(g) surface 7

Figure 7. Seven different landing scenarios: the blue ellipse indicates the reachable terrain; the different markers refer to retargeting points selected by each subject; the yellow square marker corresponds to the algorithm.

as a retargeting point, and if not, no new re-targeting point is selected and the controller updates the trajectory in order to keep descending towards the site selected at the previous step.

Distance from the centroid [m]

1500

The distinction between the first step and the successive one ensures that the initial planning of the trajectory always pinpoints to a specific site chosen according to the three criteria (saliency, safety and reachability). High retargeting occurrence can result in higher fuel consumption for manoeuvre replanning. To face this problem, we introduce in the reasoning process (3) a simulated annealing mechanism, inspired by the Metropolis algorithm [19], according to which the chance that the selected point will be used for retargeting follows the criteria

1000

500

0

0

1

2

3

4 Surfaces

5

6

7

e−d/T > R(0, 1),

8

(4)

where d represents the Euclidean distance between the selected point and the current final point of the trajectory, T is a synthetic temperature and R(0, 1) is a random number in the interval [0, 1]. We impose the temperature to increase at each

Figure 8. Mean distance from the centroid to the final points of retargeting: red shows the error bars; the blue circles indicate the distance to the final target selected by the algorithm. 7

G Schiavone et al 90

90

80

80

80

70

70

70

60

60

60

50 40

Steps

90

Steps

Steps

Bioinspir. Biomim. 7 (2012) 025007

50 40

40

30

30

30

20

20

20

10

10

10

5

10

15

20

25

5

10

Subjects

15

20

25

5

(b) surface 2 80

70

70

70

60

60

60

Steps

90

80

Steps

90

40

50 40

30

20

20

20

10

10

10

15

20

25

5

15

20

25

Subjects (e) surface 5

90

90

80

80

70

70

60

60

50

50

Steps

Steps

10

40

30

20

20

10

10 0 10

15

20

25

Subjects (g) surface 7

5

10

15

25

Subjects (f ) surface 6

40

30

5

20

40

30

Subjects (d) surface 4

25

50

30

10

20

(c) surface 3

80

50

15

Subjects

90

5

10

Subjects

(a) surface 1

Steps

50

0

20

40

60

80 100 120 140 160 180 200

Subjects (h) Histogram

Figure 9. Instants of retargeting. (a)–(g) Each plot refers to a different surface and indicates at which step of the trajectory each subject and the algorithm (subject 26) decided to retarget. (h) Histogram showing the timing of retargeting of all the subjects (in this case, the algorithm is excluded): at step 1, all the subjects are instructed to select a point for landing; the successive retargetings occur at any time each subject decides to change landing site; the histogram shows that these retargetings mainly occur towards the end of the descent.

• initial position: x = 50 m; y = −200 m; z = 3000 m, • initial velocity: vx = 10 m s−1 ; vy = 0 m s−1 ; vz = −50 m s−1 , • initial mass: m = 7472.06 kg, • gravity of the celestial body: Moon’s gravity gpl = 1.623 m s−2 , • engine specific impulse: Isp = 311 s, • maximum thrust: maxthrust = 45760 N, • simulation steps: 90, • period for trajectory replanning: 500 ms (see section 3).

step of the trajectory, as in figure 5, thus raising the probability of retargeting mostly towards the end of the descent where fine manoeuvres need to be performed. Finally, the output of the decision-making unit, corresponding to the desired touchdown point, is used to close the loop of the control. The module integrating spacecraft dynamics and control is, indeed, responsible for recalculating the trajectory and the next position of the spacecraft in order to reach the desired site with minimum amount of fuel consumption.

In figure 6, we show the entire trajectory profile. Three retargetings occurred during the descent: the first one at the beginning, the second at almost the middle of the landing path and the third towards the end. At the last step of the simulation,

5. Landing simulation Here, we present the results of a simulation. We impose the following initial settings: 8

Bioinspir. Biomim. 7 (2012) 025007

(a)

G Schiavone et al

7

Mean occurrence

6 5 4 3 2 1 0 0

(b)

5

10

15

20

25

Subjects 7

Mean occurrence

6 5 4 3 2 1 0 0

1

2

3

4

5

6

7

8

Surfaces Figure 10. Mean frequency of retargeting. (a) Across subjects, the algorithm is identified as subject 26; (b) across surfaces.

indicated by the elliptic landing footprint, according to what they would judge as salient and safe area to explore. After this selection, the spacecraft would modify its trajectory to reach the desired point, using the optimal control strategy presented in section 3. Throughout the descent, successive images gathered from the surface simulator would appear as a zoomed and slightly shifted version of the previous one, thus allowing the subjects to see the terrain with more details. During the descent, the user was free to change its target at any time, by selecting other salient and safe landing sites. We ran our algorithm using the same planetary surfaces. Both the algorithm and human subjects exploit the control module for the spacecraft; this allowed us to compare their capability of selecting saliency and safety leaving to the controller the optimization of mass consumption. Figure 7 shows the seven simulated surfaces, all at the starting point of the descent. Inside the reachable area (blue ellipse), we show the retargeting points selected by each subject with markers of distinct colours and shapes. Among them, we include also the targets selected by the algorithm. We challenge the reader to identify which markers correspond to the algorithm’s choice. At a first sight, we can observe that the algorithm’s selections are indistinguishable from the human choices, given the high variability among the users. Despite this variability, it is possible to note that there are well-defined areas within the surface where the subjects decided to land. We performed a more detailed analysis to determine if the algorithm took the spacecraft into the same areas. For this analysis, we consider only the final landing point chosen by each subject. For each surface, we clustered all the

the spacecraft is at about 350 m from the surface, with a mass of 7257.7 kg; the controller on-board was estimating a time to touchdown of 13 s with a final mass of 7085.5 kg, not dissimilar from the plan at the beginning of the descent where a final mass of 6993.5 kg was estimated. This result shows the robustness of our controller.

6. Comparison with human subjects In the previous section, we showed how the developed integrated system was able to control the spacecraft descent according to saliency, safety and reachability criteria, without any a priori knowledge of the planetary surface. In order to evaluate if the sites selected by the algorithm for landing were compatible with human choice, we developed an interface for gathering human decisions on the same set of landing scenarios. Seven planetary surfaces were generated with PANGU, differing in the density and dimension of craters and boulders, the presence of dunes and roughness of the terrain. Since the aim of the study does not address specific scientific criteria, the subjects within the test group were selected randomly, among different educational profiles (i.e. biologists, system engineers, economists, computer scientists, etc) inside the European Space Research and Technology Centre. A total of 25 subjects participated in the experiment. The interface presented to the subjects showed images gathered from PANGU at each instant of the trajectory. The same settings listed in section 4 were imposed. The subjects were instructed to select an initial point for landing on the planetary surface within a reachable area 9

Bioinspir. Biomim. 7 (2012) 025007

G Schiavone et al

solutions in one group and we estimated the centroid location. We measured the mean distance to all the points from the centroid and the standard deviation. As shown in figure 8, the final target selected by the algorithm is very close to the centroid for surfaces 1 and 6; it is within the standard deviation for surfaces 3, 4 and 7; it is very far from the centroid for surfaces 2, 5, where it seems that there are two preferred zones for landing. In the latter case, the centroid is not representative of the human subjects’ preferences (figure 7);darker a grouping into two clusters would have been more appropriate. We could speculate that if more clusters were considered and a higher number of subjects participated in the test, we would not be able to distinguish between the human subjects’ and the algorithm’s selections. Moreover, because of the configuration of the experiment, the players were not limited in time for taking a decision and this introduced a bias due to high-level decision-making processes, not modelled in the saliency-based algorithm. Other findings emerge from the analysis of the data with respect to occurrence and timing of retargeting. First of all, we found that the frequency of retargeting does not depend in average on the specific surface but is subject dependent (see figure 10). Secondly, although compared to humans the algorithm retargets a higher number of times, on average, it is interesting to note that the instants of retargeting are mostly concentrated at the second half of the landing path, both for humans and the algorithm. This result is obtained, in our model, thanks to the simulated annealing mechanism, introduced in section 4. We observed that also humans prefer to adjust the spacecraft trajectory more towards the end of the descent than at the beginning (this is clearly shown in the histogram in figure 9). More details in the images gathered during the end of the descent allow us to detect better landing sites, reachable with fine manoeuvres.

images from planetary surfaces seem much less complex than landscapes on Earth. Finally, we note that, though the described framework has been proposed for autonomous landing, it could also be readapted for application to navigation and exploration in unknown environments.

References [1] Arechavaleta G, Laumond J P, Hicheur H and Berthoz A 2008 An optimality principle governing human walking IEEE Trans. Robot. 24 5–14 [2] Bourdarias C, Da-Cunha P, Drai R, Sim˜oes L F and Ribeiro R A 2010 Optimized and flexible multi-criteria decision making for hazard avoidance 33rd Annual AAS Rocky Mountain Guidance and Control Conf. (Breckenridge, CO, February 2010) (American Astronautical Society) [3] Bruce N and Tsotsos J 2006 Saliency based on information maximization Adv. Neural Inform. Process. Syst. 18 155 [4] Cheng Y and Ansar A 2005 Landmark based position estimation for pinpoint landing on mars Proc. 2005 IEEE Int. Conf. on Robotics and Automation, 2005: ICRA 2005 (IEEE) pp 4470–5 [5] Cheng Y, Johnson A E, Matthies L H and Olson C F 2003 Optical landmark detection for spacecraft navigation Adv. Astronaut. Sci. 114 1785–803 [6] Da Costa A, Davighi A, Bernardi S and Finzi A E 2005 Hazard avoidance during planetary landing by on-line neural network images analysis 28th Annual AAS Rocky Mountain Guidance and Control Conf. (Breckenridge, CO, February 2005) (American Astronautical Society) [7] Goferman S, Zelnik-Manor L and Tal A 2010 Context-aware saliency detection 2010 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) (IEEE) pp 2376–83 [8] Guo C, Ma Q and Zhang L 2008 Spatio-temporal saliency detection using phase spectrum of quaternion Fourier transform IEEE Conf. on Computer Vision and Pattern Recognition, 2008: CVPR 2008 (IEEE) pp 1–8 [9] Harel J, Koch C and Perona P 2007 Graph-based visual saliency Adv. Neural Inform. Process. Syst. 19 545 [10] Hou X and Zhang L 2007 Saliency detection: a spectral residual approach IEEE Conf. on Computer Vision and Pattern Recognition, 2007: CVPR’07 (IEEE) pp 1–8 [11] Howard A and Seraji H 2004 Multi-sensor terrain classification for safe spacecraft landing IEEE Trans. Aerosp. Electron. Syst. 40 1122–31 [12] Indiveri G 2008 Neuromorphic VLSI models of selective attention: from single chip vision sensors to multi-chip systems Sensors 8 5352–75 [13] Itti L and Koch C 2001 Computational modeling of visual attention Nature Rev. Neurosci. 2 194–203 [14] Itti L, Koch C and Niebur E 1998 A model of saliency-based visual attention for rapid scene analysis IEEE Trans. Pattern Anal. Mach. Intell. 20 1254–9 [15] Izzo D, Weiss N and Seidl T 2011 Constant-optic-flow lunar landing: optimality and guidance J. Guid. Control Dyn. 34 1383–95 [16] Koch C and Mathur B 1996 Neuromorphic vision chips IEEE Spectr. 33 38–46 [17] Koch C and Ullman S 1985 Shifts in selective visual attention: towards the underlying neural circuitry Hum. Neurobiol. 4 219–27 [18] Liu T, Yuan Z, Sun J, Wang J, Zheng N, Tang X and Shum H Y 2011 Learning to detect a salient object IEEE Trans. Pattern Anal. Mach. Intell. 33 353–67

7. Conclusions In this work, we proposed an integrated system for guiding unmanned spacecrafts during descent onto unknown celestial bodies. We introduced in the guidance system a model of human pre-attentive vision which allows the spacecraft to autonomously select salient landing sites, without any a priori knowledge of the planetary surface. We showed how salient sites are often located in dangerous zones, where the terrain is highly heterogeneous. A safety map is built to overcome this issue, allowing for the identification of safe areas close to as many salient points as possible. The described control system, based on a simplified model of the spacecraft and on optimality principles, guarantees that the spacecraft reaches the desired final point with minimized mass consumption, resulting in an efficient and robust control architecture. The comparison with human judgements on saliency and safety showed that our algorithm can produce similar results to those of human subjects in terms of selected areas for landing, frequency and timing of retargeting. Further studies in this direction will include comparison with experts. This could give a measure of how saliency differs from specific scientific criteria, especially in the context of space exploration, where 10

Bioinspir. Biomim. 7 (2012) 025007

G Schiavone et al

[23] Ploen S R, Seraji H and Kinney C E 2009 Determination of spacecraft landing footprint for safe planetary landing IEEE Trans. Aerosp. Electron. Syst. 45 3–16 [24] Serrano N 2006 A Bayesian framework for landing site selection during autonomous spacecraft descent 2006 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IEEE) pp 5112–7 [25] Sim˜oes L F, Bourdarias C and Ribeiro R A 2012 Real-time planetary landing site selection—a non-exhaustive approach Acta Futura 5 39 [26] Todorov E 2004 Optimality principles in sensorimotor control Nature Neurosci. 7 907–15

[19] Metropolis N et al 1953 Equation of state calculations by fast computing machines J. Chem. Phys. 21 1087 [20] Orchard G, Bartolozzi C and Indiveri G 2009 Applying neuromorphic vision sensors to planetary landing tasks IEEE Biomedical Circuits and Systems Conference, 2009: BioCAS 2009 (IEEE) pp 201–4 [21] Parker G A and Smith J M 1990 Optimality theory in evolutionary biology Nature 348 27–33 [22] Parkes S M, Martin I, Dunstan M and Matthews D 2004 Planet surface simulation with PANGU 8th Int. Conf. on Space Operations (SpaceOps 2004) (Montreal, Canada, May 2004) (American Institute of Aeronautics and Astronautics) pp 1–10

11