Biol Cybern (2010) 102:45–55 DOI 10.1007/s00422-009-0349-y

ORIGINAL PAPER

A neural mechanism of synergy formation for whole body reaching Pietro Morasso · Maura Casadio · Vishwanathan Mohan · Jacopo Zenzeri

Received: 17 August 2009 / Accepted: 2 November 2009 / Published online: 25 November 2009 © Springer-Verlag 2009

Abstract The present study proposes a computational model for the formation of whole body reaching synergy, i.e., coordinated movements of lower and upper limbs, characterized by a focal component (the hand must reach a target) and a postural component (the center of mass must remain inside the support base). The model is based on an extension of the equilibrium point hypothesis that has been called Passive Motion Paradigm (PMP), modified in order to achieve terminal attractor features and allow the integration of multiple constraints. The model is a network with terminal attractor dynamics. By simulating it in various conditions it was possible to show that it exhibits many of the spatiotemporal features found in experimental data. In particular, the motion of the center of mass appears to be synchronized with the motion of the hand and with proportional amplitude. Moreover, the joint rotation patterns can be accounted for by a single functional degree of freedom, as shown by principal component analysis. It is also suggested that recent findings in motor imagery support the idea that the PMP network may represent the motor cognitive part of synergy formation, uncontaminated by the effect of execution. Keywords Whole body reaching · Equilibrium point hypothesis · Passive motion paradigm · Synergy formation

P. Morasso · M. Casadio · V. Mohan · J. Zenzeri Department of Robotics, Brain and Cognitive Sciences, Italian Institute of Technology, Genoa, Italy P. Morasso (B) · M. Casadio · J. Zenzeri Department of Informatics, Systems, Telecommunications (DIST), University of Genoa, Via Opera Pia 13, 16145 Genoa, Italy e-mail: [email protected] P. Morasso National Institute of Neuroscience, Turin, Italy

1 Introduction Posture and movement are like Siamese twins: inseparable but, to a certain extent, independent. To simplify motor control and motor control modeling, moving joints are frequently linked together to form a functional synergy. The inverted pendulum approximation during quiet standing is an example: the whole kinematic chain from the ankle to the wrist and the head is collapsed to a single degree of freedom (DoF). Clearly this is just an approximation and multi-link effects have been reported (van Soest et al. 2003; Gage et al. 2004; Alexandrov et al. 2005; Edwards 2007; Hsu et al. 2007; Wu et al. 2009, among others) but overall it is still taken as an acceptable first-order approximation. In any case, the inverted pendulum is an unstable plant which can be stabilized either by passive mechanisms (the intrinsic stiffness of the ankle joint) or active mechanisms that imply the neural control of the ankle torque. The stiffness control hypothesis, proposed by Winter et al. (2001), was a promising simplification of the control because it could achieve stability by exploiting the intrinsic mechanical properties of the ankle muscles; unfortunately it had to be rejected after performing direct measurements of the ankle stiffness (Loram and Lakie 2002; Casadio et al. 2005) which showed that stiffness is significantly smaller than a critical value, determined by the rate of growth of the gravity-driven toppling torque. As regards possible neural control mechanisms of ankle torque, which are responsible for the generation of the Center of Pressure (CoP: the control variable) in such a way to reproduce the observed sway of the Center of Mass (the projection on the support base of the CoM: the controlled variable), two main classes of control paradigms were investigated by different researchers: (1) continuous time, linear feedback controllers similar to the classical engineering servomechanisms

123

46

(PD or PID controllers: Peterka (2000); Masani et al. (2003), among others); (2) intermittent, non-linear controllers (Bottaro et al. 2005; Loram et al. 2005; Asai et al. 2009) based on some kind of switching function of the control action. It has been shown (Bottaro et al. 2008; Asai et al. 2009) that the latter class of controllers is much more robust than the former one, considering the large transmission delays in the control loop, in the sense that they guarantee stability in a much larger region of the control parameter space (e.g., the space identified by the proportional and derivative gains in the PD feedback controller). Robustness is quite important because focal movements of the upper limb induce significant variations of the dynamics of the equivalent inverted pendulum and a robust postural controller has more chances to succeed, without complicated adaptive control mechanisms. When a focal movement is produced, starting from a quiet standing posture, the equilibrium is broken and immediately all the available DoFs of the kinematic chain are recruited in a coordinated way, in addition to the ankle. Belen’ki˘ı et al. (1967) were the first to show that changes in the electromyographic activity of postural muscles in standing humans appear prior to (and during) voluntary movement of an upper limb and are specific to this movement: Anticipatory Postural Adjustments (APAs). APAs have been studied by many people; in particular, Aruin (2002) showed that the process of generation of APAs is affected by three major factors: (1) expected magnitude and direction of the perturbation, (2) voluntary action associated with the perturbation, (3) features of the underlying postural task. Thus, APAs are not reflexes but coordinative structures superimposed on the postural stabilization processes. However, in the context of this paper we are not interested in generic APAs, i.e., the issue of compensating the self-generated postural disturbances determined by voluntary movements of the upper limbs, but in the specific class of APAs which is related to reaching beyond arm’s length, i.e., Whole Body Reaching (WBR). Research on WBR is more than a decade long (Stapley et al. 1999; Pozzo et al. 2002; Kaminski 2007). The main results which emerged from this activity, in particular as regards “large movements”, can be summarized as follows: 1. The traditional dichotomy between posture and movement is blurred, indicating that the legs and trunk, for example, play a dual role: not only are they responsible for maintaining postural stability, but they also contribute to transporting the hand to the target; 2. As target distance increases, the reach and postural synergies become coupled resulting in the arms, legs, and trunk working together as one functional unit to move the whole body forward; 3. Analysis of the CoM indicates that it is shifted progressively forward, as reached distance increases, and is synchronized with the finger’s movement.

123

Biol Cybern (2010) 102:45–55

Although WBR has been studied extensively, the analysis was mainly focused on the statistical properties of the recorded signals, by means of the principal components analysis (Stapley et al. 1999; Pozzo et al. 2002; Kaminski 2007). No computational model has been proposed so far for implementing the formation of the coordinative structures or synergies that emerge from the recorded data. The proposed model is based on two main elements: (1) the Equilibrium Point Hypothesis (EPH: Feldman 1966; Bizzi et al. 1976) and (2) Motor Imagery (Decety 1996; Crammond 1997). It builds upon the concept of Passive Motion Paradigm (PMP: Mussa Ivaldi et al. 1988) and nonlinear terminal attractor dynamics (Tsuji et al. 1996). In the concluding session we shall also discuss the relationship between the proposed synergy formation model and the uncontrolled manifold concept applied to synergy formation (Scholz and Schöner 1999; Latash et al. 2007).

2 The model The proposed model is a further development of the concept of PMP (Mussa Ivaldi et al. 1988), which may be considered as a generalization of the EPH. The recent work in the field of motor imagery provides the neurophysiological framework for this kind of computational model. 2.1 From the EPH to the PMP Generally speaking, the underlying (biomechanical) motivation of the EPH is the understanding that during movements energy can be stored passively in the biomechanics of the muscle system and controlling the flow of such energy can improve/simplify the motor control task for the brain. For example, it is possible to exploit such stored energy from the point of view of energy efficiency in locomotion: consider kangaroo hopping patterns (Alexander and Vernon 1975) and biomimetic hopping robots (Raibert 1990). The same energy can also be considered in a Lyapunov sense, as a property of a conservative system that indirectly specifies a desired configuration/position as an equilibrium point in the state space. In the case of the motor system this is equivalent to say that an equilibrium point (EP) is a configuration at which, for each joint, agonist and antagonist torques cancel out. In this framework, trajectory formation can be formulated as the consequence of shifting the EP of the system. Feldman (1966); Bizzi et al. (1976) pioneered this approach, although with significant differences (e.g., Feldman’s formulation included afferent feedback, whereas Bizzi’s did not). EPH was investigated later on by a large number of research groups and indeed it is a general language for describing motor control in humans as well as more recently in humanoid robots (Gu and Ballard 2006; Mohan and Morasso 2007; Mohan et al.

Biol Cybern (2010) 102:45–55

47

Fig. 1 Left panel: Computational metaphor of the PMP as a wooden marionette animated by a master. Right panel: Causality/information flow in the PMP network

2009). However, it is a language with many “dialects” which tend to obscure the common background. It is a pity, for example, that much of the debate has been focused on the issue of compensating arm dynamics by means of end-effector stiffness [see, for example, Gomi and Kawato (1997) vs. Bizzi et al. (1992) and Feldman and Levin (1995)]. However, we think that the power of the EPH is mainly related to its ability to address what Bernstein (1967) called the “degrees of freedom problem”, although, as shown by Hermens and Gielen (2004), an EP-model does not necessarily reproduce a unique solution to the problem. Bernstein proposed that any movement is uniquely adapted to the context by a hierarchical set of control systems, each exerting input into the motion generation process in sequence as well as in parallel, in such a way to reduce significantly the number of control variables. In the framework of the EPH, the sequential part may imply a discrete chain of EPs and the parallel part the combination of the corresponding force fields. In other words, we believe that it is redundancy (managing and/or exploiting it) the key point of motor control, not the compensation of intrinsic dynamics. Summing up, we think that the energy function and the associated force fields is the key computational idea underlying EPH. Although such “energy” can be attributed to the mechanical properties of tendons and muscles, by no means this is the only possibility. A plausible generalization of the EPH concept is a network or set of networks characterized by attractor dynamics in a similar way to Hopfield networks (Hopfield 1982). In Hopfield networks, used as associative memories, connectivity arises from the combination of a number of constraints and the stored information is not searched explicitly but implicitly, by running the network and relying on its attractor dynamics for reconstructing the sought information. In the case of associative memories, only the final equilibrium state is relevant, not the trajectory

generated by the network for reaching equilibrium. In motor control, instead, the final point and the trajectory are both equally relevant, as well as the timing of the process. The general idea is that the multiple constraints that characterize any given task can be implemented as multiple superimposed force fields that collectively shape an energy function, whose equilibrium point drives the trajectory formation process of the different body parts participating to a task. In particular, the PMP (Mussa Ivaldi et al. 1988) posits that the coordination of a highly redundant set of DoFs is similar to the (passive) motion of an internal Body Model under the action of task-related virtual force fields, combined with additional constraint-related fields (Fig. 1). In the specific implementation of this concept, described in the following section, the body model, the force fields, the timing mechanism, and the task-related constraints are integrated in the same network structure. Such computational model is also inspired by previous neural network studies about force fields and topology representing networks (Morasso et al. 1997) and dynamic field theory (Erlhagen and Schöner 2002).

2.2 Computational model implementation of the PMP In the basic case of a simple kinematic chain, the PMP concept can be expressed by the network of Fig. 2 (grayshaded blocks). It contains a representation of the extrinsic space (current position of the end-effector x and position of the target xT ) and the intrinsic space (joint configuration vector q). A virtual spring attracts the end-effector to the target, thus generating an attractive force field FF = K F (xT − x), where K F plays the role of a stiffness in the extrinsic space. This field, which can be integrated with other

123

48

Biol Cybern (2010) 102:45–55

to the controlled manifold. For example, by setting to 0 a given element of the matrix it is possible to freeze a joint and thus split the kinematic chain in two independent sub-chains. The motion of the joints determines the motion of the end˙ effector according to the following relationship: x˙F = JF q. Ultimately, the motion of the kinematic chain evoked by the activation of a target is equivalent to integrating a non-linear differential equation3 that, in the simplest case in which there are no additional constraints, takes the following form: x˙ = JF A JFT K F (xT − x) Fig. 2 PMP network for the coordination of focal and balancing movements. The basic sub-network that implements the focal part corresponds to the gray-shaded blocks. The sub-network that implements the balancing part corresponds to the vertically-striped blocks. The squarepatterned block introduces terminal attractor dynamics (
(t) is the time base generator). The sub-network with horizontally-striped blocks generates a smoothly moving target xT (t) from the initial position of the hand to the designated target xF . JF is the Jacobian matrix of the overall kinematic chain (from feet to hands) used for the focal part of the task and K F is the corresponding stiffness matrix in the extrinsic space; JP is the Jacobian matrix of the subset of the kinematic chain, up to the hip, that is used for the postural part of the task; A is the admittance matrix in the intrinsic space; B is the viscosity matrix, in the intrinsic space, which aims at reducing the forward shift of the CoM. FF is the forcefield, in the extrinsic space, related to the focal part of the task; FF is the force-field, in the intrinsic space, related to the postural part of the task. K F , A, B remain constant during a movement; JF , JP are functions of q and thus change during a movement. Thus, the overall PMP network is a non-linear, time-varying dynamical system with terminal attractor dynamics

force fields that express additional “external constraints”, is mapped from the extrinsic to the intrinsic space by means of the mapping TF = JFT FF that yields an attractive torque field in the intrinsic space.1 It is worth noting that for a general, redundant kinematic chain, even if the external force field has a single point attractor, the attractor in the intrinsic space is a region, which corresponds to the null space of the kinematic transformation.2 This torque field can be mixed with other fields for expressing additional constraints in the intrinsic space. The total torque field T induces a coordinated motion of all the joints in the intrinsic space according to an admittance matrix A: q=AT. ˙ By modulating A it is possible to determine the degree of participation of each joint to the coordinated movement and thus select one specific configuration among the infinite others. In other words, A, which is typically a diagonal matrix, determines the motion in the null space or uncontrolled manifold, whereas the attractive field is related 1

(1)

This equation always allows the end-effector to reach the target, if it is located within the workspace of the kinematic chain, or settle in an equilibrium state which is at minimum distance from the target. Of course we are not suggesting that the brain may use this model in a direct way. The model is a condensed representation of the interaction between two maps, one in the extrinsic space and the other in the intrinsic space, respectively. In a corresponding distributed neural implementation (Morasso et al. 1997) the two vectors x(t) and q(t) were represented by two population codes and the Jacobian matrix by the cross-connections between the maps formed in the course of training. However, in the model of Eq. 1 there is no control of the time required for achieving the final equilibrium state. It is like a mechanism of cortical interaction without subcortical control. A way to explicitly control time, without using a clock, is to insert in the non-linear dynamics of the PMP model a suitable time-varying gain
(t), according to the technique originally proposed by Zak (1988) for speeding up the access to content addressable memories and then applied to a number of problems in recurrent neural networks. In this way, the dynamics of the network is characterized by terminal attractor properties. Our purpose, however, is not merely to speed up the operation time of the planner but to allow a control of the reaching time as well, in order to fit the human speed profile of reaching patterns and allow synchronization in multi-tasking (Tsuji et al. 1996). The time-varying gain grows monotonically as x approaches the equilibrium state and diverges to an infinite value in that state. Equation 1 is substituted by Eq. 2: ⎧ T ⎪ ⎨ x˙ =
(t)JF A JF K F (xT − x) ˙ (2)
(t) = ξ (1 − ξ ) ⎪ ⎩ ξ(t) = 6 · (t/τ )5 − 15 (t/τ )4 + 10 (t/τ )3 Here τ is the planned duration of the movement and ξ(t) is a smoothed step-like function generator that follows a minimum-jerk profile. This extension of the basic PMP network

JF is the Jacobian matrix of the kinematic transformation x = f (q).

2

The null space is the set of movements in the intrinsic space that do not affect the position of the end-effector and is characterized by the following equation: JF q˙ = 0.

123

3

The differential equation is non-linear because the Jacobian matrix is not constant but is a function of the configuration vector q and then of the end-effector position x.

Biol Cybern (2010) 102:45–55

is marked, in Fig. 2, by the introduction of square-patterned block. 2.3 Coordination of reaching with balance The model depicted in Fig. 2 can be extended in order to express multi-tasking and coordination of different body parts. This concept has been investigated in the field of robot control and reasoning (Mohan and Morasso 2007; Mohan et al. 2009). Here we explore its application to the coordination of a focal movement with balance control in standing humans. The kinematic chain that we are considering, in this case, is the total body, from the foot to the hand. For simplicity, we restrict our attention to the sagittal plane, with a symmetric participation of the left and right limbs. This simplified kinematic chain has five DoFs: ankle, knee, hip, shoulder, and elbow. Thus q is a five-dimensional vector and x (the position of the fingertip in the sagittal plane) is a twodimensional vector. In this context we also need to consider a second “end-ffector”, identified by the projection y of the CoM on the standing surface. The task of reaching beyond arm’s length or WBR has two components that must be satisfied simultaneously: 1. Coordinating the joints in order to allow the end-effector x to reach the target xT ; 2. Exploiting the redundancy of the kinematic chain in order to keep the horizontal component of the CoM y within the support base. The two sub-tasks are associated with two force fields: 1. A focal force field, related to the focal movement FF = K F (xT − x), applied to the fingertip of the hand; the field is elastic, in agreement with the EPH concept. This field also induces a concurrent forward shift of the CoM of the body that tends to destabilize the standing posture. 2. A postural force field, related to the postural component of the task, that at least partially compensates the previously mentioned tendency of the CoM to shift forward. This force field is applied to the hip, thus implementing a kind of “hip strategy”. There are several solutions for implementing the postural force field: (a) an attractive field, determined by a preferred or ‘target’ position yT of the CoM, which provides an attractive force to yT proportional to the distance from the current CoM posistion y; (b) a repulsive field, determined by a predefined ‘range of motion’ of the CoM on the support base, providing a force pushing away from the closest CoM limit; (c) a resistive field, providing a force proportional to the horizontal speed of the CoM and oriented in the opposite direction. The first two solutions require to have a sufficiently

49

precise estimate of the position of the CoM on the support base and this is a difficult measurement because, in absence of a specific and direct sensory channel, the central nervous system should carry out a complex process of multisensory fusion (proprioceptive, tactile, visual, and vestibular) in order to have access on this information. The third solution is computationally cheaper because it requires only estimating the rate of change of the CoM position, not its absolute value. Tactile and proprioceptive information is probably sufficient for this purpose. Moreover, since a resistive field is equivalent to a damping element, from the control point of view, such field would contribute to the overall stability of the PMP network. The other two alternatives, on the contrary, tend to induce additional potential energy in the system and, if not appropriately tuned, might trigger oscillatory phenomena. For these reasons we chose the third alternative (see the vertically-striped blocks of Fig. 2): FP = −B y˙ . Moreover, as we show in the following section, the simulation results are consistent with the reported behavior of the CoM in WBR. Figure 2 shows the overall PMP network used in the simulations that integrates the focal and postural sub-tasks: both force fields are mapped onto the same intrinsic space, by using the Jacobian matrix JF , for the attractive field related to the focal sub-task, and the Jacobian matrix JP , for the resistive field related to the postural sub-task. The overall dynamics of the network triggers the unfolding of a coordinated movement of the whole kinematic chain that integrates the focal and postural components. Without the postural field, the focal movement would induce a large forward shift of the CoM, generally beyond the support base. The purpose of the postural field is to constrain such shifts within the support base. The synchronization of the two sub-tasks is assured implicitly by the timing signal
(t) for both sub-networks, although it is directly applied only to the focal component. The figure also contains a network fragment responsible for generating a time-varying target, synchronized with the focal and postural processes (horizontally-striped blocks of Fig. 2). This is consistent with the finding that during reaching a target the equilibrium point is not displaced suddenly to the final position in a step-like manner but is smoothly shifted (Bizzi 1987). The PMP model was implemented in Simulink
and the simulations were carried out with a variable-step, third-order numerical integration procedure. Human-like values of the kinematic chains were used in the simulations. The duration of the movements was 1 s. In most simulations we used the following parameters: K F = 500 N/m (stiffness of the attractive field); B = 70 N/m/s (viscosity of the resistive field); A = [0.3 0.3 2.5 0.1 0.1] rad/s/Nm (admittance vector of the kinematic chain: ankle, knee, hip, shoulder, elbow). However, such values were not critical: changes of 50% or more did not affect the qualitative structure of the spatio-temporal patterns and, most important, did not induce instability.

123

50

3 Results Figure 3 shows the spatio-temporal patterns of a reaching movement that requires a forward displacement of the hand beyond arm’s length. The two graphs of the top panel show that both the hand and the CoM are shifted forward. In absence of the postural field (right graph of the top panel) the CoM would overcome the limit of the support base and

Fig. 3 Top panel: Trajectories of the hand and the COM generated by the simulation of the PMP network with or without the activation of the postural sub-network. Middle panel: time course of the joint rotations and the time base generator
(t). Bottom panel: corresponding speed profiles of the hand trajectory and the forward shift of the CoM. The middle and bottom panels refer to simulations of the complete network

123

Biol Cybern (2010) 102:45–55

thus induce a forward fall. The activation of the field (left graph of the top panel) induces the following effects: (1) a much smaller forward shift of the CoM; (2) a backward shift of the hip (typical of the so called “hip strategy”); (3) a forward tilt of the trunk associated with the lowering of the CoM. It is worth remarking that this complex control pattern is not explicitly represented but is implicitly coded by the dynamics of the PMP network. The intermediate panel

Biol Cybern (2010) 102:45–55

51

Fig. 4 Sequence of reaching movements in the forward direction to targets at increasing distance. The top panel shows two movements of the sequence, to targets T1 and T2, respectively; COM1 and COM2 indicate the corresponding trajectories of the center of mass. The bottom panel shows the relation between the forward shift of the hand and the corresponding shift of the CoM

shows the time course of the joint angles and the driving signal
(t). The bottom panel shows the speed profile of the hand movement and the corresponding profile of the forward component of the CoM shift. In spite of the complex interactions of the model non-linear dynamics, both profiles are approximately bell-shaped and are in phase. This feature remains invariant for movements of different amplitudes and directions. On the contrary, the phase relationship among the speed profiles of the different joints is not invariant and depends on amplitude and speed of the targeting movements. For a family of reaching movements to targets shifted forward at a varying distance, with respect to the initial position of the hand, Fig. 4 shows the relationship between the forward shift of the target and the corresponding shift of the CoM. It turns out that the relationship is almost linear, with a predictable ‘border effect’ when the target approaches the

border of the workspace. The slope of the line can be modulated by acting on the B parameter. The results summarized by Figs. 3 and 4 require that the postural force field is of ‘resistive’ type. With ‘attractive’ or ‘repulsive’ fields it is impossible to avoid an overshoot of the COM trajectory which is not compatible with experimental data. By acting on the K F parameter it is possible to force the hand to follow more or less closely the trajectory of the target. In the displayed simulations this parameter is rather small and thus deviations are observed that remind the patterns of human movements. Operating with rather small values of K F improves the stability of the dynamical system. By changing the admittance of the kinematic chain (A array of admittance values) it is possible to influence the type of solution chosen by the network. In the simulations we chose rather low values for A4 and A5 (0.1 rad/s/Nm) in

123

52

Biol Cybern (2010) 102:45–55

Fig. 5 Effect of changing the hip admittance from 2.5 rad/s/Nm (left panel) to 0.1 rad/s/Nm (right panel)

the PMP mechanism is characterized by a ‘graceful degradation’ feature as the task becomes more and more constrained by the environmental conditions.

4 Discussion

Fig. 6 Percentage of the variability of the joint rotation patterns accounted for by the first principal component, as a function of the distance of the reached target from the starting position of the hand. The series of movements is the same depicted in Fig. 4

order to favor final postures with almost outstretched arms (as indeed can be verified in the pictures). The leg admittances, A1 and A2 , had intermediate values (0.3 rad/s/Nm) and the hip admittance A3 had the highest value (2.5 rad/s/Nm) in such a way to enhance the key role of the hip in this integration of reaching and balancing. Figure 5 shows what happens if this value is greatly reduced (from 2.5 to 0.1) in a difficult reaching task: the pivotal role of the hip is reduced while enhancing the participation of the knee. Finally, we evaluated to which extent the PMP in fact reduces the effective degrees of freedom of the coordinated patterns. On this purpose we computed the Principal Component Analysis (PCA) of the joint rotation patterns for different movements. In all the cases the first principal component explains more than 95% of the variability of the joint rotation patterns and the first two components account for more that 99%. Moreover, we analyzed the same set of movement of increasing length already considered in Fig. 4. As shown in Fig. 6, the variability accounted for by the first component grows with length, i.e., it becomes greater as the hands approach the boundary of the reachable space. In other words,

123

The behavior of the proposed coordination model agrees with many of the features documented in experiments of WBR in the sagittal plane (Stapley et al. 1999; Pozzo et al. 2002; Kaminski 2007): (1) it produces hand trajectories which are not perfectly straight but have a bell-shaped speed profile; (2) it merges the focal and postural components in the same synergy, as shown by the PCA; (3) it induces a progressive forward shift of the CoM, synchronized with hand trajectory. In other words, WBR appears to be a flexible, unitary synergy in which the joints of the lower and upper limbs, as well as the trunk, cooperate to the combined task: reaching the target with the hand and maintaining the CoM inside the support base. The strong coupling of the lower and upper extremities found in WBR is apparently in contrast with APA experiments in which there are no targets to be reached and simple arm raises or trunk flexions/extensions are performed in order to perturb the standing posture (Bouisset and Zattara 1987; Lee et al. 1987; Crenna et al. 1987). In the absence of a target reaching task, the postural system can operate independently and fulfill the goal of minimizing the motion of the CoM. However, when moving to a target, the global reach synergy is activated and incorporates at the same time lower and upper limbs. Thus, the amount of CoM displacement may depend on whether the goal of a movement is simply to keep balance during body’s configuration changes or to interact with the environment. We should also remark that the proposed coordination model is not a controller of the standing posture. It does not address the problem of stabilizing the unstable standing body by generating appropriate motor commands to the different

Biol Cybern (2010) 102:45–55

muscles of the body kinematic chain, with main emphasis to the ankle muscles. In spite of the complex postures assumed by the body, after reaching a target, from the biomechanical point of view the body remains an inverted pendulum. The main point, however, is that the moment of inertia of the equivalent pendulum may vary greatly, considering that it depends on the square of the distance between the CoM and the ankle joint. Therefore, in the case of WBR the postural controller must stabilize a time-varying plant. It has been shown (Bottaro et al. 2008; Asai et al. 2009) that an intermittent controller of the standing posture, characterized by an appropriate switching function defined in the phase plane (see Fig. 7), can stabilize the standing posture in a robust way: this property has been demonstrated by showing that stability is preserved when we consider large variations of the control parameters, with a constant moment of inertia. We expect that the same robustness can be preserved when considering large variations of the moment of inertia, in the neighborhood of a given set of control parameters; thus we think that the class of simple intermittent controllers is a good candidate for implementing the control layer underlying the WBR synergy. In other words, we suggest that the WBR synergy generates a time-varying expected trajectory of the CoM and the intermittent motor controller uses the trajectory points as time-varying reference values of the stabilization process. In a future study we shall analyze this point in depth. As explained in previous sections, the proposed WBR synergy is based on an extension of the equilibrium point hypothesis (the PMP class of models) that is not limited to the attractive dynamics determined by the muscle properties but is extended to the dynamic interaction of the internal body schema. We believe that recent research on motor imagery supports the biological plausibility of the proposed model. Motor imagery was defined by Crammond (1997) as the mental rehearsal of a motor act in absence of overt motor output. Experimental results (in terms of EEG, fMRI, PET, NIRS) generally support the idea of a common neuronal substrate subserving both the preparation for execution and imagery of movements. These experiments also provide a broader context for the notion of a common substrate by revealing similarities in cognitive components associated with the movement tasks (Kranczioch et al. 2009; Munzert et al. 2009). In this context it is quite relevant, for the analysis of the proposed model, the Mental Simulation Theory (MST) proposed by Jeannerod (2001). MST stresses that cognitive motor processes such as motor imagery and action observation share the same representations as motor execution; it views motor images as being based on neural processes for motor execution that are inhibited at a certain stage of processing. However, this overlap of neural representations is not just found between motor imagery and motor execution. It also emerges when other cognitive motor processes such as movement observation, action planning, and action

53

verbalization are compared with motor execution. A specific element of the MST is the S-state: 1. S-states describe all those processes taken to be covert stages of action that share common representations with motor execution. 2. S-states simulate, to some degree, the real action. MST provides convincing arguments for processes that underlie mental training and explain neural activation during motor imagery as well as behavioral outcomes of mental training. The basic idea is that a neural motor network is activated while imagining motor actions. This activation includes not only premotor and motor areas such as PMC, SMA, and M1 but also subcortical areas of the cerebellum (Lotze et al. 1999) and the basal ganglia (Bonda et al. 1995). We suggest that mental simulations of S-states (motor imagery) can be conceived as the behavior of interacting neural networks characterized by attractor dynamics as a consequence of multiple force fields, thus providing a conceptual support to the PMP mechanism. Since mental simulations of covert actions correspond to the central stages of action organization, uncontaminated by the effects of execution, one can conclude that the internal processes that support motor cognition are, to some extent, segregated from body dynamics. The PMP perfectly fits into this framework. Summing up, the mounting evidence coming from motor imagery supports the idea that the EPH has a range of validity which is much wider than initially proposed and is indeed a very general mechanism for the formation of complex synergies like WBR. Moreover, the concept that the separation between movement preparation and movement execution is becoming more and more blurred is also supported by behavioral experiments, not only motor imagery: for instance, it has been found that online updating of planned movements may occur at any time during the movement preparation and execution (Goodale et al. 1986; Prablanc and Martin 1992); the continuous nature of movement preparation and the sensitivity to the task environment has been made evident in several studies (Hening et al. 1988; Favilla et al. 1990; Ghez et al. 1991). As a final question, how does the proposed model of synergy formation relates with the uncontrolled manifold concept (UCM: Scholz and Schöner 1999; Latash et al. 2007)? UCM posits that for any given set of values of the controlled variables, the joint space is divided into two orthogonal subspaces: one subspace (controlled manifold) hosts the joint configuration patterns that support the specified task and the other subspace (uncontrolled manifold) consists of all those joint configurations that lead to the same set of values of the controlled variables. In the WBR task analyzed in the present article, the joint configuration space is fivedimensional, the controlled manifold is three-dimensional

123

54

Biol Cybern (2010) 102:45–55

Fig. 7 Intermittent postural controller. y is the projection of the CoM on the support base, relative to ankle position; yref is the corresponding reference position, under voluntary control; Tc , Ts , Tg , Tn are the torque components related to active control, passive stiffness, gravity, noise, respectively. The ‘inverted pendulum’ is represented by a secondorder, linearized dynamical system. The ‘gravity’ and ‘stiffness’ blocks provide torques proportional to y (or to the sway angle), but with elas-

tic constants of opposite signs. Efferent and afferent ‘delays’ are of the order of 100 ms each. The ‘PD’ controller generates a torque Tc which is obtained by a linear combination of the position error (e = yref − y) and the corresponding time derivative, using a proportional and a derivative gain parameters. The ‘switching function’ is sensitive to the instantaneous operating point in the phase plane of the system

(two dimensions for the focal sub-task and one dimension for the postural sub-task) and, as a consequence, the uncontrolled manifold is two-dimensional. The “virtual” motion in this manifold is parameterized by the admittance array A: changing the values of this array has little or no effect on the capability of reaching the target and maintaining the CoM in the prescribed range. Therefore, A does not have to be “controlled” in a direct way, given the specifications of the WBR task. However, modulations of A have effect on the “quality” of the solution found by the network: although such qualitative aspects may be scarcely relevant for the current task, they may be important in the context of a task sequence and this may be the subject of a higher level of action representation/planning. In other words, the idea that UCM is a kind of “garbage collector” where all the unaccounted variability is dumped may only tell part of the story. The other part is that UCM may be the site where action integration occurs in a semi-controlled way. To make the point, can we suggest the acronym SCM (semi-controlled manifold) instead of UCM? This has no implications on the mechanisms of analysis which have been suggested for classifying “controlled” versus “uncontrolled” variables (Scholz and Schöner 1999; Latash et al. 2007; Hsu et al. 2007) but it suggests an alternative view of the processes of synergy formation at large, including mechanisms of reinforcement learning for the optimization of task sequences.

Humour. The writing of the paper draft was carried out by P. Morasso during his stay in the laboratory of Prof. Koji Ito at the Tokyo Institute of Technology (June 2009), with the support of the JSPS invitation programme. P. Morasso thanks Prof. Koji Ito and Prof. Toshio Tsuji (from Hiroshima University) for fruitful discussions that contributed to the formulation of the manuscript.

Acknowledgments This work was supported by the RBCS Department of the Italian Institute of Technology and the FP7 EU Project

123

References Alexander RMcN, Vernon A (1975) The mechanics of hopping by kangaroos (Macropodidae). J Zool Lond 177:265–303 Alexandrov AV, Frolov AA, Horak FB, Carlson-Kuhta P, Park S (2005) Feedback equilibrium control during human standing. Biol Cybern 93:309–322 Aruin AS (2002) The organization of anticipatory postural adjustments. J Autom Control 12:31–37 Asai Y, Tasaka Y, Nomura K, Casadio M, Morasso P (2009) A model of postural control in quiet standing: robust compensation of delayinduced instability using intermittent activation of feedback control. PLoS One 4(7):e6169 Belen’ki˘ı VE, Gurfinkel’ VS, Pal’tsev EI (1967) Control elements of voluntary movements. Biofizika 12:135–141 Bernstein N (1967) The co-ordination and regulation of movements. Pergamon Press, Oxford Bizzi E (1987) Motor control mechanisms. An overview. Neurol Clin 5:523–528 Bizzi E, Polit A, Morasso P (1976) Mechanisms underlying recovery of final head position. J Neurophysiol 39:435–444 Bizzi E, Hogan N, Mussa-Ivaldi FA, Giszter S (1992) Does the nervous system use equilibrium-point control to guide single and multiple joint movements. Behav and Brain Sci 15:603–613 Bonda E, Petrides M, Frey S, Evans A (1995) Neural correlates of mental transformations of the body-in-space. PNAS 92:1180–1184

Biol Cybern (2010) 102:45–55 Bottaro A, Casadio M, Morasso P, Sanguineti V (2005) Body sway in quiet standing: is it the residual chattering of an intermittent stabilization process. Human Mov Sci 24:588–615 Bottaro A, Yasutake Y, Nomura T, Casadio M, Morasso P (2008) Bounded stability of the quiet standing posture: an intermittent control model. Human Mov Sci 27:473–495 Bouisset S, Zattara M (1987) Biomechanical study of the programming of anticipatory postural adjustments associated with voluntary movement. J Biomech 20:735–742 Casadio M, Morasso P, Sanguineti V (2005) Direct measurement of ankle stiffness during quiet standing: implications for control modelling and clinical application. Gait Posture 21:410–424 Crammond DJ (1997) Motor imagery: never in your wildest dream. Trends Neurosci 20:54–57 Crenna P, Frigo C, Massion J, Pedotti A (1987) Forward and backward axial synergies in man. Exp Brain Res 65:538–548 Decety J (1996) Do imagined and executed actions share the same neural substrate. Cog Brain Res 3:87–93 Edwards WT (2007) Effect of joint stiffness on standing stability. Gait Posture 25:432–439 Erlhagen W, Schöner G (2002) Dynamic field theory of movement preparation. Psychol Review 109:545–572 Favilla M, Gordon J, Hening W, Ghez C (1990) Trajectory control in targeted force impulses: VII. Independent setting of amplitude and direction in response preparation. Exp Brain Res 79:530–538 Feldman AG (1966) Functional tuning of the nervous system with control of movement or maintenance of a steady posture. II. Controllable parameters of the muscle. Biophysics 11:565–578 Feldman AG, Levin MF (1995) The origin and use of positional frames of reference in motor control. Behav Brain Sci 18:723–806 Gage WH, Winter DA, Frank JS, Adkin AL (2004) Kinematic and kinetic validity of the inverted pendulum model in quiet standing. Gait Posture 192:124–132 Ghez C, Hening W, Gordon J (1991) Organization of voluntary movement. Current Biol 1:664–671 Gomi H, Kawato M (1997) Human arm stiffness and equilibrium-point trajectory during multi-joint movement. Biol Cybern 76:163–171 Goodale MA, Pélisson D, Prablanc C (1986) Large adjustments in visually guided reaching do not depend on vision of the hand or perception of target displacement. Nature 320:748–750 Gu X, Ballard DH (2006) An equilibrium point based model unifying movement control in humanoids. In: Robotics: science and systems II, Philadelphia, Pennsylvania, USA, 16–19 August 2006 Hening W, Favilla M, Ghez C (1988) Trajectory control in targeted force impulses: V. Gradual specification of response amplitude. Exp Brain Res 71:116–128 Hermens S, Gielen S (2004) Posture-based or trajectory-based movement planning: a comparison of direct and indirect pointing movements. Exp Brain Res 159:340–348 Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. PNAS USA 79:2554–2558 Hsu WL, Scholz JP, Schöner G, Jeka JJ, Kiemel T (2007) Control and estimation of posture during quiet stance depends on multijoint coordination. J Neurophysiol 97:3024–3035 Jeannerod M (2001) Neural simulation of action: a unifying mechanism for motor cognition. Neuro Image 14:103–109 Kaminski TR (2007) The coupling between upper and lower extremity synergies during whole body reaching. Gait Posture 26:256–262 Kranczioch C, Mathews S, Dean JA, Sterr A (2009) On the equivalence of executed and imagined movements. Human Brain Mapp 30: 3275–3286 Latash ML, Scholz JO, Schöner G (2007) Toward a new theory of motor synergies. Motor Control 11:276–308

55 Lee WA, Buchanan TS, Rogers MW (1987) Effects of arm acceleration and behavioral conditions on the organization of postural adjustments during arm flexion. Exp Brain Res 66:257–270 Loram ID, Lakie M (2002) Direct measurement of human ankle stiffness during quiet standing: the intrinsic mechanical stiffness is insufficient for stability. J Physiol 545:1041–1053 Loram ID, Maganaris CN, Lakie M (2005) Human postural sway results from frequent, ballistic bias impulses by soleus and gastrocnemius. J Physiol 564(1):295–311 Lotze M, Montoya P, Erb M, Hülsmann E, Flor H, Klose U, Birbaumer N, Grodd W (1999) Activation of cortical and cerebellar motor areas during executed and imagined hand movements: an fMRI study. J Cogn Neurosci 11:491–501 Masani K, Popovic MR, Nakazawa K, Kouzaki M, Nozaki D (2003) Importance of body sway velocity information in controlling ankle extensor activities during quiet stance. J Neurophysiol 90:3774–3782 Mohan V, Morasso P (2007) Towards reasoning and coordinating action in the mental space. Int J Neural Systems 17:1–13 Mohan V, Morasso P, Metta G, Sandini V (2009) A biomimetic, forcefield based computational model for motion planning and bimanual coordination in humanoid robots. Auton Robots 27:291–307 Morasso P, Sanguineti V, Spada G (1997) A computational theory of targeting movements based on force fields and topology representing networks. Neurocomputing 15:414–434 Munzert J, Lorey B, Zentgraf K (2009) Cognitive motor processes: the role of motor imagery in the study of motor representations. Brain Res Reviews 60:306–326 Mussa Ivaldi FA, Morasso P, Zaccaria R (1988) Kinematic Networks A Distributed Model for Representing and Regularizing Motor Redundancy. Biol Cybern 60:1–16 Peterka R (2000) Postural control model interpretation of stabilogram diffusion analysis. Biol Cybern 82:335–343 Pozzo T, Stapley PJ, Papaxanthis C (2002) Coordination between equilibrium and hand trajectories during whole body pointing movements. Exp Brain Res 144:343–350 Prablanc C, Martin O (1992) Autonomous control during hand reaching at undetected two-dimensional target displacements. J Neurophysiol 67:455–469 Raibert MH (1990) Trotting, pacing and bounding by a quadruped robot. J Biomech 23:79–98 Scholz JP, Schöner G (1999) The uncontrolled manifold concept: identifying control variables for a functional task. Exp Brain Res 126:289–306 Stapley PJ, Cheron G, Grishin A (1999) Does the coordination between posture and movement during human whole-body reaching ensure center of mass stabilization. Exp Brain Res 129:134–146 Tsuji T, Morasso P, Kaneko M (1996) Trajectory generation for manipulators based on artificial potential field approach with adjustable temporal behavior. In: Proceedings of IEEE/RSJ International Symposium on Intelligent Robots and Systems IROS’96, Osaka, Japan, pp 438–443 van Soest AJ, Haenen WP, Rozendaal LA (2003) Stability of bipedal stance: the contribution of cocontraction and spindle feedback. Biol Cybern 88:293–301 Winter DA, Patla AE, Riedtyk S, Ishac M (2001) Ankle muscle stiffness in the control of balance during quiet standing. J Neurophysiol 85:2630–2633 Wu J, McKay S, Angulo-Barroso R (2009) Center of mass control and multi-segment coordination in children during quiet stance. Exp Brain Res 196:329–339 Zak M (1988) Terminal attractors for addressable memory in neural networks. Phys Lett A 133:218–222

123