Robotics and Autonomous Systems 51 (2005) 81–99

Robustness of the dynamic walk of a biped robot subjected to disturbing external forces by using CMAC neural networks Christophe Sabourin∗ , Olivier Bruneau Laboratoire Vision et Robotique, Ecole Nationale Sup´erieure d’Ing´enieurs de Bourges, 10 Boulevard Lahitolle, 18020 Bourges Cedex, France Received 1 April 2004; received in revised form 26 January 2005; accepted 10 February 2005 Available online 29 March 2005

Abstract In this paper, we propose a control strategy allowing us to perform the dynamic walking gait of an under-actuated robot even if this one is subjected to destabilizing external disturbances. This control strategy is based on two stages. The first one consists of using a set of pragmatic rules in order to generate a succession of passive and active phases allowing us to perform a dynamic walking gait of the robot. The joint trajectories of this reference gait are learned by using neural networks. In the second stage, we use these neural networks to generate the learned trajectories during the first stage. The goal of the use of these neural networks is to increase the robustness of the control of the dynamic walking gait of this robot in the case of external disturbances. The first experimental results are also presented. © 2005 Elsevier B.V. All rights reserved. Keywords: Under-actuated biped robot; Dynamic walking gait; CMAC neural networks; Robustness

1. Introduction The design and the control of bipedal robots are one of the more challenging topics in the field of robotics and were treated by a great number of researchers and engineers since many years. The potential applications of this research are very important in the medium and long term. Indeed, this can lead initially to a better comprehension of the human locomotion, which can ∗ Corresponding author. Tel.: +33 2 48 48 40 75; fax: +33 2 48 48 40 40. E-mail addresses: [email protected] (C. Sabourin), [email protected] (O. Bruneau).

0921-8890/$ – see front matter © 2005 Elsevier B.V. All rights reserved. doi:10.1016/j.robot.2005.02.001

be very helpful for the design of more efficient orthosis. Secondly, the bipedal humanoid robots are intended to replace the human for interventions in hostile environments or to help him in the daily tasks. In addition to the problems related to autonomy and decision of such humanoid robots, the essential locomotion task is still today a big challenge. If it is true that some prototypes were constructed and prove the feasibility of such robots, of which most remarkable are undoubtedly today the robots ASIMO [1] and HRP-2 [2], the performances of these are still far from equalizing the human from the point of view of the dynamic locomotion process. The design of new control laws making it possible to ensure real-time control for real dynamic

82

C. Sabourin, O. Bruneau / Robotics and Autonomous Systems 51 (2005) 81–99

walking in unknown environments is thus today fundamental. Most of the time, the control of bipedal walking robots is carried out with methods based on tracking of temporal reference trajectories [3–8] generally associated with a control of the Zero Moment Point (ZMP) [9]. The torque applied at each joint is thus computed by using the difference between the desired and real trajectories. However, this kind of approaches involves some difficulties with regard to the autonomy of the robot: • Firstly, it is necessary to be able to modify on-line, for instance, the desired average velocity. If the trajectories are computed on-line, many calculations are necessary and require powerful data-processing tools in order to reduce the computing time. And if the trajectories are calculated off-line, it is necessary to memorize a set of joint trajectories for each velocity of each gait and for each transition between two different tasks. • Secondly, the tracking of reference trajectories do not ensure the global stability of the robot, especially when the ground is not regular or when a big perturbation force is applied to the robot. Indeed, in the first case the calculation of a trajectory requires to predict the position of the impact of the foot with the ground. From this constraint, which cannot easily be carried out on an irregular ground, it results that a difference between the desired and real impact point can occur. The actuator torques, generally computed by using a PD control, can then become very large and involve an instability, even the fall of the robot. In the second case, an external perturbation force can destabilize the robot because the movements are not well coordinated to react in the right way to maintain the dynamic stability of the robot. Indeed, the foot/ground contact being unilateral, it can occur one uncontrolled kinetic momentum of the system involving a rotation of the foot with regard to the ground surface. In this case, the tracking of temporal reference trajectories does not allow any more synchronization of the leg movements. • The criterion of stability ZMP, considered as dynamic criterion of stability is restrictive with regard to the irregularity of the ground and to the rapidity of the movements. Moreover, the use of the ZMP implies that the robot has feet. It is due mainly to

the fact that the dynamic stability from the point of view of the ZMP is related to the width of the support base. For instance, if the support base is lower than the foot surface (crossing of an obstacle), this one is not valid. Moreover, this criterion is also inoperative in the phases of flight during the running. Globally speaking, the control techniques using the tracking of temporal reference trajectories associated with an on-line control of ZMP are today limited, on the one hand, with regard to the complexity of calculations required to generate full dynamical motions and, on the other hand, with regard to the structure of robots (unusable for robots without feet). In fact, in order to get an efficient walking gait for biped robots, we think that it is necessary to solve four main problems: • First of all, we have to maintain the dynamic stability of the robot. This can be performed by using an instantaneous criterion, the Foot Rotation Indicator (FRI) [10], the Virtual Generalized Stabilizer (VGS) [11] or by considering the stability in the sense of the limit cycles [12,13]. • Secondly, in order to improve the autonomy of a robot, we have to minimize the energy cost. In order to carry out this, we can use the intrinsic dynamics of the robot and minimize the energy cost of the actuators [14,15]. • Thirdly, we have to decrease the computing time by using simple and discrete control strategies [16,17]. • Finally, it is essential to be able to adapt the control strategies and to modify the walking gait with regard to the environment (characteristics and geometry of the ground and of obstacles, external forces) and the state of the robot (carried load, level of energy). In this case, one solution is to use learning methods [18–20]. Commonly, these four goals are separately treated. Our aim is to propose a method in which these four points are taken into account at the same time. In this article, we present a synthesis of a new control strategy within the framework of the control of an underactuated robot: RABBIT [21,22]. The main characteristic of this robot is that it does not have a foot and consequently that its walking gait is completely dynamic. In this case, the stability criterion ZMP cannot be used.

C. Sabourin, O. Bruneau / Robotics and Autonomous Systems 51 (2005) 81–99

Our control strategy is based on two stages: • The first one uses a set of pragmatic rules making it possible to stabilize the pitch angle of the trunk and to generate the leg movements [23]. This control strategy, in the absence of disturbances, allows to generate, on the one hand, a stable dynamic walking and, on the other hand, velocity transitions and transitions of initiation and stop of walking gait. But the use of passive phases implies that this control strategy is very sensitive to the external disturbances that could occur during the progression of the robot. Moreover, frictions applied on each joint can be large and considerably limit the dynamics of the passive phases. Consequently, we propose to use a neural network allowing firstly to increase the robustness of our control strategy and secondly to compensate articular frictions on a real robot. In fact, in the first stage, the pragmatic rules are used as a reference control to learn, by using Cerebellar Model Controller Articulation (CMAC) [24,25], a set of articular trajectories. During this training phase, we consider that the robot moves in an ideal environment (without disturbance) and that frictions are negligible. • In the second stage, we use these neural networks to generate the trajectories learned during the first stage. A PD control allows us to ensure the tracking of this reference trajectories. But in our case, the trajectories learned by the neural networks are not a function of time but a function of the geometrical pattern of the robot. This implies that the movements of the legs are always coordinated and this, even in the presence of disturbances (external force, sliding, irregular ground). The robustness of the control of the walking robot is considerably improved which reduces the possibilities of robot fall. This paper is organized as follows. Section 2 presents the characteristics of the real and virtual under-actuated robot. In Section 3, we give the main characteristics of our control strategies (for more details please see [23]). Section 4 presents the method using the neural networks. In Section 5, in order to evaluate the robustness of this control strategy, we present results when the progression of the robot is disturbed. Section 6 presents the first results of our experimentation. Section 7 deals

83

with a discussion concerning our approach compared with another approach used to control RABBIT robot. Conclusions and further developments are finally given.

2. Model of the bipedal walking robot A lot of advanced humanoid bipedal robots such as ASIMO robot and HRP-2 robot, are not able to produce walking gaits with high velocity or running gaits with high dynamics. In addition to the problem of actuators, this is namely due to the fact that the stability of walking is ensured by keeping the ZMP strictly inside the basis of support. If it is true that, in its design, our robot RABBIT, shown in Fig. 1, is simpler compared to a robot with feet, from another point of view, it is more difficult to control, particularly because, in phase of single support, the robot is under-actuated. This implies to control one passive degree of freedom indirectly with the actuated degree of freedom. In this case, the analysis and the control of the intrinsic dynamics of walking (gravity, inertial effects) are of primary importance. For instance, the control of the pitch angle of the trunk permits to replace the moment generally applied to the ankle in the case of robot with actuated ankles. Consequently, the study of robot without feet can lead to an improvement of the current performances of the biped robots by the design of new control laws. 2.1. Prototype RABBIT The robot RABBIT is an experimentation of seven French laboratories (IRCCYN Nantes, LAG Grenoble, LMS Poitiers, LVR Bourges, LGIPM Metz, LRV Versailles, LIRMM Montpellier) concerning the control of a biped robot for walking and running within the framework of a CNRS project ROBEA [26]. This robot is composed of two legs and a trunk and has no foot as shown in Fig. 1. The joints are located at the hip and at the knee. This robot has four actuators: one for each knee, one for each hip. The characteristics (masses and lengths of the limbs) of RABBIT are summarized in Table 1. Motions are included in the sagittal plane by using a radial bar link fixed at a central column that allows to guide the direction of progression of the

84

C. Sabourin, O. Bruneau / Robotics and Autonomous Systems 51 (2005) 81–99 Table 1 Masses and lengths of the limbs of the robot Limb

Weight (kg)

Length (m)

Trunk Thigh Shin

20 6.8 3.2

0.625 0.4 0.4

Fig. 2. Modeling of the robot with ADAMS.

2.2. Virtual prototype

Fig. 1. Prototype RABBIT.

robot around a circle. This robot represents the minimal system able to generate walking and running gaits. Since the contact between the robot and the ground is just one point (passive DOF), the robot is under-actuated during the single support phase: there are only two actuators (at the knee and at the hip of the contacting leg) to control three parameters (vertical and horizontal position of the platform and pitch angle).

The numerical model of the robot previously described was designed with the software ADAMS.1 This software, from the modeling of the mechanical system (masses and geometry of the segments) is able to simulate the dynamic behavior of this system and namely to calculate the absolute motions of the platform and the relative motions of the limbs when torques are applied on the joints by the virtual actuators (Fig. 2). The model used to simulate the interaction between feet and ground is exposed in [27]. The normal contact force is given by Eq. (1):
0 if y > 0 n Fc = (1) −λnc |y|˙y + kcn |y| if y ≤ 0 where y and y˙ are, respectively, the position and the velocity of the foot (limited to a point) with regard to the ground. kcn and λnc are, respectively, the generalized stiffness and damping of the normal forces. They are chosen in order to avoid the rebound and to limit the penetration of the foot in the ground. The tangential 1

ADAMS is a product of MSC software.

C. Sabourin, O. Bruneau / Robotics and Autonomous Systems 51 (2005) 81–99

85

contact forces are computed with Eq. (2) in the case of a contact without sliding or with Eq. (3) if sliding occurs:
0 if y > 0 t Fc = (2) t t −λc x˙ + kc (x − xc ) if y ≤ 0
0 if y > 0 (3) Fct = n −(sgn(˙x))λg Fc − µg x˙ if y ≤ 0 where x and x˙ are, respectively, the position and the velocity of the foot with regard to the position of the contact point xc at the instant of impact with ground. kct and λtc are, respectively, the generalized stiffness and damping of the tangential forces. λg is the coefficient of dynamic friction depending on the nature of surfaces in contact and µg a viscous damping coefficient during sliding. After each iteration, the normal and tangential forces are computed from Eqs. (1) and (2). But, if Fct is located outside the cone of friction (Fct  > µs Fcn  with µs the static friction coefficient), then the tangential force of contact is computed with Eq. (3). The interest of this model is that it is possible to simulate walking with or without phases of sliding allowing us to evaluate the robustness of the control. 2.3. Actuator limits Within the framework of the control of a real robot, the morphological description of this one is insufficient. It is thus necessary to take into account the technological limits of the actuators in order to implement the control laws used in simulation on the experimental prototype. Fig. 3 gives the fields of application of servomotor RS420J used for RABBIT. This one shows that it is necessary to limit the torque according to velocity. From these characteristics we thus choose to apply the following limitations: • when velocity is included in [0 rpm; 2000 rpm], the torque applied to each actuator is limited to 1.5 N m, which corresponds to a torque of 75 N m at the output of the reducer (ration gear equal to 50); • when velocity is included in [2000 rpm; 4000 rpm], the power of each actuator is limited to 315 W; • when the velocity is bigger than 4000 rpm, the torque is imposed to be equal to zero.

Fig. 3. Field of application of the RS420J (continuous line) and chosen limits for the applied control torques (dotted line).

3. Control strategy based on a succession of passive and active phases The control strategy presented in this section is based on three points: • the observation of the relations between joint movements and the evolution of the parameters describing the motions of the robot platform; • an interpretation of the muscular behavior; • the analysis of the intrinsic dynamics of a biped. Based on these considerations, it is possible to determine a set of pragmatic rules. The objective of this strategy is to generate the movements of the legs by using a succession of passive and active phases. Fig. 4 shows the reference angles required for the development of our control and Sections 3.1 and 3.2 explain in details the laws used for the calculation of the actuator torques. Section 3.3 presents a high level control which allows to control the average velocity by adjusting the pitch angle of the trunk.

86

C. Sabourin, O. Bruneau / Robotics and Autonomous Systems 51 (2005) 81–99

the trunk: p

st v = Ktrunk (q0d − q0 ) − Ktrunk q˙ 0 Thip

(6)

where q0 and q˙ 0 are, respectively, the angle and the angular velocity of the trunk and q0d corresponds to the desired pitch angle of the trunk (Fig. 4). 3.2. Control at the knee During the swing phase, the knee joint is free and the torque is equal to zero. At the end of the knee extension, a control torque, given by Eq. (7) is applied to lock this dsw . This avoids, on the one joint in a desired position qi2 hand, an inverse knee flexion and, on the other hand, the rebound after a possible knee extension which is not the case by using a simple block stop:

Fig. 4. Parameters for angles and torques.

3.1. Control at the hip

p

During the swing phase, the torque applied to the hip given by Eq. (4) is just an impulse with a varying amplitude and a fixed duration equal to (t2 − t1 ):
pulse Khip if t1 < t < t2 sw1 Thip = (4) 0 otherwise pulse

where Khip is the amplitude of the torque applied to the hip at the beginning of the swing phase, and t1 and t2 are, respectively, the beginning and the end of acpulse tuation Khip . It should be noted that this impulse is carried out by using a polynomial interpolation of the third order during a very short time (t2 − t1 = 50 ms) pulse and that the amplitude of Khip is included in [0 N m, 75 N m]. The duration of this impulse for all the simulations presented in this paper will be always equal to 50 ms. However, this duration can be modified if needed. After this impulse, the hip joint is passive until the blocking of the swing leg in a desired position by using a PD control given by Eq. (5), which allows to ensure a regular step length: sw2 Thip

=

p dsw Khip (qr1

v − qr1 ) − Khip q˙ r1

if

qr1 >

dsw qr1

(5) dsw are, respectively, the measured and desired qr1 and qr1 relative angles between the two thighs (Fig. 4), q˙ r1 is the relative angular velocity between the two thighs. During the stance phase, the torque applied to the hip, given by the Eq. (6), is used to ensure the stability of

sw dsw v Tknee = Kknee (qi2 − qi2 ) − Kknee q˙ i2

(7)

qi2 and q˙ i2 are, respectively, the measured angular position and angular velocity of the knee joint of the dsw is a constant value included in [−20◦ ; 0◦ ] leg i. qi2 which depends on the desired propulsion during the next stance phase of this leg. During the stance phase, the torque is computed by using Eq. (8): p

st dst v = Kknee (qi2 − qi2 ) − Kknee q˙ i2 Tknee

(8)

dst = 0 at the impact with the ground in Eq. We choose qi2 dst > qdsw . (8) which contributes to propel the robot if qi2 i2 Indeed, the change of the desired position of the knee joint implies that the control law generates an impulse torque which physically acts like a spring-damper and dst − qdsw ). Indeed, just which namely depends on (qi2 i2 before the impact, qi2 is equal to the theoretical value dsw . During the continuation of the stance phase, the qi2 same control law is used to lock the knee at the position dst = 0. qi2

3.3. High level velocity control The strategy previously presented allows us to produce the dynamic walking gait of the bipedal robot without reference trajectories and with an average velocity included in [0 m/s; 1 m/s]. Fig. 5 shows, for dsw = −10◦ , a set of fixed parameters (q0d = 20◦ , qi2 dsw = 42◦ , K pulse = 60 N m), the stick diagram of the qr1 hip obtained stable dynamic walking gait.

C. Sabourin, O. Bruneau / Robotics and Autonomous Systems 51 (2005) 81–99

87

Fig. 5. Stick diagram of stable dynamic walking gait (q0d = 20◦ ,

dsw dsw qi2 = −10◦ , qr1 = 42◦ , Khip = 60 N m). pulse

The major problem is that it is necessary to regulate manually, for each desired average velocity, the four dsw , qdsw , K pulse . In order mentioned parameters: q0d , qi2 r1 hip to solve this problem, we propose to control the average velocity by adjusting the pitch angle of the trunk at each step by using the error between the average velocity VM d and of its derivative and the desired average velocity VM as described in Fig. 6. VM is simply calculated by using (9), where Lstep is the distance between the two feet at the moment of double impact and tstep is the duration of the step (from takeoff to landing of the same leg): VM =

Lstep tstep

(9)

At each step, q0d , which is computed by using the d and of its derivative (see Eq. error between VM and VM (10)), is then added to the pitch angle of the previous step q0d (n) in order to obtain the new desired pitch angle of the following step q0d (n + 1) as shown in Eq. (11): d q0d = Kp (VM − VM ) + K v

q0d (n + 1) = q0d (n) + q0d

d d (V − VM ) dt M

(10) (11)

Fig. 7 shows the evolution of the average velocity when the desired velocity changes. It must be pointed out that the used solution really allows to regulate the average velocity. We can observe that, after each modification of the desired velocity, the real velocity con-

Fig. 7. Evolution of the average desired and real velocities when the pitch angle of the trunk is controlled.

verges towards the desired velocity after around ten steps. The adjustment of the proportional Kp and derived Kv parameters are carried out in order to obtain a smooth transition velocity. Fig. 8 shows the actuator torque at the hip and at the knee during a complete cycle of walking (swing and stance phase). The swing phase is included between the stages 1 and 4, and the stance phase between the stages 4 and 8. During the active phases, the limitation of the torque and power are the constraints given in Section 2.3. At the stage 1, the torque at the hip is just an impulse. Between the stages 1 and 2, the torque at the hip is equal to zero. Between the stages 1 and 3, the torque at the knee is equal to zero too. At the stage 2, the hip is locked by using a PD control. And at the stage 3, this is the knee which is locked. It must be pointed that the locking of the knee implies a reaction at the hip. At the stage 4, there is impact with the ground. It is the end of the swing phase. After this impact, the trunk is destabilized and the torque at the hip is very large. Moreover, at this instant, an impulse torque is applied to the knee in order to propel the robot forward. During the stance phase, the dynamic coupling implies reaction forces on the stance leg due to motion of the swing leg: stage 5, Kpulse of the swing leg; stage 6, locking of the hip of the swing leg; stage 7, locking of the knee of the swing leg. At the stage 8, it is the end of the stance phase.

Fig. 6. Synoptic of the high level control for the average velocity.

88

C. Sabourin, O. Bruneau / Robotics and Autonomous Systems 51 (2005) 81–99

Fig. 8. Torques applied to the hip (continuous line) and the knee (dotted line) during a complete cycle of walking (swing and stance phases).

This control strategy allows us to produce the dynamic walking of the bipedal robot without reference trajectories and with an average velocity included in [0 m/s; 1 m/s]. The interest of this method resides on the fact that, on the one hand, the intrinsic dynamics of the system are exploited by using a succession of active and passive phases and, on the other hand, that the control strategy is very easy to implement on-line. But the use of passive phases implies that this control strategy is very sensitive to the external disturbances that could occur during the progression of the robot. Moreover, frictions of each articulation can be large on a real robot and consequently limit considerably the dynamics of the passive phases. Consequently, we propose to use a neural network allowing firstly to increase the robustness of our control strategy and secondly to compensate articular frictions on a real robot. The CMAC, the neural network that we have selected and its implementation are described in the following section.

4. Control strategy based on CMAC In this section, in the first step, the principle of the CMAC is presented. Sections 4.2 and 4.3, respectively, describe the phase of training and of the utilization of this neural network.

reduction of the training and computing times in comparison to other neural networks [28]. This is of course a considerable advantage for real-time control. Because of these characteristics, the CMAC is thus a neural network relatively well adapted for the control of complex systems with a lot of inputs and outputs and has already been the subject of some researches in the field of the control of biped robots [29,30]. The CMAC is an associative memory type neural network which is a set of N detectors regularly distributed on several C layers. The receptive fields of these detectors are distributed on the totality of the limited range of the input signal. On each layer, the receptive fields are shifted of a quantification step q. Consequently, the width of the receptive field are not always equal. The number of detectors N depends, on the one hand, on the width of the receptive fields and, on the other hand, on the quantification step q. When the value of the input signal is included in the receptive fields of a detector, this one is activated. For each value of the input signal, the number of activated detector is equal to the number of layers C (parameter of generalization). Fig. 9 shows a simplified organization of the receptive fields having 14 detectors distributed on 3 layers. Being given that there is an overlapping of the receptive fields, neighboring inputs will activate common detectors. Consequently, this neural network is able to carry out a generalization of the output calculation for inputs close to those presented during learning. The output Y of the CMAC is computed by using two mappings. The first mapping projects an input space point u into a binary associative vector A = [a1 , . . . , aN ]. Each element of A is associated with one detector and N is the number of detector. When one detector is activated, the element of A corresponding of this detector is 1 otherwise it is equal to 0. The second mapping computes the output Y of the network as a scalar product of the association vector A and the weight vector W = [w1 , . . . , wN ] (Eq. (12)): Y = A(u)T W

(12)

4.1. The CMAC neural network

4.2. Learning during a walking gait

The CMAC is a neural network imagined by Albus starting from the studies on the human cerebellum. Despite its biological relevance, its main interest is the

During the learning phase, we use the strategy developed in Section 3 to control the robot. Furthermore, the trajectories of the swing leg (in terms of joint po-

C. Sabourin, O. Bruneau / Robotics and Autonomous Systems 51 (2005) 81–99

89

Fig. 9. Description of the simplified CMAC with 14 detectors distributed on 3 layers. For each value of the input signal, the number of activated detector is equal to 3. A = [0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0], Y = A(u)T W = w3 + w8 + w12 .

sitions and velocities) are learned with several “singleinput/single-output” CMAC neural networks. Indeed, two CMAC are necessary to memorize the joint angles qi1 and qi2 and two other CMAC for angular velocities q˙ i1 and q˙ i2 . Fig. 10 shows the method used during this training phase. When leg 1 is in support (q12 = 0), the angle q11 is applied to the input of each CMAC (u = q11 ) and when leg 2 is in support (q22 = 0), this is the angle q21 which is applied to the input of each CMAC (u = q21 ). Consequently, the trajectories learned by the neural networks are not function of time but function of the geometrical pattern of the robot. The weights W of each CMAC k

(k = 1, . . . , 4) are given by Eq. (13): wik = wi−1 k +

β k ek Ck

(13)

and wik are, respectively, the weights before and wi−1 k after the training at each sample time ti . Ck is the generalization number of each CMAC and βk is a parameter included in [0, 1]. ek is the error between the desired output Ykd of the CMAC k and the computed output Yk of the CMAC. When the stance leg is leg number 1, Y d = [Y1d , Y2d , Y3d , Y4d ] = [q21 , q˙ 21 , q22 , q˙ 22 ] and when the stance leg is leg number 2, Y d = [Y1d , Y2d , Y3d , Y4d ] =

Fig. 10. Principle of the control strategy used during the training of one CMAC neural network.

90

C. Sabourin, O. Bruneau / Robotics and Autonomous Systems 51 (2005) 81–99

the receptive field is equal to 1.5◦ (qC). Consequently, the number of weigh is equal to 644. It is noticed that the choice of the step length q and the number of layers must be a compromise between precision and memory size. 4.3. Walking gait generation with the CMAC

Fig. 11. Evolution of the output of the CMAC during the training, where tj (j = 1, . . . , 6) are the instants of transition between one leg and the other (double instantaneous support phase).

[q11 , q˙ 11 , q12 , q˙ 12 ]. We have to mention that Y d are at the same time the output results of the simulation of the robot and the desired output of the CMAC that we try to obtain with the equation Y = A(u)T W of the CMAC. Furthermore, we consider that the trajectories of each leg in swing phase are identical. This makes it possible, on the one hand, to divide by two the number of CMAC and, on the other hand, to reduce the training time. Fig. 11 shows the evolution of the output of the CMAC and the desired output qi1 (alternatively q11 and q21 ) of a CMAC. One can note that after only some steps of the robot, the CMAC correctly learned the desired trajectory. In this simulation, the characteristics of each CMAC are identical. The quantification step q is equal to 0.25◦ and the number of layers C is equal to 6. The length of

After the training phase, the CMAC networks are used to generate the joint trajectories of the swing leg starting from the geometrical configuration of the stance leg. A PD control makes it possible to ensure the tracking of these trajectories. The control of the stance leg and the high level control, making it possible to control the average velocity of progression of the robot, are identical to that described in Section 3. Fig. 12 describes the control strategy used after the phase of training. Fig. 13 shows the evolution of the average velocity of the robot during and after the phase of training. At t = 7.5 s, the control strategy is modified: the CMAC networks are used to generate the joint trajectories. It should be noted that when the transition of control strategies occurs, the average velocity decreases. But the high level control (see Section 3.3) allows to readjust the pitch angle of the trunk and after some steps, the average velocity of the robot tends converges again towards that desired. After this modification of the control strategy, if the desired average velocity is modified, it can be noticed that the average velocity of

Fig. 12. Principle of the control strategy based on the use of the CMAC neural networks, when the leg 1 is in the stance phase.

C. Sabourin, O. Bruneau / Robotics and Autonomous Systems 51 (2005) 81–99

91

• The modification of the pitch angle of the trunk allows us to modify the average velocity of the robot. This one is controlled automatically by exploiting a high level control using the error between the desired and real value. In order to expose the capabilities of this strategy of control, Section 5 shows the behavior of the robot when it is subjected to external disturbances. Fig. 13. Evolution of the average desired and real velocity before and after the learning (phase of training is included in [0 s, 7.5 s]).

the robot converges towards this new desired value. In fact, one of the main advantage of this control strategy resides on the fact that the duration of the swing phase is automatically adapted to the average velocity of the robot (the step length is constant). Fig. 14 shows the actuator torque during one step. It must be noted that in this case, there is not really full passive phases but the two control strategies produce similar torques. In fact, the control strategy presented in this section can be summarized in three points: • The control of the swing leg is based on trajectories tracking. These trajectories are performed by using CMAC neural networks. The training of the CMAC is carried out during a dynamic walking achieved by a succession of passive and active phases. • The knee of the stance leg is locked. The control of the hip of the stance leg makes it possible to determine the pitch angle of the trunk. Consequently, the association of the stance leg and the trunk may be assimilated to an inverse pendulum during the phase of support.

5. Robustness of the robot during a walking gait If it is true that within the framework of the control strategies such as those presented in this article, the objective is not to prove the stability of the walking of the robot, it is however possible to evaluate the robustness of this control technique when the robot is subjected to environmental disturbances (rough terrain, perturbation forces, sliding). 5.1. Adaptation to irregular grounds As we specified in Section 1, the use of trajectories tracking techniques is not without posing problems on irregular ground. However, the surface on which the humanoid robots move are not always perfect. The control of the walking robot must be sufficiently robust to avoid the fall when the surface of the ground becomes slightly irregular. Thus, in order to evaluate our control strategy compared to this kind of disturbances, we tested the dynamic walking of our virtual robot on an irregular ground. Between t = 10 s and t = 20 s, at each new step, the height of the point of impact is computed in a random way between 0 and 15 mm with regard to an absolute reference. Fig. 15 represent our virtual robot moving on this irregular ground and Fig. 16 shows the average and instantaneous velocities of the robot. When the robot arrives on irregular ground (between t = 10 s and t = 20 s), the average velocity is disturbed. However, the robot does not fall and enters again in a normal cycle when there is no more disturbances. 5.2. Avoiding the fall of the robot subjected to an impulsive disturbing force

Fig. 14. Torques applied to the hip (continuous line) and the knee (dotted line) during a complete cycle of walking (swing and stance phases.)

During walking, a robot moving in an unknown environment can be subjected to external forces involving

92

C. Sabourin, O. Bruneau / Robotics and Autonomous Systems 51 (2005) 81–99

Fig. 15. Height of the ground in mm when a robot moves on irregular ground (between t = 10 s and t = 20 s).

an imbalance, the most critical stage being the single support phase. And if this force is too big, the robot can then fall. It is thus essential to be able to quickly react in order to find balance. Thus, in this simulation, we applied to our robot an external force along horizontal axis which pushes it in rear during its walking. At t = 12.0 s, we applied an impulse force with an amplitude of 100 N m and a duration of 0.2 s. The kinetic energy is insufficient to counter this disturbance, the instantaneous horizontal velocity decreases and becomes negative (Fig. 17). The robot falls then backwards. Fig. 18 shows the successive movements of the robot before and after this disturbance. The motion of the swing leg is then reversed, which makes it possible to stop the robot in a configuration of double support and thus to avoid the fall. This is the result of the trajectories used to generate the movement of the swing leg: they are not

functions of time but depend on the geometrical configuration of the stance leg with regard to the ground.

Fig. 16. Instantaneous horizontal velocity Vx (dotted line) and average velocity VM (continuous line) when a robot moves on irregular ground (between t = 10 s and t = 20 s).

Fig. 17. Instantaneous horizontal velocity (continuous line) and impulsive horizontal perturbation force (dotted line) (equal to −100 N when t ∈ [12 s, 12.2 s]) and 0 elsewhere.

5.3. Avoiding the fall of the robot subjected to sliding During the walking, the robots, in the same way as the humans beings, can be subjected to losses of adherence following a change of characteristics of the foot ground interaction. It is thus important to be able to evaluate the reliability of the walking of the robot during phases of sliding. The modeling of the interaction between the foot and the ground that we use in our simulations makes it possible to obtain phases of sliding followed by phase of adherence. Indeed, when the tangential force of contact is outside of the cone of friction, the tangential forces of contacts are calculated starting

C. Sabourin, O. Bruneau / Robotics and Autonomous Systems 51 (2005) 81–99

93

Fig. 18. Successive movements of the robot before and after the perturbation force.

We can observe that at the last step (stance leg is leg number 2) of the robot, the articular movements are reversed. As in the case of an external perturbation force (see Section 5.2), the robot does not fall down but returns in a stable configuration of double support. We have shown in this section the robustness of our control strategy in simulation. However, it is necessary to validate this approach with an experimentation on the real robot RABBIT. Section 6 presents the first experimental results obtained with our approach. Fig. 19. Walking of the robot with and without sliding. During the last step of the simulation, the leg number 2 is in the swing phase.

6. First experimental results

from Eq. (3). The duration of this phase of sliding then depends on the viscous damping coefficient µg . Fig. 19 shows the stick diagram of the walking of the robot with and without phases of sliding. When the sliding occurs, this one is insufficient to destabilize the robot which continues its walking. But at the following step, the slip is too large, the horizontal instantaneous velocity decreases, and the robot does not finish the step but stays stable. Figs. 20 and 21 show, respectively, the angle and the angular velocity on the leg number 2.

The objective of this section is to provide and analyze the main results obtained during the first experimental tests on the robot RABBIT. We will start by the presentation of preliminary simulations which we carried out to determine the weights of the CMAC neural networks which will be used to generate the trajectories of the swing leg on the real robot RABBIT. Finally, a comparison between the results obtained in simulations

Fig. 20. Angles q21 and q22 of the robot with and without sliding.

Fig. 21. Angular velocities q˙ 21 and q˙ 22 of the robot with and without sliding.

94

C. Sabourin, O. Bruneau / Robotics and Autonomous Systems 51 (2005) 81–99

and those obtained during experiments will be carried out. 6.1. Preliminary simulations Frictions being great on the real robot, it is not possible to carry out the training phase of the neural networks on RABBIT directly. Consequently, the learning phase was carried out in simulation with ADAMS. It should be noted that an update of the virtual model of RABBIT was necessary to take into account the final real dimensions of the trunk of the robot (weight of 12 kg and length of 20 cm). The initial reference gait that we retained are characterized by a step length of 0.4 m and an average velocity of 0.6 m/s. The main control variables used to obtain these trajectories are summarized in Table 2. The characteristics of each CMAC are identical to those used in Section 4.2 (q = 0.25◦ , N = 6). After the determination of these CMAC neural networks, they were used for the experimental tests. 6.2. Dynamic walking at 0.8 m/s After an initialization phase of the robot, it is possible to initiate the walking by pushing the robot forward. This force is necessary to break static balance in which the robot is initially located in position of stop. After some steps, the average velocity converges toward a constant value and the walking is stable (Fig. 22). The steps carried out seem to be natural and relatively regular. To stop the robot, it is necessary to apply a force on the trunk of the robot in the opposed direction of the progression. This force causes a loss of kinetic energy and the robot is then immobilized in a double support configuration. Table 2 Control parameters for a dynamic walking during the learning phase Gait control variables dst ◦ Desired knee joint angle (stance phase), qi2 ( ) dsw ◦ Desired knee joint angle (swing phase), qi2 ( ) Desired angle between the thighs (swing phase), dsw ◦ qr1 ( ) Pitch angle of the trunk, q0d (◦ ) imp Hip impulse torque, Khanche (N m)

Numerical value 0 −12 30 21 50

Fig. 23 shows the evolution of the average velocity and the step length during approximately 50 steps. The average velocity and the step length are, respectively, equal to 0.8 m/s and 0.4 m. It should be noted that there is a difference between the average velocity of the left leg and the right leg. This average velocity VM is calculated with Eq. (9). In fact, the radial bar measuring only 1.50 m, its length is insufficient to consider that the robot advances in the sagittal plane. Consequently, the movements of the inner leg and the movements of the outer leg have not the same velocity. Fig. 24 shows the evolution of the angles and angular velocities at the hip and the knee during five steps. 6.3. Comparison between simulation and experimentation The first observation which can be made is that the behavior of the robot is similar to that obtained in simulation. The walking obtained during our experimental tests is stable and regular: the geometrical configuration is namely identical at each double support phase. This is an important point because it characterizes the stability and the regularity of walking (nearly identical succession of steps). Furthermore, this confirms that our simulator is sufficiently realistic. However, it should be noted that, in spite of these good results, some points differ between simulation and experimentation: • Firstly, the average speed of progression obtained during this experimental test (0.8 m/s) is different from that obtained in simulation (0.6 m/s) in spite of the same choice of control parameters (pitch angle). In our opinion, this is explained by the fact that, on the one hand, the impacts calculated with our foot/ground interaction model are overestimated and, on the other hand, the total dynamics of the robot is different because of the great friction. • Secondly, the trajectories tracking is not exact at all. Movements of the swing leg being fast and friction being large, the PD control does not allow, in this case, to ensure a good trajectories tracking (Fig. 25). Moreover, during this experimental test, the torques were voluntarily limited to 70 N m. • Thirdly, the actuator torques are different from those obtained in simulation and the succession of

C. Sabourin, O. Bruneau / Robotics and Autonomous Systems 51 (2005) 81–99

95

Fig. 22. Successive pictures of RABBIT during the walking gait.

active and passive phases is also not performed (Fig. 26). Consequently, the good results obtained during simulations concerning the trajectories tracking are not completely reproducible on the real robot. However, it should be noted that this does not involve destabilization of the robot. Indeed, before each double support

phase, the error between the desired and real trajectories becomes relatively weak (t = 32 s; Fig. 25). The legs configuration of the robot are thus always identical before each new step (quasi-instantaneous double support stage). In fact, the amplitudes of the movements are less great, but the swing time of the hip and the knee are always lower than the duration of the step. The respect of these conditions is one of the fundamental

96

C. Sabourin, O. Bruneau / Robotics and Autonomous Systems 51 (2005) 81–99

Fig. 23. Step length and average velocity of the robot RABBIT during approximately 50 steps.

characteristics to ensure the stability of the robot and also proves that a control strategy which is not based on a rigorous trajectory tracking makes it possible to obtain a stable and robust walking. Section 7 deals with a discussion concerning our pragmatic approach compared with more theoretical approach used by other researcher.

Fig. 24. Measured angles and angular velocities at the hip (q11 , q˙ 11 ) and the knee (q12 , q˙ 12 ) during five steps of the robot.

7. Main interests of a pragmatic approach to control bipedal robots Our control strategy is an alternative solution of that presented by Grizzle et al. [31]. And if this theoretical approach seems to be rigorous, it is necessary to stress some points: • The Hybrid Zero Dynamics (HZD) makes it possible to numerically prove stability under some condi-

Fig. 25. Trajectories tracking when the robot is walking at 0.8 m/s.

C. Sabourin, O. Bruneau / Robotics and Autonomous Systems 51 (2005) 81–99

97

Fig. 26. Experimental torques when the robot is walking at 0.8 m/s.

tions which are very restrictive. Indeed, the model of foot/ground interaction used makes the assumption of rigid impacts. And, in the case of a more realistic foot/ground interaction model (compliant contact), the stability cannot be theoretically proven; it can be only shown by using simulation or experimentation. Moreover, the design of a running robot will require the integration of shock absorber and the assumption of perfectly rigid robot limbs is not valid in this case. • The trajectories are computed off-line. Consequently, it is necessary to compute and memorize the twenty parameters of the four polynomials functions for each average velocity and for each step length. And in the case of an on-line calculation, the computing time will be significant for real-time control. • The extension of this strategy to control robot with many Degrees Of Freedom (DOF) and three dimensional walking gaits seems to be much more difficult. Furthermore, the HZD is not operational for robots with feet (double support phase in not instantaneous).

• The stability of the walking robot is carried out only if the feedback control allows to get equal to zero the outputs imposed by the virtual constraints. Consequently, this approach is similar, in a certain manner, of a traditional trajectories tracking method. Our approach is radically different concerning the following points: • We try to seek to understand the principles of human walking based on a pragmatic analysis. • We consider that this stable dynamic walking can be reproduced by using a simple control strategy. In this case, the motions are not the result of a rigorous trajectories tracking: they are produced by a synergy between intrinsic dynamics and active control. A succession of almost identical steps is then carried out. • We search the means which make it possible to improve the robustness of this approach and to increase its capabilities of adaptation.

98

C. Sabourin, O. Bruneau / Robotics and Autonomous Systems 51 (2005) 81–99

If it is true that the stability of the walking robot is not theoretically proven, we have shown that the results obtained during the experimentations were very close to those obtained during the simulations (see Section 6). In our approach, we privilege the realism of the simulations compared to a purely mathematical analysis because the HZD does not make it possible to evaluate stability with regard to instantaneous disturbances unexpected by the robot. In addition, we consider that only the experimental approach makes it possible to confirm completely the validity of simulations. The interest of our approach are: • The strategy used to control only 4 DOF is very simple. Consequently, if the number of controlled DOF increases, our approach seems to be still available for real-time control. • This control strategy is very robust and its adaptation capabilities are important because of the use of neural networks.

8. Conclusion and further developments In this paper, we proposed a control strategy allowing us to perform the dynamic walking of a virtual under-actuated robot. This control strategy was based on two stages. The first one consists of using a set of pragmatic rules in order to generate a succession of passive and active phases allowing us to perform the dynamic walking of the robot. This first stage is used as a reference control to learn, by using neural networks, a set of articular trajectories. In the second stage, we use these neural networks to generate the trajectories learned during the first stage. This approach allowed us to walk on irregular grounds and to increase the robustness of the control of the walking of this robot subjected to sliding or disturbing forces. In addition, the first experimental results prove that our control strategy is usable on the real robot. Future work will focus on fast walking, running and the modification of the step length on-line.

References [1] Y. Sakagami, et al., The intelligent ASIMO: system overview and integration, Proc. IEEE Conf. Robot. Autom. (2002) 2478– 2483.

[2] F. Kaneko, et al., Humanoid robot HRP-2, Proc. IEEE Conf. Robot. Autom. (2004) 1083–1090. [3] S. Kajita, et al., Biped walking pattern generation by using preview control of Zero-Moment Point, Proc. IEEE Conf. Robot. Autom. (2003) 1620–1626. [4] K. Hirai, M. Hirose, Y. Haikawa, T. Takensa, The development of Honda humanoid robot, Proc. IEEE Conf. Robot. Autom. (1998) 1321–1326. [5] H. Lim, A. Takanishi, Online walking pattern generation for biped humanoid robot with trunk, Proc. IEEE Conf. Robot. Autom. (2002) 3111–3116. [6] K. Loffler, M. Gienger, F. Pfeiffer, Sensor and control design of a dynamically stable biped robot, Proc. IEEE Conf. Robot. Autom. (2003) 484–490. [7] Q. Huang, et al., Balance control of a biped robot combining off-line pattern with real-time modification, Proc. IEEE Conf. Robot. Autom. (2000) 3346–3352. [8] C. Avezedo, the BIP team, Control architecture and algorithms of the anthropomorphic biped BIP2000, Proc. Climbing Walking Robots (2000) 285–293. [9] M. Vukobratovic, B. Bocovac, D. Surla, D. Stokic, Biped Locomotion, Scientific Fundamentals of Robotics, vol. 7, SpringerVerlag, 1990. [10] A. Goswami, Foot rotation indicator (FRI) point: a new gait planning tool to evaluate postural stability of biped robots, Proc. IEEE Conf. Robot. Autom. (1999) 47–52. [11] J. Foret, O. Bruneau, J.-G. Fontaine, Unified approach for mstability analysis and control of legged robots, Proc. IEEE Conf. Intell. Robots Syst. (2003) 106–111. [12] A. Goswami, B. Espiau, A. Keramane, Limit cycles and their stability in passive bipedal gait, Proc. IEEE Conf. Robot. Autom. (1996) 246–251. [13] M.S. Garcia, A. Chatterjee, A. Ruina, Efficiency, scaling, and speed in two dimensional passive-dynamic walking, Dyn. Stabil. Syst. 15 (2) (2000) 75–99. [14] C. Chevallereau, A. Formal’sky, B. Perrin, Low energy cost reference trajectories for a biped robot, Proc. IEEE Conf. Robot. Autom. (1998) 1398–1404. [15] F. Yamasaki, K. Hosoda, M. Asada, An energy consumption based control for humanoid walking, Proc. IEEE Conf. Robot. Autom. (2002) 2473–2477. [16] J. Pratt, G. Pratt, Virtual model control: an intuitive approach for bipedal locomotion, Int. J. Robot. Res. 20 (2) (2001) 129– 143. [17] C. Sabourin, O. Bruneau, J.-G. Fontaine, Intuitive and fuzzy control of and under-actuated robot without references trajectories, in: Proceedings of the 11th World Congress in Mechanism and Machine Science, 2004, pp. 1736– 1740. [18] M.A. Lewis, L.S. Simo, Certain principles of biomorphic robots, Auton. Robots 11 (3) (2001) 211–216. [19] J. Hu, J. Pratt, G. Pratt, Stable adaptative control of a bipedal walking robot with CMAC neural networks, Proc. IEEE Conf. Robot. Autom. (1999) 1050–1056. [20] W. Salatain, K.Y. Yi, Y.F. Zheng, Reinforcement learning for a biped robot to climb slopping surfaces, J. Robot. Syst. 14 (4) (1997) 283–296.

C. Sabourin, O. Bruneau / Robotics and Autonomous Systems 51 (2005) 81–99 [21] C. Chevallereau, P. Sardain, Design and actuation of a 4 axes biped robot for walking and running, Proc. IEEE Conf. Robot. Autom. (2000) 3365–3370. [22] http://www.robot-rabbit.lag.ensieg.inpg.fr/. [23] C. Sabourin, O. Bruneau, J.-G. Fontaine, Start, stop and transition of velocities on an under-actuated bipedal robot without reference trajectories, Int. J. Humanoid Robot. 1 (2) (2004) 349–374. [24] J.S. Albus, A new approach to manipulator control: the cerebellar model articulation controller (CMAC), Trans. ASME (1975) 220–227. [25] J.S. Albus, Data storage in the cerebellar model articulation controller, J. Dyn. Syst. Meas. Control (1975) 228–233. [26] http://www.laas.fr/robea/. [27] O. Bruneau, F.B. Ouezdou, Distributed ground/walking robot interactions, Robotica 17 (3) (1999) 313–lastpagenumber323. [28] W.T. Miller, F.H. Glanz, L.G. Kraft, CMAC: an associative neural network alternative to backpropagation, IEEE Proc. 78 (1990) 1561–1567. [29] A.L. Kun, W.T. Miller, Adaptive dynamic balance of a biped robot using neural networks, Proc. IEEE Conf. Robot. Autom. (1996) 240–245. [30] A. Brenbrahim, J. Franklin, Biped dynamic walking using reinforcement learning, Robot. Auton. Syst. 22 (1997) 283–302. [31] J.W. Grizzle, G. Abba, F. Plestan, Asymptotically stable walking for biped robots: analysis via systems with impulse effects, IEEE Trans. Autom. Control 46 (1) (2001) 51–64.

Christophe Sabourin received his MS degree from the University of Poitiers in 1993. Since 1995, he is assistant professor in design of automated production systems at the College of Technology of Bourges. He received his PhD degree from the University of Orl´eans in 2004. During this thesis, at the LVR (Laboratoire Vision et Robotique), in ENSIB (Ecole Nationale Sup´erieure d’Ing´enieurs de Bourges), he worked on the project RABBIT within the framework of a CNRS project (Centre National de la Recherche Scientifique) called ROBEA (robotics and artificial entities). The objective of Christophe Sabourin is to develop a new strategy of real-time control allowing to generate a stable dynamic walking of the biped robot. His research interests include the study of the dynamic biped walking, soft computing and bio-inspired control.

99

Olivier Bruneau received his MS and PhD degrees from the University Pierre et Marie Curie, Paris VI, in 1994 and 1998, respectively. From 1998 to 1999, he was researcher at the LRP (Laboratoire de Robotique de Paris) in the field of dynamic bipedal robots. Since 1999, he is associate professor in robotics and theoretical mechanics at the ENSI de Bourges. Olivier Bruneau is the author of over 20 technical publications concerning the dynamic walking systems. His research interests include dynamics of locomotion, humanoid robots and learning methods of walking for the bipedal robots. Olivier Bruneau currently works at the LVR (Laboratoire Vision et Robotique). He is an active member of the research group working on the control of the biped robot called “RABBIT” for walking and running within the framework of a ROBEA CNRS project. He works on the anthropomorphic biped robot called ROBIAN in collaboration with the Laboratory LIRIS (Laboratoire d’Instrumentation et de Relation Individu Syst`eme). He was a co-organizer of the 12th International Symposium on Measurement and Control in Robotics, Towards Advanced Robot Systems and Virtual Reality, taking place in Bourges from 20 to 22 June 2002.