Exp Brain Res (1999) 128:92–100

© Springer-Verlag 1999

R E S E A R C H A RT I C L E

David A. Rosenbaum · Ruud G. J. Meulenbroek Jonathan Vaughan · Chris Jansen

Coordination of reaching and grasping by capitalizing on obstacle avoidance and other constraints

Received: 20 September 1998 / Accepted: 10 March 1999

Abstract Reaching and grasping an object can be viewed as the solution of a multiple-constraint satisfaction problem. The constraints include contact with the object with the appropriate effectors in the correct positions as well as generation of a collision-free trajectory. We have developed a computational model that simulates reaching and grasping based on these notions. The model, rendered as an animation program, reproduces many basic features of the kinematics of human reaching and grasping behavior. The core assumptions of the model are: (1) tasks are defined by flexibly organized constraint hierarchies; (2) manual positioning acts, including prehension acts, are first specified with respect to goal postures and then are specified with respect to movements towards those goal postures; (3) goal postures are found by identifying the stored posture that is most promising for the task, as determined by the constraint hierarchy, and then by generating postures that are more and more dissimilar to the most-promising stored posture until a deadline is reached, at which time the best posture that was found during the search is defined as the goal posture; (4) depending on when the best posture was encountered in the search, the deadline for the search in the next trial is either increased or decreased;

(5) specification of a movement to the goal posture begins with straight-line interpolation in joint space between the starting posture and goal posture; (6) if an internal simulation of this default movement suggests that it will result in collision with an obstacle, the movement can be reshaped until an acceptable movement is found or until time runs out; (7) movement reshaping occurs by identifying a via posture that serves as a body position to which the actor moves from the starting posture and then back to the starting posture, while simultaneously making the main movement from the starting posture to the goal posture; (8) the via posture is identified using the same posture-generating algorithm as used to identify the goal posture. These processes are used both for arm positioning and, with some elaboration, for prehension. The model solves a number of problems with an earlier model, although it leaves some other problems unresolved. Key words Reaching movements · Grasping movements · Prehension · Manual control · Computational model · Human

Introduction This work was presented at the conference, “Neural Basis of Hand Dexterity,” in Ascona, Switzerland, on 3–8 May, 1998. D. A. Rosenbaum (✉) 642 Moore Building, Department of Psychology, Pennsylvania State University, University Park, Pennsylvania 16802, USA, e-mail: [email protected] (Messages), [email protected] (Attachments) R.G.J. Meulenbroek (e-mail: [email protected]) C. Jansen (e-mail: [email protected]) Nijmegen Institute for Cognition and Information, University of Nijmegen, P.O. Box 9104, NL-6500 HE, Nijmegen, The Netherlands J. Vaughan Department of Psychology, Hamilton College, Clinton, New York 13323, USA, e-mail: [email protected]

To understand how a system works, one can observe it in action or explore its physical and physiological substrates. A complementary approach is to use simulation, embodying in one’s model the principles one believes may underlie the system’s operation. We have pursued the latter approach in the study of reaching and grasping because we believe simulation has several desirable properties: (1) it forces one to confront issues that might go unnoticed if one were not building the system from scratch; (2) rendering the model on a computer forces one to be completely explicit about one’s assumptions and parameters; (3) limitations of the simulation generally become quickly apparent when one is working with a computer, and one then has a medium for exploring means of overcoming the limitations; (4) even if the

93

simulation fails to predict in detail actual (biological) behavior, it may still be useful for other purposes (e.g., robotics or computer animation). In earlier work, we modeled manual pointing behavior (Rosenbaum et al. 1995). The theoretical issue we considered was the inverse-kinematics problem: when many possible postures allow a point along the limb segment chain to contact a specified point in external space, how is one of those postures chosen? The inverse-kinematics problem is an instance of the degrees-of-freedom problem, made famous by Bernstein (1967). The degrees-of-freedom problem arises whenever there is redundancy in a system. For example, it comes up when there are many ways of generating forces with an endeffector (the inverse dynamics problem). In vision, it arises when an infinite number of three-dimensional interpretations are possible for a two-dimensional retinal image (the retinal mapping problem). The model of Rosenbaum et al. (1995) focused on the task of arriving at any reachable location in the sagittal plane, defined by its horizontal and vertical components, with any specified point along the limb segment chain. Rosenbaum et al. explored this task using a stick figure capable of bending at the hip, shoulder, and elbow, such that bending any of these joints changed the angular position of the corresponding joint in the sagittal plane. Because there were three degrees of freedom in the stick figure and only two in the location of each target, whenever a target within the workspace had to be reached, the inverse kinematics problem remained to be solved. To solve the inverse kinematics problem, Rosenbaum et al. (1995) assumed that movements are specified both to satisfy explicit costs (getting the limb to where it needs to go) and to contain implicit costs. The notion that implicit costs are contained in movement can be traced to optimal-control theory (Bryson and Ho 1975) and has been used in motor control (e.g., Flash and Hogan 1985; Kawato 1996; Nelson 1983; Soechting et al. 1995; Uno et al. 1989). A key difference between the model of Rosenbaum et al. (1995) and the models just referred to is that Rosenbaum et al. assumed that goal postures are generally planned before movements, with implicit costs playing a role both in the planning of goal postures and in the planning of movements. The rationale for assuming that goal postures are generally planned before movements was that this method facilitates decision-making: if movements are specified first, one has to mentally “run through” the movements to see where they end up. By contrast, if goal postures are specified first based on how well they satisfy explicit and implicit task demands, once the goal postures are chosen, movements can be generated to them from the starting posture via simple interpolation. If one adopts this theoretical framework, one must ask what costs should be contained and how particular goal postures and movements should be selected. Rosenbaum et al. (1995) allowed for the possibility that more weight can be given to the containment of some costs than to others depending on the task to be performed.

They also allowed that a goal posture can be identified by taking a weighted sum of stored-posture vectors, where the weights assigned to those vectors (hereafter, the “stored postures”) depend on their relative suitability for the task. The relative suitability of each stored posture was indexed by the cost it would incur if adopted. This cost in turn was based on the sum of a “spatial error cost” (i.e., the distance between the contact point and the spatial target if a stored posture were adopted) and a “travel cost” (i.e., the cost of traveling from the starting posture to the stored posture under consideration).1 Weights associated with the spatial error cost and travel cost depended on the relative importance of minimizing each cost. Once a goal posture was found, a movement to the goal posture was generated by simple interpolation. The model of Rosenbaum et al. (1995) was successful in several respects. A stick figure embodying the model could reach any location in the workspace from any other location in the workspace and could do so from any starting posture. The stick figure could reach target positions even if one of its three joints was locked or, as seen in Fig. 1, if one its joints changed its ease of movement. This flexibility was possible because stored postures that entailed displacements of joints that had high expense factors (including infinitely high expense factors when the joint was unmovable) were assigned lower weights than stored postures that had low expense factors. The stick figure could also move at different speeds and, as reported in the literature (see Rosenbaum 1995, for a review), arrived at different postures when different movement speeds were used. The latter effect was obtained because joints for large limb segments were assumed to incur smaller travel costs when they moved at slow speeds than when they moved at high speeds, and vice versa for joints of small limb segments; supporting evidence was reported in Rosenbaum et al. (1991). Given this assumption, stored postures requiring quick displacements of small limb segments and stored postures requiring slow displacements of large limb segments were as1 The travel cost associated with a stored posture increased with the weighted sum of the angular displacements needed to reach that stored posture from the starting posture. The weight assigned to each joint’s angular displacement depended on the “expense factor” of the corresponding joint, which was defined as the rate at which the joint’s optimal movement time grew with the logarithm of the joint’s angular displacement. The expense factor for a joint was assumed to increase with the mass and moment of inertia of the limb segment moved by that joint. The travel cost index was extended to the case where individual joints did not cover their respective angular displacements in their optimal movement times. For example, if all the joints started and stopped together – the case considered in greatest detail by Rosenbaum et al. (1995) – it was possible to find an optimal common time that minimized what may here be called W, the sum of the weighted deviations of the common time from the optimal movement times for the individual joints, where the weights are again the joints’ expense factors. The travel cost for the entire movement then increased with the weighted sum of the angular displacements, as described above, at a rate that increased with W. Note that the model did not address the question of how staggering of joint motions influences travel cost. This issue remains to be investigated.

94

Fig. 1 Reaching from the same starting position (torso erect with the hand extended above and beyond the knee) to a target in front of the shin (the ring) when the hip, shoulder, and elbow could all move with equal ease (left panel) or when the expense factor for the hip was 100 times greater than the expense factor for the shoulder or elbow (right panel)

signed high weight. Yet another result that the model could account for was that final postures depend on starting postures, as shown behaviorally (e.g., Fischer et al. 1997). The model accounted for this result in the same way: stored postures whose travel cost from the starting posture were high were assigned lower weight than stored postures whose travel cost from the starting posture were low. Finally, in an extension of the model, it was shown that the model could permit writing or drawing with any part of the body (Meulenbroek et al. 1996). The model of Rosenbaum et al. (1995) ran into some problems, however, as listed briefly here: (1) a large number of stored postures was needed to achieve weighted sums that adequately spanned the workspace; (2) an ad hoc procedure (“feed forward correction”) was needed to cope with the fact that the weighted summing of postures sometimes caused the hand (or other goal-posture contact point) to be farther from spatial targets than it was in the case of the best stored posture; (3) strong assumptions were needed to devise a way of assigning weights to stored postures prior to weighted summing of stored postures; (4) a problem with assigning weights to postures was that it was impossible to assign high weight to all constraints, so long as the weights summed to one and there was no dummy weight introduced for no reason other than to solve this problem – as a result, there was no way to minimize spatial error and travel cost; (5) the method lacked a means of coping with obstacle avoidance; (6) when spatial targets were not specified for all the contact points of the stick figure (e.g., in a task such as “grab the cup with your fingers”), the task could not be performed.

Methods and Results A new model To rectify these problems, we have developed an improved model. A main idea of the improved model is that goal postures are not specified by weighted summing of stored postures. Rather, goal postures are specified through a two-stage process of stored-posture retrieval followed by new-posture generation. During storedposture retrieval, stored postures, which are defined as

Fig. 2 Overview of the new model. The large dashed-lined box surrounds the model’s planning processes. Within the dashed-line box, the two top solid-lined boxes refer to the events involved in finding a goal posture

the last m-adopted goal postures (m≥1), are evaluated for their suitability for the current task (see Fig. 2). (In the simulations to be presented here, the value of m was set to 8.) Suitability of goal postures is defined with respect to a constraint hierarchy – that is, a prioritized set of requirements established by the actor to specify the explicit and implicit requirements of the current task. The stored posture judged most suitable according to the constraint hierarchy is selected as the most promising goal posture. Once this most promising goal posture is found, new postures around it are generated until a deadline is reached (see below). These new, internally generated postures are evaluated to see whether they are even better for the current task than the most promising stored posture. The way postures are internally generated is through a diffusion process centered on the most promising stored posture. The diffusion process proceeds in joint-angle space (i.e., a space whose axes are the mechanical degrees of freedom of the modeled system) until the deadline is reached, at which time the best posture of all those considered is selected as the goal posture; “best” is defined according to the constraint hierarchy. The deadline is specified as follows. If the posture ultimately identified as the goal posture was identified be-

95

fore the deadline was reached, the deadline is reduced for the next trial, provided the deadline is greater than an acceptable minimum; the rationale is that time was wasted after the best posture was found. However, if the goal posture was identified at the deadline or if no suitable goal posture was found, the deadline is increased for the next trial, provided the deadline is less than an acceptable maximum; the rationale here is that it might have helped to spend more time searching for a goal posture. On the very first trial, the deadline is chosen randomly from within the minimum-to-maximum range. We call the entire method of adjusting deadlines dynamic deadline setting. The logic behind it is that the greater the similarity of the most promising stored posture to the optimal goal posture for a task, the less search time will be needed. Hence, if a task has been practiced a great deal recently, many stored postures for the task will be available, so even if the task changes a little (as typically occurs), planning time will continue to decrease. The preceding discussion concerned the method for selecting goal postures. Once a goal posture is found, a movement to it is generated, as in the previous model, except that some additional mechanisms are included to allow for obstacle avoidance (addressing the fifth limitation of the previous model) and to allow for grasping (addressing the sixth limitation of the previous model). The additional mechanisms for movement will be described below. Before turning to them, it is worth pointing out that the new model also solves the other limitations of the earlier model. Regarding the second, third, and fourth limitations, which concerned weighting, the diffusion process eliminates the need for any weighting of stored postures. Regarding the first limitation, which concerned the large number of stored postures, no stored postures are needed in the new model, because the search for a goal posture can, in principle, proceed through diffusion from the current starting posture or, for that matter, from some random posture. Even though stored postures are not strictly necessary, there is still a rationale for continuing to have them, however, for as mentioned above, having stored postures can lead to reduced planning times for practiced tasks. Obstacle avoidance How does the new model solve the obstacle avoidance problem? The constraint hierarchy contains an obstacleavoidance constraint. Evaluation of the possibility of collision occurs by checking for spatial overlap of the body with the object to be avoided. This check for spatial overlap occurs both when goal postures are sought and then during a movement evaluation process, which occurs in the following manner. First, the simple interpolation movement (the “main movement”) between the starting posture and the goal posture is internally generated and a subset of the postures that would be adopted during the movement is checked. The smaller the number of postures that are checked, the lower the likelihood that a pos-

Fig. 3 Superposition of a back-and-forth movement onto a main movement. Top panel Joint angle as a function of normalized movement time for one joint. The main movement (empty circles) from the start angle to the goal angle is combined with an extra movement (empty squares) from the start angle to a virtual via angle and then back to the start angle, yielding the superposed movement (filled circles) from the start angle through an actual (i.e., observed) via angle to the goal posture. Bottom panel Joint velocity as a function of normalized movement time for the same set of movements; symbols same as above

sible collision will be detected. Assuming that making more checks takes more time, this dependency predicts a speed-accuracy tradeoff for obstacle avoidance, as might be expected for human performers. If no collision is anticipated, the movement is performed, but if a collision is

96

anticipated, an alternative movement is sought. The way an alternative movement is sought is by searching for a suitable “via posture.” The via posture is a posture that serves as a subgoal body position for a back-and-forth movement, which is superimposed on the main (simpleinterpolation) movement (see Fig. 3). For a reaching task in which an obstacle must be avoided, the main movement connects the starting posture to the goal posture, such that all the joints participating in the main movement start and end their movements together and have bell-shaped angular velocity profiles. The back and forth movement consists of angular displacements of joints whose via-posture positions differ from their starting-posture positions. All such joints move from their startingposture angles to their via-posture angles and then back to their starting-posture angles, and all the joints that participate in the back-and-forth movements start and stop their movements together and use bell-shaped angular velocity profiles, simply for reasons of convenience. The main rationale for the movement superposition method is that whenever joints move to a via posture and back to a starting posture while a main movement is under way, the goal posture that is finally reached is the same as the goal posture that is reached when no extra movement is made; only the movement path differs, and this can be helpful for circumventing an obstacle. The idea that movements can be added can be traced to studies of limb oscillation Fig. 5A–D Kinematics of the movement shown in Fig. 4. A Normalized angular position of the wrist, elbow, and shoulder as a function of normalized movement time. B Normalized angular velocity profiles of the wrist, elbow, and shoulder as a function of normalized movement time. The dashed line in each panel corresponds to zero, up is positive, and down is negative. C Angle-angle graphs for each pair of joints. The minimum and maximum of each axis corresponds to the smallest and largest angle achieved by the named joint. All joints moved from their smallest angles to their largest angles, as indicated by the arrows along the axes, indicating directions of angular motion. Points generated at equal time intervals. D Normalized tangential velocity of the wrist as a function of normalized movement time

Fig. 4 Movement around an obstacle. The standing stick figure could bend the shoulder, elbow, and wrist, and moved from its starting position (arm extended with “hand” maximally to the right) to any location within the target region (the empty circle) without hitting the obstacle (the gray circle)

(Denier van der Gon and Thuring 1965; Feldman 1980; Hollerbach 1981; von Holst 1979) and to studies of movement correction (Flash and Henis 1991; Henis and Flash 1992), not to mention Fourier’s initial insights in mathematics. In our simulations, we found that superposing back-and-forth movements onto main movements led to more natural composite behaviors than did chaining individual movements from starting postures to via postures and then to goal postures.

97

Inquisitive readers will wonder at this stage how via postures are determined. Recall that, after the goal posture is determined, the main movement to it is internally generated and postures that would be adopted during that movement are checked for possible collision. If and only if a collision is anticipated, a search is begun for a via posture. The search proceeds in basically the same way as the search for a goal posture. Stored postures are evaluated for their suitability as a via posture. A most-promising via posture is found from among the stored postures, with the most-promising via posture being defined with respect to a via-posture constraint hierarchy, whose elements are, most importantly, collision avoidance, and, secondarily, travel-cost reduction. Once the most-promising via posture is found, other possible via postures are generated around it (in joint space) until a deadline is reached. When the deadline is reached, the best via posture that was found up to that time is selected as the via posture for the actual movement. A simulation showing movement to a target with an obstacle in the way is shown in Fig. 4. Corresponding kinematics are shown in Fig. 5. Qualitatively similar kinematics have been reported in studies of manual obstacle avoidance (Dean and Bruwer 1994; Rosenbaum et al. 1997; Saling et al. 1998; Schneider et al. 1989). The qualitative success of the model is attributable in part to its use of posture-based motion planning, which represents external obstacles and the limbs in a unified space; obstacles are represented as “bad” postures (LozanoPérez 1984).The model’s success with obstacle avoidance is also due to the movement-superposition scheme we have adopted. Previous attempts to model the kinematics of manual obstacle avoidance behavior have been largely unsuccessful, as noted by Bock et al. (1993). Movement superposition provides a way of achieving kinematically realistic obstacle circumvention because, among other things, it yields joint angular-velocity profiles which pass through zero crossings with constant acceleration. It is notable that movement superposition has recently been hypothesized to underlie performance in the task of switching as quickly as possible from one movement to another when a new target suddenly appears (Flash and Henis 1991; Henis and Flash 1992). Subjects’ reliance on movement superposition in this somewhat artificial situation may indicate that actors use movement superposition in the more common task of circumventing obstacles. Prehension As the title of this article suggests, we believe it is possible to model the coordination of reaching and grasping by capitalizing on obstacle avoidance as well as other constraints. We turn now to our modeling of prehension. Reaching for an object to be grasped entails obstacle avoidance as well as target attainment. When one wants to reach out and grab an object, the action can only succeed if no extraneous collisions occur. Recognizing that

prehension involves obstacle avoidance as well as object enclosure provides a basis for modeling prehension behavior. The way we have modeled prehension is to build on the idea of the constraint hierarchy. The goal posture of the hand and arm must be specified such that the hand encloses the object, and the arm must be positioned such that enclosure of the object by the hand is possible. Furthermore, the movement from the starting posture to the goal posture must permit a smooth, collision-free trajectory. Finally, the planning process must be able to deal with the fact that the task description for prehension is typically ill-posed. That is, when one decides to “grab a cup,” exact spatial targets are not specified for the individual fingers. Recall that the earlier model of Rosenbaum et al. (1995) could not deal with this problem. In the new model, several constraints are invoked for the goal postures of the hand and arm when an object is going to be grasped. For the task of taking hold of a round object with a precision grip (the only prehension act we have modeled so far), the hand must take hold of the object such that the fingers contact the object, the distance between the index finger and thumb must approximate the diameter of the object, and the tangents made by the index finger and thumb when they contact the object must be approximately parallel. Secondarily, the travel costs of the fingers during the movement of the hand from its starting posture to its goal posture must be as small as possible. An arm goal posture must also be found which permits the hand to adopt its goal posture. Thus, the arm’s goal position must permit the wrist to end up at a distance from the center of the object that allows the hand to adopt its goal shape. In addition, when the arm is in its goal posture, it must not collide with an obstacle. Of less importance, the movement of the arm from its starting posture to its goal posture must reduce travel cost as much as possible. Finally, the movement to the goal postures of the hand and arm must avoid collisions. The reason for specifying the goal hand posture before the goal arm posture is that the spatial target requirement for the arm (where to place the wrist) cannot be specified until the posture of the hand is established. In this connection, it is interesting that evidence has been presented that hand and arm postures are represented separately in the brain (Rizzolatti 1987; Rizzolatti et al. 1988). Moreover, it has been inferred from reaction-time data that hand postures are specified before arm postures (Klatzky et al. 1995). Our method for achieving prehension is basically the same as our method for achieving hand positioning without a final grasp (discussed earlier, primarily in the context of obstacle avoidance). The search for a goal hand posture consists first of a search for a most promising stored hand posture; then hand postures are generated around the most promising stored hand posture until a dynamically set deadline is reached. The search for an arm posture similarly consists of a search for a most-

98

Fig. 6A–D Simulated prehension behavior. A Reach-and-grasp movement. B Shoulder, elbow, and wrist angular velocities as a function of time. C Tangential wrist velocity (wrist speed) and aperture (distance between the tips of the thumb and index finger) as a function of time. D Angular velocities for joints responsible for bending the index finger and thumb as a function of time

promising stored arm posture, and, then, arm postures are generated around it until a dynamically set deadline is reached. After a hand goal posture and arm goal posture are found, a movement to the combined hand-arm posture is internally simulated to check for collisions. If any collisions are found, a search is carried out for a via posture in integrated hand-and-arm space within which movement collisions involving the hand and/or arm can be anticipated. One simulated reach-and-grasp movement is shown in Fig. 6. Viewed in real time, the animation looks realistic. The lifelike appearance of this and the other simulations we have generated is borne out by detailed comparisons of the model’s kinematics and the observed kinematics of human prehension. As shown in Fig. 6, the fingers spread apart wider than the object, and then they close in on the object, as seen in real behavior. This aspect of simulated performance is attributable to the obstacle-avoidance constraints used in the model. The maximum aperture occurs during the second half of the movement, as is typical in human reaching (Castiello 1996; Jeannerod 1981, 1984; Wallace and Weeks 1988). This outcome is due both to the obstacle avoidance and travel-cost constraints. As the fingers spread apart, the arm speeds up and then begins to slow down, taking less time for speeding up than for slowing down, as has been

seen behaviorally (Castiello 1996; Jeannerod 1981, 1984; Wallace and Weeks 1988). This outcome is also ascribable to the obstacle avoidance and travel-cost-reduction constraints of the model. Other features of prehension are also predicted. The model predicts that maximum finger aperture grows with object size, but with a slope less than one, as reported in human studies (Bootsma et al. 1994; Brenner and Smeets 1996; Chiefi and Gentilucci 1993; Goodale et al. 1994; Marteniuk et al. 1990; Paulignan et al. 1991; Servos et al. 1992; Zaal and Bootsma 1993). This outcome can be ascribed to obstacle avoidance: to prevent collision with the object to be grasped, the fingers generally widen more than necessary, but this extra widening need not be incremented as much as an increment in the size of the object if the fingers already clear the object easily before closing in on it. Another prediction of the model is that maximum aperture comes later in movement the larger object to be grasped, again as observed in human performance (Gentilucci et al. 1991; von Hofsten and Ronnqvist 1993; Marteniuk et al. 1990). The latter result stems as well from the emphasis on collision-avoidance and travel-cost reduction and, in particular, from the greater benefit of opening and closing the hand in a single, continuous gesture as opposed to keeping the hand open for a long time before closing it. Distance to the object has no systematic effect on maximum aperture size, as has also been reported (Bootsma et al. 1994; Chiefi and Gentilucci 1993; Paulignan et al. 1991; Zaal and Bootsma 1993). This outcome is obtained in the model because spreading the fingers more for farther objects has no beneficial effect, neither in terms of obstacle avoidance nor reduction of travel cost. Increasing movement speed results in wider finger apertures in the simulations, as has been observed behaviorally (Wing et al. 1986). This outcome can be ascribed to the model’s reliance on joints that are preferred when movement frequencies are high (the finger joints) rather than on joints that are preferred when movement frequencies are low (the shoulder and elbow). Another result of the model is that, sometimes, the simulated grasp exhibits a late low-velocity phase in the tangential velocity profile of the wrist, but sometimes it does not. Whether it does or not depends on the obstacle-avoidance requirements of the task, and thereby provides an account of the circumstances in which the low-velocity phase is present (Jeannerod 1981, 1984) or absent (Wallace and Weeks 1988). For modeled prehension movements with a low-velocity transport phase, maximum finger aperture occurs near the start of the low-velocity phase of the transport component, also as seen in human prehension (Jeannerod 1981, 1984). Finally, the model predicts new phenomena concerning the kinematics of reach-and-grasp movements, depending on the initial position of the hand vis à vis the position of the object to be grasped, especially when the object to be grasped is initially behind the hand (Meulenbroek, Rosenbaum, Jansen, Vaughan, and Vogt, in preparation).

99

Discussion In this article, we have described a model of reaching and grasping control. The model yields simulated manual behavior with many of the features of arm and hand behavior exhibited by people when they point to, reach around, and grab hold of objects. The model is similar to other models in artificial intelligence in that it contains a number of still untested assumptions. From the perspective of conventional models in psychology and neurophysiology, therefore, it is a hypothesis about the way manual control might work. The model can be compared to others that have been developed in a similar vein (e.g., Bullock et al. 1993; Hoff and Arbib 1993; Iberall and Fagg 1996; Kawato 1996; Mel 1990; Morasso and Sanguineti 1995; Soechting et al. 1995). We acknowledge that, although our model may be able to account for a wider range of kinematic competencies than other models, it is weaker than other accounts in that it does not yet address the inverse dynamics problem and has not yet been cast in neurally specific terms. The constraint hierarchy should allow the model to deal with the problem of inverse dynamics, and there is nothing in the model that would seem to prevent its neural implementation. Despite the preliminary nature of the model, it may suggest useful ways of integrating and conceptualizing a number of findings, as we have tried to show here. The model also raises new questions. One is whether detailed features of observed kinematics can be predicted quantitatively instead of just qualitatively. Preliminary attempts at fitting the model to behavior have been encouraging (Fischer et al. 1997; Vaughan et al. 1998), although more work remains to be done. Another important question is how findings in neurophysiology can be interpreted within the framework of the model. A candidate area in which it might be especially useful to pursue this question concerns brain mechanisms underlying obstacle avoidance. As we have suggested, obstacle avoidance is a fundamental requirement for adaptive motor planning. Even when no external object is immediately perceived as an obstacle, obstacle avoidance may still be necessary. For example, a part of one’s own body can be an obstacle, as when one wants to bring one’s hand from one’s lap to one’s foot, in which case one’s thigh is in the way of the hand. In terms of brain mechanisms underlying obstacle avoidance, Martin and Ghez (1993) found that reversible cooling of the rostral area of the forelimb area of the motor cortex led to defective reaches into food wells by cats. In monkeys, damage to the supplementary motor area impairs reaching under a glass to catch food morsels pushed through a hole in the glass with the other paw (Brinkman 1981). Diamond and Gilbert (1989) found that human infants have difficulty inhibiting straightahead hand motions when attempting to retrieve a toy seen through a glass, which those authors took to reflect a deficit in inhibition associated with immaturity of the frontal cortex.

Because our model provides a more detailed account of the processes involved in obstacle avoidance than has been available before, it suggests more detailed questions about brain mechanisms underlying obstacle avoidance than were possible previously. In principle, each process described in our model ought to have an analog in the brain. Similarly, if the model is correct, it should be possible to predict from the model all and only those disorders that occur within the model’s domain of application. Acknowledgements Supported by NSF grant SBR-94-96290, NIH Research Scientist Development Award KO2 MH00977, and a grant from the Research and Graduate Studies Office of the College of Liberal Arts, Pennsylvania State University.

References Bernstein N (1967) The coordination and regulation of movements. Pergamon, London Bock O, d’Eleuterio GM, Lipitkas J, Grodski JJ (1993) Parametric motion control of robotic arms: a biologically based approach using neural networks. Telem Inform 10:179–185 Bootsma RJ, Marteniuk RG, Mackenzie CL, Zaal FTJM (1994) The speed-accuracy trade-off in manual prehension: effects of amplitude, object size and object width on kinematic characteristics. Exp Brain Res 98:535–541 Brenner E, Smeets JBJ (1996) Size illusion influences on how we lift but not how we grasp an object. Exp Brain Res 111:473– 476 Brinkman C (1981) Lesions in supplementary motor area interfere with a monkey’s performance of a bimanual co-ordination task. Neurosci Lett 27:267–270 Bryson AE, Ho YC (1975) Applied optimal control. Hempshire, New York Bullock D, Grossberg S, Guenther F (1993) A self-organizing neural model of motor equivalent reaching and tool use by a multi-joint arm. J Cogn Neurosci 5:408–435 Castiello U (1996) Grasping fruit: selection for action. J Exp Psychol Hum Percept Perform 22:582–603 Chiefi S, Gentilucci M (1993) Coordination between the transport and grasp component during prehension movements. Exp Brain Res 94:471–477 Dean J, Bruwer M (1994) Control of human arm movements in two dimensions: paths and joint control in avoiding simple linear obstacles. Exp Brain Res 97:497–514 Denier van der Gon JJ, Thuring JP (1965) The guiding of human writing movements. Kybernetik 2:145–148 Diamond A, Gilbert J (1989) Development as progressive inhibitory control of action: retrieval of a contiguous object. Cogn Dev 4:223–249 Feldman AG (1980) Superposition of motor programs. I. Rhythmic forearm movements in man. Neuroscience 5:81–90 Fischer MH, Rosenbaum DA, Vaughan J (1997) Speed and sequential effects in reaching. J Exp Psychol Hum Percept Perform 23:404–428 Flash T, Henis E (1991) Arm trajectory modification during reaching towards visual targets. J Cogn Neurosci 3:220–230 Flash T, Hogan N (1985) The coordination of arm movements: an experimentally confirmed mathematical model. J Neurosci 5:1688–1703 Gentilucci M, Castiello U, Corradini ML, Scarpa M, Umilta C, Rizzolatti G (1991) Influence of different types of grasping on the transport component of prehension movements. Neuropsychology 29:361–378 Goodale MA, Jakobson LS, Keillor JM (1994) Differences in the visual control of pantomized and natural grasping movements. Neuropsychology 10:1159–1178

100 Henis EA, Flash T (1992) A computational mechanism to account for averaged modified hand trajectories. In: Moody JE, Hansen SJ, Lippmann RP (eds) Neural information processing systems, vol 4. Morgan Kaufman, San Mateo Hoff B, Arbib MA (1993) Models of trajectory formation and temporal interaction of reach and grasp. J Mot Behav 25:175– 192 Hofsten C von, Ronnqvist L (1993) The structuring of neonatal arm movements. Child Dev 64:1046–1057 Hollerbach JM (1981) An oscillation theory of handwriting. Biol Cybern 39:139–156 Holst E von (1979) The behavioral physiology of man and animals (from 1939). In: Martin R (ed) The collected papers of Erich von Holst. University of Miami Press, Coral Gables Iberall T, Fagg AH (1996) Neural network models for selecting hand shapes. In: Wing AM, Haggard P, Flanagan R (eds) Hand and brain: neurophysiology and psychology of hand movement. Academic Press, San Diego, pp 243–264 Jeannerod M (1981) Intersegmental coordination during reaching at natural objects. In: Long J, Baddeley A (eds) Attention and performance IX. Erlbaum, Hillsdale, pp 153–169 Jeannerod M (1984) The timing of natural prehension movement. J Mot Behav 26:235–254 Kawato M (1996) Trajectory formation in arm movements: minimization principles and procedures. In: Zelaznik HN (ed) Advances in motor learning and control. Human Kinetics, Champaign, pp 225–259 Klatzky RL, Fikes TG, Pellegrino JW (1995) Planning for hand shape and arm transport when reaching for objects. Acta Psychol 88:209–232 Lozano-Pérez T (1983) Spatial planning: a configuration space approach. IEEE Trans Comp C-32:108–120 Marteniuk RG, Leavitt JL, MacKenzie CL, Athenes S (1990) Functional relationships between grasp and transport components in a prehension task. Hum Movement Sci 9:149–176 Martin JH, Ghez C (1993) Differential impairments in reaching and grasping produced by local inactivation within the forelimb representation of the motor cortex in the cat. Exp Brain Res 94:429–443 Mel BW (1990) Connectionist robot motion planning. Academic Press, San Diego Meulenbroek RGJ, Rosenbaum DA, Thomassen AJWM, Loukopoulos LD, Vaughan J (1996) Adaptation of a reaching model to handwriting: how different effectors can produce the same written output, and other results. Psychol Res 59:64–74

Morasso P, Sanguineti V (1995) Self-organizing body schema for motor planning. J Mot Behav 27:52–66 Nelson WL (1983) Physical principles for economies of skilled movements. Biol Cybern 46:135–147 Paulignan Y, Jeannerod M, Mackenzie C, Marteniuk R (1991) Selective perturbation of visual input during prehension movements. 2. The effects of changing object size. Exp Brain Res 87:407–420 Rizzolatti G (1987) Functional organization of inferior area 6. Ciba Found Symp 132:171–186 Rizzolatti G, Camarda R, Fogassi L, Gentilucci M, Luppino G, Matelli M (1988) Functional organization of inferior area 6 in the macaque monkey. II. Area F5 and the control of distal movements. Exp Brain Res 71:491–507 Rosenbaum DA, Slotta JD, Vaughan J, Plamondon RJ (1991) Optimal movement selection. Psychol Sci 2:86–91 Rosenbaum DA, Loukopoulos LD, Meulenbroek RGJ, Vaughan J, Engelbrecht SE (1995) Planning reaches by evaluating stored postures. Psychol Rev 102:28–67 Rosenbaum DA, Meulenbroek RGJ, Vaughan J (1997) Reaching around obstacles. Paper presented at the 38th Annual Meeting of the Psychonomics Society, Philadelphia, Penn., USA, on November 23, 1997 Saling M, Alberts J, Stelmach G, Bloedel JR (1998) Reach-tograsp movements during obstacle avoidance movements. Exp Brain Res 118:251–158 Schneider K, Zernicke RF, Schmidt RA, Hart TJ (1989) Changes in limb dynamics during the practice of rapid arm movements. J Biomech 22:805–817 Servos P, Goodale MA, Jakobson LS (1992) The role of binocular vision in prehension: a kinematic study. Vision Res 32:1513–1521 Soechting JF, Buneo CA, Herrmann U, Flanders M (1995) Moving effortlessly in three dimensions: does Donders’ Law apply to arm movement? J Neurosci 15:6271–6280 Uno Y, Kawato M, Suzuki R (1989) Formation and control of optimal trajectory in human multijoint arm movement: minimum torque-change model. Biol Cybern 61:89–101 Vaughan J, Rosenbaum DA, Harp CJ, Loukopoulos LD, Engelbrecht SE (1998) Finding final postures. J Mot Behav 30:273–284 Wallace SA, Weeks DL (1988) Temporal constraints in the control of prehensile movement. J Mot Behav 20:81–105 Wing AM, Turton A, Fraser C (1986) Grasp size and accuracy of approach in reaching. J Mot Behav 18:245–260 Zaal FT, Bootsma RJ (1993) Accuracy demands in natural prehension. Hum Movement Sci 12:339–345