IEEE/ASME TRANSACTIONS ON MECHATRONICS, VOL. 17, NO. 1, PP. 174-182, FEBRUARY 2012

Influence of User Grasping Position on Haptic Rendering

1

φ device

´ Mildred J. Puerto, Jorge Juan Gil, Hugo Alvarez and Emilio S´anchez Abstract—This paper investigates the effect of user grasping position on the performance of haptic rendering. Two dynamic models, with seven and eleven parameters respectively, have been used to characterise the PHANToM haptic interface and the user. The parameter variability analysis shows that user grasping position significantly affects system dynamics. This variation also influences the phase margin of the system, leading to different damping factors in response to contacts with rigid virtual objects. To compensate this effect, an adaptive haptic rendering has been developed and successfully implemented, imposing a similar damping factor in the transient responses for all grip positions.

17 cm 15 cm

A user

Index Terms—Haptics and haptic interfaces, human factors, physical human-robot interaction. Fig. 1. PHANToM Premium 1.0 configuration for the experiments.

I. I NTRODUCTION Mechanical design of haptic interfaces is an important research field to ensure the usability of this kind of system [1], [2], [3], [4]. The end-effector of the interface is one of the key elements to achieve a natural immersion in the virtual task. Stylus-like end-effectors are commonly used in haptic interfaces [5]. Depending on the application, final users can hold the stylus in many different ways —probably, as he or she feels most confident. From the control point of view, this results in variable and unknown dynamic properties present in the system’s loop [6], which leads to variable stability margins. Thus, a good model of the human hand behaviour is fundamental to correctly design and tune haptic controllers. Many researchers have proposed dynamic models for the user manipulating a mechanical device [7], [8], [9]. A second-order timeinvariant system consisting of a mass-spring-damper body has been widely used to model the operator [10], [11], [12] and a fiveparameter model has also been proposed for human impedance [13], [14]. Despite the increasing complexity of previous models, some human characteristics remain unmodelled when using these approximations. Moreover, the proposed models are usually taken as time-invariant, but the user can voluntarily change his/her own dynamics. For example, different frequency responses —and consequently human wrist models— can be obtained depending on the user’s grasping force [15]. The influence of the user depends on many factors, such as the user’s physical characteristics, grasping distance, applied force and even the task instructions [16]. Recently, some authors have included uncertainties in the human model, e.g., by means of introducing a fixed boundary in the frequency response [17], or introducing a factor that may take different values within a certain range [18]. It is possible to analyse the stability of the haptic interaction without a concrete model for the user, assuming that he/she is passive [19], [20], [21]. Only the model of the haptic interface is required in this kind of study. Nevertheless, without modelling either the user or the mechanical device, it is possible to experimentally analyse the influence of some factors on the haptic interaction or perception. For example, the effect of force saturation [22] or the virtual stiffness [23] on the haptic perception of detail. The authors are with the Department of Applied Mechanics, Centro de Estudios e Investigaciones T´ecnicas de Guip´uzcoa (CEIT), and the Department of Electricity, Electronics and Control Engineering, Technological Campus of the University of Navarra (TECNUN), University of Navarra, San Sebasti´an E-20018, Spain (e-mail: [email protected]; [email protected]; [email protected]; [email protected]). Digital Object Identifier 10.1109/TMECH.2011.2116798

The purpose of this work is to analyse system dynamics, depending on user grasping position, to establish a quantitative relation between the frequency response of the system and the transient properties of the response to a haptic rigid contact. First, some physical models for the system (device and user) with concentrated parameters are derived in Section II. Experiments with multiple participants show that system dynamics strongly vary with user grasping position. A statistical study (Section III) reveals that variability depends more significantly on the grip position than on the identity of the subject. Later on, in Section IV, the phase margin of the frequency response is related to the damping factor of the transient response to a rigid contact. This response is more stable (higher damping factor) when the user is grasping the stylus closer to its tip. Finally, in Section V, an adaptive control strategy is proposed to impose a nearly constant damping factor, that is, independent of the grasping position. II. E XPERIMENTAL S YSTEM M ODELS To analyse the influence of grasping position on haptic rendering, some dynamic models for the overall system, the user and the haptic device, are estimated in this section. A system identification method based on frequency response has been applied to characterise the system’s transfer function. The protocol and details of the experimental process are explained in the following subsections. A. Apparatus and participants The first degree of freedom of the PHANToM Premium 1.0 was used as a testbed (φ-axis in Fig. 1). This device is a desktop haptic interface with low inertia and high back-drivability. To perform the experiments in the same kinematic configuration, the links of the parallelogram were mechanically locked. Thus, the centre of the gimbal (point A) was placed at 17 cm from the axis of rotation φ and at a height of 15 cm from the surface of the table. The system was controlled by a dSPACE DS1104 acquisition board running at a sampling frequency of 1 kHz. The input signal was a torque τ in the motor that moves the horizontal pulley of the device, while the measured output was the rotation of the encoder coupled to the motor. Both signals were mapped to φ-axis in SI units. Therefore, the transfer function, φ(s) , (1) G(s) = τ (s) that will be obtained, only applies to the described kinematic configuration and contains rotational parameters. However, the equivalent linear parameters (in x-axis) for the user and the device at the tip of

IEEE/ASME TRANSACTIONS ON MECHATRONICS, VOL. 17, NO. 1, PP. 174-182, FEBRUARY 2012

2

1

2

3

4

5

6

A

Gain (dB) 20 log [rad/Nm]

10

6 5 4 3

0 −10 −20 −30 −40 −50 −60 0.5 0

1 2 3 4 5 6

2 1

Position 1 Position 2 Position 3 Position 4 Position 5 Position 6

1 2

1

10

70

Fig. 2. Definition of the six positions for the experiments and user 1 grasping the stylus at position 2.

the end-effector (point A) can be derived by dividing the rotational variables by 0.172 m2 . Thirteen subjects took part in the experiments, five women and eight men, from 25 to 43 years old. All were right-handed and reported normal tactile function. All subjects had prior experience with haptic applications using the PHANToM, but they were not trained to perform the experiments. B. Procedure The participants were asked to hold the stylus of the device like a pencil (tripod grasp using thumb, index and middle fingers) with their right hand at six different positions along the stylus. Thus, six experiments were performed by each subject: one per grasping position. The stylus was divided into six segments: position 1 corresponded to placing the fingers at the tip of the stylus, while position 6 corresponded to the opposite extreme. Each segment was approximately 2 cm long. Fig. 2 shows the six different positions where the user could grasp the stylus. The user’s elbow rested on the table where the device was placed. The only instruction given to the participants was to maintain a constant, moderate grip force and keep the hand still while the torque signal was being rendered, but not to resist the motion of the device. C. Data processing and experimental frequency response A good knowledge of the system helps define the input signal. The user’s dynamics mainly influences a robotic system at relatively low frequencies (typically < 30 Hz) [14]. On the contrary, rigid mechanical devices exhibit vibration modes at relatively high frequencies. In the case of the PHANToM 1.0, the first vibration mode takes place at approximately 60 Hz [14]. Therefore, to model both the user and the first vibration mode of the device, the frequency range under analysis in this study is from 0.5 to 70 Hz. To correctly excite all the frequencies under study, the torque input was a white-noise signal lasting 20 s within a frequency range from 0.1 to 150 Hz, that is, slightly more than an octave under and above the selected limits. MATLAB’s empirical transfer function estimation algorithm (tfestimate) was used to plot the resulting Bode diagrams. A hanning window 4096 points in length with a 50% overlap was selected for data processing. The length of the experiment, 20 s, allows the tfestimate algorithm to average the resulting Bode plot, leading to a smooth and reliable frequency response. Fig. 3 shows the frequency responses of the system for grasps by user 1 at the six different positions. As expected, the first vibration mode arose at approximately 60 Hz. Although the same user was grasping the stylus, the system response strongly

Phase (º)

−45 1

−90

2 2 3

−135

−180 0.5

1

5 4 6

1

10 Frequency (Hz)

70

Fig. 3. Frequency response of the PHANToM and user 1 grasping the stylus at six different positions.

varied with the grip position. Quite similar responses were obtained for the other users. The statistical analysis on the system’s variability as a function of the users and the grasping position will be performed in Section III. D. System models and parameter fitting To estimate a transfer function for the system described by (1), an appropriate model has to be selected. The shape of the frequency responses (Fig. 3) can be used to help determine the number of poles and zeros the models should have [24]. For user 1 and grasping positions 1, 2 and 3, the shape of the experimental response is quite complex, but a sixth-order transfer function with 6 poles and 4 zeros could adequately fit the system dynamics (Fig. 4 top). However, for grasping positions 4, 5 and 6, a fourth-order transfer function with only 4 poles and 2 zeros was enough to model the system (Fig. 4 bottom). In both cases, the gain diagram is constant at low frequencies and the phase diagram tends to 0◦ . Thus, the models have to be type-0 transfer functions with position error constants of 20 log Kp (dB). 1) Eleven-parameter model: For positions 1, 2 and 3, an elevenparameter model inspired by [14], [25] was used to characterise system response (Fig. 5 top). In [14] it was successfully used for this purpose. The device dynamics is decoupled into two masses, while the user introduces additional mass to the system. The use of these concentrated parameters is only a practical assumption that allows for a physical interpretation of the parameters, but the real device and user have distributed masses. Using this physical model, the relationship between the measured output position X1 and the input force of the motor F is G(s) =

ph (s)p2 (s) − p2s (s) X1 (s) = , (2) F (s) [ph (s)p2 (s) − p2s (s)] p1 (s) − p2c (s)ph (s)

IEEE/ASME TRANSACTIONS ON MECHATRONICS, VOL. 17, NO. 1, PP. 174-182, FEBRUARY 2012

10

2 poles

20 log Kp

2 poles

2 poles

2 zeros Gain (dB)

[ XF1 ]

Gain (dB) 20 log [rad/Nm]

Gain (dB) [ XF1 ]

ω (Hz) 2 zeros

2 poles

20 log Kp

2 poles

−10 −20 −30 −40

−60 0.5 0

ω (Hz) 2 zeros Fig. 4. Schemes of pole and zero placement depending on the shape of the frequency response.

F

X2

kc

m1

m2

bc

ks

F

kc

bh

m1

bc

X2 m2

1

10

70

1

−90 2

65

−180 0.5

kh

X1

1 2

−135

mh

bs

b2

b1

Xh

2 1

Position 1 Position 2 Position 3 Position 4 Position 5 Position 6

1 2 3 4 5 6

−45 Phase (º)

user

device with a vibration mode

6 5 4 3

0

−50

X1

3

1

4

3

10 Frequency (Hz)

bs

qc (s) = bc s + kc ,

(10)

2

Fig. 5. (Top) Physical model with 11 parameters for the sixth-order transfer function (grasping positions 1, 2 and 3 for user 1). (Bottom) Physical model with 7 parameters for the fourth-order transfer function (grasping positions 4, 5 and 6 for user 1).

q1 (s) = m1 s + b1 s + qc (s),

(11)

q2 (s) = m2 s2 + bs s + ks + qc (s),

(12)

resulting in a type-0 fourth-order transfer function (with 4 poles and 2 zeros), whose position error constant is Kp = lim H(s) = s→0

where pc (s) = bc s + kc ,

(3)

ps (s) = bs s + ks ,

(4)

p1 (s) = m1 s2 + b1 s + pc (s),

(5)

p2 (s) = m2 s + b2 s + pc (s) + ps (s),

(6)

ph (s) = mh s2 + bh s + kh + ps (s),

(7)

leading to a type-0 sixth-order transfer function (with 6 poles and 4 zeros), whose position error constant is s→0

1 1 1 + + . kc ks kh

70

where

b1

Kp = lim G(s) =

1

Fig. 6. Frequency response of the six estimated models for user 1 (Table I).

ks

2

2

(8)

Therefore, the resulting Kp is the inverse of the overall stiffness of the system: kc , ks and kh in series. This value is easily identified in the experimental responses (Fig. 3) and changes significantly with user grasping position. 2) Seven-parameter model: For grasping positions 4, 5 and 6, an alternative model with only two masses and 7 parameters was used (Fig. 5 bottom). With this new model, the transfer function for the system is q2 (s) X1 (s) = , (9) H(s) = F (s) q2 (s)q1 (s) − qc2 (s)

1 1 + . kc ks

(13)

3) Parameter fitting: Model parameters have been identified by fitting the experimental responses to the proposed transfer functions with 11 and 7 variables, equations (2) and (9) respectively. A leastsquares iterative method developed in Matlab has been used. Table I shows the estimation for all the participants, while Fig. 6 shows the frequency response of the models for user 1. Null values in Table I are, in fact, very small non-null values (< 0.001) that are not shown to avoid overstating the precision with which they are known. Most of the participants had the model transition (eleven-parameter to seven-parameter model) from grasping position 3 to 4, but this was not a general behaviour. E. Discussion on the models Analysing the resulting parameters in Table I, and for farther grasping positions, variable Kp can be approximated by Kp = lim H(s) ≈ s→0

1 , ks

(14)

because the stylus/skin stiffness ks is much smaller than the stiffness associated with the first vibration mode (ks  kc ), therefore stiffness ks prevails at low frequencies.

IEEE/ASME TRANSACTIONS ON MECHATRONICS, VOL. 17, NO. 1, PP. 174-182, FEBRUARY 2012

TABLE I P HYSICAL PARAMETERS OF THE PHANT O M AND THIRTEEN USERS GRASPING AT SIX DIFFERENT POSITIONS (m gm2 , b Nms/rad, k Nm/rad) User Position 1 2 3 1 4 5 6 1 2 3 2 4 5 6 1 2 3 3 4 5 6 1 2 3 4 4 5 6 1 2 3 5 4 5 6 1 2 3 6 4 5 6 1 2 3 7 4 5 6 1 2 3 8 4 5 6 1 2 3 9 4 5 6 1 2 3 10 4 5 6 1 2 3 11 4 5 6 1 2 3 12 4 5 6 1 2 3 13 4 5 6

m1 0.9 0.8 0.9 0.8 0.8 0.8 1.1 1.4 1.3 0.9 0.9 0.9 1.3 1.4 1.3 1 1 1 1 1.4 1.3 0.8 0.9 1 1.2 1.3 1.4 0.9 0.3 0.3 1.3 1.2 1.5 1.3 0.3 0.6 1.1 1.3 1.3 1 0.5 0.5 0.8 1.3 0.9 0.8 0.6 0.3 0.9 0.9 1.2 1.3 1.1 0.9 1.2 1.3 1.2 0.7 0.7 0.4 1.2 1.4 1.2 1 0.9 0.9 1.2 1.3 1.1 1.2 0.8 0.8 0.8 1 1.3 0.8 0.8 0.9

b1 0.07 0.04 0.04 0.042 0.042 0.045 0.005 0 0.01 0.084 0.08 0.091 0.071 0 0 0.062 0.061 0.068 0.111 0.001 0.001 0.147 0.089 0.063 0.004 0.001 0.008 0.051 0.041 0.089 0 0.006 0.007 0.03 0.038 0.116 0.004 0.004 0.03 0.036 0.116 0.093 0.108 0.001 0.04 0.058 0.028 0.07 0.106 0.087 0 0 0.003 0.081 0.007 0 0 0.094 0.1 0.074 0.01 0 0 0.092 0.09 0.074 0 0.001 0.028 0 0.091 0.078 0.095 0 0.003 0.124 0.121 0.097

bc 0.15 0.17 0.07 0.04 0.06 0.06 0.12 0 0.007 0.037 0.04 0.036 0.011 0.001 0.023 0.062 0.066 0.068 0.044 0.002 0 0.041 0.033 0.045 0.001 0 0.001 0.04 0.062 0.04 0 0 0 0.013 0.055 0.04 0.087 0.006 0.002 0.047 0.055 0.044 0 0.006 0.106 0.078 0.048 0.029 0.04 0.004 0.035 0.058 0.049 0.043 0.004 0 0.029 0.04 0.04 0.05 0.16 0 0.04 0.036 0.038 0.043 0 0 0.089 0.042 0.041 0.045 0.052 0.117 0.012 0.033 0.039 0.043

kc 39 87 94 100 100 104 23 50 48 78 82 81 27 44 47 80 88 86 7 48 46 60 80 91 55 50 38 58 74 75 54 45 34 39 77 78 36 46 40 54 73 77 23 56 38 55 72 78 15 25 50 47 59 75 51 43 49 75 78 79 12 44 52 76 80 88 56 49 21 57 83 86 22 33 49 69 76 85

m2 4.2 2.3 1.3 2.95 2.95 3 2.6 0.8 0.7 2.3 2.29 2.33 0.3 0.64 0.81 2.51 2.38 2.29 2.89 0.73 0.7 1.84 2.26 2.41 1.38 1 1 0.78 0.97 2.09 2.6 1.8 1.1 0.45 1.01 2.02 3.7 1.4 0.5 0.68 1.89 1.95 2.76 0 0.8 0.62 0.76 1.98 4.6 1.44 0.8 0.76 0.79 2.27 2 1.8 0.8 2.08 2.14 2.2 1.17 1.1 0.92 2.27 2.23 2.34 1.55 1.32 1.96 0.8 2.2 2.24 2.55 2.37 0.78 2.03 2.06 2.18

b2 0.06 0.09 0.02 0.11 0.1 0.03 0 0.08 0.04 0.01 0.1 0 0.22 0.12 0.16 0.01 0.01 0 0.01 0 0 0.01 0.14 0.16 0 0 0 0.02 0 0 0.01 0 0 0 0 0 0.1 0 0 0 0.12 0 0 0 0 0 0.02 0.14 0 -

bs 0.095 0.05 0.02 0.09 0.04 0.02 0.043 0.095 0.02 0 0 0 0.02 0.028 0.004 0 0 0 0.026 0.073 0.072 0 0 0 0.115 0.094 0.048 0.001 0 0.002 0.351 0.167 0.113 0.025 0 0 0.076 0.06 0.039 0 0 0 0.116 0.251 0.001 0 0 0.018 0.063 0.083 0.002 0 0 0 0.342 0.32 0.02 0 0 0 0 0 0 0 0 0 0.245 0.227 0.075 0 0 0 0.068 0.001 0.046 0 0 0

ks 19.3 9.6 3.5 1.6 1.1 1 2.9 1.8 3.9 2 1 1.3 4.5 4.7 5.1 0.7 0.5 0.3 9.3 6.6 5.7 3.3 2 1.3 17.5 18.1 14.1 5.4 3 1.2 70.3 39.7 24.3 11.6 4.1 1.9 22.4 22.1 16.3 8.9 7.1 2.6 33.4 35.5 25.9 10.5 5.4 0.6 178.2 49.9 10.3 4.4 2.6 1.3 46.8 39.8 14.7 5.2 2.9 1.5 156.2 8.6 6.8 3.8 3.4 2.5 28.1 23.5 16.6 6.3 3.6 2.2 3.8 3.8 5.9 2.9 2.3 2

mh 4.9 5 4.2 4.2 8.7 2.6 4.3 4.2 2.2 5.6 7.4 10 6.4 8.7 6.6 4.8 3.2 16.9 12.3 8.4 7.2 3.2 7.7 7.3 7.2 5.6 17.1 41.4 6.8 6.3 3.6 27.1 32.5 1.5 1.3 0.9 40.7 49.7 1.7 8.9 4.6 1.5 9.6 12 9.2 1.5 4.3 2.7 3.5 -

bh 0.09 0.09 0.10 0.01 0.02 0.09 0.08 0.10 0.16 0.07 0.05 0.21 0.05 0.19 0.17 0.12 0.06 0.55 0.34 0.25 0.25 0.16 0.19 0.17 0.31 0.28 0.51 0.45 0.47 0.34 0.19 0.39 0.46 0.32 0.13 0.09 0.53 0.63 0.41 0.20 0.15 0.22 0.28 0.32 0.31 0.20 0.03 0.05 0.15 -

kh 3 2.5 2.3 6.1 8.1 2 2.1 2.4 1.6 4.5 3.3 4 4.8 9.6 7.7 0.5 3.6 15 9.6 7.1 8 1.1 4.3 3 3.5 3.2 6.6 7.5 1.8 3.6 2.4 4.1 3.6 1 0.5 0.7 5.1 6.4 1 3 3.3 1.5 2.7 3.1 3.6 1.7 4.1 3.6 1.2 -

4

A possible physical cause of the model simplification (from 11 to 7 variables) is precisely the fact that ks becomes smaller as the grasping position changes. Beyond a certain point and although the user has not an infinite impedance, such a weak link cannot “move” the user and therefore the human operator is seen as a perfect connection to ground from the motor’s point of view. Aside from this link to ground, the user stops intervening in the dynamics of the system. However, we stress that without the human presence, the mentioned ground connection would not exist. This is consistent with [26], where the user was modelled as a linearised endpoint stiffness. It is interesting to note that the gain diagram of Fig. 3 starts at a lower level as the user grasps the stylus closer to the tip. Thus, the equivalent stiffness of the system —the inverse of (8) and (13)— increases as the user grasps the device closer to the tip of the stylus. Not all the parameters contribute in the same way —and weight— to the transfer function. For example, only the three stiffness coefficients impose Kp (8). The combination of stiffness coefficients and masses fix the location of poles and zeros (Fig. 4), that is, their natural frequencies. Finally, the damping coefficients result in each double pole (or double zero) being more or less damped. This brief description also indicates the order of “importance” of the parameters in the transfer function (a relatively small change in one stiffness modifies the Bode diagram more than a relatively large change in one of the damping coefficients). However, the sensitivity analysis of the parameters is beyond the scope of this study. Only the variability of those parameters depending on the grip position has been analysed. III. PARAMETER VARIABILITY A NALYSIS The variability of system parameters has been statistically analysed by using the two-way balanced ANOVA test on the seven parameters with balanced data, and the General Linear Model with multivariate design on the four parameters with unbalanced data (b2 , mh , bh and kh ). In both tests, the two factors are the grip position (fixed factor) and the subject (random factor). It was found that the influence of the grasping position on ten of the eleven parameters is significant (p < 0.05). Moreover, it is extremely significant (p < 0.001) in eight of them. Only one damping coefficient is not significantly affected: bc (p = 0.313). The subject factor was found to significantly influence the parameters governing the user’s dynamics (mh , bh and kh ) and also half of the other parameters: m1 (p = 0.031), kc (p < 0.001), m2 (p = 0.043) and bs (p = 0.014). Thus, these tests show that both the user and the grip position influence the dynamics of the system. However, the grasping position affects more parameters — and in a more significant way— than the user identity. Therefore, the variability due to grasping position is greater —or more significant— than the inter-subject variability. Given the appreciable influence of the stiffness coefficients on the frequency response of the system, parameters kc and ks have been analysed more comprehensively. The box plot of parameter kc (Fig. 7 top) shows that this stiffness coefficient tends to increase with the grasping position. A positive correlation with this parameter was found (Pearson’s ρ = 0.784, p < 0.001) and therefore, among all the eleven parameters, kc is a good candidate to determine whether a grasping position is significantly different from the others. Statistically, this difference can be analysed by means of several tests. In this study, comparisons of confidence intervals have been used. Two grip positions are significantly different if their 95% confidence intervals do not overlap. These intervals for the means of kc (Fig. 7 bottom) indicate that four groups of grip positions were identified as significantly different: position 1, positions 2/3, position 4 and positions 5/6.

IEEE/ASME TRANSACTIONS ON MECHATRONICS, VOL. 17, NO. 1, PP. 174-182, FEBRUARY 2012

5

70

100

60

ks (Nm/rad)

kc (Nm/rad)

80 60 40

50 40 30 20 10

20

0

0 1

2

3

4

5

6

Fig. 7. (Top) Box plot of kc parameters. The boxes contain the middle half of the data points. The lines inside the boxes are the median values. The vertical lines cover the range of all values, unless outliers (asterisks) are present. The dashed line connects the mean values (crossed circles). (Bottom) Confidence intervals for the means of kc .

Regarding the stylus/skin stiffness ks , it tends to decrease with user grasping position (Fig. 8 upper). In fact, this parameter exhibits an exponential increment as the user grasps the stylus closer to the tip. To corroborate this, the box plot of log[ks ] has also been included (Fig. 8 middle) showing the linear tendency of this parameter with grip position. Thus, analysing log[ks ] instead of ks , a negative correlation was found (Pearson’s ρ = −0.748, p < 0.001). The 95% confidence intervals for the means of log[ks ] (Fig. 8 lower) indicate that none of the grasping positions is significantly different from its adjacent ones (except for positions 3 and 4). However, grasping positions whose relative position is at two or more locations, are always significantly different. The comparison of the confidence intervals for parameters kc and log[ks ] suggests that the stylus could have been divided into only four distinguishable grasping positions. In other words, independent of the user grasping the device, system dynamics begin to be significantly different if the subject holds the stylus in positions approximately 4 cm apart. Thus, and despite the large variability found in system parameters (Table I), given a model for the system response it is possible to identify the grasping position. This conclusion does not mean that the grasping position is the only factor that can be statistically distinguished. Other elements can be studied (grasping force levels, fist vs. tripod grasp, etc.). The experiments performed in this study have been done for a particular device, and the users held the stylus in a very similar way. The small discrepancies in the grasping between users (performing a comfortable tripod grasp) do not compromise the findings of this study. On the contrary, it seems that this kind of grasping does not present a high variability, or maybe better, it is included within user variability. Fig. 9 shows the box plots of mh , bh and kh parameters using linear units. Their median values for grasping position 1 (the most appropriate for the pen-like grasp) are mh = 420 g, bh = 7.92 Ns/m and kh = 174 N/m. Some authors [27] have reported comparable values for these physical parameters, but suggesting a human model or validating an existing one is not the aim of this study.

log[ks ] log[Nm/rad]

Individual 95% CIs for k_c Mean based on pooled StDev Pos. Mean ----+---------+---------+---------+----1 32,3346 (---*---) 2 47,7432 (---*---) 3 46,5039 (---*---) 4 65,2242 (---*---) 5 78,7207 (---*---) 6 83,3428 (---*---) ----+---------+---------+---------+----32 48 64 80

1

2

1

2

3

4

5

6

3

4

5

6

2.5

User grasping position

2.0 1.5 1.0 0.5 0.0

−0.5 User grasping position Individual 95% CIs for log_k_s Mean based on pooled StDev Pos. Mean +---------+---------+---------+--------1 1,34394 (---*--) 2 1,12758 (--*---) 3 0,98182 (---*--) 4 0,60306 (--*---) 5 0,38854 (---*--) 6 0,12562 (--*---) +---------+---------+---------+--------0,00 0,40 0,80 1,20

Fig. 8. (Top) Box plot of ks parameters and (middle) box plot of the log[ks ] parameter showing its linear tendency with the grasping position. (Bottom) Confidence intervals for the mean values of log[ks ].

IV. I NFLUENCE ON R IGID C ONTACTS The frequency response of the system can be used to derive some properties of the time response to rigid contacts. Haptic controllers consist of discrete-time loops running at high frequency. The discretetime transfer function of G(s) is defined as follows: G(z) = Z[H0 (s)G(s)],

(15)

where H0 (s) is the zero-order hold. Assuming that the haptic rendering function consist of a discrete-time closed loop without delay and stiffness K to simulate a rigid wall collision, the critical stiffness for a virtual contact [14] is the gain margin (Gm) of G(z): KCR = Gm{G(z)}.

(16)

Stiffness coefficients under this limit (K < KCR ) exhibit stable responses. The transient properties of those responses depend on the resulting phase margin (Pm) of the system. In particular, the damping factor ζ of the transient response is approximately the phase margin (in degrees) divided by 100◦ [28]: ζ≈

Pm{KG(z)} . 100◦

(17)

A. Hypothesis The shape of the frequency responses obtained in Section II shows that once a stiffness coefficient K is selected, the phase margin of the system substantially increases as the user holds the stylus closer to

IEEE/ASME TRANSACTIONS ON MECHATRONICS, VOL. 17, NO. 1, PP. 174-182, FEBRUARY 2012

Gain (dB) [ XF1 ]

1.8 1.6

Position 5 Position 3

20 log KKp

1.4

ωcg

1.2

mh ( kg )

6

0

1.0 0.8

ω (Hz)

Phase (◦ ) [ XF1 ]

0.6 0.4 0.2 0.0 1

2

3

4

5

25

−180◦

Pm

ωcg

ω (Hz)

bh ( Ns/m)

20

Fig. 10. Scheme representing the increment of the phase margin with user grasping position.

15 10 5 0 1

2

3

4

5

500

kh ( N/m)

400 300 200 100

Fig. 11. Virtual scenario used for the experiments. The sphere is the cursor controlled by the stylus and the desired collision point is marked in the wall.

0 1

2

3

4

5

User grasping position

B. Experimental collision responses Fig. 9. Box plots of mh , bh and kh parameters using linear units.

its tip. Fig. 10 shows a scheme representing this idea, inspired by the frequency response of user 1. A stiffness coefficient of, for example, K = 10 Nm/rad, moves the gain diagram up 20 dB, setting —in this particular example— the gain crossover frequency at ωcg ≈ 11 Hz (Fig. 3). For grasping positions 3 and 5, the resulting phase margins are 40◦ and 15◦ respectively. This means that the expected damping factor ζ of the transient response varies from 0.4 to 0.15, comparing the responses of those grip positions. The damping factor imposes the overshoot of the position signal. In the previous example, ζ factors of 0.4 and 0.15 lead to overshoots of 25% and 62% respectively [28]. The damping factor is also related to the settling time ts of the transient response [28]. A rough approximation is ts ≈

4 , ζωcg

(18)

which gives 0.14 and 0.38 s for the settling times of the transient oscillations (again for user 1 grasping positions 3 and 5 respectively). Thus, the hypothesis is that as the user grasps the stylus farther from its tip, the system response to the subject colliding against a virtual wall of 10 Nm/rad will exhibit a less damped transient. And this change in the damping factor will manifest itself in a higher overshoot and a longer system response settling time.

A set of experimental responses have been recorded to analyse the validity of the previous hypothesis. A unique participant (user 1, female, aged 28) took part in these experiments. To simulate rigid contacts, a scenario containing a virtual wall (Fig. 11) was implemented using the Virtual Reality toolbox for MATLAB. The virtual wall was set at φ = 0.7 rad, that exactly corresponds to the plane x = 0 (Fig. 1), with a stiffness coefficient of 10 Nm/rad. Since the device was not mechanically locked in this case, the participant was asked to collide the wall at the marked point of the scene on the screen. This point corresponds to the same kinematic configuration as in the experiments of Section II. The participant held the stylus of the device like a pencil (tripod grasp using thumb, index and middle fingers) with her right hand (as in Section II). In these experiments, only grasping positions 1, 3 and 5 were recorded. Although it is difficult to reproduce the same conditions in the multiple contacts (user’s force, contact velocity, etc.) the user was asked to try to collide with the same velocity and force against the wall. Fig. 12 shows some of the obtained responses to the rigid contact. The collisions shown are typical among all the recorded experiments and were chosen particularly because they have similar contact conditions. For example, the force exerted by the user was estimated with the final penetration in the wall (very similar in the three cases), and user’s velocity before the contact was estimated by differentiating the position signal (the slope in Fig. 12). The recorded oscillations confirm the hypothesis that the same virtual object exhibits a more or less damped transient response depending on grip position. Moreover,

IEEE/ASME TRANSACTIONS ON MECHATRONICS, VOL. 17, NO. 1, PP. 174-182, FEBRUARY 2012

7

Device and user

Displacement (rad)

0.85

Fh +

0.8

−

Grip position

0.7

F

0.65 User grasping at position 1 0

0.1

0.2

0.3

0.4

0.5

0.6

H(s) Holder

0.6

Displacement (rad)

Webcam

0.75

0.85

X1

G(s)

0.7

0.8

K +B

1−z −1 T

T

Adaptive impedance

Fig. 13. Adaptive impedance rendering depending on the grip position. TABLE II I MPEDANCES FOR THE A DAPTIVE R ENDERING

0.75

Grasping position Stiffness K (Nm/rad) Damping B (Nms/rad)

0.7

1 10 -

2 10 -

3 10 0.03

4 10 0.07

5 10 0.11

6 10 0.14

0.65 User grasping at position 3 0.6

0.75

the user is obliged to grasp the handle in a specific manner (e.g., by means of dead-man buttons or ergonomic designs) that leads to the best dynamic properties. In the case of devices with handles that allow different types of grasps, the grip position could be sensed in order to automatically modify the properties of the virtual objects. This last application is investigated in the following section.

0.7

V. A DAPTIVE H APTIC R ENDERING

Displacement (rad)

0.85

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.65 User grasping at position 5 0.6 0

0.1

0.2

0.3 0.4 Time (s)

0.5

0.6

0.7

Fig. 12. Contact against a virtual wall of 10 Nm/rad with user 1 grasping the device at positions 1, 3 and 5. Dashed line indicates the wall placement.

the experimental responses in Fig. 12 fit well with the settling times estimated in the previous subsection. Considering all the recorded trials that have similar contact conditions for grip position 5, the mean settling time is 0.401 ± 0.053 s. For grasping position 3, this mean settling time is 0.248 ± 0.036 s. The overshoots present a larger magnitude with respect to the steady-state because the initial condition for velocity was not zero (as in the case of the theoretical step response). This is also the reason why the recorded oscillation leaves the virtual wall for the third test. It is important to note that the virtual wall of 10 Nm/rad (346 N/m at collision point A) did not include any virtual damping. The resulting damping factors depend only on the phase margin of the system, and this phase margin is introduced and enlarged by the user, as she (or he) holds the stylus closer to its tip. C. Discussion on the contact response The extra phase margin provided by the user justifies a common conclusion for haptic interfaces: a firm grasp tends to stabilise the system [9], [29]. The beneficial effect of the presence of the user is greater as the user holds the stylus closer to its tip. Some non-expert users tend to hold the end-effector lightly, e.g., with only two fingers at position 5 or 6, probably afraid of being harmed by the device. However, our recommendation would be to grasp the stylus firmly, closer to the tip, in order to increase the phase margin of the system. A possible application of this knowledge for a better performance of haptic interfaces is to design the end-effector in such a way that

The results of the previous sections have inspired the development of an adaptive haptic rendering strategy to obtain collisions with similar damping factors. Although other possible algorithms could be designed for the same purpose, the proposed controller is relatively simple: it uses a number of predefined impedances (one per grasping position), but requires knowledge of the user’s grip position (to change from one impedance to another). Fig. 13 shows the scheme of the proposed technique. A. Impedances definition A viscoelastic impedance law, including a stiffness coefficient K and a damping B, was used for the adaptive haptic rendering. The stiffness was fixed to K = 10 Nm/rad for all the grasping positions. The motivation for this was to ensure a nearly constant steady-state response (that is, the final penetration in the wall) if the user was exerting the same force. Thus, the damping coefficient is the only element ensuring that the damping factor is similar for all grasping positions. The required specification was to obtain a damping factor of at least 0.5 (ζ ≥ 0.5), that is, a phase margin of at least 50◦ in the phase Bode diagram. The models proposed for user 1 (Table I) were used to tune the damping coefficients. For grasping positions 1 and 2, and a stiffness of 10 Nm/rad (an increment of 20 dB in the gain Bode diagram), the system already has a phase margin larger than 50◦ . Therefore, for those grip positions, the proposed impedance (Table II) does not include any additional damping. In the other cases, the damping coefficient was tuned by using Matlab’s margin function. As could be expected, the required damping coefficient B increases as the user holds the stylus farther from its tip. However, quite small damping values are enough to compensate the influence of the grasping position. B. Grasping position detection The online transition between impedances requires the grip position to be sensed. This was done by means of computer vision

IEEE/ASME TRANSACTIONS ON MECHATRONICS, VOL. 17, NO. 1, PP. 174-182, FEBRUARY 2012

8

Displacement (rad)

0.8 0.75 0.7 0.65 User grasping at position 1 0.6 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

techniques, using a common web camera. Image processing has been recently used to estimate haptic properties of a remote environment without force sensors [30]. In this case, only the grasping position is estimated. This strategy does not require the implementation of a sensing device in the stylus and therefore, it does not modify system dynamics. In an offline process, the vision algorithms learnt the colour appearance of the stylus and skin textures. These patterns were used to distinguish the stylus and the skin from the background during the online process. Since colour is used to detect the stylus, it was enveloped with a blue cover (Fig. 14). The user’s hand did not require any treatment to get a good skin detection. To help the algorithm identify the tip of the stylus, the gimbal link connected to the stylus was coloured in green. This piece will be called simply “gimbal” from now on. The online detecting application was implemented in C++ using the OpenCV library1 . Since the colour appearances were built using hue histograms [31], the first step of the online procedure algorithm was to convert the input image from RGB to HSV colour space. After this, three different likelihood maps were built thresholding the probability that the given pixel belongs to the stylus, gimbal and skin textures (Fig. 14). Then, a contour detection was applied to the stylus blue map, obtaining some stylus candidates. The extremes of each candidate were extended while stylus or skin textures were detected along its longitudinal direction. This refinement was necessary, as the user’s hand creates small gaps in the stylus colour (Fig. 14). To select the correct candidate, the stylus colour region having a skin texture along it and a gimbal colour in one of its extremes was chosen. If more than one candidate satisfied these restrictions, the candidate with the largest area was retained. The gimbal in green solved the ambiguity of stylus orientation. Once the stylus position and orientation were set, the stylus was covered starting from its tip until a skin texture was detected, which after subtracting the distance from point A (Fig. 14) coincided with the position of the user’s grasp. Finally, a smoothing filter [32] was applied to the estimated position to prevent possible abrupt changes due to possible partial occlusions or image blur. 1 http://sourceforge.net/projects/opencvlibrary/

0.75 0.7 0.65 User grasping at position 3 0.6 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8 Displacement (rad)

Fig. 14. (Top-left) Stylus texture map with its candidates (gray boxes). (Topright) Gimbal texture map. The circle represents the gimbal, the line the stylus and the square the tail of the stylus. (Bottom-left) Skin texture map. (Bottomright) Output image with the stylus detection superimposed. The blue circle represents the tip of the stylus and the green circle the user grasping position.

Displacement (rad)

0.8

0.75 0.7 0.65 User grasping at position 5 0.6 0

0.1

0.2

0.3 0.4 Time (s)

0.5

0.6

0.7

Fig. 15. Contact against the adaptive virtual wall with the user grasping the stylus at positions 1, 3 and 5. Dashed line indicates the wall placement.

C. Experimental responses A number of experimental collision responses to the adaptive wall have been recorded to analyse the behaviour of this strategy. The same 13 participants of the experiments of Section II were recruited for these experiments and the same virtual scenario as in Section IV-B (Fig. 11) was used. Again, the virtual wall was placed at φ = 0.7 rad (as in the previous experiments) to achieve the exact correspondence to the plane x = 0. The instructions given to the users were to collide at the marked point of the wall in a similar way (in terms of velocity and force) holding the device with the tripod grasp. They could collide as many times as they wanted (at least three times for each position). After several contacts in a specific grasping position, they were allowed to change the grip position to a farther position (outside the virtual wall) and perform more collisions (without stopping the application). An external supervisor was checking that the participants covered the six possible grasping positions. All the participants reported quite similar transition responses. The mean value of the settling times for grasping position 5 was 0.185 ± 0.04 s, and for grasping position 3 the mean settling time was 0.183 ± 0.038 s. Fig. 15 shows the collision responses of one of the users for grasping positions 1, 3 and 5. The experiments that had the most similar conditions (velocity before the contact and force exerted) were selected for the plot. The expected specification for the damping factor (ζ ≥ 0.5) was fulfilled for all users in all grasping positions. Therefore, the adaptive haptic rendering can be successfully

IEEE/ASME TRANSACTIONS ON MECHATRONICS, VOL. 17, NO. 1, PP. 174-182, FEBRUARY 2012

used to compensate the influence of user grip position. It is important to note that, although the impedances were tuned using the models for user 1, the adaptive rendering worked properly for all the subjects. This result also confirms the findings reported in Section III: system dynamics varied more significantly with the grasping position than with different users. VI. C ONCLUSIONS AND F UTURE W ORK The influence of user grasping position on haptic system dynamics and its effect on the haptic rendering of rigid contacts has been thoroughly studied. It is evident that the presence of the user in the control loop is a persistent challenge for modelling system dynamics. In this study, two different mechanical models have been used to characterise the system’s transfer function. As the user holds the stylus farther from its tip, system dynamics become simpler, in terms of the number of poles and zeros needed to fit it. This phenomenon indicates that user grasping position strongly modifies the system’s behaviour. Moreover, the analysis of the model parameters demonstrate that, despite user variability, the system is affected by the grasping position more significantly than by the different end users. Studying the phase margin of the system proved to be an adequate tool for determining the damping factor of the transient response to rigid contacts. This characteristic of the frequency response is also highly affected by the user graping position, resulting in very different transient properties to virtual collisions. This knowledge has been applied in the design of an adaptive haptic rendering strategy that was able to impose a predefined value for the damping factor, compensating for the influence of the grasping position. Furthermore this work clearly illustrates how control engineering tools are a powerful way to analyse the performance of haptic interfaces, leaving room for further studies and applications. The conclusions of this study could be extended to other devices or kinematic configurations, and also to different ways of grasping or grip force levels. R EFERENCES [1] L. J. Stocco, S. E. Salcudean, and F. Sassani, “Optimal kinematic design of a haptic pen,” IEEE-ASME Trans. Mechatron., vol. 6, no. 3, pp. 210– 220, September 2001. [2] K. Vlachos and E. Papadopoulos, “Transparency maximization methodology for haptic devices,” IEEE-ASME Trans. Mechatron., vol. 11, no. 3, pp. 249–255, June 2006. [3] U. Mali and M. Munih, “Hife-haptic interface for finger exercise,” IEEEASME Trans. Mechatron., vol. 11, pp. 93–102, 2006. [4] A. H. Zahraee, J. K. Paik, J. Szewczyk, and G. Morel, “Toward the development of a hand-held surgical robot for laparoscopy,” IEEE-ASME Trans. Mechatron. [5] A. D. Greer, P. M. Newhook, and G. R. Sutherland, “Human machine interface for robotic surgery and stereotaxy,” IEEE-ASME Trans. Mechatron., vol. 13, no. 3, pp. 355–361, June 2008. [6] M. Darainy, F. Towhidkhah, and D. J. Ostry, “Control of hand impedance under static conditions and during reaching movement,” J. Neurophysiol., vol. 97, pp. 2676–2685, 2007. [7] E. de Vlugt, A. C. Schouten, and F. C. van der Helm, “Closedloop multivariable system identification for the characterization of the dynamic arm compliance using continuous force disturbances: A model study,” J. Neurosci. Methods, vol. 122, pp. 123–140, 2003. [8] K. B. Fite, L. Shao, and M. Goldfarb, “Loop shaping for transparency and stability robustness in bilateral telemanipulation,” IEEE Trans. Robot. Autom., vol. 20, no. 3, pp. 620–624, June 2004. [9] T. Hulin, C. Preusche, and G. Hirzinger, “Stability boundary for haptic rendering: Influence of human operator,” in 2008 IEEE/RSJ Int. Conf. Intell. Robot. Syst., Nice, France, September 22-26 2008, pp. 3483–3488. [10] C. R. Wagner and R. D. Howe, “Mechanisms of performance enhancement with force feedback,” in WorldHaptics Conf., Pisa, Italy, 18-20 March 2005, pp. 21–29.

9

[11] S. Sirouspour and A. Shahdi, “Model predictive control for transparent teleoperation under communication time delay,” IEEE Trans. Robot., vol. 22, no. 6, pp. 1131–1145, December 2006. [12] H. I. Son, T. Bhattacharjee, and H. Hashimoto, “Enhancement in operator’s perception of soft tissues and its experimental validation for scaled teleoperation systems,” IEEE-ASME Trans. Mechatron. [13] J. E. Speich, L. Shao, and M. Goldfarb, “Modeling the human hand as it interacts with a telemanipulation system,” Mechatronics, vol. 15, pp. 1127–1142, 2005. [14] I. D´ıaz and J. J. Gil, “Influence of vibration modes and human operator on the stability of haptic rendering,” IEEE Trans. Robot., vol. 26, no. 1, pp. 160–165, February 2010. [15] K. J. Kuchenbecker, J. G. Park, and G. Niemeyer, “Characterizing the human wrist for improved haptic interaction,” in ASME Int. Mechan. Eng. Congress Expo., vol. 2, November 16-21 2003. [16] D. A. Abbink, “Task instruction: the largest influence on human operator motion control dynamics,” in WorldHaptics Conf., Los Alamitos, CA, USA, 2007, pp. 206–211. [17] L. Barb´e, B. Bayle, E. Laroche, and M. de Mathelin, “User adapted control of force feedback teleoperators: Evaluation and robustness analysis,” in 2008 IEEE/RSJ Int. Conf. Intell. Robot. Syst., Nice, France, September 22-26 2008, pp. 418–423. [18] A. Peer and M. Buss, “Robust stability analysis of a bilateral teleoperation system using the parameter space approach,” in 2008 IEEE/RSJ Int. Conf. Intell. Robot. Syst., Nice, France, September 22-26 2008, pp. 2350–2356. [19] J. E. Colgate and G. Schenkel, “Passivity of a class of sampled-data systems: Application to haptic interfaces,” J. Robot. Syst., vol. 14, no. 1, pp. 37–47, January 1997. [20] B. Hannaford and J.-H. Ryu, “Time domain passivity control of haptic interfaces,” IEEE Trans. Robot. Autom., vol. 18, no. 1, pp. 1–10, February 2002. [21] M. Kawai and T. Yoshikawa, “Haptic display with an interface device capable of continuous-time impedance display within a sampling period,” IEEE-ASME Trans. Mechatron., vol. 9, no. 1, pp. 58–64, March 2004. [22] M. K. O’Malley and M. Goldfarb, “The effect of force saturation on the haptic perception of detail,” IEEE-ASME Trans. Mechatron., vol. 7, no. 3, pp. 280–288, September 2002. [23] ——, “The effect of virtual surface stiffness on the haptic perception of detail,” IEEE-ASME Trans. Mechatron., vol. 9, no. 2, pp. 448–454, June 2004. [24] E. Gartley and D. Bevly, “Online estimation of implement dynamics for adaptive steering control of farm tractors,” IEEE-ASME Trans. Mechatron., vol. 13, no. 4, pp. 429–440, 2008. [25] K. J. Kuchenbecker and G. Niemeyer, “Modeling induced master motion in force-reflecting teleoperation,” in IEEE Int. Conf. Robot. Autom., Barcelona, Spain, April 2005, pp. 350–355. [26] J. Podobnik and M. Munih, “Haptic interaction stability with respect to grasp force,” IEEE Trans. Syst. Man Cybern. Part C-Appl. Rev., vol. 37, no. 6, pp. 1214–1222, November 2007. [27] M. J. Fu and M. C. C¸avuso˘glu, “Three-dimensional human arm and hand dynamics and variability model for a stylus-based haptic interface,” in IEEE Int. Conf. Robot. Autom., Anchorage, Alaska, USA, May 3-8 2010, pp. 1339–1346. [28] K. Ogata, Modern Control Engineering, 5th ed. Prentice-Hall Inc., 2009. [29] J. J. Gil, A. Avello, A. Rubio, and J. Fl´orez, “Stability analysis of a 1 dof haptic interface using the Routh-Hurwitz criterion,” IEEE Trans. Control Syst. Technol., vol. 12, no. 4, pp. 583–588, July 2004. [30] J. Kim, F. Janabi-Sharifi, and J. Kim, “A haptic interaction method using visual information and physically based modeling,” IEEE-ASME Trans. Mechatron., vol. 15, no. 4, pp. 636–645, August 2010. [31] G. Y. Jason, J. J. Corso, G. D. Hager, and A. M. Okamura, “Vishap: Augmented reality combining haptics and vision,” in IEEE Int. Conf. Syst. Man Cyb., Washington D.C., USA, October 2003, pp. 3425–3431. [32] J. J. Laviola, “Double exponential smoothing: An alternative to kalman filter-based predictive tracking,” in Immersive Projection Technology and Virtual Environments, Z¨urich, Switzerland, May 2003, pp. 199–206.