Development of a Bilateral Teleoperation System for Human Guided Spine Bone Fusion Surgery : BiTESS II Jongwon Lee1, Keehoon Kim1 Wan kyun Chung1 and Young Soo Kim2 1
Robotics and Bio-Mechatronics Lab., Pohang Univ. of Sci. and Tech., POSTECH, KOREA Center for Intelligent Surgery System, School of Medicine, Hanyang University, KOREA
CO
Abstract—Previous bone fusion surgery by surgeon has three major difficulties: lack of operation accuracy, surgeon's overexposure to radioactive contamination, and need of surgeon's intensive labor during operation. This paper proposes a bilateral teleoperation system for spine bone fusion surgery, BiTESS-II, to overcome those problems. In order to determine design specification of the system, we estimated human bone properties during gimleting and screwing process using the developed data acquisition systems. Based on the spine bone properties, we designed an end effecter, a slave robot and 2 master devices. The slave robot can perform surgical operation to gimlet cortical bone and to insert screws into human spine. Master devices are used to control the pose of the slave robot and to generate haptic information identical to the slave side. We also developed novel force reflection methods without force sensor so that the end effecter can be designed simple and light. The performance of the developed system was verified by experiments.
PY
2
O
I. INTRODUCTION
O
N
In the last few decades, many robotic systems have been applied to the field of surgical operation due to the advantages of accuracy and fatigue-free compared to surgeon’s manual operation [1]-[4]. For spine surgery, Shoham et al. developed a bone-mounted miniature robot, MARS, which guarantees relative position of needles and drills during operation [11]. Chung et al. developed SPINEBOT to guide surgeon's drilling motion [12]. Surgeons can operate spine surgery under guidance and assistance of MARS or SPINEBOT so that accuracy of surgical operations can be improved. However, in the true sense of robotic surgery, a robotic system needs to be participated in surgical operations directly interacting with a patient, since the accuracy can be more improved combined with preplanned computer aided surgical information. However, since robotic systems are not robust to unexpected disturbance and have insufficient intelligence to judge qualitative information if whole process is automated, it will cause delicate responsibility problem when there are some mistakes during operation. Therefore, a robotic system should be operated by surgeons'
D
surgery
commands in real time and teleoperated robotic surgery is a solution. Though ZEUS and da Vinci are also teleoperated robotic systems, they cannot only generate force information caused by the interaction with patients, but also they are not suitable for orthopedic surgery in the viewpoint of mechanical robustness. Since surgeons need tactile sensation positively during orthopedic operation, force reflection is important issue. In this paper, we propose a bilateral teleoperation system for spine bone fusion surgery. As shown in Fig.1(a), since unstable vertebra due to the loss of disc makes the patient suffer pain, screws are inserted into spine bone and fixed together by rods. Prior to the operation, surgeons examine CT scan images and plan how to perform the operation. Then, they incise skin and make a space for operation using k-wire and dilators. After they break protective outer shell (cortical bone) by hammering and insert screws into spine bones observing the status of screws and spine bones using fluoroscope as shown in Fig.1(b). Fig.2 shows whole procedure for spine bone fusion surgery. BiTESS-II covers the process of gimleting spine cortical bone and inserting screws into spine bones. In this procedure, the current operation method has three main difficulties; i) the operation accuracy, ii) harmful radio active exposure from fluoroscopes, and iii) fatigue from long operation time under inconvenient surgical environment 1. If a screw touches spinal
T
Keywords— Medical, surgical robot, spine surgery
(a) (b) Fig. 1 (a) Existing spine bone fusion surgery [5] (b) Environment for spine
1
It takes 2~ 2.5 hours for spine bone fusion surgery
(a) Fig. 3
PY
DOF)
Fig. 2 The task performed by BiTESS II among the steps for spine bone fusion surgery
Table I
BiTESS II SPECIFICATION
Position/Orientation accuracy
1mm/ 0.1˚ 20Nm 6 6 4 Not used
CO
Torque range for screwing Master DOF Motion Force reflection Slave DOF Motion Force sensor
(b)
BiTESS II: (a) Slave robot (6-DOF) (b) Master Devices (6-DOF/2-
(a) (b) Fig. 4 Shock Data Acquisition System: (a) Parts (b) Complete unit II. DATA ACQUISITION OF SPINE BONE PROPERTY
D
O
N
O
T
cord during operation, it makes serious problem. Moreover, since surgeons depend only on their visual and tactile sensation using fluoroscope images, their experiences are dominant factor to determine success rates. The repeated radioactive exposure makes the surgeon nervous to use fluoroscope frequently. In order to overcome such problems, we developed a bilateral human guided teleoperation system, BiTESS-II, with the specification shown in TABLE I and Fig.3. BiTESS-II consists of a haptic console with a master device operated by a surgeon and a slave robot which follows a surgeon's command to operate spine bone fusion. Using BiTESS-II, a surgeon operates the surgery apart from fluoroscopes at the patient's side using identical visual and tactile information to the slave side. Section II explains our force and torque data acquisition system to get spine bone properties. The property will be used to determine specifications to design BiTESS-II. In section III, the end effector, slave robot, and master devices in BiTESS-II are described. Section IV introduces novel methods, i.e., current monitoring method and RSP method, to generate reflecting force without force sensor. In section V, the function and performance of BiTESS-II will be verified through experiments followed by conclusion in section VI.
In order to determine the specification of a spine surgical robot system to gimlet (to break cortical bone) and to insert screws into human spine bone, we need to know how big force or torque is needed to complete the task. Unfortunately, to the best of the authors' knowledge, there have not been any studies of spine bone property during inserting a screw. This section shows how to determine the specification using a newly developed hammering and screwing system to estimate bone properties during the task. Fig.4 shows the developed hammering system. Ring type force sensor is included to get force data. The bandwidth of the system is 10(kHz) and the permissible force range is 15(kN). The system is applied to obtain pig spine bone and human spine bone properties. Though it is known that the bone property of a pig is similar to that of human, we need to confirm whether the bone property of pig and human are quantitatively coincided in order to use pig spine bone samples to test the developed robot system. Fig.5(a) shows a pig spine bone which is used in the experiment. Fig.5(b) and Table II show that the force ranges for gimleting have the value from 500(N) to 1200(N). The hammering system is also applied to human spine bones during real spine surgery. Table III shows the force ranges
Table II 3L
Subject 1
3R
4L
Avg.
Max.
Avg.
Max.
Avg.
Max.
599
1000
855.62
1172
789
1080
4R
Subject 1
PIG BONE PROPERTIES
5L
5R
Avg.
Max.
Avg.
Max.
Avg.
Max.
678
1057
817
1130
686
900
PY
(a)
Table III
HUMAN BONE PROPERTIES OF FEMALE 56 YEARS OLD (SUBJECT 1), MALE 55 YEARS OLD (SUBJECT 2), MALE 79 YEARS OLD (SUBJECT 3), FEMALE 54 YEARS OLD (SUBJECT 4) AND FEMALE 42 YEARS OLD (SUBJECT 5). 3L
3R
4L
Avg.
Max.
Avg.
Max.
Avg.
Max.
1
N/A
N/A
N/A
N/A
759
2
297
427
293
422
394
3
N/A
N/A
N/A
N/A
725
4
N/A
N/A
N/A
N/A
813
5
877
1175
777
1351
739
CO
Subject
4R
Subject
5L
956
428
1030 891
1010
5R
hammering.
Avg.
Max.
Avg.
Max.
1
841
1088
1062
1394
Max.
2
N/A
N/A
220
336
3
918
1155
524
604
4
951
1107
730
1096
798
893
5
650
1010
682
1358
714
1059
T
Avg.
(b) Fig. 5 Data Acquisition Experiment: (a) Pig spine (b) Force signal during
919
1078 338
965
O
265
793
D
O
N
of 5 subjects for each spine bone. Those maximum values are also from 400(N) to 1200(N). The values for gimleting depend on their age and gender. Therefore, we can conclude that the force larger than 1200(N) is needed to gimlet human spine bone and it is reasonable to use pig spine bone instead of human spine bone for gimleting test. After gimleting, the next procedure is insertion of screws. In order to know bone property during screwing, we developed a 1-DOF drilling bilateral teleoperation system as shown in Fig.6. When surgeons insert a screw into the spine bone, they estimate the position and orientation of the screw using fluoroscope. However, due to the radioactive exposure, it is inevitable to depend on the tactile sensing to know real-time status of the screw. Actually, the cortical bone and the sponge bone have different characteristics so that surgeons feel the relative position of screw with respect to spine bone. Therefore, we should determine the force reso-
Fig. 6 1-DOF drilling bilateral operation system.
lution to distinguish cortical bone from sponge bone via haptic interface as well as the maximum force to insert screw into spine bone. Fig.7(a) shows the maximum torques to insert a screw into sponge bone of pig spine. The value is less than 1.5(Nm) and the average value is 1.06(Nm). Fig.7(b) shows the maximum torques to insert screw into sponge bone through cortical bone of pig spine. The value is between 1.5(Nm) to 3.2(Nm) and the average value is 2.3(Nm). For the case of human spine bone, it is impossible to apply the system to real human because it is not permit-
PY Fig. 8 End effecter of the slave robot
(a)
CO
end effecter, a slave robot, two master devices, and a controller. A surgeon operates the slave robot via two master devices at the remote side and feels kinesthetic haptic information caused by interaction between the slave robot and human spine bone. A. End Effecter
T
(b) Fig. 7 Maximum
O
torque to insert a screw: (a) Maximum torques into sponge bone (b) Maximum torques into sponge bone through cortical bone.
D
O
N
ted by KFDA, yet. Therefore, it is needed to use surgeons' empirical values. Using the 1-DOF haptic device in Fig.6, we created virtual torque and 4 expert surgeons, who have many experiences of spine bone fusion surgery, compare the virtual torque with their empirical values. The range of the empirical value is between 1.5(Nm) to 3.2(Nm) as the red dotted line in Fig.7. We can conclude that the screwing forces of pig spine bone coincide with the surgeons' empirical value. Moreover, in order to generate different haptic information to distinguish sponge bone and cortical bone, our system will be designed to have relative torque sensing resolution less than 0.5(Nm). III. SYSTEM
Since the objectives of BiTESS-II are gimleting and screwing, the end effecter should be design to complete the tasks considering human spine bone property as explained in section II. In order to perform gimlet task to break cortical bone, high speed drill (10,000rpm) is used and the drill tip is changeable. For screwing task, Maxon RE-max 24 (20 Watt) with 86:1 gear ratio is used. Its stall torque is 20(Nm). Though its torque needs to be transmitted to the screw during inserting motion, the screw should be detached from the end effecter after completing the task of inserting screw into spine bone. The developed end effecter is designed to satisfy the above condition using a novel mechanism. Another good point of the end effecter is its screw motion during the screwing task so that only one motor is used to perform linear and rotary motion of the screw insertion simultaneously. The screw motion is also used to control linear motion of the high speed drill as shown in Fig.8.
DEVELOPMENT
This section shows a new developed bilateral teleoperation system, BiTESS-II, which can gimlet cortical bone and insert screws into human spine bone. It is consisted of an
B. Slave Robot The objective of a slave robot is to move the end effecter to preplanned position and orientation. If a task point of a patient's spine bone is on the medial plane as shown in Fig.10, 4-DOF is needed to complete the task. The developed slave robot with end effecter has 4-DOF except for drilling motion. Then, the slave robot also has ability to follow the movement of a patient caused by breathing in real time. Since the task is to manage hard material, i.e.,
PY
(a) (b) Fig. 11 Master Devices: (a) 6 DOF master device (b) 2 DOF master device Fig. 9 Kinematic Diagram for the Slave Robot.
CO
T and R are homogeneous transform of translation and rotation. Tx means translation in x direction and Rx means rotation about x axis. r is 500(mm). C. Master Device
T
A surgeon operates the slave robot and feels haptic information via master devices. We developed two master devices. One is to move the slave robot to position the end effecter to desired position and orientation. Another is to move high speed drill (gimleting) and controllable drill (screw insertion) at the end effecter. Fig.11 shows the developed master devices. Master-I in Fig.11(a) is the modified version of 4D4M[7]. It has 6-DOF motion space and 6DOF force reflection is available. Master-II in Fig.11(b) has 2-DOF screw motion space and 2-DOF force reflection is available.
O
Fig. 10 The task of closed loop type spine surgery robot system.
Table IV CHARACTERISTICS OF OPEN LOOP AND CLOSED
Passive mode Active mode Mobility
N
LOOP TYPE FOR SLAVE ROBOT
Stiffness Task
Low General Task
Open loop
Closed loop
O O O
O O
O
Type
△ High Specified Task
D
spine bone, strong reaction force is applied to the slave robot during drilling task, closed loop type is more suitable than open loop type. TABLE IV compares the merits and demerits of open loop type and closed loop type. From Fig.9, the forward kinematics of the developed slave robot can be described as: H= Rx(q4 )Ry (q1 )Tz (r)Ry (q2 )Tz (q3 )
IV. FORCE REFLECTION METHOD WITHOUT FORCE SENSOR Though tactile sensation is important information during spine bone fusion surgery, the end effecter becomes heavy and bulky and contamination problem arises if force sensors are used. Moreover, since we expect that the developed end effecter can be adaptable to any kind of surgical robots, the end effecter needs to be designed light and simple. This section discusses novel force reflection methods without force sensor, current monitoring method to calculate interaction torque for drilling motion during gimleting and inserting screws and RSP method to calculate interaction force between the slave robot and environment caused by collision with unexpected obstacles and workspace limitation.
PY
Fig. 12
Setup for current monitoring method.
CO
Fig. 14 Whole system diagram for BiTESS_II. Then, Fig.13 shows the results of current monitoring signal and the calculated torque. The current monitoring signal is well matched with the real torque in low frequency region. B. RSP Method
N
O
T
In teleoperation, since the operator cannot see the wholesituation at the slave side, collision between links or by unexpected obstacle can be occurred. Therefore, the interaction force and its direction should be transmitted to the human operator. Since the slave robot has no force sensor, p-p type force reflection should be used. However, since the kinematics and DOF of the master device and slave robot are different, RSP (Restriction Space Projection) method was implemented [9] [10]. RSP method using IRS(Instantaneous Restriction Space) concept is especially useful to calculate accurate direction of reflecting force when the slave robot collides with unexpected obstacles during the motion. Reflecting forces are calculated using Jacobian and joint angle errors. Jacobian of the slave robot can be calculated as follows.
Fig. 13 Current monitoring signal and calculated torque(scaled).
O
A. Current Monitoring Method
D
The idea to calculate drilling torque without sensors is to use current signal of the amplifier since the monitoring current is proportional to the load at the motor theoretically. Fig.12 shows a system to confirm if the current monitoring signal can be used instead of force sensor. Since we know the mass and inertia of slave link, the applied torque can be calculated as follows.
τe = mglsinθ + u + Iθ’’ ≈ mglsinθ + u u = Kpe + Kpe’
⎡ R cos(q1 ) ⎢ 0 ⎢ ⎢ − R sin(q1 ) J sr = ⎢ 0 ⎢ ⎢ 0 ⎢ 0 ⎣
0 0 0 0 1 0
0⎤ 0 ⎥⎥ 0⎥ ⎥, 1⎥ 0⎥ ⎥ 0⎦
(2)
.
where X sr = J sr qsr . X sr ∈ R and qsr ∈ R are the 6
3
pose and joint angles of the slave robot, respectively and the forward kinematics can be expressed as follows using Fig.9 and Fig.10.
X sr = [ xsr
ysr
zsr θ srx θ sry θ srz ]T
= [r sin(q1 ) 0 r cos(q1 ) q4
q2
0] . (3)
RG = C ( J ) ⊥ ∈ IRSG , and RE = [{C ([ J 2 , J 3 ," , J n ]) ⊕ IRSG } , ⊥
{C ([ J1 , J 3 ," , J n ]) ⊕ IRSG } ,"
(5) (6)
Where C ( J ) = C ( I − JJ ) and #
IRSG ∈ R r − C ( J ) . Then, RG and RE are calculated as
N
O
Fe 2 are external forces at the slave robot and end effecter. FRG and FRE are reflecting forces generated by RSP method. FR 2 is scaled reflecting force calculated from current monitoring signal at the end effecter. Psr and Pse mean the slave robot and the end effecter.
(8)
(9)
RG : X d 1 ∈ R 6 → FRG ∈ IRSG ,
(10)
RE : eq1 ∈ R3 → FRE ∈ IRS E ,
(11)
D
Pm 2 , respectively. IK means inverse kinematics. Fe1 and
T
0⎤ 0 ⎥⎥ 0⎥ ⎥, 0⎥ 0⎥ ⎥ 1⎦ 0⎤ 0 ⎥⎥ 0⎥ ⎥. 1⎥ 0⎥ ⎥ 0⎦
(7)
O
0 1 0 0 0 0 0 0 0 0 1 0
force command to control master-I and master-II, Pm1 and
CO
{C ([ J1 , J 2 ," , J n −1 ]) ⊕ IRSG }⊥ ],
⎡ sin(q1 ) ⎢ 0 ⎢ ⎢cos(q1 ) RG = ⎢ ⎢ 0 ⎢ 0 ⎢ ⎣ 0 ⎡ cos(q1 ) ⎢ 0 ⎢ ⎢ sin(q1 ) RE = ⎢ ⎢ 0 ⎢ 0 ⎢ ⎣ 0
Fig. 15 Experimental Setup.
(4) ⊥
⊥
PY
RSP matrices, RG and RE are defined from 2 as
where X d 1 is the pose of the master device and eq1 is joint
space error at the slave side as shown in the implemented whole system diagram (Fig.14). Fh1 and Fh 2 are human
V. EXPEREMENT
Since the developed BiTESS-II lies on the step of development, it is premature to apply to human spine bone directly. So, in this experiment, we use pig spine bone to estimate the performance of the system. The objective of the experiment is to insert screw at the preplanned position and orientation accurately less than 1mm and 0.1˚ error by using BiTESS-II as shown in Fig.15. The procedure of the experiment is as follows: 1. Using visual information at the monitor, the operator controls master device to coincide the position and orientation of the end effecter with preplanned desired pose and fix the pose of the slave robot. 2. Using master II, the operator controls high speed drill to break cortical bone feeling haptic information. 3. If high speed drilling is completed, the slave robot automatically pulls out the high speed drill and rotates the end effecter to use controllable drill. 4. Using master II, the operator controls controllable drill to insert a screw into the pig spine bone considering haptic information.
(b)
PY
(a)
based on the spine bone properties from pigs and human subjects. By using this system, surgeons can operate the surgery from the remote side avoiding radioactive exposure of fluoroscopes. A novel force reflection method without force sensor makes the slave robot simple and light. As the result of the method, surgeons feel the interaction force during screwing process with the presence of visual and tactile sensation at the slave side. Though BiTESS-II has not been applied to human alive yet, the experiment using pig spine bone verifies the function and the performance using the developed system and algorithms sufficiently.
ACKNOWLEDGMENT
CO
(c) (d) Fig. 16 Experimental Results: (a)Composition of pig’s spine bone (b) High
This research was supported in part by a grant(02-PJ3PG6-EV04-0003) of Ministry of Health and Welfare, by the International Cooperation Research Program (M6-0302-000009-03-A01-00-004-00) of the Ministry of Science and Technology, by the National Research Laboratory (NRL) Program (M1-0302-00-0040-03-J00-00-024-00) of the Ministry of Science and Technology, and by a grant(M10214-00-0116) of the Ministry of Science and Technology Republic of Korea.
speed drilling process (c) Controllable drilling process (d) End of the operation.
T
REFERENCES
O
1.
(a) (b) Fig. 17 Displayed images of preplanned operation path and the direction of
N
the end effecter (The operator controls the direction of the end effecter (green line) to match the preplanned path (black line) displayed on the spine bone image) : (a)Before the operation (b) After the operation
D
O
Fig.16 shows the procedure performed in the experiment. The movie clips for the experiment can be downloaded at the website [8]. As a result, we can conclude that the process of high speed drilling was successfully performed to break the cortical bone at the accurate desired point and direction. Our proposed high speed drilling can replace the previous hammering process, successfully. After that, screw is inserted maintaining preplanned position and orientation accurately shown in Fig.17. VI. CONCLUSION
A bilateral teleoperation system, BiTEES-II was developed to improve the accuracy of screw insertion process
M. E. Allaf, S. V. Jackman, P. G. Schulam, J. A. Cadeddu, B. R. Lee, R. G. Moore, L. R. Kavoussi, "Laparoscopic visual field Voice vs foot pedal interfaces for control of the AESOP robot," Surgical Endoscopy, vol. 12, no. 12, pp. 1415 - 1418, December 1998 2. Kevin Bonsor, "How Robotic Surgery Will Work," http://electronics.howstuffworks.com/robotic-surgery1.htm 3. Gerhardus D, "Robot-Assisted Surgery: The Future is Here," Journal of Healthcare Management, vol. 48, no. 4 pp. 242 - 251, July 2003 4. Joanne Pransky, "Surgeon's realizations of RoboDoc," Industrial Robot, vol. 25, no. 2, pp. 105 -108, 1998IFMBE at http://www.ifmbe.org 5. Kim keejung, "Hernia of a disc," http://www.snubh.org/news/contents/Disk\_kim.pdf, pp. 1 – 36 6. Kuhn, J.L., Goldstein, S.A., Choi, K., London, M., Feldkamp, L.A., and Matthews, L.S., "Comparison of the trabecular and cortical tissue moduli from human iliac crests", Journal of Orthopaedic Research, vol.7, issue 6, pp. 876-884, 1989 7. VnVnet inc. website. [Online], http://www.vnvnet.com 8. [Online], http://home.postech.ac.kr/~khk/research/research.htm 9. ``Accurate Multi-DOF Kinesthetic Haptic Display Using Instantaneous Restriction Space,'' Keehoon Kim, Wan Kyun Chung, Youngil Youm, IROS, Edmonton, Canada, Aug.3-5, 2005. 10. ``Accurate Force Reflection for Kinematically Dissimilar BilateralTeleoperation Systems Using Instantaneous Restriction Space,'' Keehoon Kim and Wan Kyun Chung, International Conference on Robotics and Automation (ICRA), Orlando, USA, May 15-19, 2006.
12. ``Development of SPINEBOT for Spine Surgery,'' G. B. Chung, S. G.Lee, S. M. Oh, B. J. Yi, W. K. Kim, Y. S. Kim, J. I. Park, and S.H. Oh, IEEE International Conference on Intelligent Robots and Systems (IROS), Sendai, Japan, 2004.
D
O
N
O
T
CO
PY
11. ``Bone-Mounted Miniature Robot for Surgical Procedures: Concept and Clinical Applications,'' Moshe Shoham, Michael Burman, Eli Zehavi, Leo Joskowicz, and Eduard Batkilin, and Yigal Kunicher, IEEE Trans. on Robotics and Automation, Vol. 19, No. 5, October, 2003.
