Design and Validation of a Robotic Control Law for Observation of Deep-Ocean Jellyfish Rife, J.H. ; Rock, S.M. ; Stanford Univ., CA This paper appears in: Robotics, IEEE Transactions on Issue Date : April 2006 Volume : 22 , Issue:2 On page(s): 282 ISSN : 1552-3098 INSPEC Accession Number: 9067085 Digital Object Identifier : 10.1109/TRO.2005.862484 Date of Current Version : 10 April 2006 Sponsored by : IEEE Robotics and Automation Society

Final Version available at http://ieeexplore.ieee.org/search/srchabstract.jsp?tp=&arnumber=4107996

© 2006 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. This work is made available on the author’s web site in compliance with the rules set forth in Section 8.1.9 of the IEEE PSPB Operations Manual. http://www.ieee.org/publications_standards/publications/rights/rights_policies.html

Rife, Rock 2004

1

Design and Validation of a Robotic Control Law for Observation of Deep-Ocean Jellyfish Jason H. Rife, Member, IEEE, and Stephen M. Rock, Member, IEEE

Abstract—Limits on the technology available to marine scientists to study jellyfish in situ motivate the development of an automated robotic tracking system for deployment in the deep ocean. This paper synthesizes a control strategy for robotic jellyfish-tracking with a remotely-operated vehicle (ROV). The control strategy employs three feedback loops tailored to the jellytracking task. A primary loop provides moderate proportionalderivative (PD) feedback to track animal swimming motion without producing excessive hydrodynamic disturbances, which might impact the behavior of the animal under study. A second boundary-control loop provides aggressive thrust, in a direction away from the target and only when needed to prevent loss of the target outside the boundaries of the vision sensor. A third disturbance-accommodation loop counters low-frequency bias forces without fighting commands issued by the human pilot. The complete system was implemented and tested in the Monterey Bay using MBARI’s ROV Ventana. The control system autonomously tracked six animals for durations longer than fifteen minutes, including a Ptychogena medusa which was tracked for 89 minutes. Index Terms—Jellyfish, remotely operated vehicle (ROV), underwater vehicle control, visual tracking

T

I. INTRODUCTION

HE primary goal of a jellyfish-tracking robot is to observe and quantify the behaviors of gelatinous animals over extended durations of hours or days. At the Monterey Bay Aquarium Research Institute (MBARI), biologists are actively interested in the ecology of a range of gelatinous animals, including jellyfish (medusae), siphonophores, ctenophores, larvaceans, and salps. Unmanned, human-piloted submersibles, called Remotely Operated Vehicles (ROVs), represent the state-of-the-art technology for studying deep-ocean animals in their native environments (Figure 1). However, longterm studies with ROVs are limited by human fatigue. Automation, in the form of an ROV pilot assist, will greatly extend the capabilities of scientists to study individual specimens in situ for hours at a time. In the future, the incorporation of jelly-tracking technology in an Autonomous Underwater VeManuscript received November, 2004. This work was supported by the Monterey Bay Aquarium Research Institute (MBARI), Moss Landing, CA, and Packard Foundation Grants 98-3816 and 98-6228. This work is part of a joint collaboration between the Stanford Aerospace Robotics Laboratory and MBARI to study advanced underwater robot technologies. The authors are with Stanford University, Stanford, CA, 94305 USA (email: [email protected]; [email protected]).

Figure 1. MBARI’s ROV Ventana

hicle (AUV) system will enable even longer observation experiments. The main goal of this paper is to describe how the specific requirements of the jelly-tracking science task distinguish it from previously investigated applications of ROV control. The current study develops a control law for robotic jelly tracking and demonstrates its effectiveness as an ROV pilot assist during deep-ocean experiments. Automated systems that control the position of a submersible with respect to a target in the midwater are new. Some preliminary experiments in ROV-based target tracking have been conducted in confined test tanks [1]-[2]. Because of the relatively benign conditions of the test tank, these experiments focused chiefly on system design rather than vehicle control. In the ocean environment, additional challenges are associated with position sensing and with the definition of the jellytracking control law. A vision sensor was used to provide relative positioning measurements of the ROV with respect to the target animal. Development of the vision sensor was guided by prior experiments in vision-based station keeping above the ocean floor ([3]-[6]), which offer substantial insight into the sensing challenges associated with jelly tracking. Development of the ROV control law was likewise guided by previous studies, which have demonstrated successful ROV control using diverse techniques including linear methods [7][9], sliding-mode control [10-12], and adaptive feedback linearization [13-15]. The control law developed in this study is a PD controller supplemented by two special-purpose feedback loops. The

Rife, Rock 2004 primary design challenge is not gain selection, which may be accomplished by a range of standard approaches. Rather, the unique challenge of jelly tracking is the effective use of system redundancies to enhance scientific data collection. System redundancies exist because the vehicle has more actuated DoFs than the minimum required to accomplish jelly-tracking. By exploiting these redundancies, the complete three-loop control system accomplishes the following design objectives: (1) maintaining the target in a conical viewing region while (2) minimizing thrust to avoid disturbing the surrounding environment and (3) rejecting infrequent but large disturbance forces on the vehicle, such as those caused by snap loading of the tether. The three-loop control law implemented in the experiments is shown in Figure 2. The primary PD control loop tracks jelly swimming motions while exploiting the redundant thrust direction to minimize total thrust output. A Boundary Control loop, BC, provides strong actuation to counter sudden transients which might cause the target to exit the camera viewing cone, illustrated in Figure 3. A Disturbance Accommodation loop, DA, rejects quasi-steady external disturbances, such as persistent loading by the ROV tether, without canceling lowfrequency commands that the Human Pilot, HP, might issue. The latter two control loops, BC and DA, both exploit the redundant vehicle DoF to reduce thrust in the direction of the target animal. The body of this paper develops each of these three control loops. First, Section II summarizes the hardware employed in the experimental system to provide a context for the control design. Section III then develops the primary PD controller. The supplementary BC and DA loops are described sequentially in Sections IV and V. Section VI summarizes the complete control law and briefly discusses control sharing with the human pilot. Finally, Section VII describes tracking experiments performed using the complete control system.

2 II. EXPERIMENTAL HARDWARE A jelly-tracking system was implemented as a pilot aid on MBARI’s ROV Ventana. This remotely operated vehicle was tethered to the surface vessel, R/V Point Lobos. The tether carried power to the ROV and provided two-way telemetry. A stereo camera pair onboard the ROV served as the principal sensor for animal tracking. Video signals were transmitted via the tether and processed using a 700 MHz Pentium III computer onboard the Point Lobos. The vision-processing algorithm relied on a gradient-based detection scheme, which is described in detail in [16]. Video illustrating the performance of the vision-processing algorithm is available online [17]. An ROV control code, running in parallel with the vision processing code, transformed vision measurements into thrust commands at 10 Hz. The human pilot was free, at any time, to sum joystick control commands with those of the automated controller or to override it, in case of an emergency. The control code received several sensor signals in addition to the stereo-vision positioning signal. Potentiometers measured the pan and tilt angles of the camera mount with respect to the vehicle. A compass and a pair of inclinometers measured the ROV’s orientation with respect to the Earth. Although data from an RDI Workhorse Doppler Velocity Logger (DVL) was recorded for analysis, the DVL was not actively incorporated into the control loop. The ROV platform was designed for motion in four degrees of freedom (DoFs): three in translation and one in yaw. The vehicle’s strong buoyancy moment provided passive stabilization around the pitch and roll axes. During operations, pitch and roll excursions did not exceed five degrees from the trim angle. With these two DoFs constrained, the thrusters of the ROV Ventana provided actuation in the remaining four DoFs. Thrusters included a lateral tunnel thruster, a pair of vertical thrusters, and a pair of fore-aft thrusters that operated differentially to provide a yaw moment. III. PRIMARY CONTROL LOOP The primary control loop is a PD controller that acts to center the target animal in the main camera by controlling three DoFs: range, bearing, and depth. Stereo vision measurements are sufficient inputs for this PD controller; no other sensor measurements are required. In mapping the three controlled DoFs into the four vehicle thrust directions, it is convenient to implement the PD control

Figure 2. Block Diagram for Control System. Control components include a classical (PD) controller, boundary control (BC), and disturbance accommodation (DA). A thruster map (TM) optimizes the PD and BC controls. A switch determines whether or not the TM applies to DA control. Automatic control commands are summed with those of the human pilot (HP) and sent to the remotely operated vehicle (ROV). A camera sensor (S) and a compass (C) measure vehicle motion. A feedback loop enables animal tracking and rejection of external disturbances (W). All positions, p, and control commands, τ, are defined in the frame centered on the target jelly, except for the reference position, pref, which is defined in the frame of the camera sensor.

Figure 3. Boundary Control Keeps Target Animal in Camera Field of View

Rife, Rock 2004

3

law in a reference frame centered on the target jelly. Section III.A introduces the intermediate frames necessary to implement the jelly-centered control law. Section III.B describes the vehicle equations of motion. Section III.C explains the choice of the error vector coordinates and justifies the use of the jelly-centered frame. Section III.D develops the gains for the PD control law written in this frame. Section III.E maps the control commands into vehicle thrusts and optimizes these thrusts using the fourth actuator DoF. A. Reference Frames and Coordinate Systems Several frames must be defined to relate sensor measurements, vehicle thrusts, and inertial accelerations to the jellycentered frame in which the control law is defined (Figure 4). Altogether, four reference frames factor into the jelly-tracking task: the jelly-centered frame (j), a sensor–fixed frame (s), a vehicle-fixed frame (v) and a water-fixed frame (w). The camera measures target position in the sensor-fixed frame, s, which is rotated and translated relative to the vehicle frame but otherwise rigidly attached. Vehicle thrusters act in the vehicle-fixed frame, v. The jellyfish swims relative to the water-fixed frame, w. The water-fixed frame is treated as an inertial frame under the assumption of uniform and irrotational current. Because the vision sensor does not measure jelly orientation, the jelly-centered frame is defined with axes aligned to the water frame. Thus the jelly-centered frame translates, but does not rotate, relative to inertial coordinates. Within the jelly-centered frame, the coordinate set that decouples the three controlled DoFs from the fourth, uncontrolled DoF is labeled as tracking coordinates. In tracking coordinates, the relative position between the jelly and the vehicle is described using range, yaw bearing, and relative depth (r, ϕ , z). The fourth tracking coordinate is defined orthogonal to the controlled coordinates, so that the kinematics for the ROV in this uncontrolled direction lie in the nullspace of the controlled kinematics. For this reason, the fourth DoF is referred to as the nullspace coordinate (n). Using tracking coordinates, the position vector for the ROV, with respect to the jelly, is given by the expression below. The leading superscript for the position vector refers to the tracking coordinate system ( ) attached to the jelly-centered frame (j). The trailing subscript indicates that the vector describes the vehicle (v) position. j

pv = ⎡⎢ ⎣

j

rv

j

ϕv

j

zv

j

T

nv ⎤⎥ ⎦

are defined with x, y, and z axes, as shown in Figure 4. The fourth coordinate of the Cartesian state vector is yaw heading, ψ. As an example, the ROV state vector for Cartesian, jellycentered coordinates is: j

pv = ⎡⎢ j x v ⎣

j

j

yv

j

zv

T ψv ⎤⎥ . ⎦

(2)

These coordinates are measured by combining compass and vision sensor measurements. Cylindrical coordinates are defined with components of radius, circumferential angle, relative depth, and yaw heading (r, γ, z, ψ ). As an example, the state vector for cylindrical, jelly-centered coordinates is: ○j

pv = ⎡⎢ ○ j rv ⎣

○j

γv

○j

zv

○j

T ψv ⎤⎥ . ⎦

The radial and depth components of the cylindrical coordinate system are identical to those of the tracking coordinate system. The angular components for the two systems are different, however, as illustrated by Figure 5(a). Mathematically, the tracking and cylindrical coordinate systems are related by the following affine transformation, which incorporates a transformation multiplier, σ, and a vector shift, χ. j

pv = σ ○ j pv + χ

⎡1 0 ⎢ ⎢ 0 −1 ⎢ σ=⎢ ⎢0 0 ⎢ ⎢⎢ 0 1 ⎣

0 0⎤ ⎡ 0 ⎤ ⎥ ⎢ ⎥ ⎥ ⎢ −π ⎥ 0 1⎥ ⎥ , χ = ⎢⎢ ⎥ 1 0 ⎥⎥ ⎢ 0 ⎥ ⎥ ⎢ ⎥ 0 ⎥ 0 1 ⎥⎥ ⎣⎢ ⎦ ⎦

(4)

(5)

B. Equations of Motion The ROV is modeled as a 4-DoF lumped inertia, subject to disturbance forces. In the 4-DoF equations of motion, the generalized force vector, F, consists of three force components and one moment. The mass-inertia matrix, M, is diagonal. Forces considered include the vehicle thrust, τ , and disturbances, W. F = Mp = τ +W

(1)

Although tracking coordinates prove useful for decomposition of the control nullspace, Cartesian and cylindrical coordinates also play a role in jelly-tracking. The two element superscript notation, seen in (1), allows the reference frame and the coordinate system of state vectors to be distinguished. The default coordinate system (no symbol) is Cartesian. A circle ( ○ ) indicates cylindrical coordinates. A target ( ) indicates tracking coordinates. Altogether, seven combinations of reference frame and coordinate system are employed in this paper: j, s, v, w, ○ j, ○ v, j. Cartesian coordinates

(3)

Figure 4. Reference Frames: Jelly (j), Sensor (s), Vehicle (v), Water (w)

(6)

Rife, Rock 2004

4

Control commands are issued as acceleration commands in the jelly-centered frame. To reflect this, (6) can be rewritten to express the thrusts in units of acceleration in the jelly-centered frame: j τ . Here, the tilde indicates that thrust and disturbance terms are normalized to units of acceleration. j

τ =

j

pv -

j

W

(7)

The disturbances, jW , include tether traction, vehicle drag, and the inertial pseudoforces associated with conversion into the jelly-centered frame (centripedal, Coriolis and jellyfish acceleration terms). Jellyfish swimming motions are the dominant transient disturbances under nominal ocean conditions. The PD control law gains are designed to reject these jelly accelerations, w p j . The neutrally buoyant tether produces infrequent but very large snap loads under rough ocean conditions. The BC control was designed primarily to handle these large but infrequent disturbances. Even under nominal ocean conditions, the tether exerts a moderate, quasi-steady loading on the ROV. The DA control term is designed to estimate and counter this steady-state bias force, along with the quasi-steady drag associated with continuous jellyfish motion. The remaining disturbances (centripedal, Coriolis and transient drag) are negligible for the jelly-tracking application. If these terms were large, they would introduce coupling between the coordinate directions of (7). However, because transient drag, centripedal and Coriolis disturbances are negligible, the coordinate axes of (7) are decoupled. C. Specifying the Error Vector The jelly-tracking control law zeros errors in three coordiants: range, bearing, and depth. Although the specification of this error vector is straightforward in concept, the details of the jelly-tracking problem motivate a particular choice of reference frame and coordinate system for defining the error. Specifically, the error vector is constructed in jelly-centered tracking coordinates ( j) so that the kinematics of the coordinate axes, including the nullspace axis, are decoupled. Range, bearing, and depth errors can also be formed in the vehicle-fixed frame. In fact, the cylindrical vehicle-fixed error, ○v perr , is a mirror image of the jelly-centered tracking error, j perr . j

⎡I3 perr = − ⎢⎢ ⎢⎣ 0

0⎤ ⎥ 0 ⎥⎥ ⎦

○v

perr

contrast, because the coordinates of the circle are not fixed in the vehicle centered frame, the nullspace coordinate, which describes the vehicle position on this circle, is inherently a function of vehicle-frame range and bearing. For this reason, the dynamic equations written in the vehicle-fixed frame resemble (7) but with the addition of a time-dependent matrix that couples the nullspace acceleration to the range and bearing accelerations. Because it is desirable to maintain decoupling of the nullspace kinematics, the control law was formulated in the jelly-centered frame, rather than the vehicle frame. A second decision in developing the error vector involves the distinction between errors based on vehicle position, j pv , and on sensor position, j ps . This distinction is also referred to as a distinction between position-based visual servoing ( j pv case) and image-based visual servoing ( j ps case) [18]. The bearing angle and range error are not the same for these two cases. For this application, the position-based visual servoing approach was selected over the image-based approach, again to maintain the axis decoupling of (7). In image-based visual servoing, coupling enters through the thruster force, which must account for the sensor-to-vehicle moment arm, and through the centripedal and Coriolis pseudoforce terms, which are non-negligible for image-based visual servoing since the camera-to-target range is, operationally, much shorter than the vehicle-to-target range [19]. Although position-based servoing was selected over imagebased servoing for this application to maintain axis decoupling, the position-based approach lacks one important feature of the image-based approach. In image-based visual servoing, the error vector only goes to zero if the target lies at the center of the visual field. In position-based visual servoing this is not true. In the experimental system for instance, such an offset initially resulted from biases in the measurement of the camera pan and tilt angles. In order to eliminate this offset, the position-based visual servoing error must be based on a reference vector and a measurement vector which are both formed in the sensor frame and transformed to the jelly-centered frame through identical transformation equations. For this purpose, the control reference vector was defined in Cartesian sensor-frame coordinates, at a distance, zref, along the camera centerline.

(8)

Figure 5(b) illustrates this relationship between the two frames. Although range, bearing and depth are effectively identical for the j and ○ v coordinate systems, the fourth DoF distinguishes them. This fourth coordinate describes the ambiguity that exists when the tracking error is driven to zero; in this zero error state, the ROV may be positioned anywhere around a circle centered on the jelly. Because the coordinates of this circle are fixed in the jelly-centered frame, a nullspace coordinate can be defined that is orthogonal to yaw bearing. By

Figure 5. Coordinate Systems: (a) Cylindrical and Tracking Coordinates in Jelly-Centered Frame and (b) Tracking Coordinates in Jelly-Centered Frame and Cylindrical Vehicle-Fixed Coordinates. In this diagram, the jellyfish is denoted by a circle and the ROV by a square.

Rife, Rock 2004 s

pref = ⎡⎢ 0 0 ⎣

5 s

T

0 ⎤⎥ . ⎦

z ref

(9)

This reference vector, s pref , and the sensor measurement, s p j , can be transformed into the cylindrical vehicle-fixed frame using the sensor-to-vehicle mapping function, vSs , and the Cartesian-to-cylindrical transformation, ○vSv . The error can then be computed in the vehicle-fixed frame as follows. ○v

perr =

○v

S v ( vSs ( s p j ) ) −

○v

Sv ( vSs ( s pref ) )

(10)

The jelly-centered error, used for control, is simply the mirror image of the vehicle-fixed error, according to (8). D. Primary Control Law A PD control law can be written in jelly-centered coordinates using the error vector defined by (8) and (10). The PD law issues acceleration commands, j τPD , using diagonal gain matrices, K p and Kd . j

τPD = − ⎡⎢ K p ⎣

⎡ Kd ⎤⎥ ⎢⎢ ⎦⎢ ⎣

j j

perr ⎤ ⎥ ⎥ perr ⎥ ⎦

(11)

The gains for the PD control law, (11), are derived to reject jelly accelerations, w p j . A physical model for these accelerations has been proposed by Daniel [20]. Daniel’s jelly model assumes pulsed-jet propulsion, which describes most free-swimming medusae and siphonophores. Although the model does not apply to gelatinous animals that propel themselves by paddling, the derived control gains are sufficient to track slow paddlers (like ctenophores) but not faster specimens (such as larvaceans that have abandoned their feeding structures). Animal motions are governed by a first-order dynamic equation for animal velocity relative to the water: ( m + ma ) w p j + D ( w p j

) = τ jelly .

(12)

The inertial term includes the animal mass, m , as well as the added mass, ma , associated with accelerating the water medium. The drag term, D, and the thrust term, τ jelly , are unsteady over the jellyfish pulsing cycle. Daniel recommends a 1-D axial jet model for thrust and a high-Reynolds number model for animal drag. ∂V ∂ V ∂t ∂ t

τ jelly = −ρAv−1

D = 21 C d ρAf

w

pj

w

2

pj

(13)

(14)

In these equations, ρ is water density, Av is the jet crosssectional area, Af is the animal frontal area, V is the internal bell volume, and Cd is the medusa drag coefficient. Given a sensor with a sufficiently broad field of view, the tracking robot need not match the target animal’s acceleration over the entire pulsing cycle. Rather, the robot observer need

only track the target’s motion in a period-averaged sense (i.e during acceleration from rest to active swimming). By filtering out oscillations over the pulsing cycle, a tracking robot can reduce the frequency of thrust reverses, potentially reducing energy consumption and turbulence generation. The periodic component of (12) is characterized by the jelly pulsation frequency, which may be as low as 0.1 Hz for large animals and as high as 3 Hz for small animals [19]. The approximate bandwidth of oscillating motions, ωpulse , thus falls between 0.6 rad/s and 19 rad/s, depending on animal size and species. The period-averaged component of (12) is approximated as a first-order linear differential equation. For a first order system, the relationship between the bandwidth, ωB , and the 90% rise time, t90 , is: ωB = − ln(0.1)/ t90 .

(15)

The rise time is a function of the pulsing frequency and N, the number of pulses required to reach 90% of cruise velocity: t90 = 2πN / ωpulse .

(16)

For an accelerating jelly, N is approximately 4-5 pulsing cycles [20]. Combining (15) and (16) results in more than a factor of 10 difference between ωB and ωpulse . This order of magnitude separation means that a linear control law can be developed that tracks period-averaged velocity while rejecting oscillatory jelly motions. The control gains for the experimental system were chosen to deliver a closed-loop bandwidth of 0.7 rad/s. This choice damps most oscillation frequencies in the 0.6-19 rad/s range while capturing most of the bandwidth associated with acceleration from rest to steady swimming, in the 0.06-1.9 rad/s range. E. The Thruster Map This section discusses the conversion of acceleration commands, j τ , into thruster forces, T. In this conversion, the three DoF acceleration command for jelly tracking (range, bearing, and depth) can be augmented with an arbitrary command in the fourth DoF (nullspace coordinate). The selection of this arbitrary, underdetermined control command offers an opportunity for system optimization. For jelly-tracking, it is useful to select this command to minimize the ROV total thrust, in order to reduce the severity of hydrodynamic disturbances which might impact the behavior of the jellyfish specimen, its predators or its prey. The conversion of acceleration commands into thruster forces occurs in three steps. First, a software thruster map (TM) is defined that modifies the fourth row (nullspace component) of the acceleration command. j

τ = [TM ]

j

τPD

(17)

Second, the mapped acceleration commands, j τ , are rotated to align with the vehicle-frame axes and mapped through the mass matrix, M, to produce the thrust force and torque vector,

Rife, Rock 2004

6

τ . This step, in effect, relates the expressions for the thrust commands in the two motion equations, (6) and (7). The coordinate conversion involves the rotation matrix between the vehicle frame and the jelly frame, j Rv , the conversion matrix between the Cartesian and cylindrical coordinate systems, ○j Rj , and the cylindrical-to-tracking coordinate transformation, σ . −1

τ = M ( σ ○ j Rj j Rv )

j

τ

(18)

The final mapping step converts the fourth component of the τ vector, the yaw moment, into a force couple through the moment arm, l. The resulting thruster force vector, T, has four components with units of force. ⎡I3 0 ⎤ ⎥ T = λτ, λ = ⎢⎢ 0 1 l ⎥⎥ ⎣⎢ ⎦

(19)

The minimization of T requires an appropriate definition for the software thrust map, TM. Because the nullspace component of the acceleration command vector, j τ , is arbitrary, the nullspace row of the thruster map, ν , is also arbitary. ⎡G ⎤ TM = ⎢ ⎥ , G = ⎢⎡ I 3 ⎣ ⎢ν ⎥ ⎣ ⎦

0 ⎥⎤ ⎦

(20)

Thruster performance is optimized by defining the vector ν . For the jelly-tracking application, it is desirable to minimize the total thrust output of the ROV. The total thrust force can be written as the one-norm of T. When this norm is minimized the resulting j τ values can be used to determine ν : −1

T

1

= λM ( σ ○ j Rj j Rv )

j

τ . 1

(21)

The form of this one-norm can be simplified by exploiting the properties of unitary matrices. The rotation matrix j Rv is unitary. Also, the nonlinear transformation matrix, ○ j Rj , may be decomposed into a scaling matrix and a unitary rotation matrix. ○j

Rj =

○j

Rj ,sc ○ j Rj ,rot

(22)

The structure of (21) allows these unitary matrices to be commuted with the λM term. Since unitary matrices do not affect the value of a vector norm, the rotation matrices may be removed from (21) to give: T

1

= λM ○ j Rj−,sc1 σ −1

j

τ 1.

(23)

Expanding the terms of (23) gives the following result. T

1

= m(

+ 12 m j rv ( +ξ

j

τ2 +

j j

τ2 − j

j

τ1 + τ4 )

j

τ4

τ3

) (24)

In this equation, the parameter ξ expresses the ratio of vehicle inertia, as described by radius of gyration, κ , to the product of the thruster moment-arm l and the reference range from vehicle to target, j rref . ξ = κ2 / ( l j rref

)

(25)

It is equation (24) which must be minimized to obtain the optimal thruster map. In other words, the minimum vehicle thrust is equal to the minimum of the thruster force one-norm over all possible values of the mapped nullspace control command, j τ 4 . The minimum value of (24) must lie at a point where the derivative of T 1 with respect to j τ 4 is zero or at a point where the slope is discontinuous. The derivative is never zero, unless ξ = 1 . Thus for the general case, with ξ ≠ 1 , the minimum one-norm occurs at one of the two slope discontinuities of (24): j

⎧⎪ + τ 4 = ⎪⎨ ⎪⎪ − ⎪⎩

j

τ2 , ξ < 1

j

τ2 , ξ > 1

.

(26)

In the first instance, with ξ < 1 , the vehicle inertia is small relative to the yaw moment. The optimal nullspace thrust, j τ 4 , is equal the yaw-bearing thrust, j τ2 . By (5), this combination of yaw-bearing and nullspace thrusts sets the circumferential thrust to zero. Effectively, the vehicle is allowed to circle around the target animal while it continually adjusts its yaw heading to point at the target. In the second case, with ξ < 1 , the optimum result is zero yaw-heading thrust. Thus, the vehicle drifts in yaw while it continually adjusts its circumferential location to point at the jelly. The ROV Ventana, falls into the former category with ξ < 1 . The vehicle parameters ( κ =0.98, l=0.64) are balanced by a large range-to-target ( j rref generally exceeds 2.4 m, with about 1 m from camera to target and 1.4 m from camera to vehicle mass center). Consequently, yaw thrust outperforms circumferential thrust for the experimental platform. In this context, the thruster map of (20) is defined with the following ν vector, which sets the experimental vehicle’s circumferential thrust component to zero. ν = ⎡⎢ 0 1 0 0 ⎤⎥ ⎣ ⎦

(27)

As shown in Figure 2, the thruster map multiplies the tracking commands from the PD, BC and DA loops to produce the final control input for the automatic control system. IV. BOUNDARY CONTROL LOOP In practice, a PD tracking law tuned to jellyfish acceleration does not act rapidly enough to prevent major disturbances from causing a loss of vision lock. These large errors occur infrequently and result from snap tether loads, vehicle drag, and unmodeled nonlinearities in the dynamics of the thruster rotors [12]. An aggressive boundary control (BC) loop counters these transients before they cause loss of the animal target

Rife, Rock 2004

7

outside the camera viewing cone. The BC loop acts only when necessary, to avoid excessive thrusting except when demanded by the vision constraint. The BC term is a nonlinear operator that acts only when PD control fails to maintain the target animal in the central region of the camera image. As shown in Figure 6(a), the image plane is divided into an interior quiescent region (unshaded) and a boundary region (shaded gray). The BC term is zero if the target remains inside the quiescent region, but steps to a nonzero value if a transient disturbance carries the target into the boundary region. In effect, there are four boundary regions, i, and four BC components, s τBC ,i , one for each vertical and horizontal edge of the image plane. s

⎧ ⎪ 0 x pix ,i ≤ xq ,i τBC ,i = ⎪ . ⎨ ⎪ nˆ x pix ,i > xq ,i ⎪ ⎩ i

(28)

Here the image pixel coordinate toward the ith edge of the image plane is x pix ,i ; the quiescent region threshold is xq ,i . Each edge component is a unit vector, nˆi , directed toward the opposite edge of the image plane. The complete boundary control term, j τBC , sums the four edge components and scales them by a gain, δ . j

τBC = δ ⋅

j

⎛ 4 ⎞ Ss ⎜⎜⎜ ∑ s τbc,i ⎟⎟⎟ ⎜⎝ ⎠⎟

When the BC control term does command thrust, the actuation is directed away from the target animal, as shown in Figure 3. By construction, the BC control law only creates forces in a plane perpendicular to the camera centerline. Otherwise, large thrust levels would produce water jets that could easily disturb, displace or deform the animal specimen. Since the BC control law only produces thrust in a direction perpendicular to the target direction, the loop avoids generating a radial thrust that would capture the target animal in a powerful hydrodynamic jet. BC falls into the general category of feedback methods called variable-structure controllers, a category which also includes the well-known sliding-mode controller (SMC). Because the stability proof for SMC does not apply to BC, a separate stability proof is derived here. In particular, stability is considered for a 1-D linear system with both BC and PD control active. Here a one-dimensional scalar, p, is defined as a single component of the position error. The boundary of the quiescent central region, projected into the jelly-centered frame, is pq. For any value of pq, the following is true: p = τx = τPD,x + τbc,x

(30)

τPD,x = −K p p − Kd p

(31)

0, p < pq ⎧⎪ τbc,x = ⎪⎨ ⎪⎪ −δ sign(p), p ≥ pq ⎩

(32)

(29)

i =1

The transformation matrix jSs rotates the thrusts from the sensor frame into the jelly-centered objective frame. For the experimental system, δ was set to 60% of the ROV’s fullscale thrust output.

The control parameters are K p , Kd and δ , as defined earlier. The potential of this system, V, is a decrescent, positive definite Lyapunov function, as depicted in Figure 6(b). 2 1 1 2 ⎧⎪ p < pq 2 Kp p + 2 p , V = ⎪⎨ 2 2 1 ⎪⎪ ( K p p + δ p + p ), p ≥ pq ⎪⎩ 2

(33)

The time-derivative of the Lyapunov function, (33), is continuous and negative semidefinite for all p, p .

(a)

(b) Figure 6. Boundary control in the image plane. (a) The unshaded zone is the quiescent region and the shaded zone is the active boundary control region. (b) The Lyapunov Potential function for 2-D boundary control.

∂V = −Kd p 2 , ∀p ∂t

(34)

Thus V is stable in the sense of Lyapunov. LaSalle’s principle may be invoked to establish global asymptotic stability. The higher-dimensional BC control employed in the experimental system is a straightforward extension of this 1D concept. The extension of the proof for DA control is cumbersome, but also straightforward, since the DA control is based on a linear estimator which is stable when its poles are placed in the left-half of the complex plane. In a broad context, the BC control term may also be viewed as an obstacle avoidance strategy, in which the camera boundaries are treated as obstacles. The Lyapunov function thus defines a potential field for obstacle avoidance, which aside from its discontinuity resembles potential fields used for

Rife, Rock 2004

8

other applications, such as the visual manipulation of a robot arm [21]-[22] or automobile lane-keeping [23]. V. DISTURBANCE ACCOMMODATION LOOP Not all large disturbances act in a rapid and transient fashion. In particular, the ROV tether, though designed with neutral buoyancy, can still generate a large, nearly steady disturbance force on the tracking vehicle [24]. Vehicle buoyancy and drag associated with persistent jelly swimming may also result in quasi-steady disturbances. At the edge of the quiescent region, these low-frequency disturbances may result in chatter from the BC control law. Within the quiescent region, low-frequency disturbances may also result in a persistent steady-state error for the PD control law. In both cases, a bias canceling control law resolves the problem by estimating and counteracting the low-frequency bias. For the jelly-tracking application, a disturbanceaccommodation (DA) loop was used in place of an integral controller to accomplish bias canceling. Disturbance accommodation is a standard estimator-based control technique described in some introductory texts, such as [25]. One of the advantages of the DA loop, which uses an estimator to measure low-frequency external forcing on the ROV, is that the DA control, unlike integral control, does not compete with commands issued by a human pilot. The reason for this difference is that the estimator includes as inputs both the actual control command, j τ , and the current error measurement, j pv . By comparison, a standard integral controller only includes the current error state and does not account for the actual thrust command, which may be modified by the input of a human pilot [19]. A secondary advantage of the DA loop is that it continually estimates bias forces in all directions, including the null space direction in which the error is defined as zero. The estimator for the DA loop uses a state vector, qˆ , which consists of vehicle position, velocity, and bias force. The estimator states are posed in the Cartesian jelly-centered frame, since the estimated low-frequency bias force, jWˆ , is approximately steady in magnitude and direction in this frame. qˆ = ⎡⎢ j pˆv ⎣

jˆ pv

Wˆ ⎤⎥ ⎦

j

(35)

The state vector includes estimated representations of vehicle position and rotation, j pˆv , and of velocity, j pˆv . The estimated position is not used for control, however, since the measured position, j pv , is readily available. The dynamic equation for the estimated quantities is driven by the estimator gains, L: qˆ = Aqˆ + B j τ + L( j pv − Cqˆ) .

measurement, along with the vision measurement, to provide full four-state information. The estimator state-space matrices, A, B, and C , are defined to incorporate the vehicle dynamics, (6), and the assumption of a zero derivative for the bias term. This assumption is appropriate for modeling quasi-steady tether forces, which tend to act in a fixed direction with respect to the jelly-centered frame. As a reasonable approximation, the A matrix treats vehicle drag as linear and isotropic, with a drag coefficient b. ⎡0 ⎡ 0 ⎤ I 0 ⎤ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ A = ⎢ 0 −M −1b M −1 ⎥ B = ⎢ M −1 ⎥ C = ⎡⎢ I ⎣ ⎢ ⎥ ⎢ ⎥ ⎢⎢ 0 ⎢⎢ 0 ⎥⎥ 0 0 ⎥⎥ ⎣ ⎦ ⎣ ⎦

The DA control term acts opposite to this bias vector, jWˆ , as transformed into the objective coordinate system. By this action, the DA controller effectively cancels out long-term biases. j

τDA = − jWˆ = −σM −1 j S ○ j

( Wˆ ) j

(38)

By default, the thruster map zeros the circumferential component of the DA control, so that a bias force may still displace the ROV in the circumferential direction. Fortuitously, this action results in quasi-steady disturbances stabilizing the vehicle’s circumferential location in a way that minimizes radial thrust toward the target. Stabilization results because the circumferential component of the bias force vanishes at two equilibrium points. These equilibrium points may be derived by taking the vehicle dynamic equation, (6), to steady state in the presence of a horizontal quasi-steady bias force. This system, mathematically similar to a gravity-driven pendulum, has one stable equilibrium and one unstable equilibrium. The stable equilibrium occurs at the circumferential angle that aligns with the direction of the bias force; the unstable equilibrium lies 180° around the circumference, opposite to the action of the bias force. At the stable equilibrium, the tether bias always pulls the vehicle away from the tracked animal. The DA control opposes the tether bias and pushes the ROV forward by expelling water jets away from the tracked animal. Because the jets point rearward, thruster action does not present a significant hazard to the science mission even when quasi-steady disturbances are large. In fact, the quasi-steady bias provides a margin for transient disturbances. The ROV need not expel water jets toward the target unless the transient disturbance is opposite and greater in magnitude than the quasi-steady bias. VI. CONTROL SHARING

(36)

The inputs to the estimator dynamic equation include the total control signal, j τ , (which incorporates pilot control commands) and the measurement vector, j pv . In contrast with the BC and PD controllers, the DA controller uses the compass

0 0 ⎤⎥ (37) ⎦

The complete jelly-tracking control sums the inputs of all three automatic control loops with the human pilot command. j

τ = TM [

j

τPD +

j

τDA +

j

τBC ] +

j

τHP .

(39)

Rife, Rock 2004 In general, the human pilot issues no command after the automated system locks onto the animal specimen. For specialized tasks, however, the human pilot may still issue control input. For instance, a scientist viewing the video feed may request that the pilot circle around the animal specimen to view it from a new angle. The human pilot can issue control commands in the null space direction to achieve this goal with no impact on automated tracking. Conversely, because of kinematic decoupling, the human pilot need not compensate for automated control commands in the nullspace direction, so long as the thruster map, TM, is disabled. For this reason, the thruster map should be set to the identity matrix when the human pilot wishes to issue control commands. VII. BIOLOGY APPLICATIONS Ocean experiments were conducted to validate the jellytracking control concept. The final system was tested with four ROV dives in the Monterey Bay, CA, during 2002. Six fully automated tracking runs were recorded that lasted longer than 15 minutes. These runs tracked a variety of animals including Ptychogena (89 min.), Solmissus (25 min., 16 min.), Benthocodon (19 min.), and two sinking Larvacean houses (34 min., 17 min.). None of these runs ended due to a failure of the control law to keep the target in the camera field of view. The longest run, which tracked the Ptychogena shown in Figure 6, did end, however, when strong tether forces saturated the ROV thrusters. In this instance, no ROV control law could have maintained tracking. Of the other five extended runs, two ended after a visual-recognition failure, one ended when a cloud of squid ink visually occluded the target, one ended because the sinking larvacean house under observation reached the ocean floor, and the last ended by manual termination at the conclusion of a dive day. This section examines the Ptychogena tracking run in more detail in order to illustrate the potential of automated jellytracking to enable a new class of long-term in situ observation experiment. During this particular run, the DA control loop, described by (38), was disabled for testing purposes, but the estimator equation, (36), was still operational as a background process. Figure 8 shows the four-axis control signals (PD plus BC) expressed in the frame of the vehicle during the 89 minute experiment. In general, the vehicle fore-aft axis pointed directly toward the target animal. Thrust was predominantly forward and downward because the tether pulled backward and upward on the ROV. Lateral (roughly circumferential) thrusts were nearly zero except when the boundary control term became active. (Boundary control events appear as sharp spikes in the control signal.) As the tracked animal neared the ocean floor (after minute 40) the disturbance forces and control magnitude began to increase. Tether forces continued to increase over the course of the dive. At the end of the experiment, the tether force eventually grew large enough to overpower the ROV thrusters, pulling the vehicle way from the target at minute 89.

9 The apparent BC chatter in the last twenty minutes of the dive underscores the significance of the disabled DA control term. The DA control term is designed to eliminate steadystate error. During the second half of the experiment, the tether bias resulted in a steady-state error which placed the vehicle on the edge of the quiescent and boundary-control regions and caused BC chatter. If the steady-state error were removed by DA control, the boundary control term would not have been consistently active during this period, and chatter would have been avoided. During the tracking run, the vehicle’s circumferential position was determined by the direction of the quasi-steady bias force, as shown by Figure 9. The ROV completed one full revolution over the course of the dive. The tether direction, as well as the ROV’s circumferential coordinate and heading, shifted twice by 180°, at approximately 30 minutes and 45 minutes through the dive. These changes are illustrated in Figure 10(b). Fluid motion near the ocean floor may have driven these changes in tether forcing. The animal came within 5 m of the ocean floor after swimming continually downward for the first 33 minutes of the dive. After the animal stopped swimming (minute 33), it continued drifting in a horizontal helical pattern (20 m in diameter), as shown by the bottom-lock DVL track of Figure 11, which was recorded during minutes 35-56 of the dive. Taken together this dataset incorporates rich information about the target animal’s behavior and environment. The pressure depth data of Figure 10(a), for instance, suggests a possible connection between the cessation of downward swimming and the vortical boundary-layer event observed near the ocean floor. Ocean hydrodynamics apparently caused the jelly to rise several meters closer to the surface even as the jelly continued swimming in a downward direction, as indicated by the anomalous depth change from minutes 30-33 shown by Figure 10(a). Did the animal detect the benthic boundary layer and stop swimming downward to avoid collision with the ocean floor? Given a large number of tracking runs, such a stimulus-response mechanism could be further investigated.

Figure 6. Ptychogena Specimen Tracked Autonomously on 9/24/2002

Rife, Rock 2004

10 VIII. CONCLUSION

This paper developed a control strategy for an ROV-based jelly-tracking pilot assist. The strategy features a PD controller that holds the ROV on a circular reference about the target. A thruster map minimizes hydrodynamic outputs which might impact the animal specimen. Supplementary boundarycontrol and disturbance-accommodation terms counter large environmental disturbance forces using thrust directed away from the tracked animal. The complete system was tested in the Monterey Bay using MBARI’s remotely operated vehicle, ROV Ventana. Ocean experiments tracked multiple animals and validated the jelly-tracking control strategy as a tool to enable new science experiments in the deep ocean. In the future, such science experiments might apply the jelly-tracking technology both as an ROV pilot-assist, for complex experiments requiring human interaction, and as an AUV control system, for simple observation experiments operating over extremely long durations. AUV operations would provide cleaner tracking by eliminating the tether force,

a source for large ROV disturbances, as evidenced by the jelly-tracking experiments described in this paper. REFERENCES [1] [2] [3]

[4] [5] [6]

R.L. Marks, “Experiments in Visual Sensing for Automatic Control of an Underwater Robot,” Ph.D. dissertation, Dept. Aero. Eng., Stanford Univ., Stanford, CA 94305, 1995. C. Silpa-Anan, T. Brinsmead, S. Abdallah and A. Zelinsky, “Preliminary experiments in visual servo control for autonomous underwater vehicle,” Proc. Int. Conf. on Intelligent Robots and Systems, 2001, pp. 1824-1829. S.D. Fleischer, S.M Rock, and R. Burton, “Global position determination and vehicle path estimation from a vision sensor for real-time video mosaicking and navigation,” Proc. IEEE/MTS OCEANS‘97, vol. 1, 1997, pp. 641-647. J.-F. Lots, D.M. Lane, E. Trucco, and F. Chaumette, “A 2D visual servoing for underwater vehicle station keeping,” Proc. IEEE Int. Conf. on Robotics and Automation, vol. 3, 2001, pp. 2773-2778. S. Negahdaripour and Xun Xu, “Mosaic-based positioning and improved motion-estimation methods for automatic navigation of submersible vehicles,” IEEE J. Oceanic Eng., vol. 27, no. 1, pp. 79-99, 2002. S. van der Zwaan and J. Santos-Victor, “Real-time vision-based station keeping for underwater robots,” Proc. IEEE/MTS OCEANS’01, vol. 2, 2001, pp. 1058-1065.

Figure 8. Thrust commands during Ptychogena tracking run of 9/24/2002 in voltage units between full scale positive, +5, and full scale negative, -5. Thrusts in vehicle-fixed frame: (a) Fore-aft, (b) Lateral, (c) Vertical, and (d) Yaw.

Figure 10. Depth and Tether Bias Direction during the 89 minute run. The vertical line corresponds to the moment at which the Ptychogena ceased swimming downward. (a) This event occurred soon after the pressure depth began to increase. (b) The estimated direction of the tether force reversed by approximately 180° both five minutes before and ten minutes after the cessation of swimming.

Figure 9. ROV position relative to the tracked animal (which is depicted as a circle). The vehicle makes one complete circuit around the Ptychogena.

Figure 11. Motion track of ROV Ventana and Ptychogena over the ocean floor, as measured by the DVL during minutes 35-56 of the experiment.

Rife, Rock 2004 [7] [8] [9] [10]

[11] [12]

[13] [14]

[15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25]

G. Campa, M. Innocenti, and F. Nasuti, “Robust control of underwater vehicles: sliding mode control vs. mu synthesis,” Proc. IEEE/MTS OCEANS’98, vol. 3, 1998, pp. 1640-1644. G. Conte and A. Serrani, “Global robust tracking with disturbance attenuation for unmanned underwater vehicles,” Proc. IEEE Int. Conf. on Control Applications, 1998, pp. 1094-1098. K. Leabourne, S.M. Rock, and S.D. Fleischer, “Station keeping of an ROV using vision technology,” Proc. IEEE/MTS OCEANS’97, vol. 1, 1997, pp. 634-640. M.L. Corradini and G. Orlando, “A discrete adaptive variable-structure controller for MIMO systems, and its application to an underwater ROV,” IEEE Trans. Control Systems Tech., vol. 5, no. 3, pp. 349-359, 1997. J.P.V.S. da Cuhna, R.R. Costa, and L. Hsu, “Design of a high performance variable structure position control of ROVs,” IEEE J. Oceanic Eng., vol. 20, no. 1, pp. 42-55, 1995. D.R. Yoerger, J.G. Cooke, and J.-J.E. Slotine, “The influence of thruster dynamics on underwater vehicle behavior and their incorporation into control system design,” IEEE J. Oceanic Eng., vol. 15, no. 3, pp. 167178, 1990. G. Antonelli, S. Chiaverini, N. Sarkar, and M. West, “Adaptive control of an autonomous underwater vehicle: experimental results on ODIN,” IEEE Trans. Control Systems Tech., vol. 9, no. 5, pp. 756-765, 2001. D.A. Smallwood and L.L. Whitcomb, “The effect of model accuracy and thruster saturation on tracking performance of model based controllers for underwater robotic vehicles: experimental results,” Proc. IEEE Int. Conf. on Robotics and Automation, 2002, pp. 1081-1087. J. Yuh, J. Nie and C.S.G. Lee, “Experimental study on adaptive control of underwater robots,” Proc. IEEE Int. Conf. on Robotics and Automation, 1999, pp. 393-398. J. Rife and S. Rock, “Segmentation methods for visual tracking of deepocean jellyfish using a conventional camera,” IEEE J. of Oceanic Eng., vol. 28, no. 4, pp. 595-608, 2003. J. Rife and S. Rock, “Video of automated robotic tracking of deep-ocean jellyfish,” http://arl.stanford.edu/~jrife/JellyTrackVideo.htm, posted May 5, 2005. This material is also available at http://ieeexplore.ieee.org. S. Hutchinson, G.D. Hager, and P.I. Corke, “A tutorial on visual servo control,” IEEE Trans. on Robotics and Automation, vol. 12, no. 5, pp. 651-670, 1996. J. Rife, “Automated Robotic Tracking of Animals in the Deep Ocean,” Ph.D. dissertation, Dept. Mech. Eng., Stanford University, Stanford, CA 94305, 2004. T.L. Daniel, “Mechanics and energetics of medusan jet propulsion,” Canadian J. of Zoology, vol. 61, pp. 1406-1420, 1983. P.I. Corke and S.A. Hutchinson, “A new partitioned approach to imagebased visual servo control,” IEEE Trans. on Robotics and Automation, vol. 17, no. 4, pp. 507-515, 2001. Y. Mezouar and F. Chaumette, “Path planning in image space for robust visual servoing,” Proc. IEEE Int. Conf. on Robotics and Automation, 2000, pp. 2759-2764. J.C. Gerdes and E.J. Rossetter, “A unified approach to driver assistance systems based on artificial potential fields,” ASME J. of Dynamic Systems, Measurement and Control, vol. 123, pp. 431-438, 2001. T.W. McLain and S.M. Rock, “Experimental measurement of ROV tether tension,” Proc. ROV ‘92, 1992. G. Franklin, J.D. Powell, A. Emami-Naeini, Feedback Control of Dynamic Systems, Addison Wesley, 1994, ch. 7.8.3.

Jason H. Rife (M’01) received the B.S. degree in mechanical and aerospace engineering from Cornell University, Ithaca, NY, in 1996, and his M.S. and Ph.D. degrees in mechanical engineering from Stanford University, Stanford, CA, in 1999 and 2004. Before commencing the M.S. degree program, he spent one year working in the turbine aerodynamics group of the commercial engine division of Pratt & Whitney in East Hartford, CT. He is currently a research engineer at Stanford University, with the Stanford GPS lab, studying precise navigation for aircraft landing and approach. Stephen M. Rock (M’94) received the B.S. and M.S. degrees in mechanical engineering from the Massachusetts Institute of Technology, Cambridge, MA, in 1972, and the Ph.D. degree in applied mechanics from Stanford University, Stanford, CA in 1978.

11 He joined the Stanford faculty in 1988, and is now a Professor in the Department of Aeronautics and Astronautics. He is also an adjunct engineer at the Monterey Bay Aquarium Research Institute. Prior to joining the Stanford faculty, he led the Controls and Instrumentation Department of Systems Control Technology, Inc. In his 11 years at SCT, he performed and led research in integrated control; fault detection, isolation, and accommodation; turbine engine modeling and control; and parameter identification. His current research interests include the development and experimental validation of control approaches for robotic systems and for vehicle applications. A major focus is both the high-level and low-level control of underwater robotic vehicles.