Caging and Swarm Manipulation Experiments: A Massive User Study with Large Swarms of Simple Robots Aaron Becker, Chris Ertel, and James McLurkin Abstract— Micro- and nano- robotics have the potential to revolutionize many applications including targeted material delivery, assembly, and surgery. We want to use large swarms of robots to perform manipulation tasks. Manipulation by caging can simplify these tasks, and is robust to control disturbances and perception uncertainty. We present user-studies with up to 100 hardware robots using compliant caging, and preliminary results from SwarmControl.net, an online game where players steer swarms of up to 500 robots to complete manipulation challenges. We record statistics from thousands of players, and use the game to explore aspects of large-population robot control. We present the game framework as a new, opensource tool for large-scale user experiments. Our results have potential applications in human control of micro- and nanorobots, supply insight for automatic controllers, and provide a template for large online robotic research experiments.

(a) Vary Number

(b) Vary Visual Feedback

I. I NTRODUCTION Micro- and nano- robotics have the potential to revolutionize many applications including targeted material delivery, assembly, and surgery. The same properties that promise breakthrough solutions—small size and large populations— present unique challenges to generating controlled motion. Limited computation and communication rules out autonomous operation or direct control over individual units; instead we must rely on global control signals broadcast to the entire robot population. It is not always practical to gather pose information on individual robots for feedback control; the robots might be difficult or impossible to sense individually due to their size and location. It is often possible to sense global properties of the group, such as mean position and density. Finally, many promising applications will require direct human control, and user interfaces to thousands—or millions—of robots is a daunting human-swarm interaction (HSI) challenge. There is currently no comprehensive understanding of user interfaces for controlling multi-robot systems with massive populations. We are particularly motivated by the sharp constraints in micro- and nano-robotics. For example, fullstate feedback with 106 robots leads to operator overload. Our work with over a hundred hardware robots and thousands of simulated robots [1] demonstrates that direct human control of large swarms is possible. Unfortunately, the logistical challenges of repeated experiments with over one hundred robots prevented large-scale tests. Our goal is to test scenarios involving large-scale human-swarm interaction A. Becker is with the Pediatric Cardiac Lab, Boston Children’s Hospital and Harvard Medical School, Boston, MA 02115 USA [email protected] C. Ertel, and J. McLurkin are with the Computer Science Department, Rice University, Houston, TX 77005 USA {cre1, jmclurkin}@rice.edu

(c) Vary Control

(d) Vary Noise

(e) Control Position

Fig. 1. Screenshots from our five online experiments controlling multirobot systems with limited, global control. Many of these tasks were best accomplished with full or partial caging. (a) Varying the number of robots from 1-500 (b) Comparing 4 levels of visual feedback (c) Comparing 3 control architectures (d) Varying noise from 0 to 200% of control authority (e) Controlling the position of 1 to 10 robots.

(HSI), and to do so with a statistically-significant sample size. Towards this end, we created SwarmControl.net, an open-source online testing platform suitable for inexpensive deployment and data collection on a scale not yet seen in swarm robotics research. Screenshots from this platform are shown in Fig. 1. All code is open-source [2]. Additionally, all experimental results are posted online. Our experiments show that numerous simple robots responding to global control inputs are directly controllable by a human operator without special training, that the visual feedback of the swarm state should be very simple in order to increase task performance, and that humans perform swarmobject manipulation faster using attractive control schemes than repulsive control schemes. II. M OTIVATION AND R ELATED W ORK A. Global-control of micro- and nano-robots More recently, robots have been constructed with physical heterogeneity so that they respond differently to a global, broadcast control signal. Examples include scratchdrive microrobots, actuated and controlled by a DC voltage signal from a substrate [3], [4]; magnetic structures with different cross-sections that could be independently steered [5]; MagMite microrobots with different resonant frequencies

increasing t −→

manipulation

3)

[22] was limited to a pool of 18 participants, [20] 5, and [21] 32. Using an online testing environment, we are conducting similar studies but with much larger sample sizes. III. T ECHNICAL A PPROACH A. User Study: Multi-Robot Manipulation Experiment

orientation control

4)

assembly

5)

To demonstrate the feasibility of human control of many simple robots, we performed experiments on three platforms, the r-one (n=8), the kilobot (n=101), and a simulated environment (n=2000). We used the r-one robot [23] to contrast three different control architectures, and the Kilobot robot platform [24] to demonstrate manipulation tasks with large populations of robots. Finally, in simulations we applied the same control techniques to control thousands of robots. We compare three n-robot system architectures with the following motion models:         x˙ i v cos(θi ) v cos(θi ) v cos(θi )  y˙ i  = δai  v sin(θi ) ,  v sin(θi ) ,  v sin(θi )  (1) i ω k sin(ψ − θi ) ω θ˙i {z } | {z } | {z } | A DDRESSABLE

L OCAL

G LOBAL

3

Fig. 2. Three tasks with 101 Kilobots: manipulating an object through a maze, manipulation with orientation control, and assembly. In each task the light source’s position was the only input to the robot population. This input is robust to individual robot failures—several robots failed each test due to glitches, poor calibration, or low batteries—but each demonstration was ultimately successful.

and a global magnetic field [6]; and magnetically controlled nanoscale helical screws constructed to stop movement at different cutoff frequencies of a global magnetic field [7]. Our previous work [8], [9] focused on exploiting inhomogeneity between robots. These control algorithms theoretically apply to any number of robots, but in practice process noise cancels the differentiating effects of inhomogeneity for more than tens of robots. We desire control algorithms that extend to many thousands of agents. B. Caging Manipulation A conceptually simple form for manipulation is to grasp a component by encircling it with robots in such a way that the grasp can resist any external force applied to the component. Rimon and Blake introduced caging to robotics [10] for nonconvex objects and two fingered gripers. Similar centralized planners include Davidson and Blake [11], Ponce et al. [12]– [15], and Wang and Kumar [16]. Kumar et al. extended this work for decentralized control [17] and obstacle avoidance [18], [19]. With uniform inputs, traditional caging algorithms may not be possible, so we investigate robot and task designs that are suited for caging-style manipulation. C. Human-Swarm Interaction A number of user studies compare methods for controlling large swarms of simulated robots, for example [20]–[22]. These studies provide insights but are limited by cost to small user studies, have a closed-source code base, and focus on controlling intelligent, programmable agents. For instance

The state of the ith robot is [xi , yi , θi ] ∈ R and the state of the system is ∈ R3n . 1) Objective: compare the three system architectures of Eqn. (1) with hardware experiments using n robots to manipulate objects to goal positions and orientations. Many users implemented caging methods to complete the task. 2) Equipment: This experiment used 1 to 8 r-one robots [23]. The r-one is a low-cost, open-hardware, differential-drive robot with a 10 cm diameter circular profile shown in Fig. 3. For each technique, the inputs are velocity and turning rate (by tilting the controller forward/backwards to control velocity and tilting it side to side to control turning rate.) The inputs are low-pass filtered to provide a slow control. 3) Task: Using each controller technique, a human user will steer the robots from a starting position in the bottom left side of the environment, to move an object to the goal region (a position and an orientation). The experimenter records the position/orientation of the object using a tracking system and the time required for each task. B. Online Game for Human-Swarm Experiments We have developed a flexible testing framework for online human-swarm interaction studies. There are two halves to our framework: the server backend and the client-side (inbrowser) frontend. The server backend is responsible for tabulating results, serving webpages containing the frontend code, and for issuing unique identifiers to each experiment participant. The in-browser frontend is responsible for running an experiment—that is to say, accepting user input, updating the state of the robot swarm, and ultimately evaluating task completion. Subjects were recruited using a combination of social network effects and coordinated news posts. Concurrently, we contacted our university’s News and Media Relations Team. They sent a writer and photographer

(a) r-one robot (b) Manipulation experiment with Velcro-skirt grippers. Sufficient robots were used to ensure caging. Fig. 3. (a) (foreground) The r-one differential-drive mobile robot used for manipulation experiments (background) many r-one robots. (b) Manipulation experiments. Each robot is wearing a circular skirt covered in Velcro fasteners. The object has the alternate type of fastener, so the robots remain attached to the object after they collide with it. These experiments demonstrate the feasibility of multi-robot manipulation with simple robots and uniform inputs.

300

E

L SAB RES ADD L GLOBA

1000 500

Objects

0

3417 experiments

250

1500

Time HsL

Time (s)

2000

1

3

5 Robots n

8

200 150 100 50 0 0

100

Number of Robots n Fig. 4. We measured the completion time for human users using the three control techniques in Eqn. (1). Users controlled n = 1 robots to manipulate the box shape, 3 for the triangle, 5 for Texas, and 8 for the amoeba shape. G LOBAL had the shortest mean completion times overall and L OCAL the longest for experiments with 5 and 8 robots. Also shown are the means and the final times for each trial (the same subject tested all three control techniques, and each shape was tested with at least 5 subjects).

to our lab, worked with us to draft a press release, and publicized with news outlets and alumni. Most universities have a media team, and this is a valuable no-cost venue to gain publicity. Given the large number of experiment sessions run (over 1,100 at the time of this writing), we see a per-experiment cost of less than two cents. IV. I NSIGHT A NALYSIS 1) Hardware Experiment Results: The completion times are shown in Fig. 4. Each shape was tested with at least 5 subjects. G LOBAL had the shortest mean completion times, while L OCAL had the longest mean completion times for experiments with 5 and 8 robots. 2) Online Experiment Results: We designed five experiments to investigate human control of large swarms of robots for manipulation tasks. Screenshots of each experiment are shown in Fig. 1. Each experiment examined the effects of varying a single parameter: 1 to 500 robots for manipulation, four levels of visual feedback, three control architectures, 200 levels of Brownian noise, and position control with 1 to 10 robots. The users could choose which experiment to try, but our architecture randomly assigned a particular parameter value for each trial. We recorded the completion time and the participant ID for each successful trial.

200 300 400 Number of Robots

500

Fig. 5. Data from Varying Number using robots to push an object through a maze to a goal location. The data indicates that this task has an optimal number of robots, perhaps due to the relative sizes of the robots, obstacles, and object. Best-fit linear and quadratic lines are overlaid for comparison.

A. Varying Number Transport of goods and materials between points is at the heart of all engineering and construction in real-world systems. This experiment varied from 1 to 500 the number of robots used to transport an object. We kept the total area, maximum robot speed, and sum force the swarm could produce constant. The robots pushed a large hexagonal object through an ‘S’-shaped maze. Our hypothesis was that participants would complete the task faster with more robots. The results, shown in Fig. 5, do not support our hypothesis, indicating rather that there is a local minima around 130 robots. B. Varying Control Ultimately, we want to use swarms of robots to build things. This experiment compared different control architectures modeled after real-world devices. We compared attractive and repulsive control with the global control used for the other experiments. The attractive and repulsive controllers were loosely modeled after scanning tunneling microscopes (STM), but also apply to magnetic manipulation [25] and biological models [26]. The experiment challenged players to assemble a three-block pyramid with a swarm of 16 robots. The results were conclusive, as shown in Fig. 6: for our

Control architecture vs. completion time 189

repulsive

èè

594

attractive

èèèèè

èè è è è è è è

èè

462

global

èèèèèè èèèèèè èèè

0

100

200

300

400

Time HsL, 1275 experiments Fig. 6. Attractive control resulted in the shortest completion time and repulsive the longest for building a three-block pyramid.

assembly task attractive control was the fastest, followed by global control, with repulsive control a distant last. The median time using repulsive control was four times longer than with attractive control. C. A Key Observation To steer an object through a maze, there must always be robots on each side of the object or it is not completely controllable. This is a partial cage. Care must be taken to preserve a sufficient number of robots for caging, or the object may become trapped along an obstacle, as shown in http://youtu.be/qIW6hPgqCRE. V. C ONCLUSION AND F UTURE W ORK We want to enable human control of large swarms of robots, under a specific control situation when every robot receives exactly the same control inputs. Caging manipulation provides a convenient abstraction for a human user, because the user can often forget about the individual robots and instead concentrate on moving the whole unit—the caging robots and the object they surround. Our online experiments Fig 1.a-c, did not enforce caging, and users spent a considerable amount of time moving robots around the object being pushed in order to apply force in the required direction. Our hardware experiments, Fig. 3 and 4, used velcro and large numbers of robots to enforce a cage. This technique was intuitive for our test subjects. We are looking for micro- and nano-robotic systems where we can enforce caging in a similar way. R EFERENCES [1] A. Becker, G. Habibi, J. Werfel, M. Rubenstein, and J. McLurkin, “Massive uniform manipulation: Controlling large populations of simple robots with a common input signal,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (Accepted), Nov. 2013. [2] C. Ertel and A. Becker. (2013, Sep.) Swarmcontrol git repository. [Online]. Available: https://github.com/crertel/swarmmanipulate.git [3] B. Donald, C. Levey, C. McGray, I. Paprotny, and D. Rus, “An untethered, electrostatic, globally controllable MEMS micro-robot,” J. of MEMS, vol. 15, no. 1, pp. 1–15, Feb. 2006.

[4] B. Donald, C. Levey, and I. Paprotny, “Planar microassembly by parallel actuation of MEMS microrobots,” J. of MEMS, vol. 17, no. 4, pp. 789–808, Aug. 2008. [5] E. Diller, J. Giltinan, and M. Sitti, “Independent control of multiple magnetic microrobots in three dimensions,” The International Journal of Robotics Research, vol. 32, no. 5, pp. 614–631, 2013. [Online]. Available: http://ijr.sagepub.com/content/32/5/614.abstract [6] D. Frutiger, B. Kratochvil, K. Vollmers, and B. J. Nelson, “Magmites - wireless resonant magnetic microrobots,” in IEEE Int. Conf. Rob. Aut., Pasadena, CA, May 2008. [7] K. E. Peyer, L. Zhang, and B. J. Nelson, “Bio-inspired magnetic swimming microrobots for biomedical applications,” Nanoscale, 2013. [8] A. Becker and T. Bretl, “Approximate steering of a unicycle under bounded model perturbation using ensemble control,” IEEE Trans. Robot., vol. 28, no. 3, pp. 580–591, Jun. 2012. [9] A. Becker, C. Onyuksel, and T. Bretl, “Feedback control of many differential-drive robots with uniform control inputs,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Oct. 2012. [10] E. Rimon and A. Blake, “Caging 2D bodies by 1-parameter twofingered gripping systems,” in IEEE Int. Conf. Rob. Aut., vol. 2. IEEE, 1996, pp. 1458–1464. [11] C. Davidson and A. Blake, “Caging planar objects with a three-finger one-parameter gripper,” in IEEE Int. Conf. Rob. Aut., 1998, pp. 2722— 2727. [12] A. Sudsang and J. Ponce, “On grasping and manipulating polygonal objects with disc-shaped robots in the plane,” in IEEE Int. Conf. Rob. Aut., 1998, pp. 2740—2746. [13] A. Sudsang, J. Ponce, M. Hyman, and D. J. Kriegman, “On manipulating polygonal objects three 2-dof robots in the plane,” in IEEE Int. Conf. Rob. Aut., 1999, pp. 2227—2234. [14] A. Sudsang and J. Ponce, “A new approach to motion planning for disc-shaped robots manipulating a polygonal object in the plane,” in IEEE Int. Conf. Rob. Aut., Apr. 2000, pp. 1068—1075. [15] A. Sudsang, F. Rothganger, and J. Ponce, “Motion planning for discshaped robots pushing a polygonal object in the plane,” IEEE Trans. Robot. Autom., vol. 18, no. 4, pp. 550–562, Aug. 2002. [16] Z. Wang and V. Kumar, “Object closure and manipulation by multiple cooperating mobile robots,” in IEEE International Conference on Robotics and Automation, vol. 1, 2002, pp. 394–399. [17] G. A. S. Pereira, M. F. M. Campos, and V. Kumar, “Decentralized algorithms for multi-robot manipulation via caging,” The International Journal of Robotics Research, vol. 23, no. 7-8, pp. 783–795, Aug. 2004. [Online]. Available: http://ijr.sagepub.com/content/23/7-8/783 [18] J. Fink, N. Michael, and V. Kumar, “Composition of vector fields for multi-robot manipulation via caging,” in Robotics Science and Systems, Atlanta, GA, Jun. 2007. [19] J. Fink, M. Hsieh, and V. Kumar, “Multi-robot manipulation via caging in environments with obstacles,” in Robotics and Automation, ICRA IEEE International Conference on, May 2008, pp. 1471–1476. [20] S. Bashyal and G. K. Venayagamoorthy, “Human swarm interaction for radiation source search and localization,” in Swarm Intelligence Symposium (SIS). IEEE, 2008, pp. 1–8. [21] A. Kolling, S. Nunnally, and M. Lewis, “Towards human control of robot swarms,” in Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction. ACM, 2012, pp. 89–96. [22] J.-P. de la Croix and M. Egerstedt, “Controllability characterizations of leader-based swarm interactions,” in 2012 AAAI Fall Symposium Series, 2012. [23] J. McLurkin, A. Lynch, S. Rixner, T. Barr, A. Chou, K. Foster, and S. Bilstein, “A low-cost multi-robot system for research, teaching, and outreach,” Distributed Autonomous Robotic Systems, pp. 597–609, 2010. [24] M. Rubenstein, C. Ahler, and R. Nagpal, “Kilobot: A low cost scalable robot system for collective behaviors,” in IEEE Int. Conf. Rob. Aut., May 2012, pp. 3293–3298. [25] I. S. M. Khalil, F. van den Brink, O. S. Sukas, and S. Misra, “Microassembly using a cluster of paramagnetic microparticles,” in IEEE International Conference on Robotics and Automation, Karlsruhe, Germany, May 2013, pp. 5507–5512. [26] M. A. Goodrich, S. Kerman, B. Pendleton, and P. Sujit, “What types of interactions do bio-inspired robot swarms and flocks afford a human?” in Robotics: Science and Systems, 2012.