WearUI: An Elegant Wearable User Interface for Mobile Devices Manish Bhardwaj, Mayank Garg College of Computing and College of Architecture Georgia Institute of Technology Atlanta, Georgia [email protected], [email protected]

ABSTRACT WearUI, abbreviation for Wearable User Interface, is embroidery over a textile which acts like a user interface for a handheld device like mobile phones. A user interacts with the embroidery through various gestures to control the volume of the speaker and handling calls while getting a haptic and auditory feedback. The piece of embroidery has been designed elegantly to foster intuitive interaction and can be sewed over any part of the clothing.

1. Introduction Within the last decade mobile devices have become ubiquitous in our daily and social life. A lot of effort has been made to put interaction for mobile devices on clothing, to help users interact with them. However most of these approaches have concentrated on porting the keypad from the device to the clothes, the examples of which are Zegna Sports Jacket, Motoroal Jacket [13][14][15][16]. Touch Pads are not socially meaningful, we do not hit answer when we want to talk to some one, nor do we hit deny when we do not. Our aim is to incorporate socially meaningful gestures to interact with mobile phones and mobile social interaction. A good example in this direction is the Cute Circuit Hug Shirt that hugs the wearer on receiving text messages from loved ones [17]. Cute Circuit has a sleeve to answer calls, however the sleeve is cumbersome to use and does not form part of the garment. WearUI is a technology that would let any user stitch embroidery over their clothes, and use it as a multi touch interface to his mobile device. The intuitive interaction with the cloth is sensed and can be configured/changed depending on users needs, desires and situations in form of widgets. The design is aesthetic which blends well with clothing without giving it a digital look.

2. Previous Work A number of techniques of Touch Sensing on clothing are present. Leah Buechley demonstrated a push switch made out of felt and conductive fabric [1]. However this switch is more push than touch, which is not good for any multi touch interface or touch interactions.

Alternatively, capacitive sensing has been used [2][11]. This approach is really complex to work, because of the complex electronic components involved. Additionally the sensing circuitry[2] has to be placed at the same point where we need to sense. Thus, this is not a single point sensing solution. Apart from all this there are still concerns about wash ability. Modifying the cloth to make touch pads has been discussed [5], however this solution used a highly complex circuitry, advanced textile design to achieve its goal and is un scalable. Our focus has been to use a simple technology for touch sensing, and use filtering in software and hardware to make it robust, as this has been a major concern with such circuits [5]. In our work, the medium of electronic textile i.e. the conductive thread, acts as a sensor on the clothes. The electronics used is detachable and design scalable needing no maintenance. We have characterized different touch pad designs and evaluated them on different surfaces for stationary and moving touch. The multi touch interface design and gestures help user interact with the mobile device to answer, make calls and increase and decrease volume. Thus making clothing a socially relevant and meaningful medium, that lets user interact with the mobile device without actually taking it out of a pocket or a bag.

3. System Overview WearUI has three major components. The embroidery based touch pad, the electronic touch detector and the mobile phone. 3.1 Embroidered Multi Touch Pad The user interacts by moving his hand over this touch pad. We have used thick and thin threads so that the embroidery provides a haptic feedback as the user touches it and is easy to locate. We have analyzed a number of touch pad design composing of only conductive thread and ones that have both conductive and non conductive thread embroidery (see Appendix I).

WearUI and once received, analyses it and takes appropriate action. 3.4 Gesture Design

Figure1. Different Thread patterns inside square touch sensors

There are two instances or behavior of the garment. Firstly When the user initiates interaction, i.e. volume control, speed dial and secondly when the device initiates interaction, i.e. incoming calls accept and decline. Gestures for the first case are multi touch as the interaction is initiated by the user whereas for second are very intuitive so that the interaction is quick (see video). The multi touch pad (figure 2) has been stitched with combination of thick and thin threads to act as haptic feedback so that the user can locate the pad easily.

3.2 Embedded Electronic System The touch sensor is a super alpha touch sensor typically used for alarms. The sensor provides a means to filter off noise and false detections. The value out is read using a Analog to Digital Converter. We used Atmega168 for our processor and made our own board to keep costs down. The processor has 6 ADC channels to sense the touch and a UART to connect to mobile device through Bluetooth.

ADC Channel 0-5

Thermo chromic Ink Embroidered touch pads

Atmega 168

Figure 2. WearUI touch pad design made of Conductive thread

Super Alpha Pair Touch Sensors

Figure 3. Sensing Electronics and Bluetooth Connection to Mobile Device

Software Filtering A single ADC Channel is dedicated to single touch pad. Each ADC is over sampled and time domain and value filtered to recognize touch, and each touch is time stamped to identify duration and sequence. The sequence then is interpreted as an interaction and a communication to the mobile device which is then initiated through the Bluetooth. 3.3 Mobile Phone Application The application for the mobile phone is designed in Dot Net Compact framework for Windows Mobile. Initially we decided to build the application for mobile phone in J2ME but realized that it does not handle incoming calls. Thus we decided to move on with windows platform. The application has a communication layer which connects to WearUI. The application is context aware, i.e. interprets the signals from the user depending on the current context the phone is in. The software application always listens for a signal from

The first four pairs of vertical lines acts as way to activate the following touch pads, This helps us prevent false positives. The sequence of touches is interpreted as a sequence and used to interact with the mobile device.

4. Future Work Our efforts, till now has concentrated on getting the underlying technologies to work. Currently our system is slow and we have problems identifying if two gestures were in quick succession. We plan on solving this issue and making it fast and come up with formal method to analyze these situations. We also plan to do a user studies to find out what specific gestures would be good for the multi touch interface and evaluate them, incorporate them into a garment and test functionality and reliability. We plan to add the functionality to control the media player in the windows mobile phone and also port the complete mobile application for different platforms like Apple iPhone, etc in form of widgets.

5. Conclusion We have evaluated and prototyped the underlying technologies that we want to use for WearUI. We have tried various thread patterns and textile types to test which one

suits the best for our goal for multi touch (see Appendix I and II). Issues of speed, and identifying subsequent sequences and formal method to analyze this situation have been identified and would focus our future work on this.

6. Acknowledgement We would like to acknowledge help we got in designing and building the project from Sidhant Gupta, College of Computing and Nick Kom, College of Architecture.

7. References [1]. Leah Buechley, Mike Eisenberg, Jaime Catchen, and Ali Crockett. The LilyPad Arduino: Using Computational Textiles to Investigate Engagement, Aesthetics, and Diversity in Computer Science Education. In Proceedings of CHI 2008 Proceedings · Aesthetics, Awareness, and Sketching, April 5-10, 2008 · Florence, Italy. [2]. Paul Holleis, Albrecht Schmidt, Susanna Paasovaara, Arto Puikkonen, Jonna Häkkilä. Evaluating Capacitive Touch Input on Clothes, Mobile HCI 2008, September 2-5, ACM. [3]. Data sheet for Atmega 168 Micrpcontroller. [4]. ElekTex® data entry sensors by ELEKSEN. www.eleksen.com [5]. Manuchehr Soleimani. Knitted switches for smart clothing using double electrode technology, a research Article, Sensor Review, 28/3 (2008) 229-232.

[6]. Gretchen Anderson and Gwanhoo Lee. Why Consumers (Don’t) Adopt Smart Wearable Electronics [7]. Bradley Rhodes and Kenji Mase. Wearables in 2005. PERVASIVE computing, Published by the IEEE CS and IEEE ComSoc. [8]. George F. Eichinger III, Kara Baumann, Thomas Martin, Mark Jones. Using a PCB Layout Tool to Create Embroidered Circuits. [9]. Jun Rekimot. GestureWrist and GesturePad: Unobtrusive Wearable Interaction Devices. [10]. Cliff Randell. Computerised clothing will benefit textile manufacturers [11] L. K. Baxter. Capacitive Sensors [12]. Emerging tech Toolkit, http://news.zdnet.co.uk/emergingtech/0,1000000183,39161 612,00.htm?r=27 [13]. Engadget, http://www.engadget.com/tag/navjacket/ [14]. Motorola Bluetooth jacket with stereo technology. http://direct.motorola.com/hellomoto/audexproducts// [15]. Interactive Biker Jacket http://www.talk2myshirt.com/blog/archives/80 [16]. Rosner Jacket With Wearable Technologies http://www.alibaba.com/catalog/11574752/Rosner_Jacket_ With_Wearable_Technologies.html [17] http://www.cutecircuit.com/