www.redpel.com +917620593389

Ultrasonic Spectacles and Waist-belt for Visually Impaired and Blind Person Shripad S. Bhatlawande Indian Institute of Technology, Kharagpur Kharagpur, India Abstract— This paper presents an electronic navigation

system for visually impaired and blind people (subject). This system understands obstacles around the subject up to 500 cm in front, left and right direction using a network of ultrasonic sensors. It effectively calculates distance of the detected object from the subject and prepares navigation path accordingly avoiding obstacles. It uses speech feedback to aware the subject about the detected obstacle and its distance. This proposed system uses AT89S52 microcontroller based embedded system to process real time data collected using ultrasonic sensor network. Based on direction and distance of detected obstacle, relevant pre-recorded speech message stored in APR9600 flash memory is invoked. Such speech messages are conveyed to the subject using earphone. Keywords- Electronic Travel Aid; Visual Impairment; Navigation Aid; Mobility; Ultrasonic spectacles; Blind navigation; I. INTRODUCTION According to survey conducted in 2009 by World Health Organization on disability, there are 269 million visually impaired and 45 million blind people worldwide [1]. Ageing populations and lifestyle changes means that chronic blinding conditions such as diabetic retinopathy are projected to rise exponentially. Without effective, major intervention, the number of blind people worldwide has been projected to increase to 76 million by 2020 if current trends continue[2]. There are many traditional and advanced navigational aids are available for visually impaired and blind people. Usage of all these travel aids for detecting obstacles for smooth navigation requires a good training. Presently several electronic travel aids (ETA) are available for visually impaired and blind people. These aids are designed using recent technological developments in automation. Some of these aids are sonic pathfinder [3], Mowat-Sensor [4], Guide-Cane[5], SonicGuide [6], NavBelt [7], vOICE [8], NAVI [9], SVETA [10],CASBLIP [11] and Electronic travel aid [12]. All these systems are either sensor (non-vision) based or vision based. In sensor based systems like sonic pathfinder, Mowat-Sensor, Guide-Cane, Sonic-Guide , NavBelt, ultrasonic or laser devices are used. In such a system, the device receives reflected waves, and produces either an audio (buzzer beep) or vibration in response to detected obstacles. Recent navigation systems use digital video cameras as vision sensor along with

Jayant Mukhopadhyay and Manjunatha Mahadevappa Indian Institute of Technology, Kharagpur Kharagpur, India other multiple sensors. These systems are quite bulky and involves physical interface with the subject. In recent systems like vOICe, NAVI, SVETA and CASBLIP, images are captured using single or stereo video cameras mounted on a wearable system. Captured images are re-sized, processed further and converted to speech, audio beeps, musical sound or vibrations. In such systems frequency of sound shares some relationship with the orientation of pixels. Some advanced systems use Global Positioning System (GPS) integration with the main system. GPS receiver is useful for understanding the current location of the subject and nearby landmarks. Although many advanced electronic navigation aids are available these days for visually impaired and blind people, very few of them are in use. Therefore user acceptability assessment of such systems is very important. The most influencing parameters in this regard are size, portability, reliability, useful functionalities, simple user interface, training time, system robustness and affordability in terms of cost. Considering all these user expectations and requirements, a tailor made low cost and reliable navigation system is proposed in this paper for visually impaired and blind people. II. DESCRIPTION OF THE SYSTEM An embedded system integrating five ultrasonic sensor pairs, APR9600 audio recording and playback flash memory, earphone with AT89S52 microcontroller. Figure 1 shows the proposed system for visually impaired and blind navigation. In this wearable system, two ultrasonic sensor pairs are mounted on the eye glasses and rest three pairs are mounted on customized waist belt. These three ultrasonic sensor pairs are placed 12 cm apart facing towards front left, center and right direction.

Figure 1: Ultrasonic spectacles and waist belt system for visually impaired and blind person

978-1-4673-0816-8/12/$31.00 ©2012 IEEE

www.redpel.com +9176205933891

Using this placement and alignment of ultrasonic sensor pairs, subject can detect obstacles from waist level height to head level height in the range of 500 cm in any direction. These five pairs of ultrasonic sensors collect real time data after every 20 msec. and send it to AT89S52 microcontroller. After processing this data, microcontroller invokes relevant speech message stored in flash memory. APR9600 audio recording and playback flash memory is used for storing pre-recorded speech messages. Variable duration of speech messages up to 60 sec. duration can be stored in this memory. Actual photographs of the systems and its components are shown in figure 2.

III. OBSTACLE DETECTION AND DISTANCE CALCULATION A. Obstacle detection: Ultrasonic sensors are used for obstacle detection and calculation of its adaptive distance from the visually impaired person. Ultrasonic sensors are used in pair as transceivers. One device which emits sound waves is called as transmitter and other who receives echo is known as receiver. These sensors work on a principle similar to radar or sonar which detects the object with the help of echoes from sound waves. An algorithm is implemented in C-language on AT89S52 microcontroller. The time interval between sending the signal and receiving the echo is calculated to determine the distance to an object. As these sensors use sound waves rather than light for object detection, so can be comfortably used in ambient outdoor application. Five ultrasonic sensor pairs are used in this system. Input Requirement: Working Voltage : 5V(DC) Working Current : 15mA Input trigger signal : 10us impulse TTL

Figure 2a: Ultrasonic spectacles

Output Signals: Echo signal: PWM signal. Time required for sound signal to travel twice between source and obstacle. Range: 5 meters. B. Distance calculation: For distance calculation following equation is used: Figure 2b: Ultrasonic waist belt

D= [(EPWHT) * (SV)/2]

… (1)

Where, D = EPWHT = SV =

Distance in cm Echo pulse width high time Sound velocity in cm/s

Before concluding the obstacle distance from the subject, repeated information sampling and averaging is performed. As ambient light conditions do not affect ultrasonic sensors, object detection and distance calculation can be performed accurately.

Figure 2c: AT89S52 and APR9600 Interface circuit Figure 2: Actual photographs of proposed system and system components

IV. COMMUNICATION BETWEEN SYSTEM AND SUBJECT This system can understand 500 meters distant object / obstacle in any direction. This system announces calculated real time distance as it is in meters or centimeters using speech messages. To make distance understanding more appealing to the subject, speech messages can be stored in an universal language.

www.redpel.com +9176205933892

A. Speech warning messages for conveying detected conditions to subject: Many researchers [10, 11, 12] used vibration array, buzzer based audio frequency clips or text to speech conversion for announcing any detected condition to the subject. This system uses pre-recorded speech messages for conveying any detected condition to the subject. It uses APR9600 audio recording and playback flash memory. It can store variable duration speech messages up to 60 sec. duration. Number of messages can be increased by reducing the duration of each message. AT89S52 microcontroller processes real-time data collected by ultrasonic sensor array and takes the correct decision. Based on processed data, correct decision is taken and relevant message is invoked from the flash memory and conveyed to the subject through earphone. Sample speech messages stored in flash memory are shown in table 1. Table 1: Sample obstacle distance speech messages

environment and their readings are recorded. Details of test carried and their distance range outcomes are given in table 2. Table 2: Response of ultrasonic sensor to different object surface Obstacle Detection range in cm surface Test 1

Test 2

Test 3

Test 4

Metal

490

485

476

480

Concrete wall

412

446

437

450

Wood

400

402

412

406

Human body

392

380

401

394

B. Results

V. RESULTS AND DISCUSSION:

A wearable system prototype is developed by integration five ultrasonic sensor pairs on customized spectacles & waist belt, APR9600 flash memory, earphone with AT89S52 microcontroller. To evaluate the performance of this wearable electronic travel aid device, testing is performed in laboratory environment on trained and novice blind folded peoples. Training of one week (16 hours) is sufficient to use this electronic travel aid comfortably. A total of eight tests have been carried out on four blind folded persons in which two were trained subjects and two subjects were novice. After blindfolding the person, he was asked to walk through the corridor where different type of obstacles has been placed within 10 meter range. During the experiment, user’s walking motion is recorded. Time taken by the users (trained and novice) for successfully walking through the obstacles is measured and travel speed for each test has been calculated as depicted in Table 3. It is apparent from the Table 3 that average speed of a trained and novice users are 0.76 and 0.38 m/s respectively. In comparison with the traveling speed of the sighted people (1.4 m/s), this result is acceptable. The accuracy of the device in finding out obstacles is also very good. This result shows that training of the user is one of the important factors for gaining high traveling speed and also to increase the user confidence to choose obstacle free path.

A. Test methodology:

Table 3: System performance analysis

Sr. No. 1 2 3 4 5 6

Distance in centimeters Less than 70 cm 70 cm to 99 cm 100 cm to 199 cm 200 cm to 299 cm 300 cm to 399 cm 400 cm to 499 cm

Formal distance scaling (with speech message) Object is very close Object is at 1 meter distance Object is at 2 meter distance Object is at 3 meter distance Object is at 4 meter distance Object is at 5 meter distance

B. Flexibility to use any language for speech warning messages: For speech assisted navigation, many researchers are using text to speech conversion. In such cases researchers are converting text into English language only. As this system uses APR9600 flash memory to store the pre-recorded speech messages, there is no barrier for usage of any language. Any appealing universal language can be used for recording speech warning messages. This system offers a simple mechanism for recording and storing such speech warning messages.

Ultrasonic sensors, AT89S52 and APR9600 are tested individually as well as an integrated system. As ultrasonic sensors work on principle of echo, study of its reflection properties on different object surfaces is very important. Four such tests are carried on concrete wall, static human body, wood and metal. Surface smoothness plays key role in obstacle detection. Smooth surface object can be detected from maximum detection range of ultrasonic sensors. Metal surface gives highest reflections and then concrete wall, wood and human body. These four surfaces are considered for testing as subject can come across any of them during navigation. All these tests are carried out in laboratory

User

Obst acles

Cleared Obstacle

Test1

Novice

6

6

Travel speed (m/s) 0.40

Test2 Test3 Test4 Test5 Test6 Test7 Test8

Novice Novice Novice Trained Trained Trained Trained

6 7 7 6 6 7 7

6 7 6 6 6 7 7

0.31 0.47 0.34 0.80 0.74 0.70 0.83

www.redpel.com +9176205933893

VI. CONCLUSION: This wearable electronic navigation system is successfully tested on blind folded subjects in indoor environment. Less training time is required to use this system. With rigorous training, system can be used for outdoor navigation also. System has following advantages and limitations: Advantages: Accurate detection of obstacles in front left and right direction. Detection of waist level height to head level height obstacles Minimum physical interface Less training time Very low cost Low power consumption

[9] G. Sainarayanan, “On Intelligent Image Processing Methodologies Applied to Navigation Assistance for Visually Impaired”, Ph. D. Thesis, University Malaysia Sabah, 2002. [10] G. Balakrishnan, G. Sainarayanan, R. Nagarajan and S. Yaacob, “Wearable Real-Time Stereo Vision for the Visually Impaired,” Engineering Letters, vol. 14, no. 2, 2007. [11] G. P. Fajarnes, L. Dunai, V. S. Praderas and I. Dunai, “CASBLiP- a new cognitive object detection and orientation system for impaired people,” Proceedings of the 4th International Conference on Cognitive Systems, ETH Zurich, Switzerland, 2010. [12] Amit kumar, M. Manjunatha and J. Mukhopadhyay, “An Electronic Travel Aid for Navigation of Visually Impaired Person,” Proceeding of the 3rd International Conference on Communication Systems and Networks, pp.1-5, 2011.

Limitations: Certain improvements are required for following: Detection of ground level obstacles Recognition of obstacles Recognition of colors Considering the expectations and requirements of the visually impaired and blind people, this system offers a low cost, reliable, portable, low power and robust solution for smooth navigation. Though the system is light weight, but hard wired with sensors and other components. Further wearable aspect of this system can be improved using wireless connectivity between the system components. This system is developed considering visually impaired and blind people in developing countries. REFFERENCES [1] Margrain, TH. Helping blind and partially sighted people to read: the effectiveness of low vision aids. British Journal of Ophthalmology. pp. 919-922, 2000. [2] Blindness and Visual Impairment: Global Facts. http://www.vision202 -0.org. [3] A. Dodds, D. Clark-Carter, and C. Howarth, “The sonic PathFinder: an evaluation,” Journal of Visual Impairment and Blindness, vol. 78, no. 5, pp. 206–207, 1984. [4] A. Heyes, “A polaroid ultrasonic travel aid for the blind,” Journal of Visual Impairment and Blindness, vol. 76, pp. 199– 201, 1982. [5] I. Ulrich, and J. Borenstein, “The guide cane-Applying mobile robot technologies to assist the visually impaired,” IEEE Transaction on Systems, Man, and Cybernetics-Part A: Systems and Humans, vol. 31, no. 2, pp. 131-136, 2001. [6] J. Barth, and E. Foulhe, “Preview: A neglected variable in orientation and mobility,” Journal of Visual Impairment and Blindness, vol. 73, no. 2, pp. 41–48, 1979. [7] S. Shoval, J. Borenstein, and Y. Koren, “The NavBelt- A computerized travel aid for the blind based on mobile robotics technology,” IEEE Transactions on Biomedical Engineering, vol. 45, no 11, pp. 1376-1386, 1998. [8] P. Meijer, “An Experimental System for Auditory Image Representations,” IEEE Transactions on Biomedical Engineering, vol.39, no 2, pp. 112-121, Feb 1991.

www.redpel.com +9176205933894