Haptic Feedback to Help Geometry Learning for Visually Impaired Child Benoît Martin*

Isabelle Pecci*

Robin Vivian**

*Université de Metz – UFR MIM LITA Ile du Saulcy 57000 Metz – France [email protected] [email protected] **Université de Metz – UFR SHA ETIC Ile du Saulcy 57000 Metz – France [email protected]

ABSTRACT. “Draw a right-angled triangle with 10.3 cm and 4.6 cm sides long. Then measure

the third side.” Visually impaired child are tackled to this problem that can seem quiet simple. How to learn to such child to trace or evaluate the exact length of a segment ? Specialized teachers are tackled to this problem too. This paper presents a study to help millimeter segment drawings. We show encouraging preliminary tests and we show that haptic feedback should be effectively used to lead child to learn, to do and to verify geometric works. KEY WORDS : visually impaired people, haptic feedback, man/machine interaction, help,

geometry.

1

Introduction

As every people with specific needs, visually impaired children need suitable aids that come from the ability of many connected fields [1]. In this paper, we present a study that experiments if a technological solution is adequate to an actual problem that aims to integrate visually impaired children at school. These children join ordinary class and must succeed as others pupils. Our goal is to provide to specialized teacher many methods and pedagogical tools using new technology. The teacher must have choice of the pedagogy to use to better adapt to visually impaired children. Another difficulty is the difference between each visually impaired child. It is impossible to be unaware of his lived past because it influences his abilities to assimilate knowledge. Each visually impaired child is different : •

•

•

by his handicap, a blind child doesn’t learn as a partial visually impaired one. The difference of learning increases if the child already saw. Precocious blinds don’t have the same strategy [2]. by the development of his disease, the progressive disappearing of vision is hard to manage in knowledge learning because it involves non-attendance for surgical reasons and psychological troubles. Besides, an visually impaired child who is not totally blind should sometimes learn without using his rest of vision because it will disappear. It’s important to accustom child to use others ways of understanding information. the age and the experience have much influence [3].

Computing tool has to succeed in palliating the loss of vision and helping to manage the development of the disease: it has to accustom child to use non visual methods to understand information without being confusing him. Among difficult fields a teacher takes up to visually impaired children, we chose the Mathematics and in particular the geometry

and the notion of precision. It seems very difficult for visually impaired children to get accurate distance, because existing methods are not suitable. However, it seems to be important to check for example that a triangle is truly right-angled or that a line is truly horizontal. We aim to study the development of touch and particularly the direct impact of interactions using only this sense. For palliating the visual deficiency, we propose to intensify this sense of touch. To complete this, we want to give more information by using this sense that is very used with visually impaired children to learn Mathematics. The study of the abilities of touch is owing to the various notices below : • some tests have been done to show that the use of touch can reduce the number of graphic [4]. • some haptic devices have already been successfully used in the understanding of spatial properties [5][6][7][8] or in the recognition of line graph [9]. • touch and manual exploration are important for children and adults [2]. 2

Haptic feedback

2.1

Definition

The term haptic features the science of touch encompassing a broad set of related sensations including proprioception and kinaesthesia [10]. Two complementary haptic feedbacks exist [7][11]: • •

Tactile feedback that affects the skin surface by stretching it or pulling it. It can give the user a feel of virtual object surface contact geometry, smoothness, slippage, and temperature. Force feedback that affects the finger, hand, or body position and movement. It can simulate the weight, hardness or inertia.

Tactile feedback seems to be the best haptic way of learning for visually impaired people since they use first their hand to explore. However, force feedback seems to give more possibilities to help

visually impaired people in learning task since it aims to modify motion and positions. This is the reason why we chose to study first a force feedback solution. However, haptic feedback applied on interactions is a complex task. It introduces a source of new information sending in new way and it uses touch recognition that is hard to manage. The problem of touch is complex and raises three first questions : • •

•

the information coding : how to code it to have an optimal use of the haptic system ? [9][12][14] the information translation : how to manage the link between graphic information and haptic ones. Such link involves large re-education of the amount of information that can be presented [12]. Some visual properties can’t be translated in haptic ones. This point raises the problem of the decomposition of information in its basic features that allow later comparisons in memory [15]. the preservation of sense : information featured with touch must keep his completeness. For example, a good aesthetic form can have no sense for a blind person [16].

In addition, the use of haptic feedback can involve sensory overloading for somebody who uses it intensively [4][6][12][13]. Several important criterions must be studied before using haptic perception as a way of perception in palliative method : amount of perceptive, cognitive and motor effort needed, the tiredness generated and the pleasure that the user has. Haptic perception seems to be too much difficult to use. However it can show new information or in another way for visually impaired people, which can be a solution for helping them to better understand more concepts. Then, it’s interesting to overcome those difficulties if the results give a true help for visually impaired people. Before propounding haptic solutions, it seems to be important to check the following hypothesis : the haptic technology, particularly force feedback, is well perceived by visually impaired people and in particular gives good results and a great pleasure to use it. For the other criterions, especially the sensory overloading, we attempt to explore them in future works.

2.2

The haptic hardware used : the PHANToM desktop®

We use a PHANToM Desktop® to give and obtain information by way of the hand (figure 1). The PHANToM is a device with haptic feedback and provides six degree of freedom : the user can interact in any plane.

y button

Real world

x z

Virtual world

Virtual and real world

Figure 1. The PHANToM This device is attractive because the hand of the user is not constrained. Even if the space of moving is limited, the user have a lot of freedom during the manipulations. Moreover, an visually impaired user interacts with a pen as a person who can see, with the difference that the pen is interactive itself. It can give haptic information to user so as to palliate the visual lack and supply a button for the user to give information.

3 3.1

Experimental conventions Hypothesis

Before to introduce haptic interactions techniques dedicated to visually impaired children to learn mathematical concepts, we planed tests to evaluate the potential of the haptic feedback, especially the force feedback. The idea was to propose to draw segments of fixed lengths with or without haptic feedback. This choice was justify by the lack of helps dedicated to visually impaired children to do such a task.

For example, the classical raised graduated rule is not accurate. It’s not possible to go down to a millimetre precision. These preliminary tests should exhibit that : • • 3.2

Haptic material doesn’t hinder participants drawings by comparing “classical” drawings with “haptic” drawings but without any dedicated or additional helps. Haptic material increases accuracy and speed by introducing new interaction with intensive use of haptic feedback. Participants

We don’t have to consider visually impaired persons like blindfolded sighted person because their growing will be done with other bases integrating blindness [17]. However, to realise our tests, we used two participants classes apparently very different : • •

Visually impaired children with degenerative diseases. Blindfolded sighted adults.

In fact, for these tests, we couldn’t work with sufficient number of visually impaired children. They are not numerous in the school which is our partner. We chose to use blindfolded sighted adults to have a larger sample. They have a lot of common points with visually impaired children for these tests : they have experienced sighting, they can’t use sighting during tests and they are not used with haptic material. We don’t want to compare behaviours between the two classes of participants but, above all, we want to identify the potential of the haptic material in some interactions where sighting can’t be used. Moreover, the number of participants for each class is low : two visually impaired children (11 and 12 years old) for the first class of participants and ten persons (students, teachers from a computer science department and a teacher specialized in visually impaired children) for the second one. So, the results can’t be generalized but they will be able to show main problems. Even with a larger sample of visually impaired children, it can be difficult to generalized because the differences are very important between each children : growing, disease, …

3.3

First experiment : Paper, rule, pen versus PHANToM

At the beginning, we try to judge the impact of a new haptic material. We want to test if haptic interfaces perturb or help participants to evaluate distances without using sighting. To evaluate this, each participant must draw without any visual help three segments with various lengths (7 cms, 19 cms and 12 cms). Two drawing methods are successively used : • •

To draw a segment on a sheet of paper with a rule without any graduating. The sheet is on a table and thus, the drawing is done in the horizontal plane. To virtually follow a horizontal line in the frontal plane with the pen of PHANToM. The participant must indicate the end of the segment with a long click with the button on the pen of PHANToM. The beginning of the segment – the starting point – is always the same. The PHANToM is restrained on the horizontal line to simulate the help brought by the rule in the previous step.

In the two methods, participants have no other help. 3.4

Second experiment : Drawings with millimeter accuracy

The second set of tests must show the efficiency of haptic interactions to learn distances without using vision. Each participant must draw a set of segments with the following lengths : 2.5 cms, 14 cms, 5.5 cms, 8 cms, 17.2 cms, 9.7 cms, 8 cms, 14 cms and 11.3 cms. The segments must be drawn in this order because we choose this order to avoid any learning between each successive drawing. On the contrary, the redundancy of lengths (8 cms and 14 cms) is voluntary, we wanted to analyze if a participant memorize something from the previous drawing with the same length. Two drawing methods are successively used : •

The participant follows virtually a horizontal line in the frontal plane as in the first set of tests. He indicates the end of the segment by a long click with the pen of the PHANToM. The beginning of the segment – the starting point – is always the

•

same. At anytime, the participant can request the covered distance (from the starting point to the current position of the stylus) by a short click with the pen of the PHANToM. This information is orally given by somebody to simulate a vocal synthesis. The participant follows virtually a horizontal line in the frontal plane as in the previous step but now, he has complementary help with haptic marks (see figure 2). We use the “graduated rule” metaphor. The horizontal line is graduated with different fixed marks (bumps) all 5 and 10 centimetres. These marks have two different heights, one for 5 centimetres and one for 10 centimetres. The user can’t control the intensity of the marks but the feeling of the marks is linked with the speed of the user : more the speed is high, less the feeling is accurate. Thanks to these marks, the participant can go very quickly in an interval close to the expected end of the segment. After, he can find the expected distance by successive try like in the previous step with a vocal synthesis (simulated by somebody).

haptic marks

virtual line (support of the segment to plot)

the beginning of the segment

Figure 2. Segment drawing with haptic marks

4

Results analysis

4.1

First experiment : Paper, rule, pen versus PHANToM

Figure 3 shows the results of paper drawings and haptic drawings. For this experiment, participants 1 and 2 are visually impaired children and participants 3 and 4 are blindfolded sighted people.

Distances (in cms)

Paper drawing

10 9 8 7 6 5 4 1

2 3 4 Participants

Haptic drawing

Expected value

13

22

12

19

11

16

10

13

9

10

1

2 3 4 Participants

1

2 3 4 Participants

Figure 3. Paper drawing vs. haptic drawing for 7 cms, 12 cms and 19 cms These preliminary tests show that haptic material didn’t hinder participants drawings. There is no special deterioration in accuracy between the paper drawings and the haptic drawings. On the contrary, after the graphics analysis we identified three cases : • • •

The haptic drawing is more accurate than paper drawing. It is the most frequent cases. The haptic drawing is less accurate than paper drawing (but less than 3 millimetres). Only two cases are in this case, the drawing of 7 centimetres for participants 1 and 3. The haptic drawing is worse compare to paper drawing. The only case is the drawing of 19 centimetres for the participant 3.

This seems to improve that haptic device and haptic feedback increase significantly the accuracy of drawings. After a discussion with participants, we have the following assumption : to draw in or near the front plane is more natural than to draw in the horizontal plane like on a sheet of paper.

For the bad results of participant 3, we think there is at least two reasons : the hand’s holding was very bad, you can see it on the figures 1 and 2. The hand doesn’t hold the pen of the PHANToM like a pen but more like a brush. In addition, the device has a physical limit for the movements : the limited space for the movement is more or less 20 centimetres. At the ends of the space, the manipulation is not so accurate with mechanical constraints. 4.2

Second experiment : Drawings with milimetrical accuracy

Results analysis of preliminary tests show that the use of marks provides a significant gain for the self-sufficiency and for the speed to achieve the goal. When the participants use the marks, the number of current position requests goes down significantly in 57% to 85% of cases. To illustrate this, we choose to show only results for one participant, an impaired child called “sample participant”. The behaviour of all other participants is really similar but the low gain of this sample participant (57%) is better to show the difficulties. We didn’t notice major difference between test people, especially between impaired childs and blindfolded sighted people. Figures 4.1 to 4.9 show drawings in the order fixed by the experimental protocol : 2.5 cms – 14 cms – 5.5 cms – 8 cms – 17.2 cms – 9.7 cms – 8 cms – 14 cms and 11.3 cms. For each figure, the X axis is the current position requests in a chronological order and the Y axis is the current position when the request occurs. Moreover, we used the following caption : With marks No marks The white curves show the different positions in haptic drawing with marks and the black curves show the different positions in haptic drawing without marks. 3,5 2,5 1,5 1

2

3

4

5

6

7

8

9

10

11

Figure 4.1. Expected value : 2,5 cms

12

13

15 14 13 12 11 10 9 8 1

2

3

4

5

6

7

8

9

10

11

12

13

11

12

13

11

12

13

11

12

13

12

13

Figure 4.2. Expected value : 14 cms 6 5 4 3 1

2

3

4

5

6

7

8

9

10

Figure 4.3. Expected value : 5,5 cms 9,5 8,5 7,5 6,5 1

2

3

4

5

6

7

8

9

10

Figure 4.4. Expected value : 8 cms 17,5 17 16,5 16 1

2

3

4

5

6

7

8

9

10

Figure 4.5. Expected value : 17,2 cms 10,5 9,5 8,5 7,5 1

2

3

4

5

6

7

8

9

10

Figure 4.6. Expected value : 9,7 cms

11

9 8 7 1

2

3

4

5

6

7

8

9

10

11

12

13

11

12

13

11

12

13

Figure 4.7. Expected value : 8 cms 15 14 13 12 11 1

2

3

4

5

6

7

8

9

10

Figure 4.8. Expected value : 14 cms 11,5 10,5 9,5 8,5 1

2

3

4

5

6

7

8

9

10

Figure 4.9. Expected value : 11,3 cms Thanks to the marks, the participant can approach his goal very quickly. However, even if he is sooner near the solution, he wavers sometimes a long time around the exact solution : this is what we called the « yoyo » effect. Our method seems to be efficient to come near the solution but it is not sufficient to reach accurately the expected measurement. The figure 4.9 is a very good example : the sample participant got excited around the solution, and although he was very quickly near this solution, all the benefit has been lost because the haptic marks are not useful to achieve the exact solution. In addition to the self-sufficiency (less of current position requests), we notice a significant increase of speed. The figure 5 shows the time to achieve the expected value in all tests in the order of the protocol, with and without marks. The X axis is the number of the experiment (number 1 corresponds to the figure 4.1 and so on) and the Y axis is the time to achieve the exact expected value.

45

Time in seconds

40 35 30 25

No marks

20

With marks

15 10 5 0 1

2

3

4

5

6

7

8

9

Experiment

Figure 5. Time for each experiment. This figure refers always to our sample participant whose the gain in term of self-sufficiency (number of current position requests) is only 57%. The haptic marks seem to be useful in 5 cases (2, 3, 4, 7 and 8) although they are unfavourable in 4 cases (1,5,6 and 9). When we studied the results closer and especially the cases 1 and 9 (the worse cases), we was able to give an explanation : figures 4.1 and 4.9 show that participant went closer to the expected value quicker than without marks. In fact, the haptic marks was really efficient. The loss of time appears after with the “yoyo” effect when the haptic marks are not used. The figure 4.9 is really significant : since the second current position request, the participant is close to the solution although he will need 10 seconds more to waver around the expected value.

5

Conclusion and future works

Our study on haptic drawings for impaired people shows very interesting results. Haptic material doesn’t perturb participant drawings. Without any training period, haptic feedback seems to give better results than with traditionally tools. The two experiments give encouraging results but, of course, more developments and experiments are required. In particular, it would be useful to know how would users perform with a graduate rule and compare results with the experimental haptic system with marks. We hope results would be confirmed by a larger test with numerous participants.

It is very important to insist on the self-sufficiency for impaired people with the proposed technique : the participant can himself control his virtual drawing without any other help. Of course, at this time, this control is limited all 5 centimetres but although this, results are encouraging for self-sufficiency but for time too. After results analysis, some difficulties appeared and require improvements. The holding of the haptic device is the first problem, especially for the large distances. It is very important for the haptic feeling to hold correctly the pen. Future works will be done in collaboration with specialists to train and help future users. Moreover, this bad holding has a very bad effect : it amplifies the “yoyo” effect we noticed when the participant must finish his drawing and achieve the expected value. In addition to the holding, we can improve our haptic technique, we can introduce a multi levels interaction technique : • Large marks to achieve quickly the expected value with a maximum freedom for the user. • Precise marks to refine the drawing. The first case refers to the marks proposed in this paper, but the precise marks must be study very precisely. Numerous solutions can be proposed : « magnifying glass » metaphor to elongate the distance and thus avoid the oscillations, « damper » metaphor to absorb the oscillations with haptic feedback, … This study will be done in future too.

6

References

1.

David L. Haury. Teaching science through inquiry. In CSMEE Digest, March, 1993.

2.

Yvette Hatwell. Les procédures d’Exploration manuelle chez l’Enfant et l’Adulte. In Yvette Hatwell, Arlette Streri, and Edouard Gentaz, editors, Toucher Pour Connaître : Psychologie Cognitive de la Perception Tactile Manuelle, chapter 4, p.71–84. Presse Universitaire de France, Novembre 2000.

3.

Yu, W., Ramloll, R. and Brewster, S. Haptic Graphs for Blind Computer Users. Actes de ACM CHI 2000, The Hague, NL, ACM Press Addison-Wesley, p. 102-107, 2000.

4.

Oakley, I., McGee, M. R., Brewster, S. and Gray, P. Putting the Feel in ‘Look and Feel’. Actes de ACM CHI 2000, The Hague, NL, ACM Press Addison-Wesley, p. 415-422, 2000.

5.

Edouard Gentaz and Yvette Hatwell. Le traitement haptique des propriétés spatiales et matérielles des objets. In Yvette Hatwell, Arlette Streri, and Edouard Gentaz, editors, Toucher Pour Connaître : Psychologie Cognitive de la Perception Tactile Manuelle, chapter 7, p.129–162. Presse Universitaire de France, November, 2000.

6.

Jason P. Fritz, Thomas P. Way, and Kenneth E. Barner. Haptic representation of scientific data for visually impaired or blind persons. In Proc. Of the 11th Annual Technology and Persons with Disabilities Conference, Los Angeles, April 1996. California State University Northridge.

7.

Emerson Foulke. Tangible graphic displays. In Symposium on High Resolution Tactile Graphics CSUN’94, Los Angeles, 1994.

8.

Erwan Malik, Benoît Martin, Isabelle Pecci and Robin Vivian. Le retour de force : une aide à l’apprentissage des mathématiques pour les enfants déficients visuels ? In proceedings of JIM’2001 p.126-135, July 4-6, 2001.

9.

R. Ramloll, W. Yu, S. Brewster, B. Riedel, M. Burton, and G. Dimigen. Constructing sonified haptic linegraph for the blind students : First steps. In Proc. Of ACM Assets 2000, p.17–25. ACM Press, November 13-15, 2000.

10. Hinckley, K. Haptic Issues for Virtual Manipulation, PhD thesis, Etats-Unis, Université de Virginie, 200 pages, December, 1996. 11. Burdea, G. C. Haptic Feedback for Virtual Reality, 7èmes journées du groupe de travail Réalité Virtuelle, Laval Virtual 1999, première rencontres internationales de la Réalité virtuelle de Laval, http://www-sop.inria.fr/epidaure/GT-RV/JT-GT-RV7/, Laval, 10 pages, June 3-4, 1999.

12. Emiliani, P.L., Stephanidis, C., Lindstrom, J.I., Jansson, G. , Hamailainen, H., Cavonius, C.R., Engelen J. and Ekberg, J. Access to Pictorial Information by Blind People. Issues in Telecommunication and Disability, S. von Tetzhner (Ed.), Bruxelles, Belgique, EC DG XIII COST 219 project (ISBN 92826-3128-1), p.396-416, October, 1991. 13. Heller, M. A. Les illusions perceptives haptiques. In Yvette Hatwell, Arlette Streri, and Edouard Gentaz, editors, Toucher Pour Connaître : Psychologie Cognitive de la Perception Tactile Manuelle, chapter 8, p.163–174. Presse Universitaire de France, November, 2000. 14. Ungar, S. Cognitive mapping without visual experience. In R. Kitchin and S. Freundschuh, editors, Cognitive Mapping : Past, Present and Future, chapter 13. London : Routledge, November, 2000. 15. Klatzky, R. L. and Lederman, S. J. L’Identification haptique des objets significatifs. In Yvette Hatwell, Arlette Streri, and Edouard Gentaz, editors, Toucher Pour Connaître : Psychologie Cognitive de la Perception Tactile Manuelle, chapter 6, p.109–128. Presse Universitaire de France, November, 2000. 16. Fricke, J. Electrorheological aids as materials for future tactile tablets. In Symposium on High Resolution Tactile Graphics CSUN’94, Los Angeles, 1994. 17. Guidetti, M. and Tourette, C. Handicap et Développement psychologique de l’enfant. ISBN 2-200-01420-1, 192