Published in Proceedings of All India Seminar on Advances in Product Development: APD 2006

SIMULATION AIDED TEACHING: DEVELOPMENT OF A SYSTEM FOR CHARACTER RECOGNITION, GRAPH GENERATION, AND AUTONOMOUS ROBOT CONTROL Priyanshu Agarwal

Rahul Gupta

Pawan Kumar Chaurasia

([email protected])

([email protected])

([email protected])

rd

3 Year Mechanical Motilal Nehru National Institute of Technology Allahabad, India

ABSTRACT In this paper, we present a Graphical User Interface (GUI) incorporating Character Recognition (CR), Graph Generation (GG) and autonomous robot control for facilitating simulation aided teaching. In this GUI a new technique of CR has been implemented using Stroke Slope Patterns (SSP) and number of break points in a mouse drawn input character. In this method the problem of stroke sequence has been overcome using a Relative Positioning System (RPS). The input is processed by an algorithm which gives the best fit match as the output. The performance of the system has been found to increase with increasing training samples. The Character Recognition Module (CRM) has the feature of providing an equation as input to Graph Generation Module (GGM). GGM uses a Priority Assigning Technique (PAT) for evaluating the given mathematical equation and plot its graph. And finally through port programming a robot which is connected to the printer port through the interface circuitry can be commanded to move over this path autonomously.

KEYWORDS Priority assigning technique, relative positioning system, character recognition, graph generation, autonomous robot control, path tracking

1. INTRODUCTION In this era of modernization when technology has revolutionized the way of teaching, simulation aided teaching can be of immense help towards proficient teaching. Past studies have shown that pictorial representation of a problem is always easy to comprehend by the human mind [1]. A system which can dynamically recognize the drawn characters, assemble it into a logical mathematical equation, and finally draw its graph, will be useful from various aspects. Intelligent industrial robots are to be governed over predefined contours with an efficient closed loop feedback system for satisfactory performance. In this direction also our system will prove to be very useful for generating user defined contours.

Developing an interactive simulation aided teaching system with Character Recognition (CR), Dynamic Graph Generation (DGG) and robot control is a challenging problem. The key and difficult task in CR is to extract “good” features of similar handwritten characters. Features of handwritten characters are mainly categorized into two categories: curvature pattern and number of strokes. Much research has been done based on these two types of features [2]. In this paper, contour curvature is employed as the feature of characters. The curvature is the essential feature of a character. There has been persistent interest on curvature-based characterization in the field of handwritten character recognition [3, 4]. It is well known in differential geometry that with the curvature and the direction of the starting point, a curve can be reconstructed accurately [5]. An important advantage of curvature representation is that with coordinate expression, two signals x(t) and y(t) are usually needed to express a contour mathematically, however, with its curvature, one signal s(t) is sufficient. Another advantage of curvature representation is that the contour curvature of an object is translation invariant. However, evaluation of curvature of a discrete signal is difficult in practice because the numerical calculation of the second derivatives is a typical ill-posed problem [6]. Graphs have always been of vital interest to human beings due to their wide variety of applications which range from solving complex mathematical problems to the advanced areas of research in different fields of science and engineering. The greatest hurdle in graph generation is to dynamically derive the mathematical equation from the equation string. Our system has successfully overcome this problem through Priority Assigning Technique (PAT).

2. FEATURES OF THE SYSTEM The system developed consists of Plot Graph Mode, Training Mode, and Tracking Mode. The whole system has been developed using C Programming. The detailed description of these modes is provided in the following sections. 2.1 PLOT GRAPH MODE This is the basic mode for plotting equation governed curves. The input can be through mouse or keyboard. An illustration of two input types is shown in fig 1 and 2. In this mode we have developed the following tools (fig 3): (1) Zooming effects for zooming in and out of the plotted graph, (2) Plotting derivative of the plotted graph which can be simultaneously superimposed over the drawn curve, (3) Integration of the recognized equation between certain limits, (4) ‘Import’ and ‘save’ features to facilitate the import of an already saved curve or to save a plotted curve, (5) Area bounded between two user provided curves, and (6) User defined path tracking by an autonomous robot.

2.2 TRAINING MODE Our system supports training for different users by providing them individual accounts. This mode allows users to login through their own accounts and allows training of the module in their own stroke sequence.

Fig1 Plot Graph Mode through mouse input.

Fig 2 Plot Graph Mode through keyboard input.

2.3 TRACKING MODE In this mode a user defined curve can be generated by specifying coordinates of certain points lying on the curve. Thus the robot can be guided over this user generated path. The additional feature of this mode is that the robot can be guided either autonomously over the predefined path or semi-autonomously over spontaneously generated path. An instance of a simulation is shown in fig 4.

Fig 3 A plot of equation Y=SIN(COS(TAN(X))) with its derivative.

3. ALGORITHMS The major algorithms used in the development of the system are character recognition algorithm, graph generation algorithm and autonomous robot control. These are described in the following section:

3.1 CHARACTER RECOGNITION Character Recognition (CR) is the technique of identifying a character on the basis of its intrinsic properties which remains independent of the scale and location of the character drawn. The algorithm used for CR consists of the several steps as described in the following section.

Fig 4 An instance of the simulation of tracking robot over Y=X^2.

3.1.1 PRE SCANNING In this module of the GUI the input given by the user is scanned and the coordinates of the drawn characters are stored in a file. These coordinates are arranged in the order of strokes drawn by the user. Since the system should be independent of the order of the strokes in a character so a post scanning is performed to rearrange the coordinates.

3.1.2 FORMATION OF BOX FILES The coordinate file is subdivided into 18 smaller files corresponding to 18 boxes provided for writing characters. These box files are then further processed to identify individual characters.

3.1.3 POST SCANNING (RELATIVE POSITIONING SYSTEM) The files obtained are arranged in the order of increasing x-coordinate and for same x in the order of increasing y-coordinate. Now the scanning of each new file is started from the point having least x and y coordinate. The technique used for scanning is as shown in figure 5. A set of nine boxes is used. Each box consists of a set of 3X3 points. Now, the scanning process starts by considering the point at the center of box 5. Further the boxes 1, 2, 3, 4, 6, 7, 8, 9 are

Fig 5 Representation of pixel box

scanned for any point(s) present in the file. Now the scanning point is shifted starting from the lowest box number to the highest box number i.e. branches corresponding to the lowest box number are scanned first than the higher ones. So, finally a file is obtained in which the coordinates are arranged in branch wise fashion.

3.1.4 SLOPE CODE GENERATION Fixed number of slopes is then calculated for each file obtained. Experimentally the most accurate results were obtained when the number of slopes was taken as 10. A Slope Code (SC) is then generated using these slopes. Slope Code (SC) is defined as a pattern of slopes of the curvature of a character. It consists of a string of 10 symbols. In this code the first symbol is the character to be recognized, second is the number of break points in the character and the remaining are the slope pattern in the curvature of the character. The slope pattern consists of the following symbols as explained in the table 1 below:

STROKE SYMBOL

STROKE DESCRIPTION

_

Negative slope

+

Positive slope

0

Almost zero slope

5

Almost infinite slope

!

No slope

Table 1 Symbols used for Slope Pattern Generation.

3.1.5 SLOPE CODE MATCHING The SC generated is then matched with a default file having standard SC for each character. Through this comparison finally the character is recognized. An equation string is then formed using these characters. Our system has been trained for the following characters: (1) Alphabets: A, C, D, E, H, I, L, M, O, S, T, X, Y, (2) Numerals: 0-9, and (3) Other Characters: +, -, /, ^, [, ], (, ), =

3.2 GRAPH GENERATION This section deals with the algorithms used for generating graphs. Logical distinction and priority assigning technique play a vital role in graph generation. The general form of the equation plotted by GGM is aY^n=f(X)

or

aX^n=f(Y), where f(X) and f(Y) are any function in X or Y

respectively and a, n are user-defined constants. f(X) or f(Y) may be a combination of the following functions with polynomials: (1) TRIGONOMETRIC FUNCTIONS ( SIN, COS, TAN, COSEC, SEC, COT), (2) INVERSE TRIGONOMETRIC FUNCTIONS (ASIN, ACOS, ATAN, ACOSEC, ASEC, ACOT), (3) HYPERBOLIC FUNCTIONS (SINH, COSH, TANH, COSECH, SECH, COTH), (4) INVERSE HYPERBOLIC FUNCTIONS (ASINH, ACOSH, ATANH, ACOSECH, ASECH, ACOTH), (5) EXPONENTIAL FUNCTION (E^X), (6) LOGARITHMIC FUNCTION (LNX), (7) MOD FUNCTION (MOD(X)), and (8) GREATEST INTEGER FUNCTION ([X]).

3.2.1 LOGICAL DISTINCTION (LD) Obtained equation string is then modified to distinguish between certain characters having similar curvature. For e.g. in the expression Y=COSX the character ‘O’ may be misidentified as ‘0’. The LD helps in overcoming this problem.

3.2.2 PRIORITY ASSIGNING TECHNIQUE (PAT) Through this technique a Coded String Equation (CSE) is generated from the original String Equation (SE). For this we have assigned certain numbers to each valid mathematical function and for each mathematical operator. SE is then converted into a valid mathematical expression which is then finally plotted. The complete procedure is described below with the help of an example. Consider the equation Y=SIN(COS(TAN(X))). The following actions are performed for plotting the graph: (1) The String Equation (SE) is first converted into the general form e.g. SE=>1Y^1= (SIN(COS(TAN(X)))), (2) The corresponding Coded String Equation (CSE) is then generated e.g. CSE=> 1Y(14)1=(15(16(17(X))), (3) The starting value of X is placed in the above expression, (4) Now the CSE is scanned for the innermost bracket and the expression in the bracket is sent to a function for evaluation and the result is placed in the SE in place of the expression e.g. SE=> 1Y^1=SIN(COS(-0.648)), (5) The CSE is now updated according to the changed expression of SE e.g. CSE=>1Y(14)1=15(16(-0.648)), (6) The evaluate function is called recursively to evaluate the whole expression and finally the value is placed on the right side of the

equal sign e.g. SE=> 1Y^1=SIN(0.797), CSE=>1Y(14)1=15(0.797), SE=> 1Y^1=0.715, CSE=>1Y(14)1=0.715, (7) Now this value is divided by the value of a, e.g. SE=> Y^1=0.715, CSE=>Y(14)1=0.715, (8) The nth root of the obtained value is then calculated and thus the value of Y is finally calculated, e.g. SE=> Y=0.715, CSE=>Y=0.715.

OPERATOR/FUNCTION IN SE CORRESPONDING NUMBER/SYMBOL IN CSE

1

Y

^

=

SIN

COS

TAN

(

)

X

1

Y

14

=

15

16

17

(

)

Value

Table 2 Conversion code for SE to CSE 3.3 AUTONOMOUS ROBOT CONCEPT The coordinates of the curve drawn are available in a file. Now using this file the slope of the curve at each coordinate is calculated. This gives the corresponding angle that the robot has to move each time. Now this gives the direction in which the robot has to move and the corresponding bit is send to the printer port which activates the robot connected through an interface circuitry. The proper time for sending the bit is obtained by comparing this angle with the already stored angle of the file. Hence, the required path is traced by the robot.

4. HARDWARE REQUIREMENT The two major hardware parts in the system are the interface circuitry and the path tracking robot. The interface circuitry acts as interface between the computer and the robot.

(a)

(b)

Fig 6 (a) The motor driver IC schematic, (b) robot schematic with corresponding bits for driving and steering.

4.1 INTERFACE CIRCUITRY

The circuit consists of an IC L293D which is connected to the printer port through a buffer IC 74125. The IC is responsible for clockwise and counter-clockwise rotation of the geared motors controlling the robot. The circuit schematic is shown in fig 6. 4.2 ROBOT The designed robot is a four-wheeled guided vehicle driven by four 12V DC geared motors which are attached to the four wheels. The turning mechanism is based on the reversing of motors.

5. CONCLUSION The main theme of our work is to encourage the use of simulations in teaching which makes teaching fun and helps in making concepts more lucid to understand. We discuss merger of techniques of character recognition, graph generation and robot control for such simulation aided teaching. The system is tested for 500 samples consisting of various characters given by 3 different users out of which 423 were correctly identified. Though in-depth study has been done on character recognition but yet world haven’t witnessed perfect CR software which gives cent percent results. An important feature of the CRM is the self updating aspect which comes in action when the GUI fails to identify the character. In such a case it prompts the user to type in the drawn character through keyboard which is then added to the standard database file. Thus, our system has various applications both in academia and industry.

6. FUTURE WORK Imagine a world where pedagogy has reached its acme in terms of advancement in technology and robots are a lot smarter than humans. Our system is the first step towards realizing such a utopian world. Such a

system can bridge the gap between students and teachers, leverage

research by generating complex curves and allowing mathematical operations to be performed on them, provide autonomous controlling of robots in nuclear power plants, industries etc for handling of hazardous materials, and generate statistical reports in stock market. Much work needs to be done for extending this approach of simulation aided teaching for more advanced physics-based simulation problems.

REFERENCES 1. Lorayne, H., ”The Secrets of Mind Power” Canada, U.S.A.,1997. 2. Wing-nin Leung and Kam-shun Cheng “A Stroke-Order Free Chinese Handwriting Input System Based On Elative Stroke Positions And Back-Propagation Networks” Department of Computing, Hong Kong Polytechnic University. 3.Hull,J.J., Commike, A., Ho, T.K., “Multiple algorithms for handwritten character recognition”, in: Proceedings of the International Workshop on Frontiers in Handwriting Recognition, Concordia University, Montreal, pp.117–124,April 1990.

4. Suen, C.Y., Nadal, C., Mai,T.A., Legault, R. L.,“ Recognition of handwritten numerals based on the concept of multiple experts”, in: Proceedings of the International Workshop on Frontiers in Handwriting Recognition, Concordia University, Montreal, pp. 131–144, April 1990. 5. Cullum,,J., “Numerical differentiation and regularization”, SIAM J. Numer. Anal. 8 (1971) pp.254–265. 6. Kirsch, A., ” An Introduction to the Mathematical Theory of Inverse Problems”, Springer, New York, 1996.