Many applications have throughput time requirements on the order of a few hundred milliseconds, and don 't require real-time image analysis.

A Single-Chip Image Processor A.L. Pai and S.H. Lin, Arizona State University, and David P. Ryan, Intel Corporation

M

ost of the research efforts on image processing focus on ex· panding the complexity and dimension of image analysis. Unfortunately, this em· phasis results in algorithms that are so computationally intensive that expensive special·purpose vector and pipeline proc· essors are required to evaluate an image fast enough to be considered "real·time." Not all applications, however, have the burdensome requirements of true real·life image analysis. Specifically, applications that have image throughput time require· ments of greater than a few hundred milli· seconds can use a lower cost, general·pur. pose microprocessor·based system. Appli· cations that have even slower frame rates are candidates for not only the use oflower cost CPUs, but also allow for replacement of video-rate flash AID converters with slower, less expensive converters. Addressing the most cost·sensitive ap· plications, the design described herein uses Intel's 16-bit microcontroller to im· plement a single-chip image processor. The on-chip 10-bit AID converter of the controller digitizes the image of a charge injection device (ClD) camera, while the chip's 16-bit CPU executes a library of standard vision algorithms and reports the results by passing a few tokens over an onchip universal asynchronous receivertransmitter (UAR1).

SYSTEM OVERVIEW A block diagram of the single-chip imSENSORS June 1987

0: W (12 MHz)

~ 0: ... ~ ... ()

o:~

lllg

::>0: Q.

!~

~~

t;~ ~W

age processor is shown in Figure I. The image is acquired by the ClD camera and input as an analog voltage to the 8096 where it is digitized and stored in one of two image buffers. The digital image is stored 'as an N x N matrix of 8-bit values corresponding to the gray level intensity at each picture element (pixel) as shown in Figure 2. After an image resides in an image buffer, the 8096 can execute a number of standard image processing algorithms available as system monitor commands. These programs perform thresholding and filtering functions on the digital image, and can analyze objects found within the image. If the 8096 were attached to a host system instead of a terminal, custom pro-

N

Wo:

~~

0: ()

0:'" ••.::> III

grams could be downloaded to the user program RAM and executed. To view the raw and processed images, a CRT controller is used to keep a video monitor updated with the images stored in the two frame buffer memories of Figure I. The 8096 updates the frame buf· fers with the data in its image buffers depending upon commands given to the system. • Hardware. The system is composed of a 128 x 128 ClD camera and Intel's 8096 (with on-chip AID) for image acquisition and analysis. A standard CRT controller was added for displaying raw and processed images as directed by the 8096. Driv· ing the decision to use a 128 by 128 digital

r' ~ ~

I-

es

8397 with 3K left. Two 32K x 8 SRAMs of the result. This provides 256 gray levels, are used to provide space for two 16K byte and greatly simplifies memory requireimage buffers, a 16K section of working ments. Line .-/ RAM, and space for user-downloadable ~ Software. In addition to the code programs that are invoked by the monitor. necessary to digitize images, the system Two 16K byte frame·buffer memories EPROM contains an extensive set of alare mapped to the same addresses as the gorithms for digital image acquisition and corresponding image buffer used by the analysis. Video operations are used to ac8096. Normally, the frame buffers are quire a digitized image. Point and arith· mapped to the CRT controller to keep the metic operations involve the pixel-by-pixel Diglta Image video monitor updated. However, when manipulation of a digital image. Neighbor· ,', , , - fthe image stored in the image buffer is hood operations produce an output image I I I I I changed, the 8096 performs a frame- that is the result of a combination of the synchronized flyby block move to refresh gray lev~l intensities around a specified neighborhood of each pixel. Measurement the frame buffer (50 to 290 ms depending on whether frame synchronization is off operations include the computation of or on). desired parameters of objects located in an To digitize an image, the 8096 monitors image for pattern recognition and other the end-of·line and end-of-frame signals applications. Finally, utility operations are from the CID camera and synchronizes necessary for system operation. the AID conversion of the pixel data to The algorithms present in the system a pixel valid signal from the camera. The monitor can be used to identify desired obLine Spacing analog output signal of the camera ranges jects in a digital image by following the apfrom 0 to I V, corresponding to the gray proach shown in Figure 3. Once an image level intensity at each pixel. This I V range is digitized, it can be enhanced by the ap· Picture Element is amplified to a 5 V range before being plication of various image processing tech(Pixel) input to one of the eight AID inputs of niques including histogramming, thresh· olding, and spatial mtering to delineate the the 8096. desired objects. The 8-directional chain code (Figure 4) can then be used to trace the boundaries of objects, and relevant object parameters can be determined and -Spatial mering • HtstOyramming compared with those of a known object - Thresholding database to identify the unknown object. oumno

I

OBJECT CLASSIFICATION

F;,ure 2. R.",..,.l4tion

of on N x N di,ildl

i""",. image was the desire to store and operate upon two images simultaneously while minimizing memory requirements. The image processing and communications software takes 5K of the 8K bytes allocated to the system monitor space, and would fit in the on-<:hipROM space of an

The 8096 converts the input voltage to a IO-bit digital representation in 22 flS. Another 18 flS are needed to store the pixel in the image buffer, update pointers, update a counter, and start another conversion. Therefore, the camera is clocked at a rate which results in a new pixel being output every 50 flS. Although the 8096 converts its analog input to 10 bits, the extemally generated analog errors (such as buffer error and noise) led to the decision to use only 8 bits 6-297

In the following example, the 8096 performs a binary thresholding operation as described earlier to set the image back· ground to pure white and the objects in the image to pure black. Then the 8096 searches the image for objects. When an object is found, the object boundary is traced and shape analysis is performed. Descriptive information about the object (or objects) is output over the on-<:hip UART of the 8096 to a terminal, or host computer. The controller, without con· sulting a host computer, can also be programmed to make,the decision to accept or reject an object on a set of prescribed rules. The sequence of processing for this example, from serial communication reception and interpretation to the reporting of the shape analysis results, takes approx· imately 1500 ms with an 8096 running at 12 MHz. The time will vary with the size· and number of objects in the field of view. SENSORS June 1987

inter

(a) A 3x3 Neighborhood

3

2

Photos I through 4 show the original 256 gray level digitized image and resul· tant binary (two-level) image of a circular object and a square object. (The circle looks like an oval when displayed due to the aspect ratio of the video monitor). Table I ~ummarizes the output of the systems shape analysis program. The objects' perimeter (P), area (A),center of mass coordinates (ex, cy), and the coordinates of the endpoints of each minimum enclosing rectangle are listed. The rectangularity and circularity of the objects were also calculated and appear in Table I. The rectangularity of the circle and the square using the actual data were ideal. Although the circularity of the digitized circle is slightly different from ideal (12.416 vs. 12.56), the digitized circle can be distinguished from the digitized square since the circularity of the square is very different from a perfect circle (15.44 vs. 12.56). From these typical results, it is clear that this image processor can be used to dis· tinguish between and identify objects placed in its field of view.

CONCLUSIONS

6 (b) 8-Directional

Chain-Code

If the stringent requirements of "realtime" image processing can be relaxed in favor of substantially reduced sytem cost, a standard 16·bit microcontroller can perform as a stand-alone image processor. Not only does the design described here demonstrate that a microcontroller can undertake two-dimensional image processing, but the surprising speed with which it accomplishes the processing should lead to the reevaluation of current microprocessor applications for possible cost reduc· tion via microcontrollers.

REFERENCES

(c) Example Chain-Code

of an Object

Fiaure 4. Sdirectiorldl boundmy eMin codt.

1. Embedded Controller Handbook. Qatest edition). Intel Corporation, Santa Clara, CA. 2. Un, S.H., A.L. Pai, and D.P. Ryan. A Mierocontroller Bosed Digitollm"8' Processor, Proceeding of the Second World Conference on Robotics Research, MS86-7~, Society of Manufacturing Engineers, Dearborn, MI. 3. Cunningham, R. "Segmenting Binary Images," Robotics Age, Vol. 5, No. 2, July/August 1981. 4. Wilf, I·M. "Chain Code," Robotics Age, Vol. 3, No.2, March/April 1981. 5. Gonzalez, R. and P. Wintz. Di&itol I",. Proceni"l/, Second Edition, Addison·Wesley Publishing Company, NY, 1987. 6. Fu, S.U., R.C. Gonzalez, and C.S.G. Lee. Robotics: Control, Serui"l/, Vision and Intelli· gence, McCraw·Hill Book Company, NY 1987.

inter A histogram gives the distribution of all the gray levels in a digital image. The image histogram is used to select a desired threshold intensity level for separating an object from the background in the digital image. A digital image can be thresholded using various threshold functions (three of which are shown in the figure) to yield an output image that contains a better definition of an object. For example, a binary (black and white) image is obtained by applying the two-level threshold function shown in (c).

rLe

Y

255

§

]

~ Cl

i;-b -

.

i3

:

gb

~

00

25

~b

°0

X

(a) Y •

80

X

o Input

Input Gray Level

Photo 2. A thresltolded binary (two·leve~ image of the same circle. The circle appears oval because of the aspect ratio of the video monitor.

••••••••

Gray Level 255

, :

o

g ~~;~re

x

•

Input Gray Level

255

(c)Y.{~~~;h~;e

In spatial filtering, the pixels adjacent to pixel (x,y) of image plane f are operated upon by the filter mask operation h. The resulting value of this spatial convolution is used to compute a replacement gray level intensity value at location (x,y) in the output image g. The following formula is used: g(x,y) = h[~x,y)) = + Wl ~x-I,y+ +

W7

~x -I, Y-I) + w2 ~x -I,y) I) + w.• ~x,y-I) + W5 ~x,y) + W6 ~x,y+ I)

[WI

~x+ I, y-I)

+ w8 ~x+ I,y) + w9 ~x+ I,Y+ I))

Various types of filter masks can be used to perform different digital image enhancement operations. A low·pass filter uses neighborhood averaging to "smooth" the digital image to remove noisy pixels. A high·pass filter accentuates noisy pixels. A high-pass filter accentuates the higher frequencies present in an image, thus "sharpening" its edges. Operators such as the Sobel masks can be used to compute the gradient at each point in an image, thus produc· ing a gradient edge-detected image. Using such filtering methods, the boundaries of objects in an image can be isolated, thus permitting the computation of useful object parameters for object identification and classification. W,

V< (x-1,y-l)

(x-l,y)

y

w3

lF

(x-l.y+1)

(x,y)

W. (x,y-l)

Vi, (X+1,y-1)

w,

W.

(x.y)

(x.y+1)

W.

W.

(x+l,y)

(X+l,y+1)

(a)A3x3 PixelW.-wilII SpelialMaok coetficienls Wi and Ccx_ICIiIIgIr1\aga

Photo of. A thresholded binary irrJ4llf of tlu ••m.

Pixel Localians

"IUdre.

7. Castleman, K.R IJWjtdllrrJ4llf Proceai"l, Prentice Hall International, Englewood Cliffs, NJ, 1985. 8. Baxe., C.A. IJWjtdll,""", Procmill,f-A Prac. tical Prim." Prentice HaU International Englewood Cliffs, NJ, 198-4. '

116 118 [

118

111J

1/4

118

118

1/16

[.: ~1 :Jl G~.~]~ ~-; ~}]1 1

1116

-1

-1

2

1

·1

(e) SolleI Gr __

(c)A a.-. •••• FIIlar

Malt

·1

2

1

OBJECT

PERIMETER P

AREA A

C.O.M. COORDINATES CX.CY

ENCLOSING RECTANGLE XMA)

XMIN

YMAX

RECTANGULARITY

YMIN

R=Aa fA R

CIRCULARITY C=r:lfA

CIRCLE

301

7297

(62.68)

111

15

116

21

0.785

12416

SQUARE

328

fJ167

(55.73)

97

14

115

32

1.0

15.440

A.L. Pai ISan ASSOCIate Professor and S.H. Lin ISa Ph.f). candidate m the Computer SCienceDept., Col·

Size Parameters

lege of Engineering, Anzond State UnJversll). Tempe,

The horizontal and vertical extent of an object and its minimum enclosing rectangle are easily computed by using the minimum and maximum line and sample numbers.

AZ 85281. David P. R.an ISa Senior ApphcalJom Engmeer for Intel Corp.. 5000 II Chandler BI\d., Chandler, AZ 85226.

The perimeter (circumferential distance) around an object boundary is obtainable from the boundary chain code by using the formula: P = NE + 1[2 No where NE and No are the number of even and odd steps in the object boundary chain code. The area of an object, which is a convenient measure of object size. is equal to the number of picture elements inside and including its boundary, multiplied by the area of a single pixel.

Shape Parameters In addition to size parameters, shape parameters can be used to distinguish objects. Some shape parameters that are easily computed are described below. The formula for computing the rectangularity R of an object is: R = AO / AR where Ao is the object area and AR is the area of its minimum enclosing rectangle. R ranges from 0 to I, with a value of 1.0 for rectangular objects, n/4 (0.785) for circular objects, and smaller values for slender. curved objects. The aspect ratio, A, which is the ratio of width to length of the minimum enclosing rectangle of an object, is used to distinguish slender objects from roughly square or circular objects. One of the commonly used circularity measures is: C = p2 / A the ratio of the square of the object perimeter to its area, which reflects the complexity of the object boundary. C has a minimum value of 4n (12.56) for a circular object. while more complex shapes have higher values.

Reprinted from Senaor8, June 1887 Copyright© 1887 by Helmer8 Publl,hlng, Inc. 174 Concord St., Peterborough NH 03451 All RIght' ReHrYed