Image Segmentation using Global and Local Fuzzy Statistics Debashis Sen, Member, IEEE, and Sankar K. Pal, Fellow, IEEE

Abstract— In this paper, criterion optimization based image thresholding techniques to perform segmentation using global and local fuzzy statistics are presented. The global and local fuzzy statistics considered for an image are the fuzzy histogram and fuzzy co-occurrence matrix of the image, respectively. A novel way of adapting the membership function, which is required to calculate the fuzzy statistics, to the local nature of the corresponding crisp first-order statistic (histogram) is suggested. The fuzzy statistics of an image obtained using such an adaptive membership function are called adaptive fuzzy statistics. Experimental results of various image segmentation techniques using crisp, fuzzy and the proposed adaptive fuzzy statistics are given. A comparative study demonstrating the usefulness of fuzzy statistics in image segmentation and the effectiveness of adapting the membership function in order to determine the fuzzy statistics is presented. Index Terms— Co-occurrence matrix, Fuzzy co-occurrence matrix, Fuzzy histogram, Histogram, Image segmentation, Membership function.

I. I NTRODUCTION EVERAL image processing and computer vision systems require an image segmentation task at a pre-processing stage to isolate the objects from the rest of the scene or to segregate the scene into different regions. Thresholding based on the histogram or co-occurrence matrix of an image, due to its simplicity and ease of implementation, has been a popular technique to carry out segmentation in images and video frames. In literature, several graylevel histogram and co-occurrence matrix based thresholding algorithms have been proposed, with most of them optimizing a criterion function to determine the threshold values [1]–[6]. Usually, images do not have well-defined boundaries separating the various regions and the objects from the background in the scene. In case of digital images, there also exists an inherent imprecision associated with the gray values of the pixels. Due to these reasons, images are of ambiguous nature and it is appropriate to take into account this nature of images while carrying out thresholding operations. Most of the popular existing thresholding techniques do not consider the ambiguity present in images [1]–[3]. In [4] and [6], the authors consider the fuzzy nature of images and hence minimize various fuzziness measures to obtain the threshold values. However, the required graylevel histogram and co-occurrence matrix are calculated considering each gray value in the image as a crisp number. In this paper, we take into account the ambiguous nature of images by considering each gray value in the image as a

S

D. Sen and S. K. Pal are with the Center for Soft Computing Research, Indian Statistical Institute, 203 B. T. Road, Kolkata, India 700108. E-mail: {dsen t, sankar}@isical.ac.in.

fuzzy number. The graylevels represented by fuzzy numbers are then used to calculate the graylevel fuzzy histogram (global statistic) and fuzzy co-occurrence matrix (local statistic) similar to those proposed in [7]. The authors in [7] used a triangular membership function in order to calculate the global and local fuzzy statistics. We suggest that the membership function that is used to represent the gray values in an image as fuzzy numbers should be adapted to the local nature of the corresponding crisp histogram (first-order statistic). In case of image segmentation such adaptation would preserve or enhance certain important features of the crisp statistics in the fuzzy statistics, while eliminating the insignificant or unwanted ones. We shall refer to the fuzzy statistics obtained using adaptive membership function as adaptive fuzzy statistics. In order to segment the image into N regions, N −1 threshold values may be obtained using global or local statistics and by optimizing various criterion functions. In this paper, we restrict our analysis to bilevel thresholding, that is N = 2, as most of the criterion optimization based techniques could be easily extended to multilevel thresholding (N > 2). Bilevel thresholding is carried out to extract the objects in an image from the background. Simulation results and comparisons of the performance of various criterion optimization based thresholding techniques using crisp, fuzzy and adaptive fuzzy statistics are presented. The effectiveness of the proposed adaptation of the membership function in order to determine the fuzzy statistics is demonstrated with the help of a few qualitative results. The organization of the paper is as follows. The fuzzy statistics of digital images are presented in Section II. The concept of adapting the membership function, which is used to calculate the fuzzy statistics, to the first-order crisp statistic is introduced in Section III. The segmentation of images using fuzzy statistics and criterion optimization based thresholding techniques is explained in Section IV. In Section V, experimental results of the various thresholding techniques using crisp, fuzzy and adaptive fuzzy statistics are presented. The paper concludes with Section VI by providing an overview of the analysis and contributions made.

II. F UZZY S TATISTICS OF D IGITAL I MAGES As digital images do not have well-defined boundaries and the gray values of the pixels are imprecise, we represent the gray values in images using fuzzy numbers. Fuzzy numbers are normal convex fuzzy sets on a real line [8]. Let us consider a symmetrical triangular fuzzy number n ˜ , whose membership

grade at a crisp number x on the real line is given as 
|x − n| µn˜ x = max 0, 1 − α

(1)

where n is the crisp counterpart of the fuzzy number n ˜ and α is a real positive constant. We see that at x = n, the membership grade is 1, and the membership grade decreases with increase in the distance between x and n till |x − n| = α. Now, we present the first- and second-order fuzzy statistics of digital images similar to those given in [7]. A. Fuzzy Histogram (F.H) First-order statistic of an image is it’s histogram that gives the information about the distribution of the gray values in it’s dynamic range [0, L − 1], where L is the number of gray levels. Since, the histogram furnishes information about the image as a whole, it is a global statistic. As mentioned in [7], a gray value g in a digital image can be g +1 or g −1, without any significant change in the visual perception. The classical framework that defines a histogram as a sequence of integers pg with each integer representing the frequency of occurrence of a particular gray value g does not consider such ambiguity in images. On the other hand, the fuzzy histogram is defined as a sequence of real numbers with each real number representing the frequency of occurrence of gray values that are around a particular gray value, say g, or in other words frequency of occurrence of a fuzzy gray value g˜. Using (1), the fuzzy histogram, which is a sequence of real numbers represented as qg , is calculated as  µg˜I(m,n) , g ∈ {0, 1, · · · , L − 1} (2) qg = m

where s and t takes all the values in the set {0, 1, · · · , L−1}, with the constraint that |s − t| ≥ d. In this paper, we consider the simplest case, d = 1. III. F UZZY S TATISTICS USING AN A DAPTIVE M EMBERSHIP F UNCTION (A DAPTIVE F UZZY S TATISTICS ) A membership function for each fuzzy gray value is required to calculate it’s membership grades at all the crisp graylevels in the image. In the previous section and in [7], a triangular fuzzy number or membership function (refer 1) has been considered. Similar to the authors in [7], let us assume that the bandwidth (2α) of the membership function is a global constant throughout the image. However, by using the triangular membership function we inherently also assume that the membership grade of all the fuzzy gray values is similar at the crisp graylevels at same graylevel distance within it’s bandwidth. Such an assumption may not be true for an image. Moreover, we find that in case of image segmentation by thresholding, the usage of such a membership function might result in the elimination of a few significant features such as ‘valleys’, ‘shoulders’ [9] from the local and global statistics. Hence we suggest that the membership function used to calculate fuzzy graylevels for image segmentation purposes may be adapted to the nature of the crisp first-order statistics (histogram) in a local neighborhood. In these regards, we formulate a symmetrical membership function or fuzzy number n ˜ that is dependent on two parameters, say α and γ, and whose membership grade at a crisp number x is given as
   |x − n|  1 · 1 − exp − 1 − γ µn˜ x = 1 − exp(−γ) α

n th

th

where I(m, n) is the gray value at m row and n column of the image I under consideration. This fuzzy histogram can represent the ambiguity in information in a much better manner than its crisp counterpart. B. Fuzzy Co-occurrence Matrix (F.C) A gray level co-occurrence matrix is a second-order statistic of an image that gives the joint probability of occurrence of two gray values separated by a particular distance, say d. Since the co-occurrence matrix furnishes information about the local distribution of gray values in an image, it is a local statistic. This local statistic  of digital images is represented in matrix form H = hdst , where s and t are the concerned gray values. The classical framework defines a co-occurrence matrix as a matrix of integers representing the frequency of occurrence of a particular gray value s followed by another gray value t. Here, we present a fuzzy co-occurrence matrix that gives the frequency of occurrence of the gray values around a value s followed by the gray values around another value t. In other words, it gives the frequency of occurrence of a fuzzy gray ˜ value s˜ followed by another fuzzy  d gray value t. The fuzzy co-occurrence matrix, say F = fst is calculated as  d fst = min(µs˜I(m,n) , µt˜I(ρ,κ) ) (3) m

n

ρ

κ

for 0 ≤ =

0 otherwise

|x − n| ≤1 α (4)

where n is the crisp counterpart of the fuzzy number n ˜, 2α is the bandwidth of the membership function and γ is a parameter calculated from the local neighborhood of n in the crisp histogram under consideration. The expression for γ is given as
 |σ 2 − ν 2 | −1 γ=M 2 (5) max[σ 2 , ν 2 ] where −M ≤ γ ≤ M , with M a positive real number, σ 2 is the sample variance in a local neighborhood in the crisp histogram and ν is obtained from the bins pg of the crisp histogram as ν

=

1.48 × median([|pg − median([pg ])|]), g ∈ {0, 1, · · · , L − 1}

(6)

The expression of γ given in (5) is based on the concept of the adaptive Wiener filter proposed in [10]. The expression for ν given in (6) is well known as an estimate of the variance of a Gaussian distributed random variable. The value of ν 2 indicates the non-smoothness in the overall crisp histogram, whereas the value of σ 2 indicates the non-smoothness in a local neighborhood in the crisp histogram. Figure 1 shows

optimization techniques work on histograms or co-occurrence matrices which could be calculated using crisp, fuzzy or adaptive fuzzy gray values. We consider the following popular criterion optimization techniques that have been applied on crisp histogram (C.H) and crisp co-occurrence matrix (C.C) of digital images and apply them on fuzzy and adaptive fuzzy statistics for threshold selection. A. Maximization of separability

Fig. 1.

The proposed adaptive fuzzy number

the membership function given in (4) for different values of γ. As can be seen, at γ = 0, we consider the membership function to be the triangular membership function. This is due to the reason that as γ → 0 the proposed membership function reduces to the triangular membership function, which can be easily proved by using the L’hospitals rule on µn˜ x given in (4) and obtain lim µn˜ x

γ→0

|x − n| |x − n| for 0 ≤ ≤1 α α = 0 otherwise

= 1−

(7)

The membership function is designed by comparing the global and local smoothness of the crisp histogram (refer (5)). The concept behind this method is that if the local smoothness is less compared to the global smoothness then the uncertainty or ambiguity is more in the local neighborhood. The dependency of the membership function on the parameter γ would preserve or sometimes enhance the significant features in local and global statistics. Now in order to prove that the expression in (4) is indeed a valid fuzzy number, we shall show that it is normal and convex in nature. Similar to the triangular fuzzy number given in (1), the proposed adaptive membership function also has a membership grade 1 at x = n and the membership grade decreases with the increase in the distance between x and n till |x − n| = α. Hence, the proposed membership function is always normal. In the proposed membership function, µn˜ x1 ≥ µn˜ x2 for x2 ≥ x1 . Hence it can be easily shown that µn˜ (λx1 +(1−λ)x2 ) ≥ min[µn˜ x1 , µn˜ x2 ], where λ ∈ [0, 1] (8) This proves that the proposed membership function is convex and as it is also normal, it qualifies as a fuzzy number [8]. The histogram and co-occurrence matrix calculated from the fuzzy graylevels represented by the proposed adaptive fuzzy number shall be called the adaptive fuzzy histogram (A.F.H) and Adaptive fuzzy co-occurrence matrix (A.F.C). IV. S EGMENTATION OF I MAGES USING F UZZY S TATISTICS AND C RITERION O PTIMIZATION We consider various criterion optimization techniques in order to determine the threshold value to carry out segmentation of images for object extraction. These criterion

Otsu obtained an optimal threshold value by maximizing the separability of resultant classes of gray values [1]. This method utilizes the zeroth- and first-order cumulative moment of the crisp graylevel histogram. We instead use the fuzzy and adaptive fuzzy histogram and maximize the separability of resultant classes of fuzzy gray values to obtain the corresponding optimal thresholds. B. Entropic thresholding Kapur et al. obtained an optimal threshold value by maximizing the information between the object and background distributions in the image [2]. The information is given by the sum of the Shannon’s entropies of the two resultant classes of gray values with the probabilities given by the crisp graylevel histogram. Again, we use the fuzzy and adaptive fuzzy histogram and maximize the information between the fuzzy object and background distributions in the image to arrive at an optimal threshold. Pal et al. maximized the local and conditional entropies of an image using its graylevel co-occurrence matrix to determine the threshold value [5]. In this paper, we calculate the local and conditional entropies using the fuzzy and adaptive fuzzy co-occurrence matrix and maximize them to obtain the corresponding threshold values. C. Moment preserving thresholding Tsai proposed a thresholding technique that retains the graylevel moments of the original image in the thresholded image [3]. In other words, the threshold is selected in such a fashion that the difference between the graylevel moments of the original image and that of the threshold image is minimized. In this paper, we calculate the graylevel moments from the fuzzy and adaptive fuzzy histogram of the original image and find a threshold value retaining these moments in the thresholded image. D. Ambiguity minimization Pal et al. in [4] and [6], optimized various measures of fuzziness in order to minimize the ambiguity present in images and obtain a suitable threshold value. In these methods, each graylevel in the image is associated with a degree of brightness measure using a suitable function. Then, these degree of brightness measures and the image histogram or co-occurrence matrix are used to calculate various fuzziness measures such as index of fuzziness and fuzzy entropy. The optimization of these measures of fuzziness results in the selection of a graylevel as the suitable threshold value.

As can be inferred that although the ambiguity in images have been considered in these methods and represented by the degree of brightness, crisp histograms and co-occurrence matrices were used for the calculation of the fuzziness measures. We instead use the fuzzy and adaptive fuzzy histogram and co-occurrence matrix in order to calculate the fuzziness measures and hence carry out the thresholding process. V. E XPERIMENTAL R ESULTS In this section, we present a few results of the extensive simulations carried out in order to study the process of image segmentation by thresholding using fuzzy and adaptive fuzzy histograms and co-occurrence matrices.To analyze the effectiveness of using fuzzy statistics, we compare these results with those obtained using crisp statistics. The image statistics shall be used by various criterion optimization based

(a) Bacteria Image

(c) C.H & (i)

(f) C.H & (ii)

(i) C.H & (iii)

(j) F.H & (iii)

(k) A.F.H & (iii)

(l) C.H & (iv)

(m) F.H & (iv)

(n) A.F.H & (iv)

(o) C.C & (v)

(p) F.C & (v)

(q) A.F.C & (v)

(r) C.C & (vi)

(s) F.C & (vi)

(t) A.F.C & (vi)

(u) C.C & (vii)

(v) F.C & (vii)

(w) A.F.C & (vii)

(b) Various Histograms

(d) F.H & (i)

(g) F.H & (ii)

(e) A.F.H & (i)

(h) A.F.H & (ii)

thresholding techniques to carry out the segmentation process. The criterion optimization thresholding techniques that we have considered in this paper are (i) Otsu’s maximization of separability method [1], (ii) The entropic method by Kapur et al. [2], (iii) Tsai’s moment preserving method [3], and (vii) The higher order fuzzy entropic method [4]. These methods

Fig. 2. Qualitative results of different image segmentation processes based on various global and local statistics

shall be henceforth referred using their corresponding number in this paper. Figure 2 gives the qualitative results obtained using the above mentioned methods for image segmentation based on crisp, fuzzy and adaptive fuzzy statistics. Figure 2 (b) shows the crisp, fuzzy and adaptive fuzzy histograms of the Bacteria image.In order to calculate the adaptive fuzzy statistics, we have used a local neighborhood size equal to the size of the bandwidth α and the value of M required in the (iv) The ambiguity minimization method by Pal et al [4], (v) The higher order entropic method by Pal and Pal [5], (vi) The higher order conditional entropic method by Pal and Pal [5], expression of γ (refer (5)) equal to half the bandwidth α2 . As can be seen, the fuzzy histogram is much smoother than the crisp histogram with most of the local minima eliminated. However, at the same time the fuzzy histogram may not have some essential features such as ‘valleys’, which might have got eliminated during the fuzzification process. It is also evident from the figure that the adaptive fuzzy histogram retains certain important features present in the crisp histogram, while smoothing out the insignificant local minima. From the figure we find that for the bacteria image, the results obtained using crisp, fuzzy and adaptive fuzzy techniques produce similar qualitative results. Figure 3 gives a few qualitative results of image segmentation on different images where the effectiveness of using an adaptive membership function to calculate the fuzzy statistics is evident. In Figure 3 (a)-(e), it is evident that the use of adaptive fuzzy statistics results in better segmentation. As can be seen in Figure 3 (b), the reason behind the better performance is that the adaptive fuzzy statistics has a local minima at a ‘shoulder’, which is a feature that is enhanced in the adaptive fuzzy statistics when compared to the crisp statistics. In Figure 3 (f)-(j), we see that again a ‘shoulder’ feature is enhanced which helps in the proper selection of threshold value in order to extract out the chromosomes from the background. On the other hand, an insignificant local minima present in both the crisp and fuzzy histogram is suppressed in the adaptive fuzzy histogram. In Figure 3 (i)(o), the threshold value (gray value 90) obtained using the various image statistics produce similar results. However, as can be seen from Figure 3 (l) the credible valley (gray value 90) is much more prominent in the adaptive fuzzy histogram than the others. Due to a similar reason, in Figure 3 (p)(t), the segmentation results obtained using the adaptive fuzzy histogram is slightly better than the others.

(a) Brain Image

(c) C.H & (vii)

(b) Various Histograms

(d) F.H & (vii)

(f) Chromosome Image

(h) C.H & (iv)

(e) A.F.H & (vii)

(g) Various Histograms

(i) F.H & (iv)

(j) A.F.H & (iv)

VI. C ONCLUSION Criterion optimization based image thresholding techniques to perform segmentation using global and local fuzzy statistics has been explored in this paper. The fuzzy statistics, namely, fuzzy histogram and fuzzy co-occurrence matrix have been calculated by considering the gray values in the image as fuzzy numbers. A membership function adapted to the crisp first-order statistic or histogram is proposed to calculate the fuzzy gray values from the crisp gray values. The concept behind this adaptation has been that more the local smoothness

(k) SAR Image

(l) Various Histograms

(m) C.H & (iv)

(n) F.H & (iv)

(p) A Standard Image

(r) C.H & (i)

(o) A.F.H & (iv)

(q) Various Histograms

(s) F.H & (i)

(t) A.F.H & (i)

Fig. 3. Qualitative results showing the effective usage of adaptive fuzzy statistics for image segmentation

compared to the global smoothness in the crisp histogram lesser the ambiguity in the local neighborhood. Fuzzy statistics obtained using such an adaptive membership function are called adaptive fuzzy statistics. Experimental results of various image segmentation techniques using crisp, fuzzy and adaptive fuzzy statistics have been given and a comparative study showing the effectiveness of adapting the membership function to determine the fuzzy statistics has also been presented. R EFERENCES [1] N. Otsu, “A threshold selection method from gray-level histogram”, IEEE Trans. Syst., Man, Cybern., vol. 9, no. 1, pp. 6266, 1979. [2] J. N. Kapur, P. K. Sahoo, and A. K. C. Wong, “A new method for graylevel picture thresholding using the entropy of the histogram”, Computer Vision, Graphics, and Image Processing, vol. 29, pp. 273-285, 1985. [3] W.-H. Tsai, “Moment-preserving thresholding: a new approach”, Computer Vision, Graphics, and Image Processing, vol. 29, pp.377-393, 1985. [4] S. K. Pal, R. A. King, and A. A. Hashim, “Automatic grey level thresholding through index of fuzziness and entropy”, Pattern Recognition Letters, vol. 1, no. 3, pp. 141-146, 1983. [5] S. K. Pal and N. R. Pal, “Object extraction from image using higher order entropy”, in Proceedings of the IEEE International Conference on Pattern Recognition, vol. 1, pp. 348-350, 1988.

[6] S. K. Pal and A. Ghosh, “Image segmentation using fuzzy correlation”, Information Sciences, vol. 62, no. 3, pp. 223-250, 1992. [7] C. V. Jawahar and A. K. Ray, “Fuzzy statistics of digital images”, IEEE signal processing letters, vol. 3, no. 8, pp.225-227, 1996. [8] G. Klir and B.Yuan, Fuzzy Sets and Fuzzy Logic: Theory and Applications. New Delhi, India: Prentice Hall, 2005. [9] A. Rosen¯feld and P. D. L. Torre, “Histogram concavity analysis as an aid in threshold selection”, IEEE Trans. Syst., Man, Cybern., vol. 13, no. 1, pp.231-235, 1983. [10] J. S. Lee, “Digital image enhancement and noise filtering by use of local statistics”, IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-2, no. 2, pp.165-168, 1980.

Debashis Sen (S’05, M’06) received his M.A.Sc degree in Electrical Engineering from Concordia University at Montreal, Canada in 2005 and B. Engg. (Hons.) degree in Electronics and Communication Engineering from the University of Madras at Chennai, India in 2002. He was associated as a research assistant with the Center for Signal Processing and Communications (CENSIPCOM) and the Video Processing and Communications Group at the Electrical and Computer Engineering Department of Concordia University. He is currently pursuing research towards a doctoral degree at the Center for Soft Computing Research, Indian Statistical Institute at Kolkata, India, where he holds the position of a research scholar. His research interests encompasses image processing, video processing, probability theory and soft computing.

Sankar K. Pal (M’81, SM’84, F’93) is the Director and a Distinguished Scientist of the Indian Statistical Institute. He founded the Machine Intelligence Unit, and the Center for Soft Computing Research: A National Facility in the Institute in Calcutta. He received a Ph.D. in Radio Physics and Electronics from the University of Calcutta in 1979, and another Ph.D. in Electrical Engineering along with DIC from Imperial College, University of London in 1982. He worked at the University of California, Berkeley and the University of Maryland, College Park in 1986-87; the NASA Johnson Space Center, Houston, Texas in 1990-92 & 1994; and in US Naval Research Laboratory, Washington DC in 2004. Since 1997 he has been serving as a Distinguished Visitor of IEEE Computer Society (USA) for the Asia-Pacific Region, and held several visiting positions in Hong Kong and Australian universities. Prof. Pal is a Fellow of the IEEE, USA, The Academy of Sciences for the Developing World, Italy, International Association for Pattern recognition, USA, and all the four National Academies for Science/Engineering in India. He is a co-author of thirteen books and about three hundred research publications in the areas of Pattern Recognition and Machine Learning, Image Processing, Data Mining and Web Intelligence, Soft Computing, Neural Nets, Genetic Algorithms, Fuzzy Sets, Rough Sets and Bioinformatics. He has received the 1990 S.S. Bhatnagar Prize (which is the most coveted award for a scientist in India), and many prestigious awards in India and abroad including the 1999 G.D. Birla Award, 1998 Om Bhasin Award, 1993 Jawaharlal Nehru Fellowship, 2000 Khwarizmi International Award from the Islamic Republic of Iran, 2000-2001 FICCI Award, 1993 Vikram Sarabhai Research Award, 1993 NASA Tech Brief Award (USA), 1994 IEEE Trans. Neural Networks Outstanding Paper Award (USA), 1995 NASA Patent Application Award (USA), 1997 IETE-R.L. Wadhwa Gold Medal, the 2001 INSA-S.H. Zaheer Medal, and 2005-06 P.C. Mahalanobis Birth Centenary Award (Gold Medal) for Lifetime Achievement. Prof. Pal is an Associate Editor of IEEE Trans. Pattern Analysis and Machine Intelligence, IEEE Trans. Neural Networks, Pattern Recognition Letters, Applied Intelligence, Information Sciences, Fuzzy Sets and Systems, Fundamenta Informaticae, Int. J. Computational Intelligence and Applications, and Proc. INSA-A; a Member, Executive Advisory Editorial Board, IEEE Trans. Fuzzy Systems, Int. Journal on Image and Graphics, and Int. Journal of Approximate Reasoning; and a Guest Editor of IEEE Computer.