Fuzzy Logic based Protection for Image Resizing by Seam carving Animesh Banerjee, Bibhu Prasad Mishra, Debdutta Bhattacharya, Kunal Kishore, Kundan Kumar and Sushil Subramanian Department of Electronics and Electrical Communication Engineering Indian Institute of Technology, Kharagpur – 721302, India

Abstract Seam carving for resizing images has been proven to be more effective for image compression and enlargement than conventional methods (such as scaling and column removal) used by most image manipulation software [2]. Seam carving searches for connected seams with the least energy in the picture. For this cause, the seam carving algorithm computes the energy function using a simple gradient method and removes connected seams with minimal energy (see Fig. 1).

(a)

(b)

Figure 1. (a) Energy function of an image using the gradient method (b) Least energy seam As these seams are removed, the content of the image is retained, as the least energy seam is least noticeable. However, in some cases the content based seam carving algorithm passes through many noticeable objects even though their energy is low. Fig. 1(b) shows such this unwanted effect (notice seam passing through human being). A test on many other images reveals that this case most occurs in two cases: where highly noticeable objects having low energy than the background, or the number of seams being removed is large. When both these effects combine, the results in seam carving are disastrous, as shown in Fig. 2. It has important objects in the foreground with heavy energy in the background. The number of seams removed is as little as 20. Thus, seam carving is not completely protection free for low energy objects with high content. Examples of such objects may be human beings, buildings, small objects etc. For this reason, we introduce a fuzzy logic based image segmentation method and neural network based skin detection method to specifically protect humans, who are generally the most noticeable objects in pictures. This method may be extended for other objects too with their own pattern recognition techniques. However, our method is tailor-made for the seam carving algorithm and works very well with simple and high energy images.

(a)

(b)

Figure 2. (a) the original picture (b) the picture with 20 seams removed using seam carving. Note that this image has low energy on the human being thus distorting his face Fuzzy Image Segmentation The images are divided into segments by the fuzzy c-means algorithm [1]. The subtractive clustering algorithm is used to predict the number of segments in the image. The fuzzy c-means algorithm uses the number of clusters and performs iterative clustering to re-compute the centers of the segments. The segmentation assigns higher weights to intensity values as compared to proximity. The fuzzy cmeans algorithm assigns membership values to each pixel for each region. We then define three fuzzy membership functions which use the region membership values to classify each pixel into a particular region. The membership functions include region pixel distribution, closeness to a region’s average intensity and pixel spatial relations that is relation with surrounding pixels. Each pixel has a membership value in each region. Weights are assigned to each function. The key weighing factor is obtained by data-mining from the picture. The pixel gets classified into a particular region based on the simple fuzzy “If–Then” rules for regions using weights for the values of three membership functions. Thus, we get a segmented image as our output. Fuzzy logic segmentation divides the picture into segments and classifies the picture into regions based on those segments, thus facilitating the manipulation of the weights introduced by the energy picture. Fuzzy logic segmentation has been shown to classify images better than most fast performing [4] crisp methods . To detect human we introduce the skin detecting neural network. Skin Detecting Neural Network For skin detection we use a neural network with skin and non-skin pixels as inputs. We use a simple back propagation algorithm without momentum, owing to its characteristics of fast computation. We use about 12000 skin and non-skin pixels for our training, obtained from the World Wide Web. The training of the pixels is done after normalizing the RGB values so that two of them can be input thus decreasing the network size. The network is then trained with the transfer functions as log sigmoid to detect skin and non-skin. Since the red tone lies between Caucasian and African tones, RGB network fails to detect the red color as non-skin. Similarly, the pixels in the HSV format fail to detect dark wood brown as non-skin. A similar normalization is also done in HSV images. Thus, we train networks for both pixel formats and then find an intersection of both images to remove these tones in the images. Isolated impulse noise was visible in the picture. This is removed using a window based impulse noise removal method by scanning the entire picture for noise in the form of a window proportional to the size of the image. After the noise is removed, the picture is ready for merging with segmented image. We have tested this algorithm with pictures of various faces and human skin. By visual inspection, the accuracy is very high.

Merging of the segmented image and the skin detected image is done simply by seeing which regions in the segmented image have more skin. The corresponding regions are assigned normalized weights (between 0 and 1) proportional to the amount of skin which they carry. Therefore, an entire region with lots of skin gets more weight, thus successfully protecting the human being in it. These weights are added to the original energy function thereby producing a new weight picture. Seam carving on this new picture (with revised weights from image and color segmentation) results in very good protection. We find that the image segmentation spreads out the effect of a human being’s presence in a picture thus safely making the algorithm compatible for seam carving. Skin detection identifies the human regions. For example, see Fig. 3.

(a)

(b)

(c)

Figure 3. (a) The original picture (b) Picture with 100 seams removed using seam carving; notice, for example, the person second from left is thinner (c) Picture with our improved seam carving and 100 seams removed; the human features are protected

The above method proposed also works for Fig. 2 (b), with excellent results. Future directions in this method include firstly, optimizing the running time of all algorithms. Also, better pattern recognition techniques for protecting other objects may be used. We propose the use of automatic protection in videos as a very prospective application, where frames may be resized and protected, and user inputed protection is impossible owing to large number of frames. References 1. Albayrak, S. and Amasyali, F. (2003). “Fuzzy c-means clustering on medical diagnostic th systems”. 12 International Turkish Symposium on Artificial Intelligence and Neural Networks, TAINN 2003, Çanakkale, Turkey. 2-4th July, 2003. 2. Avidan, S. and Shamir, A. (2007). “Seam carving for Content Aware Resizing”. The 34th International Conference on Computer Graphics and Interactive Techniques, San Diego, California, 7-9th August 2007, Vol. 26, Issue 3. 3. Chen, L., Zhou, J., Liu, Z., Chen, W. and Xiong, G. (2002). “A skin detector based on neural network”. IEEE 2002 International Conference on Communications, Circuits and Systems and West Sino Expositions, 29th June – 1st July 2002, Vol. 1, Page No. 615-619. 4. Karmakar G.C. and Dooley L.S. (2002). “A generic fuzzy rule based image segmentation algorithm”. Pattern Recognition Letters, August 2002, Vol. 23, Issue 10, Page No. 1215-1227. 5. Kim, I., Shim, H.J. and Yang, J. (2003). “Face Detection”. Face Detection Project, Course No. EE368, Stanford University, 28th May, 2003. 6. Seow, M., Valaparla, D. and Asari, V.K. (2003). “Neural Network Based Skin Color Model for Face Detection”. Proceedings of the 32nd Applied Imagery Pattern Recognition Workshop, 15-17th October 2003, Page No. 141-145. 7. Web Reference: http://en.wikipedia.org/wiki/Cluster_analysis, “Cluster Analysis”.