Automatic Recognition of Fruits and Vegetables and Detection of Fruit Diseases A thesis submitted in partial fulfilment of the requirements for the award of the degree of
Master of Technology in
Computer Science and Engineering By Shiv Ram Dubey Roll No: 108150013
Department of Computer Engineering & Applications Institute of Engineering & Technology
GLA University Mathura- 281406, INDIA July, 2012
Department of computer Engineering and Applications GLA University, Mathura 17 km. Stone NH#2, Mathura-Delhi Road, P.O. – Chaumuha, Mathura – 281406 U.P (India)
Declaration I hereby declare that the work which is being presented in the M.Tech. thesis “Automatic Recognition of Fruits and Vegetables and Detection of Fruit Diseases”, in partial fulfillment of the requirements for the award of the Master of Technology in Computer Science and Engineering and submitted to the Department of Computer Engineering and Applications of GLA University, Mathura, is an authentic record of my own work carried under the supervision of Mr. Anand Singh Jalal, Associate Professor, GLA University, Mathura. The contents of this thesis, in full or in parts, have not been submitted to any other Institute or University for the award of any degree and are free from plagiarism.
Signature of Candidate: Name of Candidate: Shiv Ram Dubey Roll. No. 108150013
Certificate This is to certify that the above statement made by the candidate is correct to the best of my/our knowledge and belief.
Signature of Supervisor(s): Date: Name & Designation of Supervisor(s): Mr. Anand Singh Jalal, Associate Professor, Dept. of CEA, GLAU, Mathura
ii
Dedicated To My Parents and Teachers
iii
Abstract Images are the important source of data and information in the agricultural sciences. The use of image processing techniques is of great significance for the analysis of agricultural operations. Fruit and vegetable classification is one of the major applications that can be used in the supermarket to automatically detect the kind of the fruit or vegetable purchased by the customer and to generate the prices for it. Training on-site is the fundamental requirement for this type of system, which is mostly done by the users having little or no technical knowledge. In this thesis, we have addressed this problem and designed a methodology for these types of problem which requires less number training examples. In addition, we have presented an efficient improved sum and difference histogram (ISADH) texture feature which are based on the sum and difference of the intensity values of the neighboring pixels of an image. Our experimental result suggest that proposed ISADH feature shows very high accuracy and outperform other color and texture feature. Fruit disease recognition is our second contribution that aims to detect and classify the diseases present in the fruit images. Precise defect segmentation is required to segment the infected area in the image. We have proposed a framework for the automatic detection and classification of fruit diseases from the images. To achieve good result, we have used K-means clustering based segmentation approach. Our proposed method is able to distinguish between those diseases which are very similar in color and texture.
iv
Acknowledgments I have taken efforts in this thesis. However, it would not have been possible without the kind support and help of many individuals. I would like to extend my sincere thanks to all of them. I am highly indebted to Mr. Anand Singh Jalal for their guidance and constant supervision as well as for providing necessary information regarding the thesis & also for their support in completing the thesis. I sincerely thank to the chancellor and vice-chancellor of GLA University to provide a research oriented infrastructure. I would like to express my special gratitude and thanks to the Head of Department, Computer Engineering and Applications, Prof. Krishna Kant for giving me attention and time. I would like to thank the members of the Research Group at Computer Engineering and Applications, namely Prof. Charul Bhatnagar, Mr. Ashish Sharma, and Mr. Manas Kumar Mishra for their valuable suggestions and helpful discussions. My thanks and appreciations also go to my colleague in developing the thesis and people who have willingly helped me out with their abilities. I would like to express my gratitude towards my parents for their kind co-operation and encouragement which help me in completion of this thesis.
Shiv Ram Dubey GLA University, Mathura
v
Contents Certificate
ii
Abstract
iv
Acknowledgments
v
List of Tables
viii
List of Figures
ix
1. Introduction
1
1.1
Motivation and Overview ………………………………........
1
1.2
Issues and Challenges ….………………………………….…
3
1.3
Objectives …………………………………………………....
4
1.4 Contribution of the Thesis …………………………………...
4
Outline of the Thesis ….……………………………………..
5
1.5
2. Literature Review
7
2.1 Introduction …………………………………………………..
7
2.2 Previous Work ……………..…………………………………
7
Fruit and Vegetable Recognition …..…………………
7
2.2.2 Fruit Disease Detection ……………………...……….
11
2.3 Summary ……….…………………………………………….
15
2.2.1
3. Fruit and Vegetable Classification
16
3.1
Introduction ……………………….…………………………
16
3.2
Proposed Framework ……………………..………………….
17
3.2.1 Image Segmentation ………...………..……………...
18
3.2.2
Feature Extraction ...…………..……………………..
19
3.2.3
Training and Classification ........…………………….
22
Results and Discussion ……………....…...………………….
23
3.3
Contents
3.4
3.3.1
Data Set ……………………………………………...
24
3.3.2
Result Discussion ……………………………………
25
Summary ……………..……………………………………...
29
4. Automatic Detection and Classification of Fruit Diseases
30
4.1
Introduction ………….………………………………………
30
4.2
Proposed Framework ……………………..………………….
31
4.2.1
Detection Segmentation ……………………….….....
32
4.2.2
Feature extraction ……….…………………………...
34
4.2.3
Supervised Learning and Classification ...…………...
36
Results and Discussion ...…………………………………….
36
4.3.1 Data Set Preparation …………….…………………...
37
4.3.2
Result discussion ...………………………..…………
38
Summary ……………………..……………………………...
42
4.3
4.4
5. Conclusions and Future Directions
43
5.1
Summary and Contributions ……………………………........
43
5.2
Future Work ….……………………………………………...
44
References
45
List of Publications based on the Research Work
52
vii
List of Tables
3.1
Simple histogram for both matrixes (8-bin) ……………………..
21
3.2
Unser feature for both matrixes (8-bin) ………………………….
21
3.3
ISADH feature for both matrixes (8-bin) ………………………..
21
3.4
Difference in feature of Matrix ‘A’ and Matrix ‘B’ ……………..
22
3.5
Unique ID of each class …………………………………………
22
3.6
Accuracy (%) in HSV color space when trained with 40 images per class ………………………………………………………….
28
viii
List of Figures
3.1
Fruit Recognition System ….…………………………….………
3.2
Extracting region of interest from the images (a) before segmentation, (b) after segmentation …………………………….
17 18
3.3
Data set used ……………………………………………………..
24
3.4
Illumination differences, Kiwi category …………………………
25
3.5
Pose differences, Cashew category ……………………………...
25
3.6
Variability on the no. of elements, Orange category …………….
25
3.7
Examples of cropping and partial occlusion …………………….
25
3.8
Comparison of GCH, CCV, BIC, Unser, and proposed ISADH features using SVM as a base learner in RGB color space ……...
3.9
Comparison of GCH, CCV, BIC, Unser, and proposed ISADH features using SVM as a base learner in RGB color space ……...
3.10
Comparison of results in RGB and HSV color space using (a) GCH, (b) CCV, (c) BIC, (d) Unser, and (e) ISADH feature …….
4.1
Proposed Approach for the Fruit Disease Recognition ………….
4.2
K-Means clustering for an apple fruit that is infected with apple
26
26
27
32
scab disease (a) The infected fruit image, (b) first cluster, (c) second cluster, (d) third cluster, and (e) fourth cluster,
33
respectively, (f) single gray-scale image colored based on their cluster index ……………………………………………………... 4.3
Image segmentation results (a) Images before segmentation, (b) Images after segmentation ……………………………………….
ix
34
List of Figures
4.4
Sample images from the data set of type (a) apple scab, (b) apple rot, (c) apple blotch, and (d) normal apple ………………………
4.5
37
Accuracy (%) for the GCH, CCV, LBP, and CLBP features derived from RGB and HSV color images considering MSVM
38
classifier …………………………………………………………. 4.6
Comparison of the accuracy achieved in RGB and HSV color space for the GCH, CCV, LBP, and CLBP features considering
39
MSVM classifier ………………………………………………… 4.7
Accuracy per class for the LBP features in RGB and HSV color spaces using MSVM as a classifier ……………………………...
4.8
Accuracy per class for the CLBP features in RGB and HSV color spaces using MSVM as a classifier ……………………………...
40
41
x
Chapter 1 Introduction
1.1
Motivation and Overview
In agricultural science, images are the important source of data and information. To reproduce and report such data photography was the only method used in recent years. It is difficult to process or quantify the photographic data mathematically. Digital image analysis and image processing technology circumvent these problems based on the advances in computers and microelectronics associated with traditional photography. This tool helps to improve images from microscopic to telescopic visual range and offers a scope for their analysis. Several applications of image processing technology have been developed for the agricultural operations. These applications involve implementation of the camera based hardware systems or color scanners for inputting the images. We have attempted to extend image processing and analysis technology to a broad spectrum of problems in the field of agriculture. The computer based image processing is undergoing rapid evolution with ever changing computing systems. The dedicated imaging systems available in the market, where user can press a few keys and get the results, are not very versatile and more importantly, they have a high price tag on them. Additionally, it is hard to understand as to how the results are being produced. We have tried to develop a solution which presents classification problem in a most realistic way possible. Recognition system is a ‘grand challenge’ for the computer vision to achieve near human levels of recognition. The fruits and vegetables classification is useful in the super markets where prices for fruits purchased by a customer can be determined
Chapter 1
Introduction
automatically. Fruits and vegetables classification can also be used in computer vision for the automatic sorting of fruits from a set, consisting of different kinds of fruit. Recognizing different kind of vegetables and fruits is a recurrent task in the supermarkets, where the cashier must be able to identify not only the species of a particular fruit or vegetable (i.e., banana, apple, pear) but also identify its variety (i.e., Golden Delicious, Jonagold, Fuji), for the determination of it’s price. This problem has been solved by using barcodes for packaged products but most of the time consumers want to pick their product, which cannot be packaged, so it must be weighted. Assignment of codes for each kind of fruit and vegetable is a common solution to this problem; but this approach has some problems such as the memorization, which may be a reason for errors in pricing. As an aid to the cashier, a small book with pictures and codes is issued in many supermarkets; the problem with this approach is that flipping over the booklet is time-consuming. This research reviews several image descriptors in the literature and presents a system to solve the problem by adapting a camera at the supermarket that recognizes fruits and vegetables on the basis of color and texture cues. Formally, the system must output a list of possible type of species and variety for an image of fruit or vegetable. The input image contains fruit or vegetable of single variety, in random position and in any number. Objects inside a plastic bag can add hue shifts and specular reflections. Given the variety and impossibility of predicting which types of fruits and vegetables are sold, training should be done on site by someone having little or no technical knowledge. The solution of the problem is that the system must be able to achieve higher level of accuracy by using only a few training examples. Monitoring of health and detection of diseases is critical in fruits and trees for sustainable agriculture. To the best of our knowledge, no sensor is available commercially for the real time assessment of trees health conditions. Scouting is the most widely used method for monitoring stress in trees, but it is expensive, timeconsuming and labor-intensive process. Polymerase chain reaction which is a molecular technique used for the identification of fruit diseases but it requires detailed sampling and processing procedure. Early detection of disease and crop health can facilitate the control of fruit diseases through proper management approaches such as vector control through
Dept. of CEA, GLAU, Mathura
2
Chapter 1
fungicide
Introduction
applications,
disease-specific chemical applications
and
pesticide
applications; and improved productivity. The classical approach for detection and identification of fruit diseases is based on the naked eye observation by experts. In some of the developing countries, consultation with experts is a time consuming and costly affair due to the distant locations of their availability. Automatic detection of fruit diseases is of great significance to automatically detect the symptoms of diseases as early as they appear on the growing fruits. Fruit diseases can cause significant losses in yield and quality appeared in harvesting. For example, soybean rust (a fungal disease in soybeans) has caused a significant economic loss and just by removing 20% of the infection, the farmers may benefit with an approximately 11 million-dollar profit [1]. An early detection system of fruit diseases can aid in decreasing such losses caused by fruits diseases and can stop further spread of diseases. The various types of diseases on fruits determine the quality, quantity, and stability of yield. The diseases in fruits not only reduce the yield but also deteriorate of the variety and its withdrawal from the cultivation. Fruit diseases appear as spots on the fruits and if not treated on time, cause the severe loss. Excessive uses of pesticide for fruit diseases treatment increases the danger of toxic residue level on agricultural products and has been identified as a major contributor to the ground water contamination. Pesticides are also among the highest components in the production cost their use must be minimized. Therefore, we have tried to give such an approach which can detect the diseases in the fruits as soon as they produce their symptoms on the fruits such that proper management treat can be applied. A lot of work has been done to automate the visual inspection of the fruits by machine vision with respect to size and color. However, detection of defects in the fruits using images is still problematic due to the natural variability of skin color in different types of fruits, high variance of defect types, and presence of stem/calyx. To know what control factors to consider next year to overcome similar losses, it is of great significance to analyze what is being observed. Some fruit diseases also infect other areas of the tree causing diseases of twigs, leaves and branches. The precise segmentation is required for the defect detection. The early detection of fruit diseases (before the onset of disease symptoms) could be a valuable
Dept. of CEA, GLAU, Mathura
3
Chapter 1
Introduction
source of information for executing proper pest management strategies and disease control measures to prevent the development as well as the spread of fruit diseases.
1.2
Issues and Challenges
A number of challenges had to be addressed to enable the system to perform automatic fruits and vegetables recognition of the types of the fruit or vegetable present in the images from the camera. Many kinds of fruits and vegetables are subject to significant variation in shape, texture and color, depending upon their ripeness. For example, Orange ranges from being green, to yellow, to patchy and brown. Using just one image descriptor to secure the classes’ separability might not be sufficient, so it is necessary to extract and combine those features which are useful to the fruit and vegetable recognition problem. Sometimes, the object may be inside the plastic bag that can add hue shifts and specular reflections. Different classifier may produce different results, so the selection of classifier must also be addressed. In the literature, available classifiers works on two classes only, but in the produce classification problem we consider more than two classes, so it is a major issue to use binary classifier in a multiclass scenario. Background subtraction may become necessary to reduce the scene complexities such as illumination variation, sensor capturing artifacts, background clutter, shading, and shadows. The result of the system heavily depends upon the efficient working of the image segmentation method, so efficient image segmentation must be used. It might be interesting to consider the number of training examples because more number of training examples require more time to train the system. The system must perform better in situations where system is trained with less training examples.
1.3
Objectives
The objective of this thesis is to use the image processing techniques in the agriculture. The main objectives are as follows:
To recognize species and variety of fruits and vegetables from the images.
To classify the input image into one of the classes of fruits and vegetables.
Dept. of CEA, GLAU, Mathura
4
Chapter 1
Introduction
To identify the image descriptor that is suitable for the fruits and vegetables classification problem.
To automatically detect and classify the fruit diseases present in any image using image processing techniques.
To explore features that suited the fruit disease recognition problem.
To use SVM in the Multi-class scenario.
1.4
Contribution of the Thesis
The contribution of this thesis is to perform automatic image processing in the field of agriculture. The work presented in this thesis can be used for designing automatic systems for agricultural process using images from distant farm fields. The major contributions of this thesis are as follows:
In this thesis we have developed a framework for the fruits and vegetables classification problems to recognize the kind of the fruit present in the input image with its species and variety.
We have also proposed an efficient texture feature from the intensity values of neighboring pixels in the image.
We have also introduced a solution for the problem of detection and classification of fruit diseases from the images.
1.5
Outline of the Thesis
The thesis is outlined as follows: Chapter 2: Literature Review
In this chapter, we provide some of the core concepts used in image categorization and present a survey of efforts in the past to address this problem.
Chapter 3: Fruit and Vegetable Classification
In this chapter, we describe the framework for the fruit and vegetable classification problem and also introduce an efficient improved sum and difference histogram texture feature.
Dept. of CEA, GLAU, Mathura
5
Chapter 1
Introduction
Chapter 4: Automatic Detection and Classification of Fruit Diseases
In this chapter, we develop a framework for the automatic detection and classification of fruit diseases and evaluate the proposed methodology upon the apple diseases using some state of the art image descriptors.
Chapter 5: Conclusions and Future Directions
In this chapter, we summarize the novelties, achievements, and limitations of the proposed solutions in this thesis, and indicate some future directions.
Dept. of CEA, GLAU, Mathura
6
Chapter 2 Literature Review
2.1
Introduction
In this chapter, we focus on the previous work done by several researchers in the area of image categorization, fruits recognition, fruit diseases identification. Fruits and vegetables classification and fruit disease identification can be seen as an instance of image categorization. Most of the researches in the field of fruit recognition or fruit disease detection have considered color and texture properties for the categorization. Most of the works for fruit recognition are done on the fruits located on trees but we restrict our self to the classification of fruits and vegetables amongst the several kind of fruits and vegetables. Most of the work for the fruit disease detection using images done in the literature is restricted to the detection of single type of disease only. In the next section we discuss several approaches used by researchers with the aim of being aware to the latest research carried out, which are related to the formulated problems in this thesis.
2.2
Previous Work
2.2.1 Fruit and Vegetable Recognition Recently, a lot of activity in the area of Image Categorization has been done. In respect of produce fruit and vegetable classification problem, Veggie-vision [2] was the first attempt of a fruit and vegetable recognition system. The system uses texture, color and density (thus requiring some extra information from the system). This system does not take some advantage of recent developments, because it was created some time ago. The reported accuracy was around 95% in some scenarios but it uses
Chapter 2
Literature Review
the top four responses to achieve such result. Our data set is more demanding in some respects; while the data set of Veggie-vision had some extra classes, the hardware that captures the images gave a suppressed specular lights and more uniform color. The data set gathered in the super market has more illumination differences and much color variation among different images, and there is no measure for the suppression of specularities. In [3], the author has shown that the sum and difference of two random variables with same variances are de-correlated and define the principal axes of their associated joint probability function. Hence, the author introduces sum and difference histograms as an alternative to the usual co-occurrence matrices for image texture description. An approach to compare images based on color coherence vectors is presented by Pass et al. [4]. They have defined color coherence as the degree to which image pixels of that color are members of a large region with homogeneous color. They refer to these significant regions as coherent regions. Coherent pixels are part of some sizable contiguous region, and incoherent pixels are not. In order to compute the CCVs, the method blurs and discretizes the image’s color-space to eliminate small variations between neighboring pixels. Then, it finds the connected components in the image in order to classify the pixels of the image within a given color type as either coherent or incoherent. After classifying the image pixels, CCV computes two color histograms: one for coherent pixels and another for incoherent pixels. The two histograms are stored as a single histogram. The border/interior pixel classification (BIC), a compact approach to describe images is presented in [5]. BIC relies on the RGB color-space uniformly quantized in 4 × 4 × 4 = 64 colors. After the quantization, the image pixels are classified as border or interior. A pixel is classified as interior if its 4-neighbors (top, bottom, left, and right) have the same quantized color. Otherwise, it is classified as border. After the image pixels are classified, two color histograms are computed: one for border pixels and another for interior pixels. In general, the produce fruit and vegetable classification problem can be seen as an instance of object’s categorization. In [6] the author employed Principal Component Analysis (PCA) and obtained the reconstruction error of projecting the
Dept. of CEA, GLAU, Mathura
8
Chapter 2
Literature Review
whole image to a subspace then returning to the original image space. However, it depends heavily on pose, shape and illumination. A new image descriptor for broad Image Categorization, the Progressive Randomization (PR) [7] is introduced in the literature by Rocha and Goldenstein that uses perturbations on the values of the Least Significant Bits (LSB) of images. The authors have shown that different classes of images have a distinct behavior under their methodology. They have introduced a methodology that captures the changing dynamics of the artifacts inserted between a perturbations process in each of the broad-image classes. The most important features in the PR descriptor are its low dimensionality and its unified approach for different applications (e.g., the class of an image, the class of an object in a restricted domain) even with different cameras and illumination. With few training examples, PR still has good separability, and its accuracy increases with the size of the training set. Computer graphics rendering software is capable of generating highly photorealistic images that can be impossible to differentiate from photographic images. As a result, the unique stature of photographs as a definitive recording of events is being diminished (the ease with which digital images can be manipulated is, of course, also contributing to this demise). To overcome this problem, a method for differentiating between photorealistic and photographic images is described in [8]. In this method, it is shown that based on first-order and higher order wavelet statistics a statistical model reveals subtle but significant differences between photorealistic and photographic images. The drawback of this, from a rendering point of view, is that these models do not necessarily give any insight into how one might render more photorealistic images. A unified approach [9] is presented in Rocha et al. that can combine many features and classifiers. The author approaches the multi-class classification problem as a set of binary classification problem in such a way that one can assemble together diverse features and classifier approaches custom-tailored to parts of the problem. They define a class binarization as a mapping of a multi-class problem onto two-class problems (divide-and-conquer) and referred binary classifier as a base learner. For Nclass problem N ( N 1) / 2 binary classifiers will be needed where N is the number of different classes. According to the author, the ijth binary classifier uses the patterns
Dept. of CEA, GLAU, Mathura
9
Chapter 2
Literature Review
of class i as positive and the patterns of class j as negative. They calculate the minimum distance of the generated vector (binary outcomes) to the binary pattern (ID) representing each class, in order to find the final outcome. They have categorized the test case into that class for which the distance between ID of that class and binary outcomes was be minimum. Photographs differ from paintings in their edge, color, and texture properties. In [10] the problem of automatically differentiating photographs of paintings from photographs of real scenes is addressed. The authors used color, edge, and texture properties to demonstrate the problem. Using single features they have achieved 70– 80% correct discrimination performance, whereas they have achieved more than 90% correct discrimination results by a classifier using multiple features. But they did not observe any differences in the edge structure of paintings and photographs in our images. Low- and middle-level features are used to distinguish broad classes of images in [11]. The author has shown that a low complexity, low dimensional feature set can, in fact, be used to achieve a high indoor/outdoor classification rate when applied in a two stage SVM classification scheme. The combination of low-level and semantic features increases the classification accuracy. In addition, an approach to establish image categories automatically using histograms, shape and colors descriptors with an unsupervised learning method is presented in [12]. Recently, Agarwal et al. [13] and Jurie and Triggs [14] adopted approaches that take categorization problem as the recognition of specific parts that are characteristic of each object class. Marszalek and Schmid [15] have extended the category classification with bag-of-features, which represents an image as an order less distribution of features. They have given a method to exploit spatial relations between the features by utilizing object boundaries when supervised training is in progress. They increase the weights of features that agree on the shape and position of the objects and suppress the weights of features that used in the background. Sets of local features which are invariant to image transformations are used effectively when comparing images. Grauman and Darrel [16] have presented an approach for efficiently comparing images without clustering descriptors, based on their discrete distributions of distinctive local invariant features. With an
Dept. of CEA, GLAU, Mathura
10
Chapter 2
Literature Review
approximation of Earth Mover’s Distance, similarity between images is measured, which quickly computes correspondences between two bags of features having minimal cost. Feature distribution of each image’s is mapped with a low-distortion embedding of EMD into a normed space. Most similar examples to a novel query image are retrieved via approximate nearest neighbor search in the number of examples in the embedded space in time sub-linear. The object categories presented in a set of unlabelled images are discovered in [17]. It is achieved in the statistical text literature: probabilistic Latent Semantic Analysis by using a model developed. This is used to discover topics using the bagof-words document representation in a corpus in the text analysis. Here object categories are equivalent to the topics, so that an image having instances of several categories is constructed as a mixture of topics. By using a visual analogue of a word, the model is applied to images, formed by vector quantizing SIFT-like region descriptors. Such techniques, called as bag-of-features showed better results even though they do not try to model spatial constraints among features. Another interesting technique was proposed in [18]. The author has calculated feature points from the gradient image. By a joining path, the points are connected and a match is finalized if contour found is similar enough to the one presents in the database. A very serious drawback of such method for produce fruit and vegetable classification is that such method requires a nonlinear optimization step for the finding of best contour; still it relies too heavily on the silhouette cues, which are not very informative cues for fruits like lemons, melons and oranges. Using a generative constellation model, Weber [19] has taken spatial constraints into account. The algorithm can work with occlusion in a very good manner, but very costly (exponential with the number of parts). A further work made in [20], introduced pre knowledge for the estimation of the distribution, so it reducing the number of examples used for the training to around 10 images while having a good recognition rate. The problem of exponential growth in the number of parts persists even with this improvement that makes it unpractical for the classification problem presented in this paper.
Dept. of CEA, GLAU, Mathura
11
Chapter 2
Literature Review
2.2.2 Fruit Disease Detection Major works performing defect segmentation of fruits are done using simple threshold approach [21], [22]. A globally adaptive threshold method (modified version of Otsu’s approach) to segment fecal contamination defects on apples are presented in [23]. Classification-based methods attempt to partition pixels into different classes using different classification methods. Bayesian classification is the most used method by researchers Kleynen et al. [24] and Leemans et al. [25], where pixels are compared with a pre-calculated model and classified as defected or healthy. Unsupervised classification does not benefit any guidance in the learning process due to lack of target values. This type of approach was used by Leemans et al. [26] for defect segmentation. In [27], Ojala et al. used the Local Binary Pattern histogram for rotation invariant texture classification. Local Binary Pattern is a simple yet very efficient operator to define local image pattern, and it has reported impressive classification outcomes on representative texture databases. Local Binary Pattern has also been adapted by other applications, such as face recognition [28] dynamic texture recognition [29] and shape localization [30]. A Complete Local Binary Pattern is presented by Z Guo [31] as the completed modeling of Local Binary Pattern. Recent developments in agricultural technology have lead to a demand for a new era of automated non-destructive methods of plant disease detection. It is desirable that the plant disease detection tool should be rapid, specific to a particular disease, and sensitive for detection at the early onset of the symptoms [32]. The spectroscopic and imaging techniques are unique disease monitoring approaches that have been used to detect diseases and stress due to various factors, in plants and trees. Current research activities are towards the development of such technologies to create a practical tool for a large-scale real-time disease monitoring under field conditions. Various spectroscopic and imaging techniques have been studied for the detection of symptomatic and asymptomatic plant diseases. Some of the methods are: fluorescence imaging used by Bravo et al. [33]; Moshou et al. [34]; Chaerle et al. [35], multispectral or hyperspectral imaging used by Moshou et al. [36]; Shafri and Hamdan [37]; Qin et al. [38], infrared spectroscopy used by Spinelli et al. [39]; Purcell et al. [40], fluorescence spectroscopy used by Marcassa et al. [41]; Belasque et
Dept. of CEA, GLAU, Mathura
12
Chapter 2
Literature Review
al. [42]; Lins et al. [43], visible/multiband spectroscopy used by Yang et al. [44]; Delalieux et al. [45]; Chen et al. [46], and nuclear magnetic resonance (NMR) spectroscopy used by Choi et al. [47]. Hahn [48] reviewed multiple methods (sensors and algorithms) for pathogen detection, with special emphasis on postharvest diseases. Several techniques for detecting plant diseases is reviewed in [49] such as, Molecular techniques, Spectroscopic techniques (Fluorescence spectroscopy and Visible and infrared spectroscopy), and Imaging techniques (Fluorescence imaging and Hyper-spectral imaging). In [34], a ground-based real-time remote sensing system for detecting diseases in arable crops under field conditions and in an early stage of disease development is developed. The authors have used hyper-spectral reflection images of infected and healthy plants with an imaging spectrograph under ambient lighting conditions and field circumstances. They have also used multi-spectral fluorescence images simultaneously using UV-blue excitation on the same plants. They have shown that it was possible to detect presence of disease through the comparison of the 550 and 690 nm fluorescence images. In [35], the authors have aimed at the quantification and description of the kinetics of fluorescence (blue-green) by using fluorescence imaging compared to the visual development of diseases at regular intervals of tobacco mosaic virus infection in resistant tobacco. The authors have detected plant stress caused by disease infestation and to discriminate this type of stress from nutrient deficiency stress in field conditions using spectral reflectance information [36]. Yellow Rust infected winter wheat plants were compared to nutrient stressed and healthy plants in this paper. They have taken in-field hyper-spectral reflectance images with an imaging spectrograph and applied a normalization method based on reflectance and light intensity adjustments. For achieving high performance stress identification, they have introduced Self-Organising Maps (SOMs) and Quadratic Discriminant Analysis (QDA). Large scale plantation of oil palm trees requires on-time detection of diseases as the ganoderma basal stem rot disease was present in more than in Peninsular Malaysia 50% of the oil palm plantations. To deal with this problem, airborne hyperspectral imagery offers a better solution [37] in order to detect and map the oil palm trees that were affected by the disease on time. Airborne hyper-spectral can provide
Dept. of CEA, GLAU, Mathura
13
Chapter 2
Literature Review
data on user requirement and has the capability of acquiring data in narrow and contiguous spectral bands which makes it possible to discriminate between healthy and diseased plants better compared to multispectral imagery. Citrus canker is among the most devastating diseases that affect marketability of citrus crops. In [38], a hyper-spectral imaging approach is developed for detecting canker lesions on citrus fruit and hyper-spectral imaging system is developed for acquiring reflectance images from citrus samples in the spectral region from 450 to 930 nm. In [40], the authors have investigated the power of NIR spectroscopy as an alternative to rate clones of sugarcane leaf spectra from direct measurement and examined its potential using a calibration model to successfully predict resistance ratings based on a chemometrics approach such as partial least squares. To populate the nature of the NIR sample, they have undertaken a scanning electron microscopy study of the leaf substrate. Marcassa et al. [41] have applied laser-induced fluorescence spectroscopy to investigate biological processes in orange trees. They have investigated water stress and Citrus Canker, which is a disease produced by the Xanthomonas axonopodis pv. citri bacteria. They have detected the stress manifestation by the variation of fluorescence ratios of the leaves at different wavelengths. The fluorescence ratios present a significant variation, showing the possibility to observe water stress by fluorescence spectrum. They have discriminated the Citrus Canker’s contaminated leaves from the healthy leaves using a more complex analysis of the fluorescence spectra. However, they were unable to discriminate it from another disease. Belasque et al. [42] have investigated the detection of mechanical and disease stresses using laser-induced fluorescence spectroscopy in citrus plants. They have investigated a canker disease in the citrus, which is caused by a bacteria namely Xanthomonas axonopodis pv. citri. In the plant’s infection by such bacteria, mechanical stress plays an important role. They have used a laser-induced fluorescence spectroscopy system, composed of a spectrometer and an excitation laser to perform fluorescence spectroscopy. They have shoan that this ability to discriminate may be an important application to detect citrus canker infected trees. Lins et al. [43] have developed an optical technique to detect and diagnose citrus canker in citrus plants with a portable field spectrometer unit. They have used laser induced fluorescence spectroscopy (LIF) method to detect citrus canker. They
Dept. of CEA, GLAU, Mathura
14
Chapter 2
Literature Review
have populated that the length of time a leaf has been detached is an important variable. They have concluded that LIF has the potential to be applied to citrus plants. In [45], the authors have investigated the use of hyper-spectral methods caused by apple scab for early detection of plant stress to move towards more reduced and efficient application of fertilizers, pesticides or other crop management treatments for the apple orchards. Apple leaves of the susceptible cultivar, Braeburn and the resistant cultivar, Rewena, were artificially inoculated in a controlled greenhouse environment with conidia of V inaequalis. They have differentiated leaves infected with V inaequalis from non-infected leaves. They have investigated the developmental stage at which V inaequalis infection can be detected. They have also selected wavelengths that best differentiated infected leaves with non-infected leaves. They have used treebased modeling, partial least squares logistic discriminant analysis, and logistic regression to select the hyper-spectral bands that differentiate infected leaves with non-infected leaves and shown that the spectral domains between 2200-2500 nm and 1350-1750 nm are the most important parts for separating healthy from stressed leaves immediately after infection.
2.3
Summary
A detailed literature survey is presented in this chapter which gives an idea that what has been done till now and what is the scope of current research for the image categorization problems. We have discussed various approaches and several image descriptors used by several researchers in the image categorization problem. In the section 2.2.1 of this chapter, we have considered previous works related to the fruits and vegetable classification problem. Recent researches for the detection and defect segmentation of diseases of the fruits from the images using image processing technology are studied in the section 2.2.2 including several imaging techniques to automate the system.
Dept. of CEA, GLAU, Mathura
15
Chapter 3 Fruit and Vegetable Classification
3.1
Introduction
Contemporary Vision and Pattern Recognition problems such as face recognition, fingerprinting identification, image categorization, and DNA sequencing often have an arbitrarily large number of classes and properties to consider. Recognition system is a grand challenge for the computer vision to recognize fruits and vegetables with near human levels of recognition. The fruit and vegetable classification can be used in the super market for the automatic generation of prices for the fruits purchased by a customer. Fruit and vegetable classification can also be used in computer vision for the automatic sorting of fruits from a set, consisting of different kinds of fruit. Image categorization, in general, relies on combinations of structural, statistical and spectral approaches. Structural approaches describe the appearance of the object using well-known primitives, for example, patches of important parts of the object. Statistical approaches represent the objects using local and global descriptors such as mean, variance, and entropy. Finally, spectral approaches use some spectral space representation to describe the objects such as Fourier spectrum. [50]. In this chapter, we discuss a method which exploit statistical color and texture descriptors to categorize fruits and vegetables in a multi-class scenario. There are a number of challenges that must be addressed to perform automatic recognition of the different kinds of fruit or vegetable using the images from the camera. Many types of fruits are subjected to significant variation in color and texture, depending on how ripe they are. Color and texture are the fundamental characteristic of natural images, and plays an important role in visual perception.
Chapter 3
Fruit and Vegetable Classification
Instead of considering color and texture feature separately, this chapter introduces a texture feature derived from the color images. In this chapter, we discuss a framework for the automatic classification of fruits and vegetables which will take an image of fruit or vegetable as input and identify the kind of fruit or vegetable in the image. The input image contains fruit or vegetable of single variety in arbitrary position and in any number. Section 3.2 explores the framework for the fruits and vegetable recognition problem. K-means clustering based image segmentation approach is presented in section 3.2.1. An efficient texture feature for image categorization problems is proposed in section 3.2.2. Section 3.2.3 explains how a set of binary classifiers can be used as multiclass classifier for the training and classification. To validate the proposed approach, a supermarket data set is used in this experiment as discussed in the section 3.3.1. Section 3.3.2 presents the experimental results and discussion in various aspects. Finally, section 3.4 summarizes the chapter.
3.2
Proposed Framework
The proposed framework for the fruit recognition system, shown in Figure 3.1 operates in two phases, training and testing. Both require some preprocessing (i.e. image segmentation and feature extraction).
Training Fruit Images
Test Fruit Image
Image Segmentation
Image Segmentation
Feature Extraction
Feature Extraction
Training by Multiclass SVM
Classification by Multiclass SVM
Recognized Fruit
Figure 3.1: Fruit Recognition System
Dept. of CEA, GLAU, Mathura
17
Chapter 3
Fruit and Vegetable Classification
The proposed approach is composed of three steps, in the first step fruit images will be segmented into foreground and background. In the second step feature extraction process is carried out. We also propose a texture feature to achieve more accurate result for the fruits and vegetables classification. In the last step fruits and vegetables are classified into one of the classes using support vector machine with the trained system. 3.2.1 Image Segmentation Image segmentation is a convenient and effective method for detecting foreground objects in images with stationary background. Background subtraction is a commonly used class of techniques for segmenting objects of interest in a scene. This task has been widely studied in the literature. Background subtraction techniques can be seen as a two-object image segmentation and, often, need to cope with illumination variations and sensor capturing artifacts such as blur. Specular reflections, background clutter, shading and shadows in the images are major factors which must be addressed. Therefore, in order to reduce the scene complexity, it might be interesting to perform image segmentation focusing on the object‟s description only. For a real application in a supermarket, background subtraction needs to be fast, requiring only fractions of a second to carry out the operation. The best channel to perform the background subtraction is the S channel of HSV-stored images. This is intuitive from the fact that the S channel is much less sensitive to lighting variations than any of the RGB color channels.
(a)
(b) Figure 3.2: Extracting region of interest from the images (a) before segmentation, (b) after segmentation
Dept. of CEA, GLAU, Mathura
18
Chapter 3
Fruit and Vegetable Classification
We use a background subtraction method based on K-means clustering technique [9]. Amongst several image segmentation techniques, K-means based image segmentation shows a trade-off between efficient segmentation and cost of segmentation. Some examples of image segmentation are shown in figure 3.2. Algorithm for Image Segmentation using K-Mean: 1. I ←Down-sample the image using simple linear interpolation to 25% of its original size. 2. Extract the S channel of I and represent it as 1-d vector V of pixel intensity values. 3. Perform clustering D ←K-Means (V, k = 2). 4. M ← D back to image space by linear scanning of D. 5. UP ←Up-sample the generated binary M to the input image size. Close small holes on UP using the Closing morphological operator with a disk structuring element of radius 7 pixels. 3.2.2 Feature Extraction In this section, we are presenting a texture feature for the image categorization problems. Unser (1986) has defined sum and difference histogram of an image which are calculated from the sum and difference of two intensity values with a displacement of (d1, d 2) [3]. Unser has considered the displacement in x- and ydirections simultaneously, but by doing this he missed some information which is present in the x- and y-directions. In this section we are improving the Unser‟s descriptor by considering information present in x- and y-direction separately. To use the information present in both x- and y-directions, first we calculate the sum and difference in x-direction and then simulate this result in the y-direction. Simulation is carried by taking the sum and difference on outcome of x-direction. Proposed Improved Sum and Difference Histogram (ISADH) Texture Feature In this section, we propose an Improved Sum and Difference Histogram (ISADH) texture feature which is an improvement in the Unser‟s descriptor. We also analyze the accuracy of proposed ISADH texture feature and compare with other color and
Dept. of CEA, GLAU, Mathura
19
Chapter 3
Fruit and Vegetable Classification
texture features in the multi-class fruits and vegetables classification scenario in the result and discussion section of this chapter. ISADH Feature Algorithm (1) Find the sum S and difference D for the 1st channel of an image I with a displacement of (1, 0) as: S ( x, y) I ( x, y) I ( x 1, y)
(3.1)
D( x, y) I ( x, y) I ( x 1, y)
(3.2)
(2) Find the sum S1 and difference D1 of S with a displacement of (0, 1) as: S1( x, y) S ( x, y) S ( x, y 1)
(3.3)
D1( x, y) S ( x, y) S ( x, y 1)
(3.4)
(3) Find the sum S2 and difference D2 of D with a displacement of (0, 1) as: S 2( x, y) D( x, y) D( x, y 1)
(3.5)
D2( x, y) D( x, y) D( x, y 1)
(3.6)
(4) Find the histogram for the 1st channel by concatenating the histograms of S1, D1, S2, and D2. (5) Repeat step1 to step 4 for the 2nd and 3rd channel of the color image. (6) Concatenate the histograms of all three channels in order to find the ISADH texture feature of the input image I.
ISADH texture feature relies upon the intensity values of neighboring pixels. The histogram of two images of the same class may vary significantly. On the other hand, the ISADH feature has less difference for these images. If the difference in feature of two images is less, then images are more likely to belong to the same class. But if the difference is significant, then images are more likely to belong to the different class. This can be illustrated by an example of two 5×5 matrix having intensity values in the range of 0 and 7. Let Matrix „A‟ and Matrix „B‟ is as: Matrix: A 3 2 2 2 2
5 4 6 4 5
3 2 4 5 4
4 6 1 2 4
Dept. of CEA, GLAU, Mathura
Matrix: B 6 1 7 6 5
3 2 2 2 2
7 3 7 4 3
3 2 4 3 4
4 4 1 2 4
6 1 7 6 3
20
Chapter 3
Fruit and Vegetable Classification
Calculate three features (1) simple histogram, (2) Unser‟s feature, and (3) ISADH feature. The length of each feature calculated is 8-bin. Table 3.1 shows the simple histogram of Matrix „A‟ and Matrix „B‟, Table 3.2 shows the Unser‟s feature of Matrix „A‟ and Matrix „B‟, and Table 3.3 shows the proposed improved sum and difference histogram (i.e. ISADH feature) of Matrix „A‟ and Matrix „B‟.
Table 3.1: Simple histogram for both matrixes (8-bin) Simple Histogram
I0
I1 I2 I3 I4 I5 I6 I7
Matrix „A‟ Matrix „B‟
0 0
2 2
6 6
2 6
6 6
4 0
4 2
1 3
Table 3.2: Unser feature for both matrixes (8-bin) Unser‟s Feature Matrix „A‟ Matrix „B‟
I0
I1
I2 I3
0.5 2.5 7 0.5 5 5
I4
I5
I6
I7
2.5 0.5 3.5 6 2.5 2 0.5 3.5 5.5 3
Table 3.3: ISADH feature for both matrixes (8-bin) ISADH Feature Matrix „A‟ Matrix „B‟
I0
I1
I2
I3
I4
I5
I6
I7
0 6.25 2 4.25 0.5 5.75 1.25 5 1.25 5 1.75 4.5 1.25 5 1.5 4.75
In Table 3.1, 3.2, and 3.3, I0 to I7 represents the intensity levels (i.e. 0 to 7 for 8-bin). Let the difference between the feature of Matrix „A‟ and the feature of Matrix „B‟ are defined as the sum of square of difference of the values for each intensity level and can be calculated using equation 3.7. 7
Diff ( F A(i ) FB(i )) 2
(3.7)
i 0
Where, FA is feature of Matrix „A‟, FB is feature of Matrix „B‟, and Diff is the difference between FA and FB. Table 3.4, shows the values of difference for three features Simple Histogram, Unser‟s Feature, and ISADH Feature for the matrix „A‟ and matrix „B‟. From the
Dept. of CEA, GLAU, Mathura
21
Chapter 3
Fruit and Vegetable Classification
Table 3.4, it is clear that ISADH Feature has the lowest value of difference for matrix „A‟ and matrix „B‟. The value of Diff will be minimal if „A‟ and „B‟ are more likely belongs to the same class and that is achieved in the case of ISADH feature. Table 3.4: Difference in feature of Matrix „A‟ and Matrix „B‟ Feature Simple histogram Unser‟s feature ISADH feature
Difference 40 11 4.5
3.2.3 Training and Classification Recently, a unified approach was presented in [9] that can combine many features and classifiers. The author approached the multi-class classification problem as a set of binary classification problem in such a way one can assemble together diverse features and classifier approaches custom-tailored to parts of the problem. They define a class binarization as a mapping of a multi-class problem onto two-class problems (divideand-conquer) and referred binary classifier as a base learner. For N-class problem N ( N 1) / 2 binary classifiers will be needed where N is the number of different
classes. According to the author, the ijth binary classifier uses the patterns of class i as positive and the patterns of class j as negative. They calculate the minimum distance of the generated vector (binary outcomes) to the binary pattern (ID) representing each class, in order to find the final outcome. Test case will belong to that class for which the distance between ID of that class and binary outcomes will be minimum. Table 3.5: Unique ID of each class xy
xz
yz
x
+1
+1
0
y
-1
0
+1
z
0
-1
-1
Their approach can be understood by a simple three class problem. Let three classes are x, y, and z. Three binary classifiers consisting of two classes each (i.e., xy,
Dept. of CEA, GLAU, Mathura
22
Chapter 3
Fruit and Vegetable Classification
xz, and yz) will be used as base learners, and each binary classifier will be trained with training images. Each class will receive a unique ID as shown in Table 3.5. To populate the table is straightforward. First, we perform the binary comparison xy and tag the class x with the outcome +1, the class y with −1 and set the remaining entries in that column to 0. Thereafter, we repeat the procedure comparing xz, tag the class x with +1, the class z with −1, and the remaining entries in that column with 0. In the last, we repeat this procedure for binary classifier yz, and tag the class y with +1, the class z with -1, and set the remaining entries with 0 in that column, where the entry 0 means a “Don‟t care” value. Finally, each row represents unique ID of that class (e.g., y [1, +1, 0]). Each binary classifier results a binary response for any input example. Let‟s say if the outcomes for the binary classifier xy, xz, and yz are +1, -1, and +1 respectively then the input example will belongs to that class which have the minimum distance from the vector [+1, -1, +1]. So the final answer will be given by the minimum distance of
min dist 1, 1, 1 , 1, 1,0 , 1,0, 1 , 0, 1, 1
We have used Multi-class Support Vector Machine (MSVM) as a set of binary Support Vector Machines (SVMs) for the training and classification.
3.3
Results and Discussion
In this section, we describe the data set of fruits and vegetables, evaluate the proposed approach over the 15 types of fruits and vegetables and discuss various issues regarding the performance and efficiency of the system. In the section 3.3.1, we describe the data set used in this experiment and highlight several difficulties present in the data set. In the section 3.3.2, the performance of proposed ISADH texture is presented and compared with other color and texture feature. In order to show the efficiency of the proposed texture feature, we have compared it with four state-of-theart features. We also consider and compare the performance of the system in two color-spaces (i.e. RGB and HSV color-space).
Dept. of CEA, GLAU, Mathura
23
Chapter 3
Fruit and Vegetable Classification
3.3.1 Data set To demonstrate the performance of the proposed approach, we have used a supermarket data set [51] of fruits and vegetables, which comprises 15 different categories: Plum(264), Agata Potato(201), Asterix Potato(181), Cashew(210), Onion(75), Orange(103), Taiti Lime(104), Kiwi(157), Fuji Apple(212), Granny-smith Apple(155), Watermelon(192), Honeydew Melon(145), Nectarine(247), Spanish Pear(158), and Diamond Peach(211): totaling 2615 images. Figure 3.3 depicts the classes of the data set.
Agata Potato
Asterix Potato
GrannySmith Apple Honneydew Melon
Orange
Plum
Fuji Apple
Cashew
Nectarine
Kiwi
Taiti Lime
Spanish Pear
Watermelon
Onion
Diamond Peach
Figure 3.3: Data Set Used Figure 3.4 shows an example of Kiwi category with different lighting. Figure 3.5 shows examples of the Cashew category with differences in pose. Figure 3.6 shows the variability in the number of elements for the Orange category. Figure 3.7
Dept. of CEA, GLAU, Mathura
24
Chapter 3
Fruit and Vegetable Classification
shows the examples of cropping and partial occlusion. Presence of these features makes the data set more realistic.
Figure 3.4: Illumination differences, Kiwi category
Figure 3.5: Pose differences, Cashew category
Figure 3.6: Variability on the no. of elements, Orange category
Figure 3.7: Examples of cropping and partial occlusion 3.3.2 Result Discussion To evaluate the accuracy of the proposed approach quantitatively, we have compared our results with the results of Global Color Histograms (GCH) [50], Color Coherence Vectors (CCV) [4], Border/Interior Classification (BIC) [5] and Unser‟s descriptor [3]. A GCH is a set of ordered values, one for each distinct color, representing the probability of a pixel being of that color. Uniform quantization and normalization are used to reduce the number of distinct colors and to avoid scaling bias [50]. In this experiment, we use a 64-d GCH feature vector.
Dept. of CEA, GLAU, Mathura
25
Chapter 3
Fruit and Vegetable Classification
In order to compute the CCVs, the method finds the connected components in the image aiming to classify the pixels within a given color bucket as either coherent or incoherent. After classifying the image pixels, CCV computes two color histograms: one for coherent pixels and another for incoherent pixels. The two histograms are stored as a single histogram. In this work, we use a 64-d CCV feature vector. In order to compute the BIC, the method classifies image pixels as border or interior. A pixel is classified as interior if its 4-neighbors (top, bottom, left, and right) have the same quantized color. Otherwise, it is classified as border. After the image pixels are classified, two color histograms are computed: one for border pixels and another for interior pixels. The two histograms are stored as a single histogram with 128- bins. Average error(%)
25
GCH_RGB CCV_RGB BIC_RGB UNSER_RGB ISADH_RGB
20 15 10 5 0 20
25
30
35
40
45
50
55
60
Training examples per class
Figure 3.8: Comparison of GCH, CCV, BIC, Unser, and proposed ISADH features using SVM as a base learner in RGB color space Average error(%)
25 GCH_HSV CCV_HSV BIC_HSV UNSER_HSV ISADH_HSV
20 15 10 5 0 20
25
30
35
40
45
50
55
60
Training examples per class
Figure 3.9: Comparison of GCH, CCV, BIC, Unser, and proposed ISADH features using SVM as a base learner in HSV color space In order to extract the Unser feature, first the method finds the sum and difference of intensity values over a displacement of (d1, d2) of an image, then it
Dept. of CEA, GLAU, Mathura
26
Chapter 3
Fruit and Vegetable Classification
calculates two histograms sum and difference histogram and stores both histograms as a single histogram. We use a 64-bin Unser‟s feature in this experiment. In the experiment, we have used different number of images per class for the training. The average error is computed by calculating the sum of average error of each class divided by total number of classes. Figure 3.8 and 3.9 shows the average error for the Fruits and Vegetables classification for different features. The x-axis represents the number of images per class for the training and y-axis represents the
Average error(%)
average error. 25
GCH_RGB GCH_HSV
20 15 10 5 0 20
25
30 35 40 45 50 Training exmples per class
55
60
25 20
CCV_RGB CCV_HSV
15 10 5 0 20
25
30 35 40 45 50 Training examples per class
55
Average error(%)
Average error(%)
(a)
60
25 20
BIC_RGB BIC_HSV
15 10 5 0 20
25
20
UNSER_RGB UNSER_HSV
15 10 5 0 20
25
55
60
(c)
30 35 40 45 50 Training examples per class
55
60
Average error(%)
Average Error(%)
(b)
30 35 40 45 50 Training examples per class
40
ISADH_RGB ISADH_HSV
30 20 10 0 20
(d)
25
30 35 40 45 50 Training examples per class
55
60
(e)
Figure 3.10: Comparison of results in RGB and HSV color space using (a) GCH, (b) CCV, (c) BIC, (d) Unser, and (e) ISADH feature We have calculated the average error for each feature in RGB and HSV color space. The average error for each feature in RGB color space is shown in Figure 3.8. The result illustrates that Global Color Histogram (GCH) has the highest average
Dept. of CEA, GLAU, Mathura
27
Chapter 3
Fruit and Vegetable Classification
error because, it has only the color information and it does not consider the relation among the neighboring pixels. The average error for the Color Coherence Vector (CCV) is less than the average error for the GCH feature because CCV feature exploits the concept of coherent and incoherent regions. Border/Interior Classification (BIC) feature has low average classification error than the CCV feature because BIC feature takes the values of 4-neighboring pixel into account.
Table 3.6: Accuracy (%) in HSV color space when trained with 40 images per class S. No.
Fruit or Vegetable
No. of Test Images
Accuracy (%) in HSV color space when system is trained with 40 images per class GCH
CCV
BIC
UNSER
ISADH
1.
Agata Potato
161
100
100
99.38
100
100
2.
Asterix Potato
141
100
100
100
100
100
3.
Cashew
170
100
100
100
100
100
4.
Diamond Peach
171
98.25
98.83
98.83
97.66
100
5.
Fuji Apple
172
45.93
45.93
47.67
70.35
95.93
6.
Granny Smith Apple
115
100
100
100
100
100
7.
Honneydew Melon
105
98.10
99.05
99.05
100
98.10
8.
Kiwi
117
97.44
98.29
96.58
98.29
98.29
9.
Nectarine
207
91.30
94.69
97.58
97.10
97.10
10.
Onion
35
94.29
100
100
100
100
11.
Orange
63
90.48
96.83
100
95.24
98.41
12.
Plum
224
97.77
98.66
99.11
99.11
99.11
13.
Spanish Pear
118
96.61
98.31
99.15
96.61
96.61
14.
Taiti Lime
64
100
98.44
100
100
100
15.
Watermelon
152
98.68
100
100
100
100
93.62
95.27
95.82
96.96
98.90
Average Accuracy (%)
Unser feature has the low average classification error than GCH, CCV, and BIC. Figure 3.8 and 3.9 illustrate that proposed improved sum and difference histogram (ISADH) outperform the other features because ISADH feature considers not only the sum and difference of neighboring pixel but also the neighboring information of sum and difference of neighboring pixel. Figure 3.9 depicts the average classification error for the features derived in HSV color space. GCH feature achieves the highest classification error in HSV color
Dept. of CEA, GLAU, Mathura
28
Chapter 3
Fruit and Vegetable Classification
space also. Both BIC and CCV features have low average classification error than GCH feature. Unser feature performs better in HSV color space also than GCH, CCV, and BIC features. ISADH feature also outperform and shows highest average accuracy among all features. We have also observed across the plots that HSV color space is better than the RGB color space. One possible explanation is that in HSV color space S channel is very less sensitive to the illumination differences. As shown in Figure 3.10 all the features perform better in HSV color space. Table 3.6 shows the accuracy (%) of fruits and vegetables classification problem when features are extracted from HSV color images and training is done with 40 training images per class. From this table it is clear that ISADH feature shows a better result in the case of Fuji Apple whereas other features fail to produce good result in the case of Fuji Apple.
3.4
Summary
A framework for the fruits and vegetables classification is described and an efficient texture feature from the sum and difference of intensity values of neighboring pixel on the colored images is developed and proposed in this chapter. The given framework operates in three phases, image segmentation, feature extraction and training and classification. In the proposed approach, we compute the ISADH feature from the sum and difference histogram for each channel of the color image and combine these to make a single histogram. The fusion of neighborhood information with the color information makes this feature more discriminative than any other color and texture feature individually. The proposed feature is validated for the fruits and vegetables classification and show fairly more accurate results compared to other features.
Dept. of CEA, GLAU, Mathura
29
Chapter 4 Automatic Detection and Classification of Fruit Diseases
4.1
Introduction
Diseases in fruit cause devastating problem in production and economy in agricultural industry worldwide. Till now experts identify the presence of the disease in the fruits manually, but it is expensive for a former to consult an expert due to their distant availability, so it is required to detect the symptoms of the fruit diseases automatically as early as they appear on the growing fruits. Apple fruit diseases can cause major losses in yield and quality appeared in harvesting. To know what control factors to take next year to avoid losses, it is crucial to recognize what is being observed. Some disease also infects other areas of the tree causing diseases of twigs, leaves, and branches. Some common diseases of apple fruits are apple scab, apple rot, and apple blotch [52]. Apple scabs are gray or brown corky spots. Apple rot infections produce slightly sunken, circular brown or black spots that may be covered by a red halo. Apple blotch is a fungal disease and appears on the surface of the fruit as dark, irregular or lobed edges. In this chapter, we propose and experimentally evaluate a framework for the detection and classification of fruit diseases. The proposed framework is composed of the following steps; in first step the fruit images are segmented using K-means clustering technique, in second step, some state of the art features are extracted from the segmented image, and finally, fruit diseases are classified using a Multi-class Support Vector Machine. We show the significance of using clustering technique for the disease segmentation and Multi-class Support Vector Machine as a classifier for
Chapter 4
Automatic Detection and Classification of Fruit Diseases
the automatic detection and classification of fruit diseases. In order to validate the proposed approach, we have considered three types of the diseases in apple; apple blotch, apple rot and apple scab. The experimental results show that the proposed approach can significantly achieve accurate detection and automatic classification of fruit diseases. This chapter is structured as section 4.2 presents the proposed framework and methodology which takes the fruit images as the input and produces the kind of disease present in the fruit as the output. In the section 4.2.1 an efficient K-means clustering based image segmentation approach is presented which detect the infected region of the fruit in the image. In the section 4.2.2 we discuss some state of the art features which are used in this experiment to evaluate the proposed method. Section 4.2.3 describes the supervised learning techniques in the literature and uses them in the multiclass scenario. Section 4.3.1 shows some examples of the data set of apple fruit diseases with three types of disease apple scab, apple rot and apple blotch. Section 4.3.2 presents the experimental results and discussion in various aspects for the produce fruit disease classification problem. Finally, section 3.4 summarizes the chapter.
4.2
Proposed Framework
The steps of the proposed approach are shown in the Figure 4.1. Defect segmentation, feature extraction, training and classification are the major task in this approach. For the fruit disease classification problem, precise image segmentation is required; otherwise the features of the non-infected region will dominate over the features of the infected region. In this approach K-means based image segmentation is used to detect the region of interest which is the infected part only. The proposed approach operates in two phases training and classification. Training is required to learn the system with the characteristics of each type of disease. First we extract the feature from segmented portion of all the images that are used for the training and store it in a feature database. Then we train the support vector machine with the features stored in the feature database. Finally any input image can be classified into one of the classes using feature derived from segmented part of the input image and trained support vector machine.
Dept. of CEA, GLAU, Mathura
31
Chapter 4
Automatic Detection and Classification of Fruit Diseases
Training Images
Feature Database
Defect Segmentation and Feature Extraction
Trained Multiclass SVM
Training
Defect Segmentation and Feature Extraction
Image for Testing
Classification Kind of Disease in the Fruit
Figure 4.1: Proposed Approach for the Fruit Disease Recognition 4.2.1 Defect Segmentation K-means clustering technique is used for the defect segmentation. In the chapter 3 we have used image segmentation based on the K-means clustering technique but there are some differences between the algorithm used in this and previous chapter. In previous chapter we have used only S-channel of the HSV image for segmentation because we were interested only in foreground and background of the image so only a single channel was sufficient with two clusters one for foreground and one for background. But in disease detection we have to identify only the infected region in the image so using only a single channel and two clusters are not sufficient, so here we use more than two clusters and consider more than one channel of the color
Dept. of CEA, GLAU, Mathura
32
Chapter 4
Automatic Detection and Classification of Fruit Diseases
images for the precise disease segmentation. In this experiment images are partitioned into four clusters in which one cluster contains the majority of the diseased parts of the image. Input Image
(a) objects in cluster 3
(d)
objects in cluster 1
(b) objects in cluster 4
(e)
objects in cluster 2
(c) image labeled by cluster index
(f)
Figure 4.2: K-Means clustering for an apple fruit that is infected with apple scab disease (a) The infected fruit image, (b) first cluster, (c) second cluster, (d) third cluster, and (e) fourth cluster, respectively, (f) single gray-scale image colored based on their cluster index.
(a)
(b) Figure 4.3: Image segmentation results (a) Images before segmentation, (b) Images after segmentation
Dept. of CEA, GLAU, Mathura
33
Chapter 4
Automatic Detection and Classification of Fruit Diseases
K-means clustering algorithm was developed by J. MacQueen [53] later by J. A. Hartigan and M. A. Wong [54]. The K-means clustering algorithm classifies the objects (pixels in our problem) into K number of classes based on a set of features. The classification is carried out by minimizing the sum of squares of distances between the data objects and the corresponding cluster. Algorithm for the K-Means Image Segmentation Step 1: Read input image. Step 2: Transform image from RGB to L*a*b* color space. Step 3: Classify colors using K-Means clustering in 'a*b*' space. Step 4: Label each pixel in the image from the results of K-Means. Step 5: Generate images that segment the image by color. Step 6: Select the segment containing disease. In this experiment, squared Euclidean distance is used for the K-means clustering. We have used L*a*b* color space because the color information in the L*a*b* color space is stored in only two channels (i.e. a* and b* components), and it causes reduced processing time for the image segmentation. In this experiment input images are partitioned into four segments. From the empirical observations it is found that using 3 or 4 cluster yields good segmentation results. Figure 4.2 demonstrates the output of K-Means clustering for an apple fruit infected with apple scab disease. Figure 4.3 also depicts some more image segmentation results using the K-mean clustering technique. 4.2.2 Feature Extraction In the proposed approach, we have used some state of the art color and texture features to validate the accuracy and efficiency. The features used for the apple fruit disease classification problem are Global Color Histogram, Color Coherence Vector, Local Binary Pattern, and Completed Local Binary Pattern. We have already discussed Global Color Histogram (GCH) and Color Coherence Vector (CCV) in the previous chapter in the section 3.3.2. In this section we discuss Local Binary Pattern and Completed Local Binary Pattern in brief.
Dept. of CEA, GLAU, Mathura
34
Chapter 4
Automatic Detection and Classification of Fruit Diseases
Local Binary Pattern (LBP) Given a pixel in the input image, LBP [27] is computed by comparing it with its neighbors: n1 1, x 0 LBPN ,R s(vn vc )2 n , s( x) n 0 0, x 0
(4.1)
Where, vc is the value of the central pixel, vn is the value of its neighbors, R is the radius of the neighborhood and N is the total number of neighbors. Suppose the coordinate
of
vc
is
(0,
0),
then
the
coordinates
of
vn
are
( R cos(2 n / N ), R sin(2 n / N )) . The values of neighbors that are not present in the
image grids may be estimated by interpolation. Let the size of image is I*J. After the LBP code of each pixel is computed, a histogram is created to represent the texture image: I
J
H (k ) f ( LBPN , R (i, j ), k ), k [0, K ], i 1 j 1
1, x y f ( x, y ) 0, otherwise
(4.2)
Where, K is the maximal LBP code value. In this experiment the value of ‘N’ and ‘R’ are set to ‘8’ and ‘1’ respectively to compute the LBP feature. Completed Local Binary Pattern (CLBP) LBP feature considers only signs of local differences (i.e. difference of each pixel with its neighbors) whereas CLBP feature [31] considers both signs (S) and magnitude (M) of local differences as well as original center gray level (C) value. CLBP feature is the combination of three features, namely CLBP_S, CLBP_M, and CLBP_C. CLBP_S is the same as the original LBP and used to code the sign information of local differences. CLBP_M is used to code the magnitude information of local differences: n 1 1, x c CLBPN , R t (mn , c)2 n , t ( x, c) n 0 0, x c
Dept. of CEA, GLAU, Mathura
(4.3)
35
Chapter 4
Automatic Detection and Classification of Fruit Diseases
Where, c is a threshold and set to the mean value of the input image in this experiment. CLBP_C is used to code the information of original center gray level value: 1, x c CLBPN , R t ( g c , c I ), t ( x, c) 0, x c
(4.4)
Where, threshold cI is set to the average gray level of the input image. In this experiment the value of ‘N’ and ‘R’ are set to ‘8’ and ‘1’ respectively to compute the CLBP feature. 4.2.3 Supervised Learning and Classification Supervised learning is a machine learning approach that aims to estimate a classification function f from a training data set. The trivial output of the function f is a label (class indicator) of the input object under analysis. The learning task is to predict the function outcome of any valid input object after having seen a sufficient number of training examples. In the literature, there are many different approaches for supervised learning such as Linear Discriminant Analysis (LDA), Support Vector Machines (SVMs), Classification Trees, Neural Networks (NNs), and Ensembles of Classifiers [55]. We use Multiclass Support Vector Machine (MSVM) for the training and classification. The procedure for the conversion of binary classifier to multiclass classifier is discussed in the section 3.2.3.
4.3
Results and Discussion
In this section we show data set of apple fruit diseases and present a detailed result of the produce fruit disease classification problem and discuss various issues regarding the performance and efficiency of the system. In the section 4.3.1, we discuss about the data set used in this work and difficulties that may appear in the processing. In the section 4.3.2, the performance of proposed framework is presented by using four different descriptors and a comparison is made for these features. We also consider and compare the performance of the system in two color-spaces (i.e. RGB and HSV color-space). This section also compares the result for different types of diseases
Dept. of CEA, GLAU, Mathura
36
Chapter 4
Automatic Detection and Classification of Fruit Diseases
present in the apple with the aim to find out those diseases which are very difficult to detect and classify from the images. 4.3.1 Data Set Preparation To demonstrate the performance of the proposed approach, we have used a data set of normal and diseased apple fruits, which comprises four different categories: Apple Blotch (104), Apple rot (107), Apple scab (100), and Normal Apple (80). The total number of apple fruit images (N) is 391. Figure 4.4 depicts the classes of the data set. Presence of a lot of variations in the type and color of apple makes the data set more realistic.
(a)
(b) ()
(c)
(d) Figure 4.4: Sample images from the data set of type (a) apple scab, (b) apple rot, (c) apple blotch, and (d) normal apple
Dept. of CEA, GLAU, Mathura
37
Chapter 4
Automatic Detection and Classification of Fruit Diseases
4.3.2 Result Discussion In the quest for finding the best categorization procedure and feature to produce classification, this chapter analyzes some color and texture based image descriptors derived from RGB and HSV stored images considering Multiclass Support Vector Machine (MSVM) as classifier. If we use M images per class for training then remaining N-4*M are used for testing. The accuracy of the proposed approach is defined as, Accuracy(%)
Total number of images correctly classified *100 Total number of images used for testing
100
Accuracy (%)
90 80 70
GCH_RGB CCV_RGB LBP_RGB CLBP_RGB
60 50
10
15
20 25 30 35 40 Training Examples per Class
45
50
(a) Using RGB color image 100
Accurcy (%)
90 80 70
GCH_HSV CCV_HSV LBP_HSV CLBP_HSV
60 50
10
15
20 25 30 35 40 Training examples per class
45
50
(b) Using HSV color image Figure 4.5: Accuracy (%) for the GCH, CCV, LBP, and CLBP features derived from RGB and HSV color images considering MSVM classifier. Figure 4.5 (a-b) shows the results for different features in the RGB and HSV color spaces respectively. The x-axis represents the number of images per class in the
Dept. of CEA, GLAU, Mathura
38
Chapter 4
Automatic Detection and Classification of Fruit Diseases
training set and the y-axis represents the accuracy for the test images. This experiment shows that GCH does not perform well and reported accuracy is lowest for it in both the color spaces. One possible explanation is that, GCH feature has only color information, it does not consider neighboring information. GCH uses simply frequency of each color, however CCV uses frequency of each color in coherent and incoherent regions separately and so it performs better than GCH in both color spaces.
100
Accuracy (%)
90 80 70 60 50
10
15
20 25 30 35 40 Training examples per class
GCH_RGB GCH_HSV CCV_RGB CCV_HSV 45 50
(a) GCH and CCV feature 100 Accuracy (%)
90 80 70
LBP_RGB LBP_HSV CLBP_RGB CLBP_HSV
60 50
10
15
20 25 30 35 40 Training examples per class
45
50
(b) LBP and CLBP feature Figure 4.6: Comparison of the accuracy achieved in RGB and HSV color space for the GCH, CCV, LBP, and CLBP features considering MSVM classifier. From the Figure 4.5 (a-b), it is clear that LBP and CLBP features yield better result than GCH and CCV features because both LBP and CLBP uses the neighboring information of each pixel in the image. Both LBP and CLBP are robust to illumination differences and they are more efficient in pattern matching because they use local differences which are computationally more efficient. In HSV color space
Dept. of CEA, GLAU, Mathura
39
Chapter 4
Automatic Detection and Classification of Fruit Diseases
with 50 training examples per class, the reported classification accuracy is 80.94% for GCH, 86.47% for CCV, 90.97% for LBP, and 93.14% for CLBP feature. The LBP feature uses only the sign information of the local differences, even then, LBP reasonably represents the image local features because sign component preserves the major information of local differences. The CLBP feature exhibits more accurate result than LBP feature because CLBP feature uses both sign and magnitude component of local differences with original center pixel value (i.e. CLBP considers additional discriminant information).
100
Accuracy (%)
90 80 70
Apple Blotch Apple Normal Apple Rot Apple Scab
60 50
10
15
20 25 30 35 40 Training examples per class
45
50
(a) LBP in RGB color space 100
Accuracy (%)
90 80 70 60 50
10
15
20 25 30 35 40 Training examples per class
Apple Blotch Apple Normal Apple Rot Apple Scab 45 50
(b) LBP in HSV color space Figure 4.7: Accuracy per class for the LBP features in RGB and HSV color spaces using MSVM as a classifier We also observe across the plots that each feature performs better in the HSV color space than the RGB color space as shown in the Figure 4.6 (a-b). For 45 training examples and CLBP feature, for instance, reported classification error is 88.74% in
Dept. of CEA, GLAU, Mathura
40
Chapter 4
Automatic Detection and Classification of Fruit Diseases
RGB and 92.65% in HSV. One important aspect when dealing with apple fruit disease classification is the accuracy per class. This information points out the classes that need more attention when solving the confusions. Figure 4.7 and 4.8 depicts the accuracy for each one of 4 classes using LBP and CLBP features in RGB and HSV color spaces. Clearly, Apple Blotch is one class that needs attention in both color spaces. It yields the lowest accuracy when compared to other classes in both color spaces. Figure 4.7 and 4.8 also shows that, the behavior of Apple Rot is nearly same in each scenario.
100
Accuracy (%)
90 80 Apple Blotch Apple Normal Apple Rot Apple Scab
70 60 50
10
15
20 25 30 35 40 Training examples per class
45
50
(a) CLBP in RGB color space 100
Accuracy (%)
90 80 70
Apple Blotch Apple Normal Apple Rot Apple Scab
60 50
10
15
20 25 30 35 40 Training examples per class
45
50
(b) CLBP in HSV color space Figure 4.8: Accuracy per class for the CLBP features in RGB and HSV color spaces using MSVM as a classifier Normal Apples are very easily distinguishable with diseased apples and a very good classification result is achieved for the Normal Apples in both color spaces as shown in Figure 4.7 and 4.8. For CLBP feature and HSV color space, for instance,
Dept. of CEA, GLAU, Mathura
41
Chapter 4
Automatic Detection and Classification of Fruit Diseases
reported classification accuracy are 89.88%, 90.71%, 96.66%, and 99.33% for the Apple Blotch, Apple Rot, Apple Scab, and Normal Apple respectively, resulting average accuracy 93.14% when training is done with 50 images per class.
4.4
Summary
An image processing based approach is proposed and evaluated in this chapter for fruit disease detection and classification problem. The proposed approach is composed of mainly three steps. In the first step defect segmentation is performed using K-means clustering technique. In the second step features are extracted. In the third step training and classification are performed on a Multiclass SVM. We have used three types of apple diseases namely: Apple Blotch, Apple Rot, and Apple Scab as a case study and evaluated our program. Our experimental results indicate that the proposed solution can significantly support automatic detection and classification of apple fruit diseases. Based on our experiments, we have found that normal apples are easily distinguishable with the diseased apples and CLBP feature shows more accurate result for the classification of apple fruit diseases and achieved more than 93% classification accuracy. Further work includes consideration of fusion of more than one feature to improve the output of the proposed method.
Dept. of CEA, GLAU, Mathura
42
Chapter 5 Conclusions and Future Directions
5.1
Summary and Contributions
In this thesis, we have addressed the problem of automatically recognizing fruit or vegetable from images and the problem of automatically detecting the type of diseases present in the fruit using images. Chapter 3 presented a framework for the automatic recognition of fruit or vegetable from the images. The proposed framework is composed of three main steps; background subtraction, feature extraction and training and classification. We have used K-means clustering based approach for the image segmentation with two clusters; one for the foreground (fruits region) and one for the background (non-fruit region). To achieve the higher classification accuracy, we have presented a texture feature in that chapter and compared the result with some state of art features. From the experiment it is shown that the proposed texture feature outperform the other color and texture features. We have used Multiclass SVM as the base learner for this experiment. Chapter 4 proposed a framework for the automatic detection and classification of fruit diseases from the images. The final outcome of this research is heavily depend upon the precise image segmentation, because the infected part in the image are being segmented in this step. We have used K-means clustering with four clusters for the precise defect segmentation. We have analyzed several features to classify the type of diseases and found that Completed Local Binary Pattern performs very well and suited for this problem and achieves good classification accuracy. In this problem also, we have considered the Multiclass SVM for the learning and classification.
Chapter 5
Conclusions and Future Directions
In this experiment, we have used data set with a lot of variations. We have considered the images with illumination differences; pose differences; different number of fruits; arbitrary position; cropped fruits. Presence of these difficulties makes the problem more real-time and realistic.
5.2
Future Work
For both the produce classification problem in this thesis, we have used single learning machine and single feature at a time. The future work includes the consideration of more than one feature at a time and more than one classifier at a time. More than one feature can be fused to enhance the performance of the system. Ensembles of classifier may increase the accuracy of the produce classification problem. One of future work includes the consideration of the shape feature with the color and texture features to improve the result. For the fruit disease classification problem a more precise defect detection technique is required.
Dept. of CEA, GLAU, Mathura
44
References
[1] M. J. Roberts, D. Schimmelpfennig, E. Ashley, M. Livingston, M. Ash and U. Vasavada, “The Value of Plant Disease Early-Warning Systems,” Economic Research Service, no. 18, 2006, United States Department of Agriculture. [2] R.M. Bolle, J.H. Connell, N. Haas, R. Mohan and G. Taubin, “Veggievision: A Produce Recognition System,” Proc. 3rd IEEE Workshop on Applications of Computer Vision (WACV), Sarasota, USA, 1996, pp. 1-8. [3] M. Unser, “Sum and Difference Histograms for Texture Classification,” IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 8, no. 1, pp. 118-125, 1986. [4] G. Pass, R. Zabih and J. Miller, “Comparing Images Using Color Coherence Vectors,” ACM Multimedia (ACMMM), 1997, pp. 1-14. [5] R. Stehling, M. Nascimento and A. Falcao, “A Compact and Efficient Image Retrieval Approach Based on Border/Interior Pixel Classification,” Proc. ACM Conference on Information and Knowledge Management (CIKM), 2002, pp. 102-109. [6] M. Turk and A. Pentland, “Eigen Faces for Recognition,” Journal of Cognitive Neuroscience, vol. 3, no.1, pp. 71-86, 1991. [7] A. Rocha and S. Goldenstein, “PR: More than Meets the Eye,” Proc. Eleventh IEEE International Conference on Computer Vision (ICCV), 2007, pp. 1-8. [8] S. Lyu, and H. Farid, “How Realistic is Photorealistic,” IEEE Transactions on Signal Processing (TSP), vol. 53, no. 2, pp. 845-850, 2005.
References
[9] A. Rocha, C. Hauagge, J. Wainer and D. Siome, “Automatic Fruit and Vegetable Classification from Images,” Computers and Electronics in Agriculture, vol. 70, pp. 96-104, 2010. [10] F. Cutzu, R. Hammoud and A. Leykin, “Distinguishing Paintings from Photographs,” Computer Vision and Image Understanding (CVIU), vol. 100, no. 3, pp. 249-273, March 2005. [11] N. Serrano, A. Savakis and J. Luo, “A Computationally Efficient Approach to Indoor/Outdoor Scene Classification” Proc. 17th International Conference on Pattern Recognition (ICPR), 2004, pp. 146-149. [12] G.
Heidemann,
“Unsupervised
Image
Categorization,”
Proc.
16th
International Vacuum Congress (IVC), October 2004, vol. 23, no. 10, pp. 861-876. [13] S. Agarwal, A. Awan and D. Roth, “Learning to Detect Objects in Images Via a Sparse, Part-Based Representation,” IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 26, no. 11, pp. 1475-1490, November 2004. [14] F. Jurie and B. Triggs, “Creating Efficient Codebooks for Visual Recognition,” Proc. Tenth IEEE International Conference on Computer Vision (ICCV), 2005, vol. 1, pp. 604-610. [15] M. Marszalek and C. Schmid, “Spatial Weighting for Bag-of-Features,” Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2006, pp. 2118-2125. [16] K. Grauman and T. Darrel, “Efficient Image Matching with Distributions of Local Invariant Features,” Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2005, pp. 627-634. [17] J. Sivic, B. Russell, A. Efros, A. Zisserman and W. Freeman, “Discovering Objects and Their Location in Images,” Proc. Tenth IEEE International Conference on Computer Vision (ICCV), 2005, pp. 370-377.
Dept. of CEA, GLAU, Mathura
46
References
[18] A. Berg, T. Berg and J. Malik, “Shape Matching and Object Recognition using Low Distortion Correspondences,” Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2005, vol. 1, pp. 26-33. [19] M. Weber, “Unsupervised Learning of Models for Object Recognition,” PhD Thesis, Caltech, Pasadena, US, May 2000. [20] L. Fei-Fei, R. Fergus and P. Perona, “One-Shot Learning of Object Categories,” IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 28, no. 4, 594-611, April 2006. [21] Q. Li, M. Wang and W. Gu, “Computer Vision Based System for Apple Surface Defect Detection,” Computers and Electronics in Agriculture, vol. 36, pp. 215-223, Nov. 2002. [22] P. M. Mehl, K. Chao, M. Kim and Y. R. Chen, “Detection of Defects on Selected Apple Cultivars using Hyperspectral and Multispectral Image Analysis,” Applied Engineering in Agriculture, vol. 18, pp. 219-226, 2002. [23] M. S. Kim, A. M. Lefcourt, Y. R. Chen and Y. Tao, “Automated Detection of Fecal Contamination of Apples Based on Multispectral Fluorescence Image Fusion,” Journal of food engineering, vol. 71, pp. 85-91, 2005. [24] O. Kleynen, V. Leemans and M. F. Destain, “Development of a MultiSpectral Vision System for the Detection of Defects on Apples,” Journal of Food Engineering, vol. 69, pp. 41-49, 2005. [25] V. Leemans, H. Magein and M. F. Destain, “Defect Segmentation on „Jonagold‟ Apples using Color Vision and a Bayesian Classification Method,” Computers and Electronics in Agriculture, vol. 23, pp. 43-53, June 1999. [26] V. Leemans, H. Magein and M. F. Destain, “Defect Segmentation on „Golden Delicious‟ Apples by using Color Machine Vision,” Computers and Electronics in Agriculture, vol. 20, pp. 117-130, July 1998.
Dept. of CEA, GLAU, Mathura
47
References
[27] T. Ojala, M. Pietikäinen and T. T. Mäenpää, “Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Pattern,” IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 24, no. 7, pp. 971-987, 2002. [28] T. Ahonen, A. Hadid and M. Pietikäinen, “Face Recognition with Local Binary Patterns: Application to Face Recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 28, no. 12, pp. 2037-2041, 2006. [29] G. Zhao and M. Pietikäinen, “Dynamic Texture Recognition using Local Binary Patterns with an Application to Facial Expressions,” IEEE Transactions On Pattern Analysis and Machine Intelligence (TPAMI), vol. 27, no. 6, pp. 915-928, 2007. [30] X. Huang, S. Z. Li and Y. Wang, “Shape Localization Based on Statistical Method using Extended Local Binary Pattern,” Proc. International Conference on Image and Graphics (ICIG), 2004, pp.184-187. [31] Z. Guo, L. Zhang and D. Zhang, “A Completed Modeling of Local Binary Pattern Operator for Texture Classification,” IEEE Transactions on Image Processing (TIP), vol. 19, no. 6, pp. 1657-1663, 2010. [32] M. M. López, E. Bertolini, A. Olmos, P. Caruso, M. T. Gorris, P. Llop, R. Penyalver and M. Cambra, “Innovative Tools for Detection of Plant Pathogenic Viruses and Bacteria,” International Microbiology, vol. 6, no. 4, pp. 233-243, 2003. [33] C. Bravo, D. Moshou, R. Oberti, J. West, A. McCartney, L. Bodria and H. Ramon, “Foliar Disease Detection in the Field using Optical Sensor Fusion,” Agricultural Engineering International: the CIGR Journal of Scientific Research and Development, vol. 6, pp. 1-14, December 2004. [34] D. Moshou, C. Bravo, R. Oberti, J. West, L. Bodria, A. McCartney and H. Ramon, “Plant Disease Detection Based on Data Fusion of Hyper-Spectral
Dept. of CEA, GLAU, Mathura
48
References
and Multi-Spectral Fluorescence Imaging using Kohonen Maps,” Real-Time Imaging, vol. 11, no. 2, pp. 75-83, 2005. [35] L. Chaerle, S. Lenk, D. Hagenbeek, C. Buschmann and D. V. D. Straeten, “Multicolor Fluorescence Imaging for Early Detection of the Hypersensitive Reaction to Tobacco Mosaic Virus,” Journal of Plant Physiology, vol. 164, no. 3, 253-262, 2007. [36] D. Moshou, C. Bravo, S. Wahlen, J. West, A. McCartney and J. De Baerdemaeker, H. Ramon, “Simultaneous Identification of Plant Stresses and Diseases in Arable Crops using Proximal Optical Sensing and SelfOrganising Maps,” Precision Agriculture, vol. 7, no. 3, pp. 149-164, 2006. [37] H. Z. M. Shafri and N. Hamdan, “Hyperspectral Imagery for Mapping Disease Infection in Oil Palm Plantation using Vegetation Indices and Red Edge Techniques,” American Journal of Applied Sciences, vol. 6, no. 6, pp. 1031-1035, 2009. [38] J. Qin, F. Burks, M. A. Ritenour and W. G. Bonn, “Detection of Citrus Canker using Hyper-Spectral Reflectance Imaging with Spectral Information Divergence,” Journal of Food Engineering, vol. 93, no. 2, pp. 183-191, 2009. [39] F. Spinelli, M. Noferini and G. Costa, “Near Infrared Spectroscopy (NIRs): Perspective of Fire Blight Detection in Asymptomatic Plant Material,” Proc. 10th International Workshop on Fire Blight, Acta Horticulturae 704, 2006, pp. 87-90. [40] D. E. Purcell, M.G. O‟Shea, R. A. Johnson and S. Kokot, “Near-Infrared Spectroscopy for the Prediction of Disease Rating for Fiji Leaf Gall in Sugarcane Clones,” Applied Spectroscopy, vol. 63, no.4, pp. 450-457, 2009. [41] L. G. Marcassa, M. C. G. Gasparoto, J. Belasque Junior, E. C. Lins, F. Dias Nunes and V. S. Bagnato, “Fluorescence Spectroscopy Applied to Orange Trees,” Laser Physics, vol. 16, no. 5, pp. 884-888, 2006.
Dept. of CEA, GLAU, Mathura
49
References
[42] L. Belasque, M. C. G. Gasparoto and L. G. Marcassa, “Detection of Mechanical and Disease Stresses in Citrus Plants by Fluorescence Spectroscopy,” Applied Optics, vol. 7, no. 11, pp. 1922-1926, 2008. [43] E. C. Lins, J. Belasque Junior and L. G. Marcassa, “Detection of Citrus Canker in Citrus Plants using Laser Induced Fluorescence Spectroscopy,” Precision Agriculture, vol. 10, pp. 319-330, 2009. [44] C. M. Yang, C. H. Cheng and R. K. Chen, “Changes in Spectral Characteristics of Rice Canopy Infested with Brown Planthopper and Leaffolder,” Crop Science, vol. 47, pp. 329-335, 2007. [45] S. Delalieux, J. van Aardt, W. Keulemans, E. Schrevens and P. Coppin, “Detection of Biotic Stress (Venturia Inaequalis) in Apple Trees using Hyper-Spectral
Data:
Non-Parametric
Statistical
Approaches
and
Physiological Implications,” European Journal of Agronomy, vol. 27, no. 1, pp. 130-143, 2007. [46] B. Chen, K. Wang, S. Li, J. Wang, J. Bai, C. Xiao and J. Lai, “Spectrum Characteristics of Cotton Canopy Infected with Verticillium Wilt and Inversion of Severity Level,” Computer and Computing Technologies in Agriculture, vol. 2, pp. 1169-1180, 2008. [47] Y. H. Choi, E. C. Tapias, H. K. Kim, A. W. M. Lefeber, C. Erkelens, J. T. J. Verhoeven, J. Brzin, J. Zel and R. Verpoorte, “Metabolic Discrimination of Catharanthus Roseus Leaves Infected by Phytoplasma using 1H-NMR Spectroscopy and Multivariate Data Analysis,” Plant Physiology, vol. 135, pp. 2398-2410, 2004. [48] F. Hahn, “Actual Pathogen Detection: Sensors and Algorithms—A Review,” Algorithms, vol. 2, no. 1, pp. 301-338, 2009. [49] S. Sankarana, A. Mishraa, R. Ehsania and C. Davisb, “A Review of Advanced Techniques for Detecting Plant Diseases” Computers and Electronics in Agriculture, vol. 72, pp. 1-13, 2010.
Dept. of CEA, GLAU, Mathura
50
References
[50] R. Gonzalez and R. Woods, Digital Image Processing, 3rd edition, PrenticeHall, 2007. [51] http://www.liv.ic.unicamp.br/~undersun/pub/communications.html,
viewed
on December 2011. [52] J. Hartman, “Apple Fruit Diseases Appearing at Harvest,” Plant Pathology Fact Sheet, College of Agriculture, University of Kentucky, April 2010. http://www.ca.uky.edu/agcollege/plantpathology/ext_files/PPFShtml/PPFSFR-T-2.pdf, viewed on December 2011. [53] J. MacQueen, “Some Methods for Classification and Analysis of Multivariate Observations,” Proc. of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, 1967, vol. 1, pp. 281-297, University of California Press. [54] J. A. Hartigan and M. A. Wong, “Algorithm AS 136: A K-Means Clustering Algorithm,” Journal of the Royal Statistical Society, Series C (Applied Statistics), vol. 28, pp. 100-108, 1979. [55] C.M. Bishop, Pattern Recognition and Machine Learning, 1st edition, Springer, 2006.
Dept. of CEA, GLAU, Mathura
51
List of Publications based on the Research Work
[1] Shiv Ram Dubey and Anand Singh Jalal, “Robust Approach for Fruit and Vegetable Classification”, In the Proceedings of the International Conference on Modeling Optimization and Computing (ICMOC-2012), Tamilnadu, India, April 2012. (Accepted for Publication in Elsevier Procedia Engineering) [2] Shiv Ram Dubey and Anand Singh Jalal, “Detection and Classification of Apple Fruit Diseases using Complete Local Binary Patterns”, In the Proceedings of the 3rd IEEE International Conference on Computer and Communication Technology (ICCCT-2012), MNNIT Allahabad, India, November 2012. (Accepted for Publication)
Dept. of CEA, GLAU, Mathura
52
