Assessment the quality of apricots in the process of drying with neural networks and support vector machines

The paper presents an analysis of the assessment the quality of apricots during the drying process using two types of classifires: ANNs and SVMs. The quality of apricots is categorized in three classes according to the color and b-carotene content through the process of drying. The classification is made by using ‘CIE Lab’ color model and spectral characteristics in the VIS range. Neural networks are BPN and PNN, and classifiers are kernel and linear SVM. The spectral characteristics are pre-processed with SNV, MSC, First derivative and PCA. According to the results for color features, BPN and SVM with “rbf” kernel have the best performance while PNN has the worst performance. When using spectral characteristics the BPN network performs well: eavg = 4.1% and emax = 12.1% but the SVM linear (eavg = 3.4%, emax =5.3%) and SVM with “rbf” kernel (eavg = 2.4%, emax =5.2%) classifiers have better results. As a conclusion, it could be said that classifiers using spectral features perform well with errors at about 2-5%. Classification with color features is an alternative method, which is less complex, cheaper and with acceptable errors.


INTRODUCTION
It is well known that the quality is a critical issue facing the modern food industry because consumers always expect food products with excellent quality and reasonable price. In order to meet consumers' expectations to maintain food quality, the food manufacturers also require some suitable detection techniques and methods to ensure the quality of their products. However, the current traditional detection techniques and methods such as sensory evaluation or physical and chemical approaches are time-consuming, arduous and expensive. Therefore, rapid, nondestructive, and effective quality assessment techniques are needed. As an alternative to traditional methods for evaluation of food quality, noncontact optical methods based on analysis of color images, spectrophotometric and hyperspectral analysis are most perspective in the recent years. On the other hand, data mining is the process of automatic discovery of useful information in databases. Data mining techniques are useful for quality assessment tasks, particularly those that perform classification tasks [1][2][3][4][5][6][7][8]. These techniques use well-known methods of machine learning, pattern recognition, and statistics to find patterns that uncover hidden and useful information in data sets. Many successful applications used these techniques for food quality classification according to the characteristics of a product [9][10][11][12].
Some of the non-invasive techniques for predicting the food characteristics are done through computer vision and spectral analysis in a combination with artificial neural networks (ANN).
The application of thermal treatments in the food industry is one of the main stages to obtain a good quality product, whether applied in the pre-treatment or transformation stage. A feedback-type ANN was used to evaluate the concentration and temperature in a solution of fructooligosaccharides in parameters such as mass, moisture, volume, and solids in dehydrated yacon cubes [11]. In another study, a multilayer perceptron with a layer at the entrance with four neurons and an exit layer with two neurons was used to predict percentages of water loss and solids gain in the osmotic dehydration of aubergines [13]. Azadbakht et al. [14] found that for the determination of dehydration parameters of potato cubes, such as energy and exergy efficiency and the use of energy, the use of ANN provided better response approaches than the traditional mathematical models. On the other side, a study conducted by Khawas et al., [15] evaluated the influence of temperature, thickness and type of pretreatment, texture of the color and texture during vacuum dehydration of banana (MusaABB).
Amoriello et al. [16] study the quality of apricots. They demonstrated the possibility of sorting fresh apricots into uniform ripening classes using Vis and Vis-NIR spectrometers. Obtained results highlight how molecular spectroscopy coupled with multivariate analysis and statistical algorithms can identify and sort homogenous fruit batches with similar characteristics. Khojastehnazhand et al. [17] classified apricot fruits based on maturity levels and size by the use of image processing. The apricots were classified into three maturity stages (i.e. unripe, ripe, and overripe) and the volume was estimated. They said that the LDA and QDA could categorize the apricots with the accuracy of 0.904 and 0.923, respectively based on color features. Results showed that the algorithm can properly classify the fruits using the image properties of apricots. Sani et al. [18] show the potential of using FT-NIR spectroscopy for the rapid and non-destructive measurement of the moisture, water activity, firmness and SO2 content in sulphureddried apricots. They demonstrate that the major quality parameters of sulphured-dried apricots can be measured in a short time by FT-NIR spectroscopy without any need for the sample preparation. Camps et al. [19] studied the ability of portable Near Infrared Spectroscopy to determine apricot fruit quality. Their calibration models allow determination of different apricot's quality features with variable precisions. They claim that NIRS technology could be applicable to apricot quality evaluation.

Problem Formulation
Apricots are cultivated world-wide mainly for their high-quality fruit, which is consumed fresh, processed by the food industry, or preserved by drying. Fruit quality is a combination of physical and chemical characteristics accompanied by sensory properties (appearance, texture, taste and aroma), nutritional values, chemical compounds, mechanical properties, and functional properties. The comprehensive term "quality" comprises of a number of parameters of the drying material, in a mid-state or after the completion of the drying process. These parameters are important for characterization and prediction of the quality of dried product. The quality related properties could be grouped into the following: structural (density, poriosity), optical (color, appearance), textural (compession test, stress relaxation), sensorial (aroma, taste), thermal (glassy, cristaline), nutritional (vitamins, proteins). The most promising methods are the optical methods. The phisicochemical changes that occur during the drying seem to affect the quality properties of the dehydrated product. The color of the material changes during dehydration not only due to evaportion of the surface water but also due to certain reactions, such as enzymatic browning, non-enzymatic browning and caramelization reactions. These reactions may be undesirable for many products as well as apricots. Thus, the examination of color and some internal content like b-carotene in the process of apricot drying is important.

Problem Solution
The aim of this research is to classify apricots in the process of drying in three different categories (classes) according to the color change and content of b-carotene. The quality of the apricots is classified in three main color categories -apricots with typical color, apricots with half-dried light-brown color and full-brown color apricots. The corresponding classes for b-carotene content are as follow: 210mg/kg, 93.5mg/kg and 41.7mg/kg. To evaluate the quality of apricots in the process of drying, attributes such as color and spectal characteristics are used. The color and spectral characteristics are measured with Ocean Optics QE65000 spectrophotometer during the process of drying of the apricots. The results are processed using data mining techniques. The classification is done by using color features obtained from the 'CIE Lab' color model and spectral characteristics obtained in a VIS range (200 -1000nm).

Sample preparation and feature extraction
The apricot samples were obtained from local markets in the city of Ruse (Bulgaria). The species of apricots is "New Jersey". They were cut in two pieces and dried in the oven with a fixed temperature of T = 65о and drying air speed of v = 2m/s. The drying process of apricots is shown on Fig.1.
Color and spectral features are obtained by Ocean optics -"QE65000" spectrophotometer with its ability to measure color. The features (color or spectrum) are measured in three specific points of a drying process: before the drying process begins, with 55% moisture content and with 22% moisture content. These three batches of samples are used as groups (classes) in the process of drying. The following numbers of samples (points from the surface of apricots) are measured for the first, second and third class: 96, 61, and 128. The color and spectrum are measured form the apricot surface. In this study the three parameters: L, a, b, of 'CIE Lab' color model are used as an input parameters to the classifiers. The spectral characteristics are obtained in the same way of the drying process as color features, and some pre-processing methods are applied to spectral data before it can be used for model preparation. These methods are standard normal variate (SNV), multiplicative scatter correction (MSC) and first derivative (Savitzky-Golay). After that, PCA is applied to the data. The Principal Component Analysis (PCA) is applied to the spectral characteristics in order to obtain an exploratory data analysis. PCA is a frequently employed unsupervised pattern recognition technique based on the linear combination of optimally-weighted observed variables (principal components) enabling data dimensionality reduction, while retaining maximum variability of the data. In this study, the first three principal components are used.

Classification models
In this study, two types of models are structured. The first model is based on neural networks: BPN and PNN, and the second -on the support vector machines -SVM (radial basis unction) and SVM (linear). Training and testing of neural networks and Support Vector Machine (SVM) classifiers is done in a "Matlab™" environment.

Performance measures
A classification task maps (in other words, classifies) a data item into one of several predefined classes based on data variables. To perform that, data mining uses machine learning to train algorithms, and uses the training data to build a classification model that is capable of predicting the class of unknown samples. One way to measure the performance of the classification model is by accuracy, which is the overall number of (1) e i gives the relative part of objects from some class i, which are assigned incorrectly to other classes k=1..N, where FNi is the number of elements from the i-th class classified incorrectly to other classes, TPi is the number of correctly classified elements from the i-th class.
(2) g i gives the relative part of objects from other classes, which are assigned to class i, where FPi is the number of elements from other classes, assigned to the i-th class.
(3) e o (classification error rate) gives the relative part of all incorrectly classified objects where N is the number of classes.

Neural Networks
Multilayer Perceptron (MLP) is a classifier inspired by the way the human brain stores and processes knowledge. Neural networks can model real world complex relationships being able to predict unknown sample classes. Given a training set the neural network can learn the classification between the classes yi based on input data xi. The most commonly used neural network is a Multilayer Perceptron (MLP). The MLP can be used for linear and nonlinear modelling of inputoutput data. MLP trained with backpropagation algorithm is called BPN.
Other popular neural network is PNN. Probabilistic neural network (PNN) is closely related to Parzen window pdf estimator. A PNN consists of several subnetworks, each of which is a Parzen window pdf estimator for each of the classes. The input nodes are the set of measurements. The second layer consists of the Gaussian functions formed using the given set of data points as centres. The third layer performs an average operation of the outputs from the second layer for each class. The fourth layer performs a vote, selecting the largest value. The associated class label is then determined.

Support Vector Machines
Support Vector Machines (SVM) is a classifier which aims to find a hyperplane with maximum margin to separate the classes of data. A linear SVM is a classifier that searches the hyperplane with the widest margin. The goal is to maximize the margin of this hyperplane, maximizing the distance between the samples in its limit. In non-separable cases it is necessary to add slack variables and a kernel trick to separate data. The kernel trick is a commonly used technique to solve linearly inseparable problems. The kernel function performs a non-linear mapping into feature space. In this study, we used the Gaussian Radial Basis Function (RBF) kernel, defined as the following equation:

Classification results
In this study, cross-validation "k-fold" method will be used. The cross-validation gives the model an opportunity to test on multiple splits so we can get a better idea on how the model will perform on unseen data. In order to train and test our model using crossvalidation, we will use a cross-validation value of 5.

Quality assessment using ANN with color features
The quality assessment of apricots in three classes by color features is based on two types of artificial neural networks: BPN and PNN.
BPN neural network designed for color feature classification is presented on Fig.2 inputs corresponding to each of the color model components (L, a, b) shown on Fig.3.
The output layer defines three classes (three quality groups), with the corresponding grouping being encoded through 0-for not belonging to a class or 1-for belonging to a class. Because of that, the output layer has three neurons. Activation function is 'tangsig' and other parameters are as follows: epoches = 1000 and μ = 0.001.
The network training process includes a backpropagation algorithm. In the case of the BPN network, training is within 8 epochs. One of the best training, validation and testing is presented in Fig. 4. The average error over k-folds is 15.1% and the confusion matrix of the biggest error is given in Table 1.
After that, input data is presented to train PNN neural network. It does not need validation but evaluates its perormance by the test data. The average error over kfolds is 54.5% and the confusion matrix of the biggest error is given in Table 2.

Quality assessement using SVM with color features (Lab color model)
Two types of classifiers are choosen for the present research: SVM linear and SVM kernel with radial basis function. At first a capabilities of linear classifier is investigated. The average error over k-folds is 17.3% and the confusion matrix of the biggest error is given in Table 3. After that, kernel SVM with radial basis function is modeled. The average error over k-folds is 15.1% and the confusion matrix of the biggest error is given in Table 4.

Quality assessment using ANN with spectral features
The spectral characteristics are pre-processed with SNV, MSC and first derivative (Savitzky-Golay) in this sequence. Three principal components PC1, PC2 and PC3 are obtained by principal component analysis (PCA) Fig.8. These principal components are used as an input vector to develop a classifier model. The k-fold cross-validation is applied to the dataset with 5 disjoint subsets from all number of observations. The BPN neural network structured to model the classifier using principal components is shown in Fig.5. The input layer of the neural network has three inputs corresponding to each of the principal components.  Training, validation and testing of the BPN classifier for the best fold is given on the Fig.6. The average error over k-folds is 4.1% and the confusion matrix of the biggest error is given in Table 5.  The PNN neural network structured to model the classifier using principal components is shown in Fig.7. The input layer of the neural network has three inputs corresponding to each of the principal components. The three outputs correspond to the probability of the testing vector to belong to the output class. The results show that after training the PNN classifier, the average error over k-folds is 61.0% and the confusion matrix of the biggest error is given in Table 6.

Quality assessment using SVM with spectral features
As mentioned above two types of classifiers, SVM linear and SVM with radial basis function are applied to the dataset, which is compound of transformed spectral characteristics.

Fig. 8. Three principal components obtained from preprocessed spectral features
The three principal components (PC1, PC2 and PC3) obtained from pre-processed spectral features are shown on Fig.8. The three colors correspond to three classification groups. Training of the classifier is done with a multi-svm algorithm, which uses "one against the rest" method. The training and testing of linear SVM is done over the 5 k-fold tests. The average error over kfolds is 3.4% and the confusion matrix of the biggest error is given in Table 7. The average error over k-folds when training and testing SVM with radial basis function is 2.4% and the confusion matrix of the biggest error is given on Table 8.   The use of spectral characteristics is the second research initiative. The trend here is preserved. The PNN network could not classify correctly again, with following errors: e avg = 61.0% and maximum error e max = 62.1%. The interesting thing is that, BPN network performs very well with errors: e avg = 4.1% and maximum error e max = 12.1%. The superior from the classifiers is SVM with its two versions. SVM linear classifiers has the following errors: e avg = 3.4%, e max = 5.3% and SVM with rbf kernel: e avg = 2.4%, e max = 5.2%. As a conclusion, it could be said that the use of spectral characteristics for apricots quality assessment in a combination with SVM classifier is more appropriate than the use of color features. This is because the classification error is 7 times smaller. All classifiers using spectral features perform well except PNN. Errors are at about 2-5%. SVM classifiers and BPN neural network in a combination with spectral features are best in evaluating the quality of apricots during the drying process. Classification with color features is an alternative method, which is less complex, cheaper and for some tasks with acceptable errors.