A Study on SVM Based on the Weighted Elitist Teach-ing-Learning-Based Optimization and Application in the Fault Diagnosis of Chemical Process

] . This algorithm has become a hot research direction in recent years on account of its advantages such as high precision, rapid convergence and less parameters. The algorithm thought is to stimulate the process of students acquiring new knowledge from teachers as well as obtaining knowledge from interactions among students. Through continuous learning and performance improvement, objective functions of students achieve the global optimum as far as possible.


INTRODUCTION
With the ever-increasing expansion of the modern process industrial scale and the continuous complication of the control system, safety and stability during the process of production have become a major problem that must be taken into account by technicians.The fault diagnosis technology has been rapidly developed in recent decades.Many mature theoretical systems have been applied in practical production control systems.Support Vector Machine (SVM) is a classical pattern classification method proposed by Vapnik [1] , which solves traditional machine learning problems with the help of the optimization idea of mathematics [2] .SVM is widely used in fault diagnosis on account of its excellent processing capability of small sample problems [3][4][5] .Accuracy and performance of SVM depend on two parameters [6][7] , that is, the penalty parameter C and the kernel function parameter g.In order to find the optimal parameter accurately, the swarm intelligence optimization algorithm is normally adopted to carry out the parameter optimization during the training process of SVM.The accuracy of the swarm intelligence algorithm influences the performance of SVM to a large extent.
Teaching-Learning-Based Optimization [8] (TLBO) is a new swarm intelligence optimization algorithm.TLBO only has two parameters that need to be set compared with other swarm intelligence optimization algorithms, the number of cluster number and iterative algebra.This avoids problems like calculation amount increase and local optimum caused by improper parameter setting.However, the precision and stability of TLBO is still not high enough.A Weighted Elitist TLBO (WETLBO) is put forward in this paper, which adopts different inertia weight decreasing strategies in line with different features of elite and ordinary individuals during the process of iterative operation.Six unconstrained functions are selected in this paper to verify the performance of the improved algorithm and the algorithm is applied in the parameter optimization process of SVM.The optimized support vector classifier has obtained good classification results in several common fault diagnoses in the Tennessee Eastman chemical process.

TLBO ALGORITHM
TLBO is a new swarm intelligence optimization algorithm proposed by Rao in 2011 [9] .This algorithm has become a hot research direction in recent years on account of its advantages such as high precision, rapid convergence and less parameters.The algorithm thought is to stimulate the process of students acquiring new knowledge from teachers as well as obtaining knowledge from interactions among students.Through continuous learning and performance improvement, objective functions of students achieve the global optimum as far as possible.
Suppose the number of students in a class is NP (population size) and each student is required to learn D courses (design parameters), the learning result of each student is represented by the fitness value.TLBO is mainly composed of two phases, that is, Teacher Phase and Learner Phase.The teacher phase refers to the process that students improve themselves by learning from teachers with rich knowledge while the learner phase refers to mutual interactions and knowledge learning among students.The output of the teacher phase is the input of the learner phase.The solution with the best fitness value, that is, the global optimum, can be obtained by the repetition of the teaching process.

Teacher phase
As for the arbitrary i th iteration, M i is the current average and T i is the student with the best fitness value of NP students in the i th iteration, that is, the teacher of the current iteration.As for the j th course, (j=1,2,…,D), the average value of the j th course is defined as and M i should be determined by multiple subjects, which is shown in Formula (1).
The objective of the teacher phase is to make the average value of the whole class M i get close to T i as much as possible.Thus, fitness functions of other students can gradually get to the final global optimum through iterative operation.So, the formula of the teacher phase update can be obtained by taking T i as the expected objective mean value that is then substituted by M new .The Formula (2) is shown below: , , ( ) Where, , X new i and , X old i respectively stands for students' performances (values of fitness functions) after the teacher phase update and before the teacher phase; i r refers to the learning step, values of which are random numbers between 0 and 1; F T is the learning factor, which is mainly used to determine the varying degree of the average value.In order to simplify the calculation, the value of F T can be randomly selected as 1 or 2 according to Formula (3).
[1 (0,1){2 1}] F T round rand After being updated through the teacher phase in Formula (7), , X new i will be compared to the former value , X old i before the update.If the fitness value of , X new i makes a progress, the grade of each subject will be updated as well.Otherwise, original grades will be reserved.

Student phase
In TLBO, a population individual obtains knowledge not only from teachers but also from interaction among individuals.The process is the Learner Phase.After the update of students' states in the teacher phase, students will be randomly selected in the learner phase to carry out interactions.Each individual of the population is able to obtain new knowledge from students who have more knowledge.Select two students randomly in the learner phase, X p and X q , , p 1,2 ,NP ,NP and p q z .Compare the objective function values f(X) of the two students, which is also the fitness function value.There will be two situations: a. ( ) If the value of the objective function of student X p is smaller than that of student X q , student X p is superior to student X q and new X gets close to X p so as to complete the update of a student individual, which is shown in Formula (4): , , ( ) If the value of the objective function of student X p is larger than that of student X q , student X q is superior to student X p and X new gets close to X q so as to complete the update of a student individual, which is shown in Formula (5): , , ( ) Where, r i is the learning step, the value of which is a random number between 0 and 1.After the above two judgments, make a comparison between X new and X old .If the fitness function's value of the new X new is superior to X old , reserve X new , or X old will be reserved otherwise.
The optimal solution and the optimal value can be obtained ultimately by the repetition of the teacher phase and the learner phase through iterative operation.

WEIGHTED ELITIST TEACHING-LEARNING BASED OPTIMIZATION (WETLBO)
Elitist TLBO [10] introduces the elitist thought in TLBO so that the optimal solution of each generation can be reserved to the next generation.Inferior individuals in each generation will be replaced by elite students so as to accelerate the convergence speed of the algorithm.In order to avoid the fact that the algorithm is caught in the local optimal solution, elitist TLBO will conduct the mutation operation randomly on repetitive elitist individuals before the next iteration.Elitist TLBO not only retains advantages of TLBO but also accelerates the convergence speed of the algorithm by introducing the elitist solution.Parameters of the algorithm only involve the population quantity, the iteration time and the number of individuals.
In Elitist TLBO, the elitist solution normally refers to several population individuals with the largest fitness value of the current iterations.However, elite students and ordinary students need to be considered separately if the concept of inertia weight is introduced.From the angle of bionics, weights considered respectively can be regarded as different learning plans formulated in line with different learning abilities of elite students and ordinary students.As for elite students, it may need to find the local optimal solution Web of Conferences MATEC nearby.However, for ordinary students, larger weights are generally needed for further learning and the seeking of the optimal solution in the global range [11] .According to the classical method of part inertia weights definition [12][13] , the inertia weight of ETLBO changes with the increase of the iteration time in line with a nonlinear relation.The improved algorithm is called Weighted Elitist Teaching-Learning Based Optimization (WETLBO).Specific calculation steps of WETLBO are provided below.

Teacher phase of WETLBO
The initial value of each population individual should be set before the algorithm, which is similar to TLBO.And the population quantity, the iteration time and the number of individuals should also be determined.The formula of WETLBO that calculates and updates the fitness value of each student in the teacher phase is shown in Formula ( 6), which is transformed from Formula (2)., , ( ) Where, w is the inertia weight.As for the elitist solution, the best student or several best students of each generation will be reserved to the next generation.As the current default optimal solution, the elitist solution can adopt a relatively small weight so as to search the optimal solution nearby at the early stage of the algorithm.As for ordinary students, the weight is expected to be relatively large at the early stage of the algorithm so that it can be endowed with s strong global searching ability.But the weight decreases constantly with the algorithm, so the definition of w is shown in Formula (7).
As for ordinary students, the specified weight decreases in the index form, which is shown in Formula (8).Where, i is the current iteration time, max i is the total iteration times set before the start of the algorithm, and the preset values of max w and min w are 0.9 and 0.1.As for elite students, the weight remains relatively small during the whole process of the algorithm.Here, w e is 0.1.After being updated through the teacher phase in Formula (7), , X new i will be compared to the former value , X old i before the update.If the fitness value of , X new i makes a progress, the grade of each subject will be updated as well.Otherwise, original grades will be reserved.

WETLBO learner phase
The objective of enhancing knowledge level can be achieved through interactions between students in the learner phase.This is similar to the original TLBO.Select two students randomly in the learner phase, X p and X q , , p 1,2 ,NP ,NP and p q z .Compare the objective function values f(X) of the two students, which is also the fitness function value.There will be two possible situations: If the value of the objective function of student X p is smaller than that of student X q , student X p is superior to student X q and X new gets close to X p .Elite students and ordinary students should be considered respectively and processed in a similar way shown in Formula ( 9).The improved updating algorithm is provided in Formula (9).
If the value of the objective function of student X p is larger than that of student X q , student X q is superior to student X p and X new gets close to X q .Different situations of elite students and ordinary students should be considered respectively and processed in line with Formula (10)., Where, r i is the learning step, the value of which is a random number between 0 and 1. Definitions of w o and w e are consistent with those in the teacher phase.Make a final comparison between X new and X old after the above two judgments.If the fitness function of the new value X new is superior to X old , reserve X new , or X old will be reserved otherwise The pseudo-code of the learner phase is presented below: For p 1,2,..., NP , Randomly select two learners X p and X q , where p q z

If Accept X new if it gives a better function value
It is necessary to delete duplicates after the teacher phase and the learner phase.That is to say, if two neighboring students have the same scores in two subjects after the ranking of students' performance, one of the two is moved randomly so as to maintain the diversity of the algorithm population.A number of elite individuals with the best value of fitness function will be reserved to the next iteration so as to improve the operating efficiency of the algorithm.

Simulation and analysis of test functions
The performance of the improved WETLBO is verified through test functions and compared with several other kinds of common swarm intelligence algorithms, such as the differential evolution DE, the particle swarm optimization PSO, the original TLBO and the Elite TLBO.Six commonly used unconstrained test functions are involved in this paper, which search the optimal solution in the case of 30 and 100 dimensions respectively.In the PSO algorithm, c 1 = c 2 =1.49; the mutagenic factor of the DE algorithm is 0.5 and the crossover probability is 0.1.It is stipulated that iterations of these algorithms are 1000 and the population size is 50.Five optimization algorithms are adopted respectively in the case of 30 and 100 dimensions.Results obtained by six unconstrained test functions through different intelligence optimization algorithms are provided in Table 2.
WETLBO shows its perfect performance of global optimal search in test functions.It is able to identify "deceptions" of numerous local optimal solutions in  Web of Conferences MATEC test functions.Because there are only a small number of parameters in WETLBO need to be set, it is able to avoid the computational complexity increase caused by improper parameter selection as well as the fact that the global optimal solution is searched within the specified iterations caused by the impossibility of jumping out of the local optimum.Compared to other traditional algorithms, the WETLBO proposed in this paper not only maintains the population diversity of the operation but also provides meticulous search and improved capability of the global optimal search.It effectively reduces the possibility of being trapped in the local optimum.The operational efficiency and accuracy of the algorithm is guaranteed to some extent.

SVM BASED ON THE ALGORITHM OPTIMI-ZATION OF WETLBO
SVM is one of the classical methods of the pattern recognition.The basic idea is to map the original space to the high-dimensional feature space (Hilbert space) [14][15] , search the Optimal Hyperplane in the high-dimensional space, and select proper kernel functions [16] so as to accomplish the conversion from the inner product operation in the feature space to the nonlinear operation in the low-dimensional space.
In order to increase the accuracy of the classifier and search the optimal classification hyperplane, the sum of slack variables of the objective function is introduced with a penalty coefficient C. Therefore, according to the principle of structural risk minimization, SVM of nonlinear separable problems can be expressed with Formula (11).Formula (12) and Formula (13) are constraint conditions. ( The objective function can be converted with Lagrange operators: In Formula ( 14), i a is a positive Lagrange multiplier.Solve partial derivatives of w, b and i [ respec- tively.The minimization problem of L can be transformed into the maximization problem of a.
In Formula ( 15), ( ) is the kernel function that avoids dot products of solving high-dimensional feature vectors directly.RBF kernel function 2 1 ( , ) exp( ( ) ) ¦ is adopted in this paper for substitution.
When TLBO is used in the parameter optimization of SVM of RBF kernel function, each student can be expressed as Parameter C and parameter g stand for two subjects' performances of a student.In the process of the learning and training set of a support vector classifier, suppose the initial settings of WETLBO are defined--population size (the number of students NP), iterations i and the number of elites E (E<NP), specific procedures of optimizing parameters of SVM are as follows: (1) Initialize the population individuals and randomly set the initial values of two subjects C and g of NP students.
(2) Complete the fitness value evaluation and start the teacher phase: select the individual with the optimal value of the fitness function as the teacher T i and calculate the average value i M of all current individuals.Use Formula (6) and Formula (7) to complete the update of student individuals of the teacher phase.
(3) In the learner phase, select two students p X and q X randomly and make a comparison between the two.Use Formula (9) and Formula (10) to complete the update of student individuals of the learner phase.
(4) Remove duplicates after the teacher phase.E elite individuals with the best value of the fitness function will be reserved to the next iteration.
(5) Examine the end condition.If the condition is satisfied, end the parameter optimization process and output the optimal parameters C and g.Otherwise, it goes back to the step (2).provided in Table 2. 0 stands for the data of normal working conditions.In this part of the experiment, the data of 500 normal working conditions and five different kinds of fault data (each fault contains 480 data) are taken as the training set so as to obtain five different kinds of support vector classifiers.However, for the testing dataset, data of 800 normal working conditions and the collected data of all 800 faults are selected in the experiment of each kind of fault for test.The experimental results are shown in Table 3.It can be concluded that, compared to the SVM optimized by other four methods (grid search, particle swarm, traditional TLBO, elitist TLBO), the SVM optimized by WETLBO is improved to some extent in terms of fault diagnosis accuracy.This also indicates that WETLBO has certain validity and feasibility in solving parameter optimization problems of SVM.

Web of Conferences MATEC
Classification results of other common support vector classifiers with optimized parameters in dealing with multiple faults classification problems are provided in Table 4. Compared to Grid search-SVM, PSO-SVM and traditional TLBO-SVM, the classification accuracy of WETLBO-SVM for fault 4, fault 9 and fault 11 reaches 94%.

A
Study on SVM Based on the Weighted Elitist Teaching-Learning-Based Optimization and Application in the Fault Diagnosis of

ICETA 2015
In conclusion, the flow chart can be shown inFigure 1.

5 5 . 1
APPLICATION EXAMPLES OF SVM BASED ON WETLBO IN FAULT DIAGNOSIS According to the data of normal working conditions and the data of faulty working conditions, use SVM optimized by WETLBO to deal with single fault classification diagnosis problems and multiple faults classification diagnosis problems respectively.All tests are carried out in the following environment: 64-bit operating system of Windows 7; Intel Core2 Duo CPU 2GHz x2 processor; 4G RAM; Matlab R2010a version.Single fault classification diagnosis problems Failure data of 1, 3, 5, 10 and 20 Tennessee Eastman chemical industry processes (TE process) and normal working condition data are selected to conduct the first part of the experiment.Fault descriptions are ICETA 2015 05016-p.5

5. 2
Multiple faults classification diagnosis problemsIt can be found from the study on fault data of the TE process that fault 4, fault 9 and fault 11 have a relatively high similarity.The classification performance of each classifier can be verified through the comparison of the three fault classification results of SVMs optimized by different schemes.As for each fault, SVM is trained with 480 data (1440 fault data in total) and 800 data is taken as the test data for each kind of fault (2400 fault data in total).Firstly, optimize parameters of SVM with WETLBO proposed in this paper.The classified prediction results of the support vector classifier after training on three kinds of faults are presented in Figure2.The horizontal ordinate is the serial number of data and the vertical ordinate is mark number of fault type.Actual types are as follows: data 1-800 belongs to fault 4, data 801-1600 belongs to fault 9, and data 1601-2400 belongs to fault 11.The predicted classification results of the SVM optimized by WETLBO on 2400 test fault data are provided in Figure2.It can be seen that the classification accuracy of fault 4 and fault 9 reaches up to 98%.

Table 1 .
Comparison among different optimizing algorithm in test functions

Table 2 .
Instructions of some TE process faults

Table 4 .
Comparisons among Weighted elitist TLBO-SVM and some other SVM classifiers An improved weighted elitist TLBO is mainly put forward in this paper.The global search ability of the algorithm in different periods is improved by different inertia weight decreasing strategies.The improved WETLBO not only verifies the searching performance of the algorithm in the test function optimization process but also obtains ideal results with SVM optimized by WETLBO in the fault diagnosis classification of TE process.Experimental results indicate that WETLBO-SVM is a feasible and effective fault diagnosis method, which also provides a new idea for parameter estimation problems of nonlinear models.

Table 3 .
Fault classification of different methods in TE process Figure 2. Fault diagnosis result of multi-classification by weighted elitist TLBO ICETA 2015