Credit Risk Assessment Model Based Using Principal component Analysis And Artificial Neural Network

Credit risk assessment for bank customers has gained increasing attention in recent years. Several models for credit scoring have been proposed in the literature for this purpose. The accuracy of the model is crucial for any financial institution’s profitability. This paper provided a high accuracy credit scoring model that could be utilized with small and large datasets utilizing a principal component analysis (PCA) based breakdown to the significance of the attributes commonly used in the credit scoring models. The proposed credit scoring model applied PCA to acquire the main attributes of the credit scoring data then an ANN classifier to determine the credit worthiness of an individual applicant. The performance of the proposed model was compared to other models in terms of accuracy and training time. Results, based on German dataset showed that the proposed model is superior to others and computationally cheaper. Thus it can be a potential candidate for future credit scoring systems.


Introduction
Credit risk assessment has gained increasing attention in recent years.Banks and financial institutions have extensively started to consider the credit risk of their customers in order to make decisions to grant credit to new applicants, extend credit or increase credit limit for existing customers and under what terms.If the customer starts to fall behind in his repayments what actions should the financial institution take?Techniques that help with such decisions are called credit and behavioral scoring models [1].The information that is available in making a credit scoring decision includes both of the applicant's application form details and the information held by a credit reference agency on the applicant.Credit scoring models analyze the applicant's information and classify him into either applicant with good credit or applicant with bad credit.Applicant with good credit has great possibility to repay financial obligation, while, applicant with bad credit has high possibility of failing.The accuracy of credit scoring model is critical for financial institution's profitability.A one percent improvement on the accuracy of credit scoring of applicants may protect the financial institution from a great loss.Parametric statistical methods particularly Linear Discriminate Analysis (LDA) and Logistic Regression (LR) have been utilized in developing credit scoring models [2][3][4][5][6].The weakness of the LDA and LR is due to the assumption of linear relationship between response and predictor variables, which is usually nonlinear and the sensitivity to the deviation from the multivariate normality assumption.Recently, artificial intelligence (AI) and data mining approaches have been utilized in developing credit scoring models, and numerous studies revealed that AI based models outperform the classical statistical models.Neutral networks is one of the AI techniques that have been applied to credit scoring due to their ability to model both of linear and non-linear functions [8][9][10][11][12][13][14][15][16].However, ANNs are often criticized as being a ''black box'', computationally expensive and may suffer over fitting problem.Support vector machines SVMs, firstly developed by Cortes and Vapnik (1995) ).A layer of ''input'' units is connected to a layer of ''hidden'' units, which is connected to a layer of ''output'' units.The input nodes represent the raw information that is fed into the network.The input of each hidden node is determined by the outputs of the input units and the weights on the connections between the input and the hidden units.The output layer consists of nodes that represent the system's classification decisions.The outputs of the output nodes are determined based on the outputs of the hidden layer and the weights between the hidden and output layers.Figure 1 shows an example of three-layer neural network including input, output, and one hidden layer.The weight adjustments process of the ANN is known as training the ANN.The training process consists of running input values over the network with predefined classification output nodes.The predicted outputs for each training sample are calculated, and compared to the corresponding target outputs.The weights are then adjusted so that the error rate will be reduced next time when the training samples are presented to the network.Thus, the training algorithm finds out the properties of the ANN that are most relevant to learning the classification model.Testing samples are used to verify the performance of the trained network.

Principal component analysis (PCA)
Due to the multivariate nature of the samples, the computational time of designing and operating a classifier is challenging.Therefore, a PCA is applied to the dataset to find the best low dimension space that conveys maximum useful information, and to define the underlying attributes that influence the samples.only).If the attributes will be fed to PCA, normalization guarantees that PCs will be independent as PCA is sensitive to the relative scaling of the original attributes.Normalization is carried out by removing the feature's average and then dividing by its standard deviation using equation 1.
xƍ = xെmean standard deviation (1) Where x and xƍ are the old and new value of each feature in the data set respectively.
Afterwards, the data were divided into 65% for training the model, 15% for validation, and 20% for testing the performance of the trained model.

Performance Metrics
A number of performance metrics could be used to report the performance of the credit scoring classifiers including predictive accuracy, confusion matrix and AUC metrics.

Predictive accuracy
Predictive accuracy metric measures how accurately the model classifies credit applicants on the dataset and is defined by equation 2.

Predective accuracy =
TP +TN P+N Where, TP: true positive (the number of correct predictions that a customer is good); TN: true negative (the number of correct predictions that a customer is bad); P, N : are the number of positive and negative samples respectively.Predictive accuracy is not a reliable metric for the real performance of the classifiers specially, in case of imbalanced data sets (this is the case of credit scoring data).

Confusion matrix
Confusion matrix is one of the methods that provides more detailed analysis than the accuracy metric to the results of the classifiers.Confusion matrix is explained in figure 3. Where, FP: false positive (the number of bad customers who are predicted as good customers); and FN: false negative (the number of good customers who are predicted as bad customers)

ANN model
The neural network was designed of three layers, input, hidden, and output layers with 24 neurons in the input layer for the 24 input credit scoring attributes.Fifteen neurons in the hidden layer with

PCA-ANN models
PCA was applied on the normalized data set.Figure 6 shows the significance of the resulting PCs with respect to data variation.The Y axis represents the normalized weight of each PC.It was found that the highest weight is equal to 861.75 and the lowest weight is equal to 0.015.It could be observed that the first three PCS account for as much of the variability in the data; their cumulative weights is equal to 1073.512.Five PCA-ANN models were trained.The models differ from each other in the number of PCs that were fed to a three layer ANN.The structures of the five trained PCA-ANN models are listed in table 2. Figure 7 shows the AUC metric values for each of the ANN model and the five PCA-ANN models.It is clearly observed that the best model based on the AUC metric is the ANN model (AUC = .856).While the worst performance model is PCA-ANN#5 which uses 3 PCs only and results in AUC value equal to .625.PCA-ANN#4 which uses 6 PCS improves the performance dramatically (AUC = 0 .779), the more PCs included the closer the performance to the ANN model.Figure 8 shows the ROC curves of the models PCA-ANN#4 and PCA-ANN#5 and it is obvious the less the number of PCs included the closer the ROC curve to the diagonal which indicated less capability on diffrentitaing between good and bad clients.

Credit scoring attributes analysis
Although data mining techniques are useful in analyzing the credit worthiness, the selection of the best set of independent attributes to be included in the model building is essential to improve the performance of the model, speed up the learning process and save the storage especially with the large amount of data.Same as PCA is a technique that can find a smaller size attribute space that represents a set of observations; it can also be helpful in analyzing an existing attribute space and determine the significance of each of these attributes.This significance is determined based on the dominant attributes in the dominant PCs, which are responsible for the most variability in the dataset.Figure 11 shows the significance, weights, of the different attributes in each of PC1, PC2 and PC3.It is obvious that some attributes contribute minimally to PC1, PC2 and PC3; consequently, their contribution to the credit-scoring model is minimal too and could be ignored.It is noteworthy that when a NNET was trained using the attribute space of the German dataset (without pre-PCA), it gave a weight to each input attribute without any consideration to the actual impact of each attribute (as shown in figure 12), and this is in general a drawback of any ANN based model, where it develops a models that maps any nature of inputs to any nature of outputs.But with PCA analysis, significance of each input attribute is taken into consideration to develop a realistic model.

Conclusion
Credit scoring model is a mean to measure the risk related to a potential customer, by analyzing his data to determine the probability of him paying his debits back to the financial institution.The paper discussed the development of a credit-scoring model based on PCA and ANN to achieve better performance than existing models and reduce the computational complexity of the ANN through using PCA analysis. .Numerous studies have proven that ANN perform remarkably better than any other statistical approach, such as logistic regression or discriminant analysis however they are computationally expensive.So a PCA The proposed model showed its ability to generalize well to the credit-scoring problem and its performance, in terms of AUC, outweighed many other previous models.Preceding ANN with PCA reduces the number of inputs to the ANN and reduces its training time.Moreover, PCA provides a mean for analyzing the importance of each attribute to the model which helps in reducing the input space size and save storage.

Figure 1 .
Figure 1.Three layers, two outputs an ANN architecture.

Figure 2 .
Figure 2. PCA-ANN model for credit scoring

DOI: 10
function (equation 6) and one neuron in the output layer with linear transfer function to linearly scale the output values to the interval [0,1].and a are the input and output of each neuron in the hidden layer; respectively.The training algorithm is the back propagation algorithm[41].Iterations were performed on the weights and bias of each neuron in the network until the mean squared error for the validation samples dropped to the possible minimum value.Figure4shows that the minimum mean square error was 0.14 at epoch number 2. At other epochs either the mean square error is high (epochs number1 and 2).At epochs 4 and more, the mean square error of the model on the training data is decreased while it increases on the validation data which means that the generated models suffer over fitting problem.The performance of the best model (at epoch 2) was evaluated against the testing dataset.Figures 5 and table6show the ROC curve of the resulting classifier and its confusion matrix.It could be observed that the ROC curve of the trained classifier is above the diagonal line (line of no discrimination or random classifier) which implies that the ANN classifier can produce a good classification results.

Figure 4 .
Figure 4. MSE of the trained ANN models at different epochs on each of the training, validation and testing datasets.

Figure 5 .
Figure 5. ROC curve of the trained ANN classifier

6 Figure 6 .
Figure 6.Significance of the principal components with respect to the data variation

Table 2 .Figure 9 .
Figure 9. Shows the training time (in seconds) of each model, as observed PCA-ANN#5 model is the most expensive model computationally as ANN was not able to generalize quickly with the few number of input features.However, all PCA-ANN models generalize faster than ANN only and the more PCs included the more training time required.

Figure 10
Figure10shows a comparison ,based on AUC, among the performance of our 6 trained models and 8 models of Harris[33].The models of Harris are: logistic regression (LR), K-means plus logistic regression (K-means + LR), clustered support vector machine with a RBF kernel (CSVM-RBF), K-means plus support vector machine with a RBF kernel (Kmeans + SVM-RBF), support vector machine with a RBF kernel (SVM-RBF), linear clustered support vector machine (CSVM-linear), K-means plus support vector machine with a linear kernel (K-means + SVM-linear), and a linear support vector machine (SVMlinear).As observed all our models, except the model PCA-AN #5, outperform Harris models.We

Figure 9 .Figure 10 .
Figure 9. Training time (seconds) of the proposed models

6 Figure 11 .
Figure 11.Relative significance of each German dataset attribute in the calculation of PC1 , PC2 and PC3.

Figure 12 .
Figure 12.Normalized input weights of the trained ANN.
,is another modern classification technique applied to credit scoring [17-23].SVMs do not suffer from over fitting problem like ANN and consequently can generalize well.However they are still computationally expensive.Genetic algorithms was also tried in the field of credit scoring [24].
The most common type of ANN consists of three layers of units: input layer, hidden layer, and output layer.It is called multilayer perceptron (MLP

.2 Experimental Approach Two
types of models were trained for credit scoring.The first one of them is an ANN fed with the 24 attributes German data.The second type is a hybrid model consists of a PCA and ANN classifier.PCA is fed with the 24 attributes German dataset while ANN is fed with a set of PCs as shown in figure2.

Table 1 .
Confusion matrix of the trained ANN classifier