Developing Credit Scoring Models With Fuzzy Rule Based Classifiers Arijit Laha Institute for Development and Research in Banking Technology Castle Hills, Hyderabad 500 057 India ( e-mail: [email protected]) Abstract – Credit-risk evaluation is a very challenging and important problem in the domain of financial analysis. Many classification methods have been suggested in the literature to tackle this problem. Statistical and neural network based approaches are among the most popular paradigms. However, most of these methods produce so called “hard” classifiers, those generate decisions without any accompanying confidence measure. In contrast “soft” classifiers, such as those designed using fuzzy set theory produce a measure of support for the decision (and also alternative decisions) that provide the analyst with greater insight. In this paper we propose two schemes for designing fuzzy rule based credit scoring models. First fuzzy rules are extracted from the training data using a Self-organizing Map (SOM) based method. Then two classifiers, basic fuzzy rule based classifier and fuzzy rule based k-nn classifier, are developed for credit scoring. A method of seamlessly integrating business constraints into the model is also proposed.

I. I NTRODUCTION Credit scoring is a method of modelling potential risk related to a credit portfolio. Popularly statistical and neural network techniques are used with historical data to produce a scoring models, which can be used by financial institutions to evaluate portfolios in terms of risk [7]. Credit scoring tasks can be divided into two distinct types. The first type is credit application scoring, where the task is to classify applicants into “good” and “bad” risk groups. The data used for modelling is generally consisted of financial information and demographic information about the loan applicant. In contrast, the second type of tasks deal with existing customers and payment history information are also used here. This is distinguished from the first type because this reflects the customers payment pattern on the loan and the task is called behavior

scoring. The remaining sections of this paper will be focused on application scoring. However, the techniques developed here, working with appropriate data set, can be applied for behavior scoring also. Here we propose a data-driven (i.e., using learning algorithms) scheme for developing credit scoring models that incorporates a Self-organizing Map (SOM) [3] based method for fuzzy rule extraction [4], [5], [6] for classifier design. One of the two variants of model proposed here uses the fuzzy k nn rule [2] on the fuzzy labels of the test data point and its k nearest neighbors for more robust and context sensitive decision making. We also outline the design of a post-processing filter that allows a domain expert to impose business constraints on the decision according to the risk policy. The rest of the paper is organized as follows: in section 2 we describe the scheme in detail, section 3 contains the experimental results and we conclude the paper in section 4. II. D ESIGNING A CREDIT SCORING MODEL The scheme for developing the credit scoring model is depicted in Figure 1. First a fuzzy rule base is extracted (i.e., learned) from the training set using a SOM based prototype extraction algorithm, followed by a fuzzy rule extraction and tuning algorithm applied on the prototypes. Then we use the rule base to design two types of classifiers for credit scoring. The first one uses the fuzzy rules directly, while the other applies fuzzy k -nn rule to the test data and its k nearest data points from the training/reference set. Then we propose a method of incorporating the business constraints in the model. A. Extraction of prototypes Most challenging problem in prototype based classification is to find representative prototypes in

The learning system

Training Data

SOM

DYNAGEN

Fuzzy Rule Extraction

FRKNN

Decision Business Constraints

Fig. 1.

Steps in developing credit scoring model using fuzzy rule based k-nn (FRKNN) classifier

adequate number. Usually some clustering technique such as k -means algorithm [1] is used to find clusters and their centers form the set of prototypes. However, for most of the conventional clustering algorithms, the number of clusters (and hence prototypes) need to be specified by the user. The number of prototypes is the most important factor influencing the performance of the classifier. If the number is less than adequate, the performance suffers, while a larger than required number of prototype increases the computational load unnecessarily. In the proposed scheme we use the dynamic prototype generation algorithm, DYNAGEN introduced in [4], [5], that dynamically extracts adequate number of prototypes from the training data. In this method, the user need not guess or find by trialand-error, the number of prototypes adequate for the task. It is also relatively immune to underutilization of prototypes. The DYNAGEN algorithm first trains a SOM with the training data (note that, the label information is not used for training the SOM). Then the SOM prototypes are labelled and fine-tuned iteratively . In each iteration, based on their classification performance, the prototypes are deleted, split and/or merged. At the end the algorithm produce a good set of prototypes, where each prototype good degree of the properties (1) representativeness: the prototype represents significant number of points and (2) purity: majority of the points represented by the prototype comes from a single class, which is represented by it. For more details see [4], [5]. B. Designing fuzzy rule base Though the prototype based classifier produced by DYNAGEN works well with simpler data sets, its performance suffers when the data dimension is large and/or there is large variation in the variances of different components of the data. This is true

for all purely distance based classifiers, because in such a situation the distance measures fails to reflect the concept of “similarity” in a proper fashion. Fuzzy rule bases can be used to address the above problems. In our scheme we convert the set of prototypes into a set of fuzzy rules and finetune them using the method proposed in [6]. The method tunes the rules with respect to their context in the feature space. Out of two variants of the tuning algorithms proposed in [6], we use the one uses “softmin” as the conjunction operator. Using the resulting rule base {Ri }, where rule Ri for class k is of the form Ri : x1 is CLOSE TO vi1 AND · · · AND xp is CLOSE TO vip then class is k .

The fuzzy set CLOSE TO vij is modelled by a Gaussian membership function : µij (xj ; vij , σij ) = exp −(xj − vij )2 /σij 2 ,

where vij and σij are the center of the fuzzy set and a constant controlling the spread of the fuzzy set respectively. Both of them are initialized from the respective prototype and fine-tuned by the algorithm. For a data point x ∈

We can classify a data point x ∈

We shall call the classifiers using the above decision making scheme as basic fuzzy rule based classifier. C. Contextual decision making Though the above rule can be applied for classification, the rule base can be used to produce more information-rich output in form of a fuzzy or (strictly speaking) possibilistic label vector α(x) = (j) [α1 , . . . , αc ], where αj = max{αi (x)} and can be interpreted as the confidence measure of the rule base in support of the hypothesis that x belongs to class j . We can exploit this information to design more robust classifiers. The k -nn rules, while using k nearest neighbors of the data point from the training (also called reference) set, takes into account the neighborhood of the data point in the feature space for classification. In other words, they use the context information for classification. Similarly, here also we can aggregate the fuzzy labels of the test data point x and its k nearest neighbors (the contextual information) from the reference set using the fuzzy k -nn rule [2] as: P α(x) + ki=1 α(x(i) ) A α (x) = , k+1 where x(i) is the i-th nearest neighbor of x. We call the classifier using this modified rule fuzzy rule based k-nn classifier. The decision rule can be expressed as: Assign x to class i if αiA ≥ αjA ∀i 6= j, where αiA is the i-th component of the aggregated label vector αA (x). D. Introducing business constraints Since the proposed classifiers (both the basic and the one using k -nn) can generate their output in form of possibilistic label vectors, where the value its each component can be interpreted as a measure of confidence/support regarding the hypothesis that the true class of the data point is the respective class, one can calibrate the confidence values by mapping them into real situations. Thus they can be used as KPIs (Key Performance Indicators) and business constraints can be imposed on their values to take final decisions commensurate with the risk-averseness of the institution. From the system perspective it amounts to designing a filter to postprocess the classification results. For example, here we formulate a few simple constraints as follows:

Any prediction can be accepted only if αiA > T hmin ∀ i, where T hmin is a threshold value. C2 A prediction “Good” can be accepted A only if αgood ≥ T hgood . C3 A prediction “Good” can be accepted A A only if αgood ≥ T hgood , and αgood − A αbad ≥ T hdif f . All the above constraints applied together. C4 Note that, the above constraints are examples only, designed to “err in the side of caution”, i.e., to reduce the errors where a bad data is predicted as good one. Making that type of error will lead to so called “bad investments” which a banker will like to avoid earnestly. However, this may lead to predicting increased number of good data as bad one. For these cases, based on the confidence values, the banker still have the option to collect more information and analyze them further. Once a proper calibration of the system outputs is accomplished, a domain expert may come out with many other or altogether different set of constraints. C1

III. E XPERIMENTAL RESULTS For testing the proposed schemes we built credit scoring models using the German credit data, available publicly at UCI Machine Learning data repository. The data contains 1000 instances of retail loan applications. The original data has a mix of 20 categorical and numerical attributes recording various financial and demographic information about the applicants. The details of the attributes is available at the repository. In the repository a numeric version of data set, where the categorical attributes are transformed into numerical and a few indicator variables are added, which increases the dimension to 24. Here we use this numerical data set. The data instances are labelled as classes 1 (good, 700) and 2 (bad, 300). For building the models we use a randomly selected subset of data containing 300 points, 150 from each class. Rest of the data (700, 550 + 150) is used as the test data for measuring the performance of the models. To develop the model first a 5 × 5 SOM is trained with the training data. These 25 initial prototypes are labelled and the algorithm DYNAGEN is applied on them. DYNAGEN produced 6 final prototypes which are converted into 6 fuzzy rules. The rules are fine-tuned to produce the fuzzy rule base. The test results for the basic fuzzy rule based classifier and fuzzy rule based k -nn classifier is

TABLE I E RROR RATE FOR THE TEST PARTITION OF G ERMAN CREDIT DATA

Constraint None C1 T hmin = 0.1 C2 T hgood = 0.5 C3 T hdif f = 0.2 C4

Basic fuzzy rule based classifier Etotal Egood Ebad Reject 37.7% 37.1% 40.0% None 30.3% 28.7% 36.0% 13.7% 61.1% 74.0% 14.0% None 51.9% 62.7% 12.0% None 51.8% 62.7% 12.0% 13.7%

presented in table I. The column Etotal shows the overall classification errors of the classifiers, Egood depicts the percentage of class 1 (good) points classified into class 2 (bad) and Ebad shows the percentage of class 2 (bad) points classified into class 1 (good), whereas the column “Reject” gives the percentage of points remained unclassified under the constraint C1 (and C4). For the same training-test partition hard 5-nn classifier shows an overall test error rate of more than 38%. It can be observed from the table that the basic classifier exhibits comparable performance while the fuzzy rule based k -nn classifier performs significantly better in terms of overall accuracy. Between the two unconstrained classifiers, though the basic rule based classifier has slightly higher (3% approx) error rate, it has much lower value of Ebad , which is of paramount importance. Under constraint C1 the basic classifier is able to reject 13.7% of ambiguous/borderline predictions and reduce the error rates to some extent (4%). For the k -nn classifier there is marginal improvement, though the performance may improve further if the value of the threshold T hmin is changed. For constraints C2 and C3 both the classifiers are able to reduce the critical part of the error Ebad to a large extent (e.g., for C3, Ebad is 12% and 4.7% (only) respectively), though the Etotal s and Egood s increase significantly. Application of C4 (combination of C1-C3) retains the low values of Ebad while slightly reduce the Etotal s and Egood s. IV. C ONCLUSION In this paper we have proposed two fuzzy rule based schemes for developing credit scoring models. First we have generated a small but adequate fuzzy rule base using a SOM based algorithm for rule extraction and fine tuning. Next using this rule base we have developed two rule based classifiers. First of them applies the rules directly

Fuzzy Etotal 34.6% 34.3% 68.3% 68.7% 68.0%

rule based k -nn (k = 5) Egood Ebad Reject 29.6% 52.7% None 29.3% 52.7% 0.7% 85.3% 6.0% None 86.9% 4.7% None 85.3% 4.7% 0.7%

for classification while in the other scheme we design a context sensitive classifier using k -nn methodology. Effectiveness of the proposed classifiers are demonstrated with a real data set. Further, since the classifiers generate fuzzy label vectors as output, we have proposed a method of integrating business logic with the model within the same computational framework, leading to a more useful system in the context of the problem domain. We have also empirically demonstrated this feature of our proposed scheme. R EFERENCES [1] R. O. Duda, P. E. Hart, and D. G. Stork. Pattern Classification (2nd Edition). Wiley-Interscience, New York, 2000. [2] J. M. Keller, M. R. Gray, and J. A. Givens. A fuzzy k-nearest neighbor algorithm. IEEE Trans. Syst. Man Cybern, 15(4):580–585, 1985. [3] T. Kohonen. Self-Organizing Maps. Springer, Berlin, 3 edition, 2001. [4] A. Laha and N. R. Pal. Dynamic generation of prototypes with self-organizing feature maps for classifier design. Pattern Recognition, 34(2):315–321, 2000. [5] A. Laha and N. R. Pal. Some novel classifiers designed using prototypes extracted by a new scheme based on selforganizing feature map. IEEE Trans. on Syst. Man and Cybern: B, 31(6):881–890, 2001. [6] N. R. Pal, A. Laha, and J. Das. Designing fuzzy rule based classifier using self-organizing feature map for analysis of multispectral satellite images. International Journal of Remote Sensing, 26(10):2219–2240, 2005. [7] L.C. Thomas. A survey of credit and behavioural scoring: forecasting financial risk of lending to customers. International Journal of Forecasting, 16:149–172, 2000.