Introduction

Credit scoring is a technique mainly used in consumer credit to assist credit-grantors in making lending decisions. Its aim is to construct a classification rule that distinguishes between 'good' and 'bad' credit risks according to some specified definition. The rule is developed on a sample of the past applicants, whose performance is known. A number of modelling approaches have been used from classical discriminant analysis to neural networks and genetic algorithms (Rosenberg and Gleit, 1994; Hand and Henley, 1997; Thomas, 2000; Thomas et al, 2002). Most frequently the model is a weighted sum of the applicant's observed characteristics (age, marital status, etc) that produces a score, which is a summary of the applicant's creditworthiness and reflects his or her ranking relative to other applicants. The classification into 'good' and 'bad' is achieved by comparing the score to a predetermined threshold or a cut-off level.

Traditionally a credit scoring model is constructed to fit a specific credit portfolio, which normally consists of residents of one country (customized models). However, the political desire for further integration of the European Union into a single internal market opens the possibility for the lenders to compete across national borders. Therefore, the necessity arises to understand the performance of Pan-European credit scoring models as opposed to those based on separate countries. Does the predictive ability of the generic model compare favourably with national models? If it does then it will be possible to use the generic model, which is a considerably cheaper alternative to customized models, across a series of European countries. The attractiveness of generic models is further strengthened by the legal restrictions on the composition of credit scoring models. In certain countries, the use of nationality can be regarded as illegal. A more detailed discussion of legal restrictions on data used in credit scoring is given elsewhere (Andreeva et al, 2004a).

The majority of published empirical tests demonstrate the superiority of customized models (Overstreet et al, 1992; Overstreet and Bradley, 1996; Platts and Howe, 1997; Barron et al, 2000). The current paper explores the possibility of applying a single generic model to credit score the applicants for a revolving store card from three EU countries (Belgium, The Netherlands, Germany), and demonstrates the competitiveness of generic models. Several generic models are developed using logistic regression (LR) and survival analysis, and their predictive accuracy is benchmarked against the performance of equivalent national models.

While LR is the standard approach in credit industry, survival analysis is a relatively new application that offers an advantage of predicting time to the event of interest and therefore, lays the foundation for estimating the applicant's profitability (Banasik et al, 1999; Stepanova and Thomas, 2001). Applications of survival analysis in credit scoring began with Narain (1992) showing that estimates of a lifetime of a loan obtained from the exponential model can significantly improve the credit-granting decisions. Several studies (Banasik et al, 1999; Stepanova, 2001; Stepanova and Thomas, 2001; Stepanova and Thomas, 2002) explored different applications of proportional hazards (PH) models, including behavioural scoring. The dynamic exponential model was proposed for new products, when there are no historical data to develop a model on (Hand and Kelly, 2001).

These studies analysed fixed-term credit data and found the survival analysis approach competitive, and in certain applications superior to the LR. The current analysis extends the application of survival analysis to the area of revolving credit, which has not been investigated before, and explores the family of accelerated failure time (AFT) distributions that has not been addressed in detail by previous research. In this paper, comparison is made between LR, parametric AFT and PH survival models and Cox non-parametric PH model in predicting default/time to default. In addition, the sensitivity of predictive ability of survival models to the presence of heterogeneous subpopulations (applicants from different countries) is investigated.

The paper is structured in the following way. The next section describes the data and presents an overview of the basic concepts and methods used. The subsequent section compares the national survival patterns and compares predictive accuracy of national and generic models under different modelling approaches. Alternative timescales of default are investigated for national models. The following section extends the generic Cox non-parametric PH model by means of stratification. The final section concludes and outlines some directions for further research.

Data description and methodology

The data for analysis were provided by a major international credit scoring consultancy and relate to the same retail card issue in three European countries: Belgium, Germany and The Netherlands (Table 1). The performance of the accounts was observed during 25 months from October 1998 to December 2000. The life of the account was measured from the month it was opened until the account became 'bad' or it was closed or until the end of observation. The account was considered to be 'bad' if payment was not made for two consecutive months. If the account did not miss two payments and was closed or survived beyond the observation period, it was considered to be censored. The list of characteristics collected in each country was different. However, it was possible to select 16 characteristics that were collected for all three countries (Table 2), and these characteristics were used for predicting default in national and generic models.

Table 1 - Samples used in analysis.

Full table

Table 2 - List of characteristics used in analysis.

Full table

The traditional way of relating the vector of observed characteristics x to the probability of default is to fit the LR model to estimate P, the probability of becoming 'bad' within the period of observation

This approach assumes that the accounts that do not experience default are 'good', non-defaulting accounts, whilst survival analysis treats such accounts in a more conservative and realistic way, as those that proved to be 'good' so far. Survival analysis allows one to use characteristics x to estimate either time to default T or probability of surviving to a certain time

or hazard function h(t), which is the probability of the event occurring within the time interval (t, t+t), given that the event did not occur before time t:

where S₀ and h₀ are baseline survival and hazard functions and (x)=exp('x). The above given model—AFT—assumes that the covariates x act multiplicatively on time t, thus influencing the speed at which the account proceeds towards the event. In the PH model the covariates act multiplicatively on the baseline hazard rate, and so the relative ranking of risk presented by different accounts does not change with time

One can assume a particular distribution for a hazard function, the most commonly used in medicine (Collett, 1994) and reliability (Ansell and Phillips, 1994) are exponential, Weibull, log-normal, log-logistic and gamma, and they are considered in this paper and fitted as parametric AFT models. It should be noted that the Weibull family distributions (including exponential) can be both AFT and PH. So one can interpret the results reported here for parametric AFT Weibull family models also as results for parametric PH models, since the Weibull AFT coefficients can be obtained from the Weibull PH coefficients as follows:

where is a shape parameter (Kalbfleisch and Prentice, 1980). This transformation does not affect the ranking of credit applicants, and therefore predictive accuracy of the Weibull family models. However, for simplicity all parametric models are referred to as AFT further on.

Alternatively, one can use Cox PH model that does not specify the shape of the baseline hazard function (Cox, 1972). While the parametric AFT models provide more efficient estimation, Cox non-parametric PH model has an advantage of robustness and flexibility. An important property that follows from an arbitrary nature of the baseline hazard h₀(t) is that it can be allowed to vary between different groups (nations) in a population, a feature that can be useful for generic scoring models. A more detailed discussion of survival analysis methods is given elsewhere (Kaplan and Meier, 1958; Cox, 1972; Kalbfleisch and Prentice, 1980; Lawless, 1981; Cox and Oakes, 1984; Ansell and Phillips, 1994; Collett, 1994; Allison, 1995).

So Cox non-parametric PH model and several parametric AFT models (assuming the distributions described above) were fitted to three national data sets. Then three data sets were pooled together, and the generic models were built on the aggregated data. The predictive performance of the survival analysis models was benchmarked against the LR. All the data sets used in the analysis were randomly split into training (70%) and hold-out (30%) samples. The model was developed on the training sample, and its predictive ability was measured on the corresponding hold-out sample. Generic models were tested on each national hold-out sample (to allow for comparisons with national models) and on the aggregated generic hold-out.

For Cox non-parametric PH model the estimate of exp('x) was used as a score, for parametric AFT models a score was given by the probability of 'surviving' in 25 months, which is similar to the way predictions are generated from LR. However, the advantage of the survival analysis consists in the ability to produce predictions for several time periods from the same model, which LR cannot do.

Comparison of the models was made by the receiver operating characteristics (ROC) curve and percentage of incorrectly classified accounts. The area under ROC curve (AUROC) provides a measure of classification accuracy, which is not dependent on any threshold or acceptance rate (Bamber, 1975). It corresponds to the Wilcoxon or Mann–Whitney or U statistic, which estimates the probability that a ranking of a randomly selected bad account will be less than or equal to a ranking of a randomly selected good account (Hanley and McNeil, 1982). The second measure is the error rate (ER), the sum of percentages for incorrectly classified goods and incorrectly classified bads. It is based on confusion matrix, which presents the counts of good and bad accounts correctly and incorrectly classified by the model. The use of the matrix requires the choice of a cut-off level. For the purpose of this analysis the cut-off was fixed at the level of the default rate in the hold-out sample, that is, such that the observed proportion of bads equalled the predicted proportion of bads in the hold-out sample.

There is no direct correspondence between two measures, since AUROC gives the summary of predictive accuracy over all possible cut-off levels, while ER is tied to a certain cut-off, which may not be the most appropriate or 'best' for a given country, and which is, for the purpose of this research, dependent on the default rate, that differs across countries. The discussion of strategies for a cut-off selection is beyond the scope of this paper. The strategy adopted here allows one to judge the quality of prediction as benchmarked against the number of observed events, which can be easily interpreted. It is suggested that for cross-country comparisons AUROC is a preferred measure. For more information on predictive accuracy of LR generic models using a range of possible cut-offs see an earlier study (Andreeva et al, 2004b).

Survival analysis compared to logistic regression

Before modelling the relationship between application characteristics and survival times, some exploration of survival patterns was done for each country by means of fitting Kaplan–Meier estimates (Kaplan and Meier, 1958) of the survival distribution function (SDF) and generating the hazard plot, with estimates of the hazard function obtained from SDF as described by Stepanova (Stepanova, 2001).

To check whether the SDFs were statistically different across the three countries, the log-rank test and Wilcoxon test (Allison, 1995) were applied. The log-rank test involves calculating deviations of the observed number of events from the expected numbers. For country i in the population the log-rank statistic is

where d_ij and e_ij are observed and estimated numbers of events occurring in country i at time j, and r is the unique event time in the population. The Wilcoxon test calculates the weighted sum of deviations of observed numbers of events from expected numbers

where n_j is the total number at risk at each time point.

The test of equality of the k survival curves is based on the _k-1² distribution for the squared values of the log-rank or Wilcoxon statistic divided by estimated variance. Both tests for 2 degrees of freedom showed that survival curves were significantly different for the three countries (P=0.0001).

The hazard plots (Figure 1) for all three countries increase rapidly in the first months of the account life and then decrease towards an asymptote. This supports a conventional wisdom—'if they go bad, they go bad early' (Banasik et al, 1999). However, 'early' means 3 months for Belgium and Germany, and 5 months for The Netherlands. The height of peaks also differs between the countries, with Germany being the least risky in terms of early defaulters and The Netherlands being the most risky. After 9 months hazards become flat and overlap for Germany and Belgium, but The Netherlands remains slightly higher and shows a slight increase at the end of the observation period. However, this may be due to decreasing risk set.

Figure 1.

Hazard function for 'default in 25 months' by country.

Full figure and legend (16K)

The examination of plots suggested a number of possible approaches: log-normal, log-logistic and gamma are most suitable for modelling non-monotone hazards. However, one should not discard exponential and Weibull distributions, they may offer a suitable approximation since the major part of the national hazard functions are monotone and even constant. At the same time the country hazards looked roughly proportional, suggesting that Cox model would be appropriate.

So exponential, Weibull, log-normal, log-logistic distributions were fitted along with the non-parametric Cox PH model. The model fit was measured by log likelihood (Log L), with lower magnitude values indicating the better fit. This measure indicates how well the model describes the data, but it can be used only to compare nested models. Models are nested if one is a special case of another, in our case the exponential, Weibull, log-normal models are nested within the generalized gamma (Kalbfleisch and Prentice, 1980). This allows one to make tests of significance for the difference in model fit (Allison, 1995). Judgements about models not nested in gamma (log-logistic, logistic and Cox PH) can be made based on their predictive ability, which was measured by the AUROC and ER.

The results from different survival analysis models and LR are presented in Table 3. The differences in model fit were highly significant (Table 4). In terms of model fit, the leader was the gamma distribution followed by the log-normal for all three countries. However, this did not translate into superior prediction results. In fact, the gamma distribution gave the worst prediction, which may be due to overfitting when a better fit does not necessarily mean better prediction. It means that the model describes the training data set too closely, giving too much weight to variations that would be a random variation or noise in a population model. Overfitting leads to excessive variance of estimates and therefore produces poor predictions.

Table 3 - Survival analysis and logistic regression models for 'default within 25 months' by country.

Full table

Table 4 - Model fit test of significance based on log likelihood for 'default within 25 months'.

Full table

Since different distributions demonstrate very similar predictive accuracy within one country, and given the desire for a more parsimonious and therefore robust model, the exponential distribution would be most suitable from parametric models. At the same time Cox PH and LR models give identical results. Since there is no or little difference in predictive accuracy, the decision as to which approach to use should be based on additional properties that a certain approach can provide. The fact that the exponential model gives good approximation suggests that the default process is memoryless across the population for revolving credit and this explains the similarity in predictions obtained from the Cox PH and logistic models.

However, there may be periods within 25 months when certain models give superior prediction. This proposition would apply to AFT survival models since they allow for modelling the changing hazard rates between different groups over time, and therefore will produce different ranking of accounts in different time periods. To test this proposition, the survival models were applied to two alternative definitions of 'bad'. First, those that missed two payments within the first 6 months were considered to be 'bad', and the rest were treated as 'good'. Second, those that defaulted within first 12 months were classified as 'bad', the remaining customers in the hold-out sample were considered good. The parameter estimates from AFT models developed earlier were used to generate predictions of surviving within 6 and 12 months, the cut-offs were chosen to match the actual number of defaults within these time periods. For Cox PH model the estimates obtained earlier were used, since they are not time sensitive.

The results in Table 5 suggest that log-logistic, log-normal and gamma models give slightly superior prediction for Belgium and The Netherlands, especially for 'Default in 6 months'. This is in line with hazard plots (Figure 1), where early peaks in hazards are observed, so if one would expect some difference in results that would be during the first months. The hazard peak for Germany is least pronounced, and hence there is no marked difference between the predictive ability of survival models for this country. It should be noted though that even for Belgium and The Netherlands the differences are marginal and do not give enough grounds to conclude that log-logistic, log-normal or gamma should be preferred to more robust exponential and non-parametric Cox PH models. Perhaps, the superiority of AFT models may be more visible in different credit scoring applications (eg insurance) with more pronounced time structure.

Table 5 - Predictive performance of survival analysis models for alternative definitions of 'bad'.

Full table

Although the same characteristics were deliberately used, their strength of association with default varied across countries, Belgium being the country with the weakest association between default and selected characteristics. Besides, the Belgian sample was the smallest one, and this also contributed to the poorer quality of prediction as compared to other countries. As a result, same methods performed differently on different countries. Table 2 indicates most significant characteristics in predicting default using LR for each country, giving some feeling of cross-country differences. More detailed discussion of national default prediction patterns is given in an earlier study (Andreeva et al, 2004b). It should be noted that second-level interaction terms were investigated for the LR model, but although some interactions were statistically significant, there was practically no improvement in predictive accuracy due to overfitting. Therefore, this approach was not pursued in developing survival analysis models.

Previous research (Andreeva et al, 2004b) explored the application of generic LR model to the same data set as used here. The results suggested very little difference in predictive accuracy as compared to national models, in contrast to earlier findings (Platts and Howe, 1997). There are two main differences between this paper and the paper by Platts and Howe. First, different sets of countries were used. Platts and Howe used five countries that represented five different European regions and attempted to build a Pan-European scorecard for the whole Europe. Andreeva et al and this paper focus on just one part of Europe, suggesting that it is possible to segment Europe into several regions and use a generic model for several countries within each segment. Second, the data in this paper came from exactly the same product—the same retail card that was managed by one lender. This reduced the variability that otherwise might be observed. For more detailed results and discussion, see the above mentioned study (Andreeva et al, 2004b).

It would be of interest to examine how survival analysis generic models compare to LR. Table 6 compares the performance of exponential, Cox PH and LR generic models against the national LR model by means of AUROC and ER (in brackets). The customized national models give slightly better prediction, but one can argue that from the business point of view the difference in performance is not dramatic and can be offset by the lower costs of using one model instead of three different ones. Out of three generic models, LR is slightly superior, but again the difference is marginal. It should be noted that LR is given an advantage by the fact that it is fitted specifically to a definition of being 'bad' within the fixed observation period, while survival analysis is not restricted by any arbitrary time horizon. In general, it is possible to say that all three approaches are fairly insensitive to the presence of different national segments.

Table 6 - Predictive performance of generic models for 'default within 25 months'.

Full table

Stratified proportional hazards model

As was noted before, the Cox PH model can account for different subpopulations that may exist within the data, while producing a single set of parameter estimates that does not include a subpopulation indicator. This method (stratification) is commonly used for subpopulations that violate the proportionality assumption (Allison, 1995).

The following model was fitted to data:

where z is a subpopulation indicator. In this way, the hazard function was allowed to vary between specified groups. This method presents a halfway option between generic and customized models: to estimate the parameters, the country indicator is required, but the subsequent scoring of new accounts can be done without making a distinction between the countries. It should be noted, though, that in order to build the model one needs to know the applicant's country of residence. Still it will be useful in situations when 'nationality' is not legally forbidden (Andreeva et al, 2004a), and lenders can replace several national models with a generic one that accounts for different subpopulations by stratification.

The stratified Cox PH model has been fitted to the aggregated generic data set (Table 6). AUROC shows no improvement on the generic hold-out sample, but when tested separately on the national samples, there is some increase in AUROC for Belgium and The Netherlands, although not a dramatic one. ER demonstrates some superiority of the stratified approach if compared to other models, including the LR. For Belgium, the stratified model shows the ER even lower than the national LR model. In general, one can conclude that stratification brings some benefit but not a convincing one. Such modest improvements can be attributed to the fact that the hazards between the countries are roughly proportionate, so there is little scope for stratification to enhance the predictive performance.

Conclusions and further research

This paper presents the first cross-country comparison of the application of survival analysis to predict when a borrower defaults and supports the previous findings that survival analysis is competitive with the LR. The comparison of several approaches showed that there is little difference in classification accuracy between the parametric survival models, non-parametric Cox PH model and LR. The current analysis demonstrated that in spite of significant differences between the model fit measures, the predictive accuracy is not affected. Such similarity in prediction can be attributed to relatively constant hazards (apart from the first months).

The analysis was carried out on a revolving type of credit, the area where the application of survival analysis was not investigated by previous published work. It was suggested that exponential distribution offered a suitable fit for the analysed data set, therefore, implying that the data exhibits the memoryless property. This indicates that revolving credit may have a more random character than fixed-term credit, a proposition that needs to be investigated further. Early defaulters also deserve further investigation, one of the possible hypotheses is that they may represent fraudulent accounts.

Since the predictive performance of survival analysis and LR is similar, the choice of the model depends on some additional benefits any particular modelling approach can bring. The survival analysis offers a number of benefits which make it more attractive when compared to LR. Time to default can be estimated, which provides a basis for profit scoring. Alternatively, predictions can be generated that give the probability of 'survival' in certain time period. It was shown that survival analysis is also suitable for building generic models, and in addition, models that are halfway between generic and customized ones. Besides, the parametric AFT models (log-logistic, log-normal, gamma) showed some superiority in predicting early defaulters, although the magnitude of improvement is dependent on how large is the number of such 'early defaulting' accounts as compared to the total number of defaults.

Following earlier investigation of LR generic models (Andreeva et al, 2004b), this paper demonstrates that generic models can produce high-quality prediction, and segmenting on nationality is not necessarily required. It would be of interest to investigate whether segmentation on variables other than nationality can give better improvement in prediction. That forms a separate project for future research activities.

The emergence of the single European market in financial services makes generic scoring an important and timely application, and survival analysis generic models can serve as a starting point of profit scoring in integrated Europe.

References

1. Allison PD (1995). Survival Analysis: Using the SAS System. SAS Publishing: Cary, NC, USA.
2. Andreeva G, Ansell J and Crook JN (2004a). Impact of anti-discrimination laws on credit scoring. J Financial Services Market 9: 22–33.
3. Andreeva G, Ansell J and Crook JN (2004b). Credit scoring in the context of the European integration: assessing the performance of the generic models. Working paper 04/2. Credit Research Centre, University of Edinburgh.
4. Ansell JI and Phillips MJ (1994). Practical Methods for Reliability Data Analysis. Clarendon Press: Oxford.
5. Banasik J, Crook JN and Thomas LC (1999). Not if but when will borrowers default. J Opl Res Soc 50: 1185–1190.
6. Bamber D (1975). The area above the ordinal dominance graph and the area below the receiver operating characteristic graph. J Math Psychol 12: 387–415. | Article | ISI |
7. Barron JM, Ellienhausen G and Staten ME (2000). Monitoring the household sector with aggregate credit bureau data. Bus Econ 35: 63–77.
8. Collett D (1994). Modelling Survival Data in Medical Research. Chapman & Hall: London.
9. Cox DR (1972). Regression models and life-tables (with discussion). J Roy Stat Soc Series B 74: 187–220.
10. Cox DR and Oakes D (1984). Analysis of Survival Data. Chapman & Hall: London.
11. Hand DJ and Henley WE (1997). Statistical classification methods in consumer credit. J Roy Stat Soc Series A 160: 523–541. | Article |
12. Hand DJ and Kelly MG (2001). Lookahead scorecards for new fixed term credit. J Opl Res Soc 52: 989–996.
13. Hanley JA and McNeil BJ (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143: 29–36. | PubMed | ISI | ChemPort |
14. Kalbfleisch JD and Prentice RL (1980). The Statistical Analysis of Failure Time Data. John Wiley & Sons: New York.
15. Kaplan EL and Meier P (1958). Non-parametric estimation from incomplete observations. J Am Stat Assoc 53: 457–481. | Article | ISI |
16. Lawless JF (1981). Statistical Models and Methods for Lifetime Data. John Wiley & Sons: Chichester.
17. Narain B (1992). Survival analysis and the credit granting decision. In: Thomas LC, Crook JN and Edelman DB (eds). Credit Scoring and Credit Control. Oxford University Press, Oxford, pp 109–122.
18. Overstreet GA, Bradley EL and Kemp RS (1992). The flat-maximum effect and generic linear scoring models: a test. IMA J Math Appl Bus Indust 4: 97–109.
19. Overstreet GA and Bradley ELJ (1996). Applicability of generic linear scoring models in the U.S. credit-union environment. IMA J Math Appl Bus Indust 7: 291–311.
20. Platts G and Howe I (1997). A single European scorecard. In: Credit Scoring and Credit Control V, Conference Proceedings. The University of Edinburgh Management School, Edinburgh.
21. Rosenberg E and Gleit A (1994). Quantitative methods in credit management: a survey. Opns Res 42: 589–613.
22. Stepanova M (2001). Using survival analysis methods to build credit scoring models. PhD thesis, University of Southampton.
23. Stepanova M and Thomas LC (2001). PHAB scores: proportional hazards analysis behavioural scores. J Opl Res Soc 52: 1007–1016.
24. Stepanova M and Thomas LC (2002). Survival analysis methods for personal loan data. Opns Res 50: 277–289.
25. Thomas LC (2000). A survey of credit and behavioural scoring: forecasting financial risk of lending to consumers. Int J Forecasting 16: 149–172. | Article |
26. Thomas LC, Edelman DB and Crook JN (2002). Credit Scoring and its Applications. Society for Industrial and Applied Mathematics: Philadelphia, PA.

Acknowledgements

The author is grateful to ESRC for funding the dissemination of this research (PTA-026-27-0216), to Jonathan Crook and Jake Ansell (University of Edinburgh) for their guidance and advice, to the anonymous referee for his/her useful and constructive comments on the paper.