In an independent study released by Capital Services, Sioux Falls SD performed in conjunction with South Dakota State University researchers concluded that enhanced time series information recently introduced by the leading U.S. credit bureaus will significantly improve credit scoring and decision support platforms across all aspects of the consumer credit lifecycle http://www.sdstate.edu/mathstat/upload/A-Credit-Evolution_ASMBI.pdf . As a leading payment portfolio management and servicing company of more than 670,000 accounts from over 150 financial institutions Capital Services invests heavily in the development of proprietary credit scoring systems and continuously seeks new data sources and techniques to improve the performance of their models.
Time Series Credit Characteristics Are an Old Concept, Dollar Amounts Add a Different Perspective
Capital Services’ custom credit models, as most U.S. lenders, rely heavily upon credit characteristics derived from consumer credit files, which are predominately a “single-point-in- time” representation of the consumer’s profile. Prior to the recent availability of time series dollar amounts for credit balances, credit limits and payments there were very few data elements available on a consumer credit report that could yield a time series perspective of a consumer’s credit history. For example, a monthly trend of the number of satisfactory or delinquent tradelines found on a consumer’s credit report could be generated. Because consumers do not regularly open or close many accounts on a monthly basis the time series for a characteristic of this nature is relatively “flat”. As a result, incremental information from a time series based upon the number of satisfactory or delinquent tradelines may be insignificant when compared to a similar static credit characteristic.
Because credit balance and payment amounts reported on a consumer’s credit report can change substantially from one month to the next and may exhibit a trend over several months a dollar amount time series perspective may reveal temporary or long term financial stress experienced by the consumer, which may affect a variety of consumer credit behaviors. It is interesting to note that time series credit characteristics for the number of satisfactory or delinquent tradelines is not found among the thousands of predefined credit characteristic available from each of the “Big Three”.
Champion and Challenger Test Confirms the Predictive Capabilities of Time Series Information
To measure the potential incremental benefit of characteristics based upon the velocity of credit balances, credit limits and payment amounts the researchers performed a comparative analysis. A credit risk model based upon static or point in time characteristics traditionally found in Capital Services’ models (Champion) was compared to a model based upon static characteristics and time series characteristics (Challenger). Logistic regression was used to develop both test models. By limiting the analysis to logistic regression and overlooking the use of other analytic approaches, better suited to leverage the value of time series information, the researchers’ analysis does not fully describe the potential incremental information value this new information may yield. Based upon their review of numerous quantitative measures the researchers concluded that the addition of velocity characteristics within a logistic regression based model provided a significant improvement in the ability to rank order credit risk within the population studied.
An Example as to How Velocity is Important
“Understanding the historical trend of a particular credit characteristic provides important additional insight”, explained Alfred Furth, Chief Data Scientist with Capital Card Services. “Models based upon traditional static credit characteristics treat consumers with the same static characteristic identically, even though they may have exhibited different historical behaviors.” he added.
In the example below two hypothetical consumers have a current Bankcard utilization rate of .5 at time period 0. If all other credit characteristics for these two consumers were identical then the points assigned to each consumer’ credit score would be identical. However, if the historical trend for each consumer is headed is a different direction then the points assigned to each consumer, based upon this trend, and may be radically different.
Utilization Velocity Comparison
Based on the research sample analyzed, consumers with increasing bankcard utilization were riskier than consumers with decreasing bankcard utilization. For example, based upon the table below, the Challenger model scored consumer ID1 (negative bankcard utilization velocity) higher than consumer ID2 (positive bankcard utilization velocity).
Bank Card Utilization Rates
Both research models developed by the researchers included static bankcard utilization characteristic. The Champion model placed more weight on bankcard utilization than the Challenger model; however the Challenger model placed significantly more emphasis on a variety of time series utilization credit characteristics.
Promising Test Results Leads to High Expectations
When comparing test results, the Challenger model showed superior performance compared to the Champion model. The Challenger model demonstrated better KS, AUC, and GINI metrics than the Champion model on the training and hold out validation samples. On a more practical level, the researchers compared performance distributions within top and bottom deciles. Within the highest scoring decile (lowest credit risk) the interval bad rate for the Challenger model was 43% lower within the training set and 22.6% lower within the hold out validation set. Within the lowest scoring decile (highest credit risk) the interval bad rate for the Challenger model was 21.1% higher within the training set and 36.9% higher within the hold out validation set.
Although the results from this comparative analysis were impressive, expectations for even stronger results are high. As Alfred Furth explained, “Our highly focused customer base is much different from the general population. When we migrated our credit bureau based risk models from the standard characteristics available from the credit repositories to custom characteristics focusing on the unique profile and credit experiences of our customer base we witnessed a significant improvement in our models. We expect to witness similar, if not better, improvement by migrating to custom time series credit characteristics and basing our models on analytic techniques better suited for time series information.
The availability of velocity and volatility credit information should significantly improve credit behavior models developed by users and offered by credit reporting agencies. Model improvement is just one of several potential benefits time series information may provide. The potential for a wide variety of new solutions focused on data and new information delivery platforms, which have yet to be developed, can permanently alter the credit bureau landscape. Future articles will discuss some of the potential products and services concepts that time series data may facilitate.
About the Author: Chet Wiermanski is one of BIIA’s contributing editors writing on the subjects of credit scoring and decision systems. He is a Visiting Scholar at the Federal Reserve Bank of Philadelphia researching new applications of consumer credit report information. Additionally, Chet is Managing Director of Aether Analytics which specializes on leveraging hidden data sequences and time series components within consumer credit information typically ignored by traditional credit bureau based solutions. Previously Chet was the Global Chief Scientist at TransUnion LLC. Holding a variety of positions within TransUnion, during his tenure, between July 1997 and February 2012, he was responsible for identifying, evaluating and developing new technology platforms involving alternative data sources, predictive modeling, econometric forecasting and related consulting services.
