A pairwise correlation screen catches duplicates. The variance inflation factor catches the combinations that make scorecard coefficients unstable.
The variance inflation factor of a scorecard variable is 1 ÷ (1 − R²), where R² comes from regressing that variable on all the other candidates; for the whole shortlist at once, it is the diagonal of the inverse of their correlation matrix. On an illustrative six-variable shortlist where no pair is correlated above 0.60, total revolving balance still has a VIF of 5.22, because the other five, chiefly utilisation and credit limit, explain 80.9 per cent of it. Drop it and no remaining VIF is above 1.24.
Worked in full in Machine Learning for Finance by Julian R. Sterling, with every figure reproduced in a free workbook.See the book on Amazon →
Multicollinearity does not hurt a logistic regression's predictions much. It hurts the coefficients: their signs, their standard errors and the points they turn into on the scorecard. A validator who sees a variable with the wrong sign, or a points table that moves when one variable is added, will usually find a high VIF behind it. The check takes one matrix inversion.
Six candidates survive the information value screen of an illustrative credit card scorecard. They are tested in the form the model will use them, after the weight of evidence transformation, so the correlations below are between WoE values, not raw values.
| Variable | Util. | Limit | Balance | DTI | MoB | Enq. |
|---|---|---|---|---|---|---|
| Revolving utilisation | 1.00 | -0.10 | 0.60 | 0.35 | -0.15 | 0.25 |
| Total credit limit | -0.10 | 1.00 | 0.60 | 0.20 | 0.30 | 0.05 |
| Total revolving balance | 0.60 | 0.60 | 1.00 | 0.45 | 0.10 | 0.20 |
| Debt-to-income | 0.35 | 0.20 | 0.45 | 1.00 | 0.00 | 0.20 |
| Months on book | -0.15 | 0.30 | 0.10 | 0.00 | 1.00 | -0.20 |
| Enquiries in 6 months | 0.25 | 0.05 | 0.20 | 0.20 | -0.20 | 1.00 |
A pairwise screen with the common cut-off of 0.70 would pass every variable: the largest absolute correlation is 0.60. That is the trap the VIF exists to catch.
VIFj = 1 ÷ (1 − R²j)
All at once: VIFj = [R−1]jj, the j-th diagonal element of the inverse correlation matrix
R²j is the R² of an ordinary regression of variable j on the other candidates. Tolerance is its complement, 1 − R²j. In Excel, with the matrix in B2:G7, select a 6 by 6 block and enter =MINVERSE(B2:G7); the VIFs are on its diagonal, or one at a time with =INDEX(MINVERSE($B$2:$G$7),3,3).
The VIF is computed on the explanatory variables only. The default flag plays no part, which is why it is the same whether the final model is a logistic regression, a probit or a linear probability model.
Total revolving balance is, almost by construction, utilisation times limit. With two regressors the R² has a closed form:
R² = (r1² + r2² − 2 r1 r2 r12) ÷ (1 − r12²)
With r1 = 0.60 (balance and utilisation), r2 = 0.60 (balance and limit) and r12 = -0.10 (utilisation and limit): the numerator is 0.3600 + 0.3600 + 0.0720 = 0.7920, the denominator 1 − 0.0100 = 0.9900, so R² = 0.8000 and VIF = 1 ÷ 0.2 = 5.00.
The negative correlation between utilisation and limit is what pushes the R² up: the two regressors carry different information, and between them they rebuild the balance. Adding debt-to-income, months on book and enquiries lifts the R² a little further, to 0.809, and the VIF to 5.22.
| Variable | VIF | R² on the others | Tolerance | SE multiplier |
|---|---|---|---|---|
| Revolving utilisation | 3.32 | 0.698 | 0.302 | 1.82 |
| Total credit limit | 3.35 | 0.701 | 0.299 | 1.83 |
| Total revolving balance | 5.22 | 0.809 | 0.191 | 2.29 |
| Debt-to-income | 1.29 | 0.223 | 0.777 | 1.13 |
| Months on book | 1.16 | 0.141 | 0.859 | 1.08 |
| Enquiries in 6 months | 1.13 | 0.117 | 0.883 | 1.06 |
The last column is the square root of the VIF: the factor by which the variable's coefficient standard error is larger than it would be if the variable were uncorrelated with the others. That is exact in a linear regression and a close approximation in a logistic one, where the observations carry different weights. Suppose the balance coefficient is 0.250 with a standard error of 0.080 in that ideal case, a z-statistic of 3.12. With a VIF of 5.22 the standard error becomes 0.183 and the z-statistic 1.37: a genuinely predictive variable now looks insignificant, and its sign can flip from one development sample to the next.
Which variable to drop is a judgement, not an output. Removing balance brings every VIF down to between 1.13 and 1.24. Removing utilisation instead also clears the problem, leaving balance at 1.95 and limit at 1.73, but utilisation is usually the stronger and more explainable characteristic. The table tells you where the redundancy lives; information value, business sense and stability decide what stays.
For a pair of variables alone, the VIF is 1 ÷ (1 − r²). It stays small until correlations get high, then explodes:
| Pairwise correlation | VIF | SE multiplier |
|---|---|---|
| 0.50 | 1.33 | 1.15 |
| 0.70 | 1.96 | 1.40 |
| 0.80 | 2.78 | 1.67 |
| 0.90 | 5.26 | 2.29 |
| 0.95 | 10.26 | 3.20 |
A pairwise correlation of 0.70, the usual screening limit, only gives a VIF of 1.96. Our balance variable reaches 5.22 with no pair above 0.60, the same inflation as a single pair correlated at about 0.90. Thresholds in use vary: 10 in older textbooks, 5 in many bank model standards, and 2.5 in some validation teams that want stable points tables. Whatever the policy, state it before the test, not after.
Most development notebooks print a correlation heatmap, drop one of each pair above 0.70, and move on. That catches duplicates, not combinations. Balances, limits and utilisation, or income, debt and debt-to-income, are textbook combinations: each pair looks acceptable, and the three together are nearly deterministic. Only the VIF, or the condition number of the correlation matrix, sees it.
Two more slips. Compute the VIF on the variables in the form they enter the model: raw balances and WoE-transformed balances can have quite different correlations. And recompute it after every change to the shortlist, because dropping one variable changes everyone else's R², as the fall from 3.32 to 1.24 for utilisation shows.
The WoE transformation itself is covered in how to calculate weight of evidence and information value, and what the coefficients become once they are stable in how to calculate PD from a logistic regression.
Invert the correlation matrix of the shortlisted, transformed variables and read the diagonal. Anything above your policy threshold, often 5, is a variable the others can largely rebuild: here, total revolving balance at 5.22, which no pairwise screen flagged. Drop or combine it before fitting, then recheck. The development and validation file that Machine Learning for Finance builds around a credit model that decides is in the free workbooks for the book.
Many bank model standards use 5, older textbooks 10, and some validation teams 2.5. A VIF of 5 means the other variables explain 80 per cent of the variable's variance and its coefficient standard error is about 2.24 times larger than it would be without collinearity. In the illustrative shortlist, revolving balance at 5.22 breaches a limit of 5 even though no pairwise correlation exceeds 0.60.
Rarely by much. The predicted PDs and the Gini barely move, because the correlated variables share the same information. What suffers is the coefficients: with a VIF of 5.22, a coefficient of 0.250 with a standard error of 0.080 sees its standard error rise to 0.183, its z-statistic falls from 3.12 to 1.37, and its sign and scorecard points become unstable across samples.
On the variables in the form they enter the model, which in a classic scorecard means the weight of evidence values. Binning and WoE change the correlations, so a raw-variable VIF can understate or overstate the problem. Recompute after every change to the shortlist: dropping balance in the illustrative case moved utilisation's VIF from 3.32 to 1.24.
This article is one calculation from Machine Learning for Finance. The book takes the same case from first principles to the decision, chapter by chapter, and every figure it prints is a live formula in the free companion workbooks.
Get the book on Amazon →Free companion files
Also on Amazon UK · Amazon Germany · Amazon France · Amazon Canada
Reading guide: corporate finance, valuation and markets → · All 1614 articles →
If this book helped, or didn’t, a few lines on Amazon are worth more than they look: they are what the next reader goes on. Write a review. The workbook stays free either way.