Two tests, not one: bads per parameter for the regression, and bads per bin for the weights of evidence the points are built on.
A credit scorecard needs enough bads for two separate things: enough per parameter for the regression, and enough in every bin for its weight of evidence to mean something. On 6,000 SME applications with 379 bads, the regression side allows about 19 weight-of-evidence characteristics at 20 bads per parameter, but the top bin of debt service cover holds only 14 bads and its weight of 0.9352 carries a 95 per cent interval of 0.41 to 1.46. To hold every bin to a standard error of 0.20, this sample would need about 731 bads, 1.93 times what it has.
Worked in full in Machine Learning for Finance by Julian R. Sterling, with every figure reproduced in a free workbook.See the book on Amazon →
The question is usually asked as a single number, "how many defaults do I need?", and answered with a rule of thumb. The useful answer is two calculations, because a scorecard can fail either way: an overfitted regression, or a points table built on bins too thin to estimate. The second is easy to overlook, because a healthy headline count can hide a thin bin.
The sample is the SME development cohort used in the weight of evidence and information value calculation: 6,000 applications, 5,621 goods and 379 bads, a bad rate of 6.32 per cent. The characteristic is debt service cover in five bins.
| Bin | Goods | Bads | Share of bads | Bad rate |
|---|---|---|---|---|
| Below 1.00 | 893 | 106 | 27.97% | 10.61% |
| 1.00 to 1.24 | 1,007 | 97 | 25.59% | 8.79% |
| 1.25 to 1.49 | 1,204 | 73 | 19.26% | 5.72% |
| 1.50 to 1.99 | 1,988 | 89 | 23.48% | 4.29% |
| 2.00 and above | 529 | 14 | 3.69% | 2.58% |
| Total | 5,621 | 379 | 100% | 6.32% |
The development team is screening 20 candidate characteristics with an average of five bins each.
The classic guide for logistic regression is events per variable: the number of the rarer outcome, here the bads, divided by the number of parameters estimated. Peduzzi and co-authors proposed a floor of 10 in 1996; some practitioners prefer 20 or more because candidate screening, binning choices and stepwise selection all spend degrees of freedom that never appear in the final model.
Maximum parameters = bads ÷ EPV target
379 ÷ 10 = 37.9 parameters; 379 ÷ 20 = 18.9 parameters.
The coding decides how many parameters a characteristic costs:
Count the candidates screened, not the characteristics that survive. A team that tests 20 and keeps eight has still let the data choose among 20. To screen 20 dummy-coded characteristics at 20 bads per parameter it would need 1,600 bads; weight of evidence coding brings that to 400. On this sample, weight of evidence is close to the line and dummy coding is well past it.
Weight of evidence is a log ratio of two shares, and each share is estimated from counts. Its approximate variance follows from the counts alone:
Var(WoE) ≈ 1/g − 1/G + 1/b − 1/B
with g and b the bin's goods and bads, G and B the totals.
Top bin: 1/529 − 1/5,621 + 1/14 − 1/379 = 0.00189 − 0.00018 + 0.07143 − 0.00264 = 0.0705
Standard error = √0.0705 = 0.2655; 95 per cent interval = 0.9352 ± 1.96 × 0.2655 = 0.41 to 1.46.
Excel, goods in C, bads in D, totals in C15 and D15: =SQRT(1/C14-1/$C$15+1/D14-1/$D$15)
The 1/b term dominates: 0.07143 of the 0.0705. The goods hardly matter, because there are always many more of them. The precision of a weight is set by the bads in its bin.
| Bin | Bads | WoE | Std error | 95% interval |
|---|---|---|---|---|
| Below 1.00 | 106 | −0.5656 | 0.0880 | −0.74 to −0.39 |
| 1.00 to 1.24 | 97 | −0.3567 | 0.0921 | −0.54 to −0.18 |
| 1.25 to 1.49 | 73 | 0.1062 | 0.1082 | −0.11 to 0.32 |
| 1.50 to 1.99 | 89 | 0.4095 | 0.0945 | 0.22 to 0.59 |
| 2.00 and above | 14 | 0.9352 | 0.2655 | 0.41 to 1.46 |
Four bins are estimated to within about 0.2 either side. The top bin is not: its interval spans more than a full unit of weight of evidence, and overlaps the bin below it. The gap between the two weights is 0.5257 against an approximate standard error of the difference of 0.2818, a z of 1.87, short of 1.96. On this sample you cannot show that a cover of 2.00 and above is a better risk than 1.50 to 1.99, yet the points table would award it more than twice the weight.
Ignore the two small total terms and write the bin's goods as b × (1 − p) ÷ p at the sample bad rate p. The variance becomes 1 ÷ (b × (1 − p)), so the bads needed for a target standard error are:
Bads per bin = 1 ÷ ((1 − p) × SE²)
At SE 0.20: 1 ÷ (0.9368 × 0.04) = 26.7, so 27 bads.
| Target std error | Bads per bin | 95% half-width | Applications in bin |
|---|---|---|---|
| 0.30 | 12 | ±0.59 | 190 |
| 0.25 | 18 | ±0.49 | 285 |
| 0.20 | 27 | ±0.39 | 427 |
| 0.15 | 48 | ±0.29 | 760 |
| 0.10 | 107 | ±0.20 | 1,694 |
Halving the error needs four times the bads, as every square-root law does. The familiar binning rule of at least 5 per cent of observations per bin says nothing about this: the top bin holds 9.05 per cent of the applications and still only 14 bads, because good bins are, by construction, short of bads.
The bin with the smallest share of the bads sets the requirement for the whole sample. Here it holds 3.69 per cent of the bads.
| Target std error | Total bads | Applications at 6.32% | Multiple of today |
|---|---|---|---|
| 0.25 | 487 | 7,714 | 1.29× |
| 0.20 | 731 | 11,571 | 1.93× |
| 0.15 | 1,299 | 20,571 | 3.43× |
The cheaper fix is usually the binning, not the data. Merge the top bin into 1.50 to 1.99 and the combined bin holds 103 bads, a weight of 0.4994 with a standard error of 0.0854 and an interval of 0.33 to 0.67. The thinnest bin is then 1.25 to 1.49 with 73 bads and a standard error of 0.11, and the existing 379 bads are enough.
Checking only the headline count. "We have 379 bads and the rule says 10 per variable" passes the regression test and says nothing about the points table, which is built bin by bin. A scorecard can have a perfectly stable coefficient on debt service cover and still hand out points on a bin whose weight could be anywhere from 0.41 to 1.46. When that bin later drifts in monitoring, the team investigates a population shift that was never there; it was sampling noise in development.
The book builds the whole scorecard from counts, and the bin tables, weights and checks are in the free workbook for this case. Once the weights are fixed, their translation into points is in scorecard points from PDO, factor and offset.
There is no single minimum. For the regression, a common floor is 10 bads per parameter and some practitioners prefer 20, counting every candidate screened. For the bins, the standard error of a weight of evidence is roughly the square root of one over the bin's bads, so 27 bads give 0.20. On a 6.32 per cent bad rate that is about 427 applications in the thinnest bin.
Use the counts: variance is approximately 1/goods in bin minus 1/total goods plus 1/bads in bin minus 1/total bads, and the standard error is its square root. A bin with 529 goods and 14 bads out of 5,621 and 379 has a variance of 0.0705 and a standard error of 0.2655, so a weight of 0.9352 could plausibly lie anywhere from 0.41 to 1.46.
For the regression, yes: each characteristic costs one parameter instead of one per bin less one. Twenty candidates with five bins cost 80 parameters dummy coded and 20 with weight of evidence, so 379 bads give 4.7 against 18.9 bads per parameter. The weights themselves are still estimated bin by bin, so thin bins remain a problem either way.
This article is one calculation from Machine Learning for Finance. The book takes the same case from first principles to the decision, chapter by chapter, and every figure it prints is a live formula in the free companion workbooks.
Get the book on Amazon →Free companion files
Also on Amazon UK · Amazon Germany · Amazon France · Amazon Canada
Reading guide: corporate finance, valuation and markets → · All 1682 articles →
If this book helped, or didn’t, a few lines on Amazon are worth more than they look: they are what the next reader goes on. Write a review. The workbook stays free either way.