Articles

How does reject inference by parcelling work? A worked example

Parcelling turns a bad rate you can see into one you cannot, through a factor nobody can measure. The decision often turns on that factor alone.

Julian R. Sterling · from Machine Learning for Finance

Parcelling infers how rejected applicants would have performed by giving each score band's rejects the bad rate of that band's accepted applicants, multiplied by an uplift factor k for the fact that rejects were refused for a reason. On an illustrative book with k = 1.5, the 3,340 rejects carry 481.3 inferred bads and the bad rate through the door is 7.35 per cent, against 4.27 per cent on the accepted book. The same uplift turns a cut-off extension worth 680,537 into a loss of 1,387,985, and the band breaks even at k = 1.164, a number no lender can observe.

Worked in full in Machine Learning for Finance by Julian R. Sterling, with every figure reproduced in a free workbook.See the book on Amazon →

The assumptions

A lender has 10,980 applications through the door. It accepted 7,640, mostly above its cut-off of 540, plus some below it on manual override, and rejected 3,340, mostly below the cut-off, plus a few above it on policy rules. Only the accepted loans have outcomes. Band counts are illustrative. The unit economics are those of the book's case: 185,000 lent at a margin of 4.12 per cent with 640 of cost, and a loss given default of 41.7 per cent.

The book through the door

Score band	Accepted	Bads	Accepted bad rate	Rejected
Below 500	120	14	11.67%	1,480
500 to 519	260	24	9.23%	940
520 to 539	410	29	7.07%	690
540 to 559	1,350	88	6.52%	150
560 to 599	2,600	105	4.04%	60
600 and above	2,900	66	2.28%	20
Total	7,640	326	4.27%	3,340

Step 1: the break-even bad rate

The figures are per loan over its life, undiscounted. Any group of applicants that defaults below 8.24 per cent is worth approving. The question is what the rejected groups default at.

Step 2: parcel the rejects

Parcelling at k = 1.5

Score band	Reject bad rate	Inferred bads	Through the door	P&L of approving the rejects
Below 500	17.50%	259.0	17.06%	-11,621,293
500 to 519	13.85%	130.2	12.85%	-4,469,671
520 to 539	10.61%	73.2	9.29%	-1,387,985
540 to 559	9.78%	14.7	6.84%	-195,949
560 to 599	6.06%	3.6	4.08%	110,825
600 and above	3.41%	0.7	2.28%	81,765
Total	14.41%	481.3	7.35%

The P&L column is the profit of approving that band's rejects: rejects × (6,982 − inferred rate × 84,767). Through the door, 807.3 bads on 10,980 applications is 7.35 per cent, with an approval rate of 69.6 per cent. The inferred bads are what a model rebuilt on the full population would be trained on, by parcelling them at random into goods and bads, or by weighting each reject both ways as fuzzy augmentation does.

The result: the decision is the factor

On the accepted book alone, k = 1, the 520 to 539 band defaults at 7.07 per cent, comfortably under the break-even, and approving its 690 rejects adds 680,537. That is the argument for lowering the cut-off, and it is built on the most cherry-picked applicants in the book: the ones an underwriter overrode in, because something outside the score looked good. At k = 1.5 the same rejects default at 10.61 per cent and approving them loses 1,387,985.

P&L of approving the rejects, by uplift factor

k	500 to 519	520 to 539	540 to 559	Through-the-door bad rate
1.00	-792,087	680,537	218,467	5.89%
1.25	-2,630,879	-353,724	11,259	6.62%
1.50	-4,469,671	-1,387,985	-195,949	7.35%
2.00	-8,147,255	-3,456,506	-610,366	8.81%
3.00	-15,502,422	-7,593,549	-1,439,199	11.74%

The break-even factor is the break-even bad rate over the accepted rate: 1.164 for 520 to 539, 1.264 for 540 to 559, 0.892 for 500 to 519. Below 520 the rejects lose money even at k = 1. Between 520 and 559 the answer flips between k = 1.0 and k = 1.5, a gap no one can resolve from the accepted book, which means the parcelling has not answered the question; it has moved it into an assumption.

The common mistakes

• Validating on the accepted book and calling it the population. 4.27 per cent is the bad rate of the people the old policy chose. The model will score everyone, and through the door the rate is 7.35 at k = 1.5. How a cut-off goes stale even when the ranking holds is worked in what an unchanged score cut-off costs.
• Using one k for every band. Overrides near the cut-off are selected harder than policy declines far above it, so the uplift should differ by band. A single factor is a convenience, and should be shown as one.
• Treating inferred bads as data. Retraining on them and then reporting the new model's Gini on the augmented sample measures the assumption, not the model.

The cheapest way out is a test cell. Approving 10 per cent of the 520 to 539 rejects, 69 loans, costs 138,798 in expectation at k = 1.5, and then the factor is measured rather than chosen. Against a decision worth 680,537 one way and 1,387,985 the other, that is cheap information.

Takeaway

• Parcelling: reject bad rate = k × accepted bad rate, band by band.
• Here k = 1.5 moves the door's bad rate from 4.27 to 7.35 per cent.
• Solve the break-even k for each band near the cut-off before arguing about the cut-off: 1.164 for 520 to 539.

The accepted, declined and through-the-door bad rates, reconciled case by case, are in the free workbook for this case.

Questions readers ask

What uplift factor should be used in parcelling?

There is no observable answer: the factor describes how much worse rejects are than accepts in the same band, and the bank never sees rejects perform. It is set by judgement, or estimated from bureau data on rejected applicants who borrowed elsewhere. On the illustrative book here, the 520 to 539 band breaks even at a factor of 1.164, so a modest difference of judgement reverses the decision.

What is the difference between parcelling and fuzzy augmentation?

Parcelling assigns each rejected applicant to good or bad at random, in proportion to the band's inferred bad rate. Fuzzy augmentation keeps every reject twice, once as good and once as bad, weighted by the inferred probabilities. Both need the same inferred bad rate per band, which is where the uplift factor enters: at a factor of 1.5 the 3,340 rejects here carry 481.3 inferred bads.

Why does reject inference matter for the cut-off?

Because the bands just below the cut-off are where accepted applicants are most cherry-picked. Here the accepted 520 to 539 band defaults at 7.07 per cent, under the 8.2367 per cent break-even, so lowering the cut-off looks profitable. If the rejects default 1.5 times as often, they run at 10.61 per cent and approving them loses 1,387,985.

Also on this site

• What does leaving a credit score cut-off unchanged actually cost?Brechin's cut-off stayed at 550 points while the optimum moved to 580. One year of that policy cost 2,713,298, on a card nobody retrained.
• How do you calculate weight of evidence and information value?WoE = ln(share of goods / share of bads) per bin; IV sums (share difference x WoE). Five debt service cover bins give an IV of 0.2002, worked from counts.
• How do you calculate Gini and KS for a credit scorecard?Gini = 2 x AUC minus 1, with tied bads counted half; KS is the widest gap in cumulative shares. Ten score bands give 45.61% and 32.47%, worked step by step.
• How do you calculate the population stability index (PSI)?Sum (new share minus old share) x ln(new / old) over fixed score bands: 0.090517 for a card whose bad rate doubled. Under 0.10 is not proof of stability.
• How do you calculate scorecard points from PDO, factor and offset?Factor = PDO / ln 2, offset = anchor score minus factor x ln(odds): 20 points to double the odds and 600 at 50:1 give 28.8539 and 487.1229.
• How do you recalibrate a PD model to a new default rate?Shift every applicant's log-odds by one constant, solved so mean PD hits the new rate: 7.90% to 12.625% takes 0.5623, or 16.22 scorecard points.

Reading guide: corporate finance, valuation and markets → · All 1340 articles →

If this book helped, or didn’t, a few lines on Amazon are worth more than they look: they are what the next reader goes on. Write a review. The workbook stays free either way.