Leaving a score cut-off where the development sample put it cost Brechin 2,713,298 in 2020 and 2021. The optimum had moved from 550 points to 580. At 580 the book earns 2,563,882; at the inherited 550 it loses 149,416. The card was not retrained, no coefficient moved, and its ranking held: area under the curve fell only 0.016816. The price of risk moved instead, and the threshold did not follow it.
The same grid, one year later
Brechin Credit's threshold was chosen on 6,000 applications decided between 2015 and 2019, where profit peaked at 550 points. The identical card — no coefficient touched, no bin boundary moved, no points table rewritten — was then run over the 2,400 applications of 2020 and 2021. Every threshold from 480 to 550 loses money. The peak has moved to 580.
Out-of-time profit at each cut-off, on the 2,400 applications of 2020 and 2021.
Cut-off
Approved, per cent
Bad rate, per cent
Profit
540
86.54
9.58
−2,367,019
550
76.00
8.33
−149,416
560
63.29
6.98
1,620,356
570
48.17
5.80
2,391,803
580
32.50
4.36
2,563,882
590
19.17
3.70
1,770,681
At the inherited 550 the book approves 76.00 per cent and loses 149,416. At 580 it approves 32.50 per cent and earns 2,563,882, or 1,068.28 per application. The gap between the policy that was running and the policy that was available is 2,713,298.
Where the 2,713,298 sits
It sits in one slab of applicants. Between 550 and 580 there are 1,044 applications, of which 118 defaulted and 926 performed. Two prices settle the rest. An accepted case that performs is worth 6,982, being 185,000 of exposure at a margin of 4.12 per cent less 640 of origination cost. One that defaults costs 84,767 more than that.
The slab of business the inherited cut-off wrote and the moved cut-off would have declined.
Slab from 550 to 580 points
Amount
Applications accepted
1,044
Value at 6,982 each
7,289,208
Defaults
118
Cost at 84,767 each
10,002,506
Value of writing the slab
−2,713,298
The ratio is the form worth carrying. That slab delivers 926 performing loans for 118 defaults, or 7.8475 to one. Brechin needs 84,767 divided by 6,982, or 12.1408 performing loans for every default, to break even — equivalently, a pool is worth writing while it defaults at less than 8.2367 per cent. In the development years the equivalent slab, from 600 down to 550, ran at 19.9529 to one and was worth 11,751,467. Same lender, same card, same four prices, and the marginal business went from paying nearly twenty to one to paying under eight.
The band between 550 and 560 defaulted at 7.1672 per cent in development and at 15.0820 per cent out of time, 2.1043 times as high. Identical points, identical bins, identical model. The applicants sitting between 550 and 560 in 2021 were not the applicants who sat there in 2017.
The card did not break
The ranking held. Area under the curve falls from 0.735329 to 0.718513, a loss of 0.016816; Gini falls from 0.470659 to 0.437027 and KS from 0.341154 to 0.324311. The bad rate column of the new grid falls without a single reversal in seventeen rows. The card sorts the new applications into the right order.
The decision moved much further than the measurement did. The cut-off moved thirty points. On a scale where twenty points doubles the odds, thirty points is one and a half doublings, a factor of 2.8284, so the odds demanded of a marginal applicant nearly trebled while the model's ability to sort them barely changed. Part of the move is a level error rather than a ranking error: correcting the intercept by 0.573620 in log-odds moves every applicant 16.55 points, so a policy written as decline below 550 has to be rewritten as 566.55 merely to mean what it meant. That accounts for 16.55 of the thirty. The remaining 13.45 points is what a uniform shift cannot fix, because the gaps are not uniform — 9.612 points in the worst band and far less at the top, so the deterioration is concentrated exactly where the lending decision is made.
The population moved as well as the level. The stability index between the two samples is 0.090517, inside most institutions' amber threshold. At a fixed 550 the approval rate falls on its own from 83.50 per cent to 76.00 per cent, because a fixed line declines more people when more people are below it. That is 7.50 points of automatic tightening. The optimum required 51.00 points of it.
What to do with it
Ask three things of any cut-off in front of a committee: what population it was optimised on, over what period, and what it earns on the most recent population available. Here those questions produce 17,348,284 and a loss, from a card nobody touched.
Publish the observed bad rate of the slab immediately below the live cut-off, on matured cohorts, against the break-even rate of 8.2367 per cent. On this portfolio that slab went from 7.1672 to 15.0820 per cent, which no reasonable trigger would have missed.
State the lag rather than assuming it away. Bad means ninety or more days past due within eighteen months, so 2020 lending could not be judged on outcomes until 2022. A policy that reacts only to matured outcomes reacts a year and a half late.
Carry that interval on leading indicators — approval rate at a fixed cut-off, score distribution on development bands, population stability, early arrears. They are weaker evidence and they are the only evidence available in time.
Then price the proposal against the alternatives on either side of it, in money, over the whole grid rather than at the single row being recommended. Out of time, 550 against 580 is 2,713,298. A threshold is not a property of a model. It is a property of a model and a year.
The workbooks behind this article
Every figure above is a live formula in the free companion files for
Machine Learning for Finance. Each workbook ends with a Checks sheet
setting the printed figure beside the computed one. No account and no email address.
At what wealth does a family office pay?A five-person office costs the same $2,442,000 at $100m as at $2bn. Against a tiered multi-family schedule the two curves cross exactly once.
Does an undrawn revolver count as liquidity?67.4567 per cent of the headroom, and it leaves at a fall in annual revenue of 8.4127 per cent — not the 13.3333 per cent the board deck implies.
Does supply chain finance count as debt?66,049,315 released, and leverage of 2.7622× or 3.1362× depending on which balance the test is asked of — the second is a breach at today's EBITDA.
Does the fulcrum move with enterprise value?A worked capital structure at three enterprise values. The break migrates from the first lien to the subordinated notes without a single document changing.
How many companies does a reserve pool defend?Holding pro rata to the Series D costs $14.76m per company. A $40m reserve defends 2.71 of forty — and 61 per cent of the cost is the last round.
How much room a 1.15x downside actually leaves79 basis points of margin. The leverage covenant fails first, five basis points before the coverage covenant the chapter reports on, and 4.6 per cent of EBITDA is the whole cushion.
How the GP catch-up is actually solvedMost write-ups state the catch-up formula then hardcode the answer. Set it as a solve and it moves on its own when the carry rate does.
How do you set a minimum cash balance?Three components, 124,596,281 in total — and the double count that turns a margin of 27,403,719 into an apparent shortfall of -3,988,281.
Is a 2/10 net 60 discount worth taking?An annualised 14.8980 per cent against a revolver at 4.50 per cent — worth 5,471,890 a year, and 0.3132 turns of reported leverage.
Should corporate debt be fixed or floating?78.1818 per cent of gross debt fixed, and a net floating exposure after cash of -608,000.00 — the proportion, not the instrument.
Six ways to state the same fund's return7.12 per cent, 18.04 per cent, or 1.323 times the public market — one fund, one set of flows. The eleven-point gap is the valuation, and it is not cash.
The IRR at which a closed-ended fund merely tiesA drawdown fund advertising 13.25 per cent ties an evergreen netting 8.64. On committed rather than called capital, 78.7 per cent of the edge vanishes.
The month a remediation trigger has to be armed byMonth 8, on a one-year window and a 14-week hiring lead time. The date is 12W − L/4.33 and holds for any portfolio size — the book only changes the euros.
Two routes to the same NAVFive findings in one quarter-end close, and not one of them was a modelling error. Every one was a control that did not exist.
What a CBAM certificate buffer costs to holdCarry and protection are both linear in the buffer, so the break-even probability is identical at 5 per cent and at 30 — and 2027 costs exactly twice 2026.
What a clawback actually collateralises24 million recovers 13.20 net of tax and 7.20 is escrowed. The rule is e ≥ 1 − t, and every euro won on the tax clause lands uncollateralised.
What a co-investment programme actually saves37 basis points, and a 22.8 per cent total-loss rate erases it. At the usual per-deal cap a single failure is 37 per cent of the annual cohort.
What a diligence exercise actually recoversEighteen basis points is only the part that reaches the yield. Three of the four instruments never do, and the exercise is worth thirty-one.
What a follow-on reserve is actually worth0.019 turns under plain dilution, 0.285 under pay-to-play. The optimum is a property of the shareholders’ agreement, not of the portfolio.
What a lease mark-to-market is actually worth5,788,000 becomes 807,365 once the over-market stream, the expiry dates and the cost of re-letting are priced — and nothing at all below a 43.7 per cent renewal rate.
What a minimum payment actually does£81.53 leaves the account and £20.46 reaches the debt. On minimums alone the set clears in 258 months and repays 1.78 times what was borrowed.
What a month of delay costs in an enforcement463,000 euros of present value a month. No single error about enforcement, and no pair of errors, moves the indifference point from 58.7 cents to the 72 on the table.
What a pro rata cheque actually costsDefending costs 1.25 times the initial cheque across two rounds, not more in each — and below a 521 exit the follow-on dollars come back worth less than they cost.
What a subscription line does to the IRR20 per cent becomes 38.41 on the same asset while the multiple falls. Two dollars of interest buys 18.4 points, and the longer the line runs the cheaper each point gets.
What one point of fees actually costsOne point on the annual charge turns 738,846 into 567,961 over forty years — 23 per cent of the pot, and 2.43 of wealth lost per 1 of fee.
When the promote stops clearingCarried interest is not proportional to performance near the hurdle. It is a step function, and $2m of headroom is what stands between a promote and none.
Why a secondaries bid-ask gap has no zoneThe seller’s floor and the buyer’s ceiling are one formula at two rates. At equal required returns the zone is exactly zero — here it is 11.02 points apart.
Why refining a due diligence rubric makes it weakerThree of twelve managers carry a terminal flaw and the average approves all three. Splitting eight domains into sixteen makes it easier to hide, not harder.