The Kupiec test only counts exceptions; the Christoffersen test asks whether they arrive together, and on the same count it can turn a green model into a rejected one.
Classify every day of the backtest as an exception or not, count the four possible transitions from one day to the next, and compare the probability of an exception after an exception with the probability after a clean day. On 500 days of a 99 per cent VaR with six exceptions, a spread pattern and one with five exceptions in a single week give the same Kupiec result, an LR of 0.19 and a pass, but the clustered pattern gives an independence LR of 31.32 against a 3.841 critical value, a p-value of 0.00000002. Same count, opposite verdict.
Worked in full in Financial Risk Management by Julian R. Sterling, with every figure reproduced in a free workbook.See the book on Amazon →
The fund and its two exception histories are illustrative. Both histories have exactly the same number of exceptions; only the dates differ.
| Input | Pattern A: spread | Pattern B: clustered |
|---|---|---|
| Observation days | 500 | 500 |
| Confidence level | 99.00% | 99.00% |
| Expected exceptions (1.00% of 500) | 5.00 | 5.00 |
| Observed exceptions | 6 | 6 |
| Exception days | 41, 118, 203, 287, 362, 449 | 97, then 311 to 315 |
The Kupiec proportion-of-failures test compares the observed exception rate, 6 / 500 = 1.20 per cent, with the promised 1.00 per cent:
LRpof = −2 ln[ (1 − p)n−x px / ((1 − x/n)n−x (x/n)x) ] = 0.19
Read against a chi-squared with one degree of freedom: critical value 3.841 at 5 per cent, p-value 0.66.
Both histories are green. The test never sees the dates, so it cannot tell them apart. This is where most backtest reports stop, and where the traffic-light zones stop too.
Turn the 500 days into a string of zeros and ones and look at each of the 499 pairs of consecutive days. Each pair is one of four types: n00 (clean, then clean), n01 (clean, then exception), n10 (exception, then clean) and n11 (exception, then exception).
| Count | Pattern A | Pattern B |
|---|---|---|
| n00 | 487 | 491 |
| n01 | 6 | 2 |
| n10 | 6 | 2 |
| n11 | 0 | 4 |
| π01 = n01 / (n00 + n01) | 1.22% | 0.41% |
| π11 = n11 / (n10 + n11) | 0.00% | 66.67% |
| π = (n01 + n11) / 499 | 1.20% | 1.20% |
In pattern B, once an exception has happened the next day is an exception two times in three. After a clean day the chance is 0.41 per cent. If exceptions were independent, those two probabilities would be the same, and both would be close to π.
L0 = (1 − π)n00+n10 πn01+n11 (one probability for every day)
L1 = (1 − π01)n00 π01n01 (1 − π11)n10 π11n11 (a probability that depends on yesterday)
LRind = −2 ln(L0 / L1), chi-squared with one degree of freedom
In Excel, with the counts in B2:B5 and the logs taken term by term to avoid underflow, the p-value is =CHISQ.DIST.RT(LR, 1).
Take logs before multiplying: (0.9880)493 is a small number, and spreadsheets lose precision long before they show an error. Where n11 is zero, the term π110 is 1, not an error.
| Statistic | Pattern A | Pattern B | 5% critical value |
|---|---|---|---|
| Kupiec LRpof | 0.19 | 0.19 | 3.841 |
| Christoffersen LRind | 0.15 | 31.32 | 3.841 |
| p-value, independence | 0.70 | 0.00000002 | |
| Conditional coverage LRcc = LRpof + LRind | 0.34 | 31.51 | 5.991 |
Pattern A passes everything. Pattern B passes the count and fails independence by a factor of eight over the critical value. The conditional coverage test, which reads the sum against a chi-squared with two degrees of freedom, rejects it with a p-value of 0.00000014. A correct 99 per cent model would put five exceptions into one given week with a probability of one in 10,000,000,000.
Keep six exceptions in 500 days and move them together one at a time: k is the number of back-to-back pairs, built as one run of k + 1 days with the remaining exceptions isolated.
| Back-to-back pairs (n11) | π11 | LRind | p-value | LRcc | p-value |
|---|---|---|---|---|---|
| 0 | 0.00% | 0.15 | 0.702 | 0.34 | 0.845 |
| 1 | 16.67% | 3.71 | 0.054 | 3.90 | 0.142 |
| 2 | 33.33% | 10.86 | 0.001 | 11.05 | 0.004 |
| 3 | 50.00% | 20.07 | 0.000 | 20.26 | 0.000 |
| 4 (pattern B) | 66.67% | 31.32 | 0.000 | 31.51 | 0.000 |
| 5 | 83.33% | 45.17 | 0.000 | 45.36 | 0.000 |
One back-to-back pair sits just inside the 5 per cent line, at a p-value of 0.054. A run of three exception days is already a rejection at 0.001. The test is sensitive because exceptions are rare: with π at 1.20 per cent, two in a row should almost never happen.
With few exceptions the chi-squared approximation is rough, and the conditional coverage test dilutes a strong independence signal with a weak count signal. Report the independence statistic on its own, and beside the total count carry the largest number of exceptions in any five-day window.
Count the transitions, compute π01 and π11, and take the log-likelihood ratio. Six exceptions in 500 days pass the count test whether they are spread across two years or packed into one week, but the second pattern gives an independence statistic of 31.32 and a p-value of 0.00000002. The exception counter, with the zones drawn by the binomial, is in the free workbooks for this book, and the VaR and expected shortfall template produces the daily series to test.
Kupiec tests the number of exceptions: six in 500 days against five expected gives an LR of 0.19 and a p-value of 0.66, a pass. Christoffersen tests their timing, comparing the chance of an exception after an exception with the chance after a clean day. Combined, the conditional coverage statistic is the sum of the two, read against a chi-squared with two degrees of freedom and a 5.991 critical value.
On 500 days with six exceptions at 99 per cent, one back-to-back pair gives an independence LR of 3.71 and a p-value of 0.054, just inside the 5 per cent line. Two pairs, a run of three days, give 10.86 and a p-value of 0.001. In practice, any run of three exception days in a year or two of data should be treated as a rejection.
Because clustered losses arrive while the position is still on and the funding is under pressure. Five exceptions in a week on a correct 99 per cent model happen with a probability of one in 10,000,000,000. A model that misses volatility regimes will pass a count test over 500 days and still fail in exactly the week that matters.
Clustered exceptions are the subject of the case the companion files attach to chapter 6 of Financial Risk Management. The book takes the same case from first principles to the decision, chapter by chapter, and every figure it prints is a live formula in the free companion workbooks.
Get the book on Amazon →Free companion files
Also on Amazon UK · Amazon Germany · Amazon France · Amazon Canada
Reading guide: corporate finance, valuation and markets → · All 1340 articles →
If this book helped, or didn’t, a few lines on Amazon are worth more than they look: they are what the next reader goes on. Write a review. The workbook stays free either way.