PD Segmentation (2/3): Monitoring Calibration, Drift, and Migration

A segmentation remains useful only while its assignment rules, data, and estimated risk stay fit for their intended use. Monitoring should answer three separate questions: are accounts still assigned correctly; has the population changed; and do matured outcomes remain consistent with the PDs? Looking only at portfolioA defined group of credit exposures analyzed together under stated inclusion rules.Open full glossary entry ↗-wide defaultA defined credit event used for measurement; the exact triggers depend on the applicable accounting, regulatory, or internal policy.Open full glossary entry ↗ rate can hide a failing segment behind a stable average.

Start by fixing the unit (obligorA borrower or counterparty whose credit quality or default risk is assessed.Open full glossary entry ↗ or facilityA contractual credit arrangement, such as a loan, card account, or committed line.Open full glossary entry ↗), default definition, forecast horizon, reporting date, population perimeter, and outcome-maturity rule. A one-year forecast should be compared with a fully observed one-year outcome window, or explicitly marked immature/censored. Do not treat loans with incomplete follow-up as non-defaults. If repeated observations are used, define weights and prevent repeated borrowers from distorting counts.

1. Data and assignment controls

By segment and period, report record counts, unique obligors, exposure, missing/unmapped rates, invalid PD values, category changes, assignment failures, and rule/model version. Reconcile segment totals to the source portfolio. Track overrides, manual movement, and changes in score cutoffs.

2. Population mix

Monitor each segment’s share by obligor and exposure, plus product, geography, vintage, channel, rating, and key feature distributions. A simple segment-share shift is current share minus reference share. PSI can summarize changes across fixed bins: PSI = Σ (current share − reference share) × ln(current share / reference share). Handle zero shares with a documented smoothing convention. PSI is a screening statistic: it does not prove calibrationAgreement between predicted probabilities and observed event frequencies for a matched population and horizon.Open full glossary entry ↗ deterioration and has no universal alert cutoff. Keep bins and reference period fixed for comparability.

3. Migration and transitions

Count movements between grades or pools from one date to the next. Report upgrades, downgrades, exits, new originations, and changes caused by model or policy versions separately. A transition matrixA matrix of probabilities that a borrower moves between rating, delinquency, or default states over a defined interval.Open full glossary entry ↗ can reveal drift or operational churn that a one-date mix misses. When comparing cohorts, distinguish migration of existing obligors from changing origination mix.

4. Calibration by segment

For a matured cohort of n accounts, observed default rate (ODR) = D / n. Expected defaults = Σ PDᵢ. Observed-to-expected (O/E)A ratio of realized events to the sum of predicted event probabilities for a matched sample and horizon.Open full glossary entry ↗ = D / Σ PDᵢ. O/E above one means observed defaults exceed the model’s expected count in that sample; below one means fewer defaults occurred. Always display D, n, expected count, horizon, and the ODR alongside the ratio. Small denominators make O/E volatile.

Compare observed rate with mean predicted PD by segment and by relevant subgroups such as vintage, product, geography, or score band. A calibration plot and confidence interval are more informative than a single pass/fail flag. Check ranking separately: AUC or rank-order tests can show whether higher-risk groups rank worse, but a good ranking does not establish accurate PD levels. Conversely, recalibration can correct levels without fixing poor ordering.

5. Stability and risk context

Compare changes with economic conditions, underwriting policy, collections, portfolio growth, and data capture. For PIT PD, changes may be expected as current conditions change. For TTC PD, cyclical changes may be dampened by design, but the score distribution, long-run calibration, and grade migration still need review. State the model concept before interpreting movement.

A fixed segment has 400 matured one-year accounts, 16 observed defaults, and a sum of forecasts of 12 expected defaults. ODR = 16/400 = 4%; O/E = 16/12 = 1.33. A Wilson 95% interval for the observed rate is approximately 2.5% to 6.4%. Because the forecasted 3% lies within that broad interval, the sample alone gives limited evidence of a statistically clear difference. It is a monitoring signal: check adjacent periods, segment mix, economic context, and the calibration sample before deciding whether action is needed. This example is synthetic and the interval is an approximate binomial interval; it does not resolve dependence, weighting, or portfolio-design issues.

A practical table might show: Period | Segment | Matured n | Defaults | Mean PD | ODR | O/E | ODR interval | exposure share | unmapped %. Include an overall row and stable comparison periods. If the segment was created or redefined mid-period, do not present the resulting discontinuity as pure risk drift.

Set investigation triggers through model purpose, materiality, default frequency, exposure, and risk appetiteThe amount and types of risk an organization is prepared to accept while pursuing its objectives.Open full glossary entry ↗. Do not adopt a borrowed PSI, O/E, or minimum-default cutoff as if it were universal. When a trigger fires: first verify labels, maturity, joins, population, weights, and the version of segment rules; then identify which features and subgroups drove the change; compare observed and predicted outcomes with uncertainty; consider benchmarking or a challenger; and document the proposed action, owner, approval, and follow-up date.

Possible responses include correcting a data pipeline, reviewing a mapping, recalibrating PD levels, revising a segment boundary, gathering more evidence, applying an approved override, or restricting use. A metric crossing a threshold is an investigation trigger, not an automatic instruction to alter a model. Assess customer and business effects before changing eligibility, pricing, or credit decisions.

An internal monitoring report, an IFRS 9The International Financial Reporting Standard for financial instruments, including classification, measurement, hedge accounting, and impairment requirements.Open full glossary entry ↗ ECL estimate, and Basel IRB PD validation can share data but have different objectives and rules. IFRS 9 requires accounting measurement under the reporting entity’s applicable standards and forward-looking assumptions; Basel IRB requirements apply to banks and exposures in their prudential scope; internal monitoring thresholds should match their approved use. A one-year observed default rate is not by itself a lifetime ECLExpected credit losses resulting from all possible default events over the expected life of the financial instrument under applicable requirements.Open full glossary entry ↗ backtest or a regulatory capital result.

Basel Framework, CRE 36 — IRB approach minimum requirements (rating assignment, validation, PD estimation; applicability is limited to its prudential scope): https://www.bis.org/committees/bcbs/basel-framework/standard/cre/36/inforce/2023-01-01/published/2020-03-27

Part 3 shows how to calculate segment-level calibration metrics in Python and test assignment, ordering, and time-based performance using synthetic outcomes.

Educational guide only. Use matured, correctly defined outcomes and portfolio-specific governance. The example is synthetic and is not regulatory advice.