Credit-risk data engineering

Time-based splits, leakage, and class imbalance

Validate on later periods and keep resampling or feature selection inside the training process.

A visual overview of Validate on later periods and keep resampling or feature selection inside the training process. See the guide for assumptions and implementation details.

Purpose

Protect credit-risk performance evaluation from temporal leakage and rare-event distortion.

Key terms

Hover over or focus a term for a quick definition, or open its glossary entry.

Key definitions

Leakage occurs when training information contains outcome-related data unavailable at intended scoring time.
Class imbalance occurs when defaults are much less frequent than non-defaults.

Formula

Chronological split: train on earlier observation dates; validate/test on later dates with enough label follow-up.

Worked example

If a default label runs through December, a January validation start may require excluding overlapping training observations to prevent shared outcome windows.

Detailed explanation

Credit outcomes are temporal. Randomly splitting account-month rows can place the same borrower, nearby periods, or overlapping performance windows in both train and test. Prefer time-based splits aligned to label maturity, and group by obligor when repeated entities could leak.

Rare defaults also make accuracy misleading. Examine ROC and precision-recall metrics, event counts, calibration, and confidence intervals. If using class weights or resampling, fit them inside training folds and recalibrate on a sample reflecting the target population. Freeze preprocessing and feature selection within each training period, and reserve a later period for final evaluation.

Try a reproducible Python example

These examples use synthetic inputs and run locally in a browser worker. Edit the code, run it, and compare its output with the published example.

Runnable Python · local in your browser

Detect a point-in-time feature leak

A miniature data preparation check compares each feature's observation date with the prediction date.

Editable Python example

Output

Run the code to see the output.

Runtime loads only after you press Run. Code runs in a separate browser worker with a 10-second limit and a 20,000-character output cap. Use synthetic examples only; browser execution is not a secure environment for secrets or sensitive records.

Runnable Python · local in your browser

Engineer numeric credit-risk features

Use a training-only median, explicit missingness flags, a log transform, a payment-to-income ratio, and a delinquency indicator. Amounts are synthetic CAD; transforms are illustrative and should be validated for the intended population.

Editable Python example

from math import log1p

training = [
    {"income": 48000, "monthly_payment": 420, "utilization": 0.35, "dpd": 0},
    {"income": 72000, "monthly_payment": 900, "utilization": 0.72, "dpd": 12},
    {"income": 56000, "monthly_payment": 650, "utilization": 0.51, "dpd": 0},
]
scoring = [
    {"account": "A", "income": 60000, "monthly_payment": 750, "utilization": 0.60, "dpd": 0},
    {"account": "B", "income": None, "monthly_payment": 500, "utilization": None, "dpd": 35},
]

# Fit the imputation value on training data only; reuse it unchanged at scoring time.
income_values = sorted(row["income"] for row in training if row["income"] is not None)
training_income_median = income_values[len(income_values) // 2]
utilization_values = sorted(row["utilization"] for row in training if row["utilization"] is not None)
training_utilization_median = utilization_values[len(utilization_values) // 2]

for row in scoring:
    income_missing = row["income"] is None
    utilization_missing = row["utilization"] is None
    income = row["income"] if not income_missing else training_income_median
    utilization = row["utilization"] if not utilization_missing else training_utilization_median
    features = {
        "log_income": round(log1p(income), 4),
        "income_missing": int(income_missing),
        "payment_to_monthly_income": round(row["monthly_payment"] / (income / 12), 4),
        "utilization": utilization,
        "utilization_missing": int(utilization_missing),
        "dpd_30_plus": int(row["dpd"] >= 30),
    }
    print("{}: {}".format(row["account"], features))

Output

Run the code to see the output.

Assumptions

Training and evaluation boundaries respect observation and label-maturity windows.

Limitations

Random splits can overstate performance when adjacent observations share time, borrower, or economic conditions.
Resampling changes class proportions and may distort probability calibration.

References

Metrics and scoring: quantifying the quality of predictions — scikit-learn documentation
Basel Framework: IRB approach and credit-risk components — Basel Committee on Banking Supervision

Educational material only. Methods and examples are simplified; state and validate the assumptions required for your use case.