Calibration API¶

Post-hoc calibration methods that adjust a trained model's confidence to match observed accuracy.

Public objects¶

TemperatureScaling
VectorScaling
IsotonicCalibration

deepuq.methods.calibration ¶

Post-hoc calibration methods for classification models.

IsotonicCalibration ¶

Per-class isotonic regression calibration.

Fits an isotonic regression model per class on validation set predictions, mapping raw softmax probabilities to calibrated probabilities.

Parameters:

Name	Type	Description	Default
`model`	`Module`	A PyTorch model that outputs raw logits.	required

fit ¶

fit(val_loader: DataLoader) -> IsotonicCalibration

Fit isotonic regression per class on validation data.

Parameters:

Name	Type	Description	Default
`val_loader`	`DataLoader`	DataLoader yielding (inputs, targets) batches.	required

Returns:

Type	Description
`self`

predict_calibrated ¶

predict_calibrated(x: Tensor) -> torch.Tensor

Return calibrated probabilities for input x.

Parameters:

Name	Type	Description	Default
`x`	`Tensor`	Input tensor.	required

Returns:

Type	Description
`Calibrated class probabilities (normalized to sum to 1).`

predict_uq ¶

predict_uq(x: Tensor) -> UQResult

Return a UQResult with calibrated probabilities and entropy-based uncertainty.

TemperatureScaling ¶

Bases: Module

Post-hoc temperature scaling for classification models.

Learns a single scalar T that divides logits before softmax, minimizing NLL on a held-out validation set.

Parameters:

Name	Type	Description	Default
`model`	`Module`	A PyTorch model that outputs raw logits.	required

fit ¶

fit(
    val_loader: DataLoader,
    max_iter: int = 100,
    lr: float = 0.01,
) -> TemperatureScaling

Optimize temperature on a validation set using LBFGS.

Parameters:

Name	Type	Description	Default
`val_loader`	`DataLoader`	DataLoader yielding (inputs, targets) batches.	required
`max_iter`	`int`	Maximum LBFGS iterations.	`100`
`lr`	`float`	Learning rate for the optimizer.	`0.01`

Returns:

Type	Description
`self`

predict_calibrated ¶

predict_calibrated(x: Tensor) -> torch.Tensor

Return calibrated probabilities for input x.

Parameters:

Name	Type	Description	Default
`x`	`Tensor`	Input tensor.	required

Returns:

Type	Description
`Calibrated class probabilities.`

predict_uq ¶

predict_uq(x: Tensor) -> UQResult

Return a UQResult with calibrated probabilities and entropy-based uncertainty.

Parameters:

Name	Type	Description	Default
`x`	`Tensor`	Input tensor.	required

Returns:

Type	Description
`UQResult with mean=calibrated probs, epistemic_var=predictive entropy.`

VectorScaling ¶

Bases: Module

Per-class temperature and bias calibration.

Learns a vector of temperatures W and biases b such that calibrated logits = logits * W + b.

Parameters:

Name	Type	Description	Default
`model`	`Module`	A PyTorch model that outputs raw logits.	required

fit ¶

fit(
    val_loader: DataLoader,
    max_iter: int = 100,
    lr: float = 0.01,
) -> VectorScaling

Optimize per-class parameters on a validation set using LBFGS.

Parameters:

Name	Type	Description	Default
`val_loader`	`DataLoader`	DataLoader yielding (inputs, targets) batches.	required
`max_iter`	`int`	Maximum LBFGS iterations.	`100`
`lr`	`float`	Learning rate for the optimizer.	`0.01`

Returns:

Type	Description
`self`

predict_calibrated ¶

predict_calibrated(x: Tensor) -> torch.Tensor

Return calibrated probabilities for input x.

predict_uq ¶

predict_uq(x: Tensor) -> UQResult

Return a UQResult with calibrated probabilities and entropy-based uncertainty.