Abstract:

A deployed model does not come with labels, so its quality cannot be measured where it matters most. The usual substitutes — distances between input distributions, shifts in the predicted distribution, the model's own confidence — are easy to compute but need not track the quantity of interest. We ask whether a different signal does: the change in where in the input the model takes its evidence from.

For a frozen gradient-boosted detector of mineral-dust events we define a sensitivity profile, the share of the model's output variation attributable to each of twelve spectral blocks, measured by permuting a block and recording the mean absolute change in predicted probability. The profile requires no labels. The monitoring signal is the distance between the profile of a deployment batch and the profile of the training period.