Fast Predictive Uncertainty for Classification with Bayesian Deep Networks
release_xuzva4pzjrcgfj5zsmyorfx4hy
by
Marius Hobbhahn, Agustinus Kristiadi, Philipp Hennig
2022
Abstract
In Bayesian Deep Learning, distributions over the output of classification
neural networks are often approximated by first constructing a Gaussian
distribution over the weights, then sampling from it to receive a distribution
over the softmax outputs. This is costly. We reconsider old work (Laplace
Bridge) to construct a Dirichlet approximation of this softmax output
distribution, which yields an analytic map between Gaussian distributions in
logit space and Dirichlet distributions (the conjugate prior to the Categorical
distribution) in the output space. Importantly, the vanilla Laplace Bridge
comes with certain limitations. We analyze those and suggest a simple solution
that compares favorably to other commonly used estimates of the
softmax-Gaussian integral. We demonstrate that the resulting Dirichlet
distribution has multiple advantages, in particular, more efficient computation
of the uncertainty estimate and scaling to large datasets and networks like
ImageNet and DenseNet. We further demonstrate the usefulness of this Dirichlet
approximation by using it to construct a lightweight uncertainty-aware output
ranking for ImageNet.
In text/plain
format
Archived Files and Locations
application/pdf 2.0 MB
file_3v4kwforivgwvi4xwcqjsxeehy
|
arxiv.org (repository) web.archive.org (webarchive) |
2003.01227v4
access all versions, variants, and formats of this works (eg, pre-prints)