Energy-Based Open-World Uncertainty Modeling for Confidence Calibration release_vp46pyf32ze2fffr67gndz7w5m

by Yezhen Wang, Bo Li, Tong Che, Kaiyang Zhou, Dongsheng Li, Ziwei Liu

Released as a article .

2021  

Abstract

Confidence calibration is of great importance to the reliability of decisions made by machine learning systems. However, discriminative classifiers based on deep neural networks are often criticized for producing overconfident predictions that fail to reflect the true correctness likelihood of classification accuracy. We argue that such an inability to model uncertainty is mainly caused by the closed-world nature in softmax: a model trained by the cross-entropy loss will be forced to classify input into one of K pre-defined categories with high probability. To address this problem, we for the first time propose a novel K+1-way softmax formulation, which incorporates the modeling of open-world uncertainty as the extra dimension. To unify the learning of the original K-way classification task and the extra dimension that models uncertainty, we propose a novel energy-based objective function, and moreover, theoretically prove that optimizing such an objective essentially forces the extra dimension to capture the marginal data distribution. Extensive experiments show that our approach, Energy-based Open-World Softmax (EOW-Softmax), is superior to existing state-of-the-art methods in improving confidence calibration.
In text/plain format

Archived Files and Locations

application/pdf  629.7 kB
file_2ieuh3mzibhhrdxrf676wwyhua
arxiv.org (repository)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  article
Stage   submitted
Date   2021-07-27
Version   v1
Language   en ?
arXiv  2107.12628v1
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: b49ffd8a-6182-4eeb-8f82-395811f296e6
API URL: JSON