Trust Issues: Uncertainty Estimation Does Not Enable Reliable OOD Detection On Medical Tabular Data release_mgukmgna6re7folppuxzg7g35u

by Dennis Ulmer, Lotta Meijerink, Giovanni Cinà

Released as a article .

2020  

Abstract

When deploying machine learning models in high-stakes real-world environments such as health care, it is crucial to accurately assess the uncertainty concerning a model's prediction on abnormal inputs. However, there is a scarcity of literature analyzing this problem on medical data, especially on mixed-type tabular data such as Electronic Health Records. We close this gap by presenting a series of tests including a large variety of contemporary uncertainty estimation techniques, in order to determine whether they are able to identify out-of-distribution (OOD) patients. In contrast to previous work, we design tests on realistic and clinically relevant OOD groups, and run experiments on real-world medical data. We find that almost all techniques fail to achieve convincing results, partly disagreeing with earlier findings.
In text/plain format

Archived Files and Locations

application/pdf  4.1 MB
file_2nnny6j4vffw7iowf2cceglvoi
arxiv.org (repository)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  article
Stage   submitted
Date   2020-11-06
Version   v1
Language   en ?
arXiv  2011.03274v1
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: c99a2cfc-c17f-4731-8267-1e6dc7e561a9
API URL: JSON