Trust Issues: Uncertainty Estimation Does Not Enable Reliable OOD Detection On Medical Tabular Data
release_mgukmgna6re7folppuxzg7g35u
by
Dennis Ulmer, Lotta Meijerink, Giovanni Cinà
2020
Abstract
When deploying machine learning models in high-stakes real-world environments
such as health care, it is crucial to accurately assess the uncertainty
concerning a model's prediction on abnormal inputs. However, there is a
scarcity of literature analyzing this problem on medical data, especially on
mixed-type tabular data such as Electronic Health Records. We close this gap by
presenting a series of tests including a large variety of contemporary
uncertainty estimation techniques, in order to determine whether they are able
to identify out-of-distribution (OOD) patients. In contrast to previous work,
we design tests on realistic and clinically relevant OOD groups, and run
experiments on real-world medical data. We find that almost all techniques fail
to achieve convincing results, partly disagreeing with earlier findings.
In text/plain
format
Archived Files and Locations
application/pdf 4.1 MB
file_2nnny6j4vffw7iowf2cceglvoi
|
arxiv.org (repository) web.archive.org (webarchive) |
2011.03274v1
access all versions, variants, and formats of this works (eg, pre-prints)