HYCEDIS: HYbrid Confidence Engine for Deep Document Intelligence System
release_kqfwjpjrjjd73ifge4xs5kubqm
by
Bao-Sinh Nguyen, Quang-Bach Tran, Tuan-Anh Nguyen Dang, Duc Nguyen, Hung Le
2022
Abstract
Measuring the confidence of AI models is critical for safely deploying AI in
real-world industrial systems. One important application of confidence
measurement is information extraction from scanned documents. However, there
exists no solution to provide reliable confidence score for current
state-of-the-art deep-learning-based information extractors. In this paper, we
propose a complete and novel architecture to measure confidence of current deep
learning models in document information extraction task. Our architecture
consists of a Multi-modal Conformal Predictor and a Variational
Cluster-oriented Anomaly Detector, trained to faithfully estimate its
confidence on its outputs without the need of host models modification. We
evaluate our architecture on real-wold datasets, not only outperforming
competing confidence estimators by a huge margin but also demonstrating
generalization ability to out-of-distribution data.
In text/plain
format
Archived Files and Locations
application/pdf 1.8 MB
file_4ycjcvfiazaxbm563wuur63qae
|
arxiv.org (repository) web.archive.org (webarchive) |
2206.02628v1
access all versions, variants, and formats of this works (eg, pre-prints)