Deep Multi-modal Fusion of Image and Non-image Data in Disease Diagnosis and Prognosis: A Review
release_yaebqxjvg5avjfnpjmgtw37lzu
by
Can Cui, Haichun Yang, Yaohong Wang, Shilin Zhao, Zuhayr Asad, Lori A. Coburn, Keith T. Wilson, Bennett A. Landman, Yuankai Huo
2022
Abstract
The rapid development of diagnostic technologies in healthcare is leading to
higher requirements for physicians to handle and integrate the heterogeneous,
yet complementary data that are produced during routine practice. For instance,
the personalized diagnosis and treatment planning for a single cancer patient
relies on the various images (e.g., radiological, pathological, and camera
images) and non-image data (e.g., clinical data and genomic data). However,
such decision-making procedures can be subjective, qualitative, and have large
inter-subject variabilities. With the recent advances in multi-modal deep
learning technologies, an increasingly large number of efforts have been
devoted to a key question: how do we extract and aggregate multi-modal
information to ultimately provide more objective, quantitative computer-aided
clinical decision making? This paper reviews the recent studies on dealing with
such a question. Briefly, this review will include the (1) overview of current
multi-modal learning workflows, (2) summarization of multi-modal fusion
methods, (3) discussion of the performance, (4) applications in disease
diagnosis and prognosis, and (5) challenges and future directions.
In text/plain
format
Archived Files and Locations
application/pdf 742.0 kB
file_4jnzqxmzonclvivxwmklih2p6y
|
arxiv.org (repository) web.archive.org (webarchive) |
2203.15588v2
access all versions, variants, and formats of this works (eg, pre-prints)