Working memory inspired hierarchical video decomposition with transformative representations
release_dzdteas6ufgsvlovst3mndkcby
by
Binjie Qin, Haohao Mao, Ruipeng Zhang, Yueqi Zhu, Song Ding, Xu Chen
2022
Abstract
Video decomposition is very important to extract moving foreground objects
from complex backgrounds in computer vision, machine learning, and medical
imaging, e.g., extracting moving contrast-filled vessels from the complex and
noisy backgrounds of X-ray coronary angiography (XCA). However, the challenges
caused by dynamic backgrounds, overlapping heterogeneous environments and
complex noises still exist in video decomposition. To solve these problems,
this study is the first to introduce a flexible visual working memory model in
video decomposition tasks to provide interpretable and high-performance
hierarchical deep architecture, integrating the transformative representations
between sensory and control layers from the perspective of visual and cognitive
neuroscience. Specifically, robust PCA unrolling networks acting as a
structure-regularized sensor layer decompose XCA into sparse/low-rank
structured representations to separate moving contrast-filled vessels from
noisy and complex backgrounds. Then, patch recurrent convolutional LSTM
networks with a backprojection module embody unstructured random
representations of the control layer in working memory, recurrently projecting
spatiotemporally decomposed nonlocal patches into orthogonal subspaces for
heterogeneous vessel retrieval and interference suppression. This video
decomposition deep architecture effectively restores the heterogeneous profiles
of intensity and the geometries of moving objects against the complex
background interferences. Experiments show that the proposed method
significantly outperforms state-of-the-art methods in accurate moving
contrast-filled vessel extraction with excellent flexibility and computational
efficiency.
In text/plain
format
Archived Files and Locations
application/pdf 5.3 MB
file_wx3mepzx75g3fnv3jgded3jxzq
|
arxiv.org (repository) web.archive.org (webarchive) |
2204.10105v1
access all versions, variants, and formats of this works (eg, pre-prints)