Hybrid Sequence to Sequence Model for Video Object Segmentation
release_77dteeadxfbknetlljy67tcafe
by
Fatemeh Azimi and Stanislav Frolov and Federico Raue and Joern Hees and Andreas Dengel
2020
Abstract
One-shot Video Object Segmentation (VOS) is the task of pixel-wise tracking
an object of interest within a video sequence, where the segmentation mask of
the first frame is given at inference time. In recent years, Recurrent Neural
Networks (RNNs) have been widely used for VOS tasks, but they often suffer from
limitations such as drift and error propagation. In this work, we study an
RNN-based architecture and address some of these issues by proposing a hybrid
sequence-to-sequence architecture named HS2S, utilizing a hybrid mask
propagation strategy that allows incorporating the information obtained from
correspondence matching. Our experiments show that augmenting the RNN with
correspondence matching is a highly effective solution to reduce the drift
problem. The additional information helps the model to predict more accurate
masks and makes it robust against error propagation. We evaluate our HS2S model
on the DAVIS2017 dataset as well as Youtube-VOS. On the latter, we achieve an
improvement of 11.2pp in the overall segmentation accuracy over RNN-based
state-of-the-art methods in VOS. We analyze our model's behavior in challenging
cases such as occlusion and long sequences and show that our hybrid
architecture significantly enhances the segmentation quality in these difficult
scenarios.
In text/plain
format
Archived Files and Locations
application/pdf 14.5 MB
file_aevsshw745gjbcbrd7p37olx7u
|
arxiv.org (repository) web.archive.org (webarchive) |
2010.05069v1
access all versions, variants, and formats of this works (eg, pre-prints)