Unseen Object Instance Segmentation with Fully Test-time RGB-D Embeddings Adaptation
release_wsskwn6ypfaczd4beoamni3sjq
by
Lu Zhang, Siqi Zhang, Xu Yang, Zhiyong Liu
2022
Abstract
Segmenting unseen objects is a crucial ability for the robot since it may
encounter new environments during the operation. Recently, a popular solution
is leveraging RGB-D features of large-scale synthetic data and directly
applying the model to unseen real-world scenarios. However, even though depth
data have fair generalization ability, the domain shift due to the Sim2Real gap
is inevitable, which presents a key challenge to the unseen object instance
segmentation (UOIS) model. To tackle this problem, we re-emphasize the
adaptation process across Sim2Real domains in this paper. Specifically, we
propose a framework to conduct the Fully Test-time RGB-D Embeddings Adaptation
(FTEA) based on parameters of the BatchNorm layer. To construct the learning
objective for test-time back-propagation, we propose a novel non-parametric
entropy objective that can be implemented without explicit classification
layers. Moreover, we design a cross-modality knowledge distillation module to
encourage the information transfer during test time. The proposed method can be
efficiently conducted with test-time images, without requiring annotations or
revisiting the large-scale synthetic training data. Besides significant time
savings, the proposed method consistently improves segmentation results on both
overlap and boundary metrics, achieving state-of-the-art performances on two
real-world RGB-D image datasets. We hope our work could draw attention to the
test-time adaptation and reveal a promising direction for robot perception in
unseen environments.
In text/plain
format
Archived Files and Locations
application/pdf 1.1 MB
file_qkcwrsnj6nespknegezrkido7y
|
arxiv.org (repository) web.archive.org (webarchive) |
2204.09847v1
access all versions, variants, and formats of this works (eg, pre-prints)