ADeLA: Automatic Dense Labeling with Attention for Viewpoint Adaptation in Semantic Segmentation
release_2eqwcsmrazgt3myczbbtlglksm
by
Yanchao Yang, Hanxiang Ren, He Wang, Bokui Shen, Qingnan Fan, Youyi Zheng, C. Karen Liu, Leonidas Guibas
2021
Abstract
We describe an unsupervised domain adaptation method for image content shift
caused by viewpoint changes for a semantic segmentation task. Most existing
methods perform domain alignment in a shared space and assume that the mapping
from the aligned space to the output is transferable. However, the novel
content induced by viewpoint changes may nullify such a space for effective
alignments, thus resulting in negative adaptation. Our method works without
aligning any statistics of the images between the two domains. Instead, it
utilizes a view transformation network trained only on color images to
hallucinate the semantic images for the target. Despite the lack of
supervision, the view transformation network can still generalize to semantic
images thanks to the inductive bias introduced by the attention mechanism.
Furthermore, to resolve ambiguities in converting the semantic images to
semantic labels, we treat the view transformation network as a functional
representation of an unknown mapping implied by the color images and propose
functional label hallucination to generate pseudo-labels in the target domain.
Our method surpasses baselines built on state-of-the-art correspondence
estimation and view synthesis methods. Moreover, it outperforms the
state-of-the-art unsupervised domain adaptation methods that utilize
self-training and adversarial domain alignment. Our code and dataset will be
made publicly available.
In text/plain
format
Archived Files and Locations
application/pdf 7.6 MB
file_wsqbbzow4jdm7mhal5mxwp27cu
|
arxiv.org (repository) web.archive.org (webarchive) |
2107.14285v1
access all versions, variants, and formats of this works (eg, pre-prints)