Prototype Mixture Models for Few-shot Semantic Segmentation
release_zcugqugx55czbfixsy6vyag4ym
by
Boyu Yang, Chang Liu, Bohao Li, Jianbin Jiao, Qixiang Ye
2020
Abstract
Few-shot segmentation is challenging because objects within the support and
query images could significantly differ in appearance and pose. Using a single
prototype acquired directly from the support image to segment the query image
causes semantic ambiguity. In this paper, we propose prototype mixture models
(PMMs), which correlate diverse image regions with multiple prototypes to
enforce the prototype-based semantic representation. Estimated by an
Expectation-Maximization algorithm, PMMs incorporate rich channel-wised and
spatial semantics from limited support images. Utilized as representations as
well as classifiers, PMMs fully leverage the semantics to activate objects in
the query image while depressing background regions in a duplex manner.
Extensive experiments on Pascal VOC and MS-COCO datasets show that PMMs
significantly improve upon state-of-the-arts. Particularly, PMMs improve 5-shot
segmentation performance on MS-COCO by up to 5.82\% with only a moderate cost
for model size and inference speed.
In text/plain
format
Archived Files and Locations
application/pdf 1.7 MB
file_ldmrsfd2mfadnbwnwbv2s4sede
|
arxiv.org (repository) web.archive.org (webarchive) |
2008.03898v1
access all versions, variants, and formats of this works (eg, pre-prints)