Warp and Learn: Novel Views Generation for Vehicles and Other Objects release_6ognjmxdszbwxi7kbkmwsimm2e

by Andrea Palazzi, Luca Bergamini, Simone Calderara, Rita Cucchiara

Published in IEEE Transactions on Pattern Analysis and Machine Intelligence by Institute of Electrical and Electronics Engineers (IEEE).

2020   Volume PP, p1-1

Abstract

In this work we introduce a new self-supervised, semi-parametric approach for synthesizing novel views of a vehicle starting from a single monocular image.Differently from parametric (i.e. entirely learning-based) methods, we show how a-priori geometric knowledge about the object and the 3D world can be successfully integrated into a deep learning based image generation framework. As this geometric component is not learnt, we call our approach semi-parametric.In particular, we exploit man-made object symmetry and piece-wise planarity to integrate rich a-priori visual information into the novel viewpoint synthesis process. An Image Completion Network (ICN) is then trained to generate a realistic image starting from this geometric guidance.This blend between parametric and non-parametric components allows us to i) operate in a real-world scenario, ii) preserve high-frequency visual information such as textures, iii) handle truly arbitrary 3D roto-translations of the input and iv) perform shape transfer to completely different 3D models. Eventually, we show that our approach can be easily complemented with synthetic data and extended to other rigid objects with completely different topology, even in presence of concave structures and holes.A comprehensive experimental analysis against state-of-the-art competitors shows the efficacy of our method both from a quantitative and a perceptive point of view.
In text/plain format

Archived Files and Locations

application/pdf  9.4 MB
file_xgine7oxgjcahmj6hhbu2mkydm
ieeexplore.ieee.org (publisher)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  article-journal
Stage   published
Date   2020-10-13
Language   en ?
Container Metadata
Not in DOAJ
In Keepers Registry
ISSN-L:  0162-8828
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: 534b51fd-aa47-43a2-8a58-2393607bd2e7
API URL: JSON