Monocular Depth Estimation through Virtual-world Supervision and Real-world SfM Self-Supervision
release_epazddoqqbclbijkyw5kiq6ckm
by
Akhil Gurram, Ahmet Faruk Tuna, Fengyi Shen, Onay Urfalioglu, Antonio M. López
2021
Abstract
Depth information is essential for on-board perception in autonomous driving
and driver assistance. Monocular depth estimation (MDE) is very appealing since
it allows for appearance and depth being on direct pixelwise correspondence
without further calibration. Best MDE models are based on Convolutional Neural
Networks (CNNs) trained in a supervised manner, i.e., assuming pixelwise ground
truth (GT). Usually, this GT is acquired at training time through a calibrated
multi-modal suite of sensors. However, also using only a monocular system at
training time is cheaper and more scalable. This is possible by relying on
structure-from-motion (SfM) principles to generate self-supervision.
Nevertheless, problems of camouflaged objects, visibility changes,
static-camera intervals, textureless areas, and scale ambiguity, diminish the
usefulness of such self-supervision. In this paper, we perform monocular depth
estimation by virtual-world supervision (MonoDEVS) and real-world SfM
self-supervision. We compensate the SfM self-supervision limitations by
leveraging virtual-world images with accurate semantic and depth supervision
and addressing the virtual-to-real domain gap. Our MonoDEVSNet outperforms
previous MDE CNNs trained on monocular and even stereo sequences.
In text/plain
format
Archived Files and Locations
application/pdf 12.7 MB
file_oaedlgwuwjck7lilxvhnpaxhru
|
arxiv.org (repository) web.archive.org (webarchive) |
2103.12209v2
access all versions, variants, and formats of this works (eg, pre-prints)