V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer
release_frot3yl4kzhshipwevhld3s6lm
by
Runsheng Xu, Hao Xiang, Zhengzhong Tu, Xin Xia, Ming-Hsuan Yang, Jiaqi Ma
2022
Abstract
In this paper, we investigate the application of Vehicle-to-Everything (V2X)
communication to improve the perception performance of autonomous vehicles. We
present a robust cooperative perception framework with V2X communication using
a novel vision Transformer. Specifically, we build a holistic attention model,
namely V2X-ViT, to effectively fuse information across on-road agents (i.e.,
vehicles and infrastructure). V2X-ViT consists of alternating layers of
heterogeneous multi-agent self-attention and multi-scale window self-attention,
which captures inter-agent interaction and per-agent spatial relationships.
These key modules are designed in a unified Transformer architecture to handle
common V2X challenges, including asynchronous information sharing, pose errors,
and heterogeneity of V2X components. To validate our approach, we create a
large-scale V2X perception dataset using CARLA and OpenCDA. Extensive
experimental results demonstrate that V2X-ViT sets new state-of-the-art
performance for 3D object detection and achieves robust performance even under
harsh, noisy environments. The code is available at
https://github.com/DerrickXuNu/v2x-vit.
In text/plain
format
Archived Files and Locations
application/pdf 20.9 MB
file_pzbuwbibwras7ilsh6rcw3oxdu
|
arxiv.org (repository) web.archive.org (webarchive) |
2203.10638v3
access all versions, variants, and formats of this works (eg, pre-prints)