Revisiting Deep Learning Models for Tabular Data
release_cbokm64pojhbxed42srhzl3tvm
by
Yury Gorishniy, Ivan Rubachev, Valentin Khrulkov, Artem Babenko
2021
Abstract
The necessity of deep learning for tabular data is still an unanswered
question addressed by a large number of research efforts. The recent literature
on tabular DL proposes several deep architectures reported to be superior to
traditional "shallow" models like Gradient Boosted Decision Trees. However,
since existing works often use different benchmarks and tuning protocols, it is
unclear if the proposed models universally outperform GBDT. Moreover, the
models are often not compared to each other, therefore, it is challenging to
identify the best deep model for practitioners.
In this work, we start from a thorough review of the main families of DL
models recently developed for tabular data. We carefully tune and evaluate them
on a wide range of datasets and reveal two significant findings. First, we show
that the choice between GBDT and DL models highly depends on data and there is
still no universally superior solution. Second, we demonstrate that a simple
ResNet-like architecture is a surprisingly effective baseline, which
outperforms most of the sophisticated models from the DL literature. Finally,
we design a simple adaptation of the Transformer architecture for tabular data
that becomes a new strong DL baseline and reduces the gap between GBDT and DL
models on datasets where GBDT dominates.
In text/plain
format
Archived Files and Locations
application/pdf 1.6 MB
file_bc4o72xfabd47j6dao2d43tql4
|
arxiv.org (repository) web.archive.org (webarchive) |
2106.11959v1
access all versions, variants, and formats of this works (eg, pre-prints)