Deep invariant networks with differentiable augmentation layers
release_ntcup2znqzgfven44ty3fe3bbe
by
Cédric Rommel, Thomas Moreau, Alexandre Gramfort
2022
Abstract
Designing learning systems which are invariant to certain data
transformations is critical in machine learning. Practitioners can typically
enforce a desired invariance on the trained model through the choice of a
network architecture, e.g. using convolutions for translations, or using data
augmentation. Yet, enforcing true invariance in the network can be difficult,
and data invariances are not always known a piori. State-of-the-art methods for
learning data augmentation policies require held-out data and are based on
bilevel optimization problems, which are complex to solve and often
computationally demanding. In this work we investigate new ways of learning
invariances only from the training data. Using learnable augmentation layers
built directly in the network, we demonstrate that our method is very
versatile. It can incorporate any type of differentiable augmentation and be
applied to a broad class of learning problems beyond computer vision. We
provide empirical evidence showing that our approach is easier and faster to
train than modern automatic data augmentation techniques based on bilevel
optimization, while achieving comparable results. Experiments show that while
the invariances transferred to a model through automatic data augmentation are
limited by the model expressivity, the invariance yielded by our approach is
insensitive to it by design.
In text/plain
format
Archived Content
There are no accessible files associated with this release. You could check other releases for this work for an accessible version.
Know of a fulltext copy of on the public web? Submit a URL and we will archive it
2202.02142v1
access all versions, variants, and formats of this works (eg, pre-prints)