Feedback Control for Online Training of Neural Networks
release_owwebgniuzcnhjtmkb767h3naq
by
Zilong Zhao, Sophie Cerf, Bogdan Robu, Nicolas Marchand
2019
Abstract
Convolutional neural networks (CNNs) are commonly used for image
classification tasks, raising the challenge of their application on data flows.
During their training, adaptation is often performed by tuning the learning
rate. Usual learning rate strategies are time-based i.e. monotonously
decreasing. In this paper, we advocate switching to a performance-based
adaptation, in order to improve the learning efficiency. We present E
(Exponential)/PD (Proportional Derivative)-Control, a conditional learning rate
strategy that combines a feedback PD controller based on the CNN loss function,
with an exponential control signal to smartly boost the learning and adapt the
PD parameters. Stability proof is provided as well as an experimental
evaluation using two state of the art image datasets (CIFAR-10 and
Fashion-MNIST). Results show better performances than the related works (faster
network accuracy growth reaching higher levels) and robustness of the
E/PD-Control regarding its parametrization.
In text/plain
format
Archived Files and Locations
application/pdf 876.4 kB
file_k456o3m2lfg6ncblbfd6dx3sse
|
arxiv.org (repository) web.archive.org (webarchive) |
1911.07710v1
access all versions, variants, and formats of this works (eg, pre-prints)