Feedback Control for Online Training of Neural Networks release_owwebgniuzcnhjtmkb767h3naq

by Zilong Zhao, Sophie Cerf, Bogdan Robu, Nicolas Marchand

Released as a article .

2019  

Abstract

Convolutional neural networks (CNNs) are commonly used for image classification tasks, raising the challenge of their application on data flows. During their training, adaptation is often performed by tuning the learning rate. Usual learning rate strategies are time-based i.e. monotonously decreasing. In this paper, we advocate switching to a performance-based adaptation, in order to improve the learning efficiency. We present E (Exponential)/PD (Proportional Derivative)-Control, a conditional learning rate strategy that combines a feedback PD controller based on the CNN loss function, with an exponential control signal to smartly boost the learning and adapt the PD parameters. Stability proof is provided as well as an experimental evaluation using two state of the art image datasets (CIFAR-10 and Fashion-MNIST). Results show better performances than the related works (faster network accuracy growth reaching higher levels) and robustness of the E/PD-Control regarding its parametrization.
In text/plain format

Archived Files and Locations

application/pdf  876.4 kB
file_k456o3m2lfg6ncblbfd6dx3sse
arxiv.org (repository)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  article
Stage   submitted
Date   2019-11-18
Version   v1
Language   en ?
arXiv  1911.07710v1
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: 7ed2af26-61df-4ecf-8f15-81ad300f015f
API URL: JSON