Enabling Binary Neural Network Training on the Edge
release_57pouok6n5eklhd3thbszu557m
by
Erwei Wang, James J. Davis, Daniele Moro, Piotr Zielinski, Claudionor Coelho, Satrajit Chatterjee, Peter Y. K. Cheung, George A. Constantinides
2021
Abstract
The ever-growing computational demands of increasingly complex machine
learning models frequently necessitate the use of powerful cloud-based
infrastructure for their training. Binary neural networks are known to be
promising candidates for on-device inference due to their extreme compute and
memory savings over higher-precision alternatives. In this paper, we
demonstrate that they are also strongly robust to gradient quantization,
thereby making the training of modern models on the edge a practical reality.
We introduce a low-cost binary neural network training strategy exhibiting
sizable memory footprint reductions and energy savings vs Courbariaux
Bengio's standard approach. Against the latter, we see coincident memory
requirement and energy consumption drops of 2–6×, while reaching
similar test accuracy in comparable time, across a range of small-scale models
trained to classify popular datasets. We also showcase ImageNet training of
ResNetE-18, achieving a 3.12× memory reduction over the aforementioned
standard. Such savings will allow for unnecessary cloud offloading to be
avoided, reducing latency, increasing energy efficiency and safeguarding
privacy.
In text/plain
format
Archived Files and Locations
application/pdf 460.7 kB
file_rpsawhticvhz5j2f7givfhlikm
|
arxiv.org (repository) web.archive.org (webarchive) |
2102.04270v1
access all versions, variants, and formats of this works (eg, pre-prints)