Automatic deep heterogeneous quantization of Deep Neural Networks for ultra low-area, low-latency inference on the edge at particle colliders
release_7ccpczec5ja6jky73m24naqxeu
by
Claudionor N. Coelho Jr., Aki Kuusela, Shan Li, Hao Zhuang, Thea Aarrestad, Vladimir Loncar, Jennifer Ngadiuba, Maurizio Pierini, Adrian Alan Pol, Sioni Summers
2020
Abstract
While the quest for more accurate solutions is pushing deep learning research
towards larger and more complex algorithms, edge devices demand efficient
inference i.e. reduction in model size, latency and energy consumption. A
technique to limit model size is quantization, i.e. using fewer bits to
represent weights and biases. Such an approach usually results in a decline in
performance. Here, we introduce a novel method for designing optimally
heterogeneously quantized versions of deep neural network models for
minimum-energy, high-accuracy, nanosecond inference and fully automated
deployment on chip. With a per-layer, per-parameter type automatic quantization
procedure, sampling from a wide range of quantizers, model energy consumption
and size are minimized while high accuracy is maintained. This is crucial for
the event selection procedure in proton-proton collisions at the CERN Large
Hadron Collider, where resources are strictly limited and a latency of
𝒪(1) μs is required. Nanosecond inference and a resource
consumption reduced by a factor of 50 when implemented on FPGA hardware is
achieved.
In text/plain
format
Archived Files and Locations
application/pdf 849.7 kB
file_nc6pzjv32jh6vfaedopeiordyu
|
arxiv.org (repository) web.archive.org (webarchive) |
2006.10159v2
access all versions, variants, and formats of this works (eg, pre-prints)