BibTeX
CSL-JSON
MLA
Harvard
How Does Batch Normalization Help Binary Training?
release_7a4t3ew5dvbsdifnajsm7rytby
by
Eyyüb Sari, Mouloud Belbahri, Vahid Partovi Nia
Released
as a article
.
2019
Abstract
Binary Neural Networks (BNNs) are difficult to train, and suffer from drop of
accuracy. It appears in practice that BNNs fail to train in the absence of
Batch Normalization (BatchNorm) layer. We find the main role of BatchNorm is to
avoid exploding gradients in the case of BNNs. This finding suggests that the
common initialization methods developed for full-precision networks are
irrelevant to BNNs. We build a theoretical study on the role of BatchNorm in
binary training, backed up by numerical experiments.
In text/plain
format
Archived Files and Locations
application/pdf 116.7 kB
file_pk6mxs7e7vglrly3kc42h5znzi
|
arxiv.org (repository) web.archive.org (webarchive) |
Read Archived PDF
Preserved and Accessible
arXiv
1909.09139v1
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
access all versions, variants, and formats of this works (eg, pre-prints)
Cite This
Lookup Links