MFAGAN: A Compression Framework for Memory-Efficient On-Device Super-Resolution GAN
release_66jm5m2q5vaprhirf7weg74u5i
by
Wenlong Cheng and Mingbo Zhao and Zhiling Ye and Shuhang Gu
2021
Abstract
Generative adversarial networks (GANs) have promoted remarkable advances in
single-image super-resolution (SR) by recovering photo-realistic images.
However, high memory consumption of GAN-based SR (usually generators) causes
performance degradation and more energy consumption, hindering the deployment
of GAN-based SR into resource-constricted mobile devices. In this paper, we
propose a novel compression framework Multi-scale Feature
Aggregation Net based GAN (MFAGAN) for reducing the memory
access cost of the generator. First, to overcome the memory explosion of dense
connections, we utilize a memory-efficient multi-scale feature aggregation net
as the generator. Second, for faster and more stable training, our method
introduces the PatchGAN discriminator. Third, to balance the student
discriminator and the compressed generator, we distill both the generator and
the discriminator. Finally, we perform a hardware-aware neural architecture
search (NAS) to find a specialized SubGenerator for the target mobile phone.
Benefiting from these improvements, the proposed MFAGAN achieves up to
8.3× memory saving and 42.9× computation
reduction, with only minor visual quality degradation, compared with ESRGAN.
Empirical studies also show ∼70 milliseconds latency on Qualcomm
Snapdragon 865 chipset.
In text/plain
format
Archived Files and Locations
application/pdf 2.8 MB
file_n4q6ka3dufat3e6ajvupk6ovry
|
arxiv.org (repository) web.archive.org (webarchive) |
2107.12679v1
access all versions, variants, and formats of this works (eg, pre-prints)