Hierarchical Semantic Regularization of Latent Spaces in StyleGANs
release_ni6mauhaojckldawnqmzg4alzy
by
Tejan Karmali, Rishubh Parihar, Susmit Agrawal, Harsh Rangwani, Varun Jampani, Maneesh Singh, R. Venkatesh Babu
2022
Abstract
Progress in GANs has enabled the generation of high-resolution photorealistic
images of astonishing quality. StyleGANs allow for compelling attribute
modification on such images via mathematical operations on the latent style
vectors in the W/W+ space that effectively modulate the rich hierarchical
representations of the generator. Such operations have recently been
generalized beyond mere attribute swapping in the original StyleGAN paper to
include interpolations. In spite of many significant improvements in StyleGANs,
they are still seen to generate unnatural images. The quality of the generated
images is predicated on two assumptions; (a) The richness of the hierarchical
representations learnt by the generator, and, (b) The linearity and smoothness
of the style spaces. In this work, we propose a Hierarchical Semantic
Regularizer (HSR) which aligns the hierarchical representations learnt by the
generator to corresponding powerful features learnt by pretrained networks on
large amounts of data. HSR is shown to not only improve generator
representations but also the linearity and smoothness of the latent style
spaces, leading to the generation of more natural-looking style-edited images.
To demonstrate improved linearity, we propose a novel metric - Attribute
Linearity Score (ALS). A significant reduction in the generation of unnatural
images is corroborated by improvement in the Perceptual Path Length (PPL)
metric by 16.19% averaged across different standard datasets while
simultaneously improving the linearity of attribute-change in the attribute
editing tasks.
In text/plain
format
Archived Files and Locations
application/pdf 13.1 MB
file_5zswstr25zffbcvsknrh5muzey
|
arxiv.org (repository) web.archive.org (webarchive) |
2208.03764v1
access all versions, variants, and formats of this works (eg, pre-prints)