Passed Spurious: Descent Algorithms and Local Minima in Spiked
Matrix-Tensor Models
release_eisfkaqvrzdgncggbvqbjvfn7u
by
Stefano Sarao Mannelli and Florent Krzakala and Pierfrancesco Urbani
and Lenka Zdeborová
2019
Abstract
In this work we analyse quantitatively the interplay between the loss
landscape and performance of descent algorithms in a prototypical inference
problem, the spiked matrix-tensor model. We study a loss function that is the
negative log-likelihood of the model. We analyse the number of local minima at
a fixed distance from the signal/spike with the Kac-Rice formula, and locate
trivialization of the landscape at large signal-to-noise ratios. We evaluate in
a closed form the performance of a gradient flow algorithm using
integro-differential PDEs as developed in physics of disordered systems for the
Langevin dynamics. We analyze the performance of an approximate message passing
algorithm estimating the maximum likelihood configuration via its state
evolution. We conclude by comparing the above results: while we observe a
drastic slow down of the gradient flow dynamics even in the region where the
landscape is trivial, both the analyzed algorithms are shown to perform well
even in the part of the region of parameters where spurious local minima are
present.
In text/plain
format
Archived Files and Locations
application/pdf 927.2 kB
file_xrsoann5zrajzhvaxfjb6i5kfe
|
arxiv.org (repository) web.archive.org (webarchive) |
1902.00139v2
access all versions, variants, and formats of this works (eg, pre-prints)