Passed Spurious: Descent Algorithms and Local Minima in Spiked Matrix-Tensor Models release_eisfkaqvrzdgncggbvqbjvfn7u

by Stefano Sarao Mannelli and Florent Krzakala and Pierfrancesco Urbani and Lenka Zdeborová

Released as a article .

2019  

Abstract

In this work we analyse quantitatively the interplay between the loss landscape and performance of descent algorithms in a prototypical inference problem, the spiked matrix-tensor model. We study a loss function that is the negative log-likelihood of the model. We analyse the number of local minima at a fixed distance from the signal/spike with the Kac-Rice formula, and locate trivialization of the landscape at large signal-to-noise ratios. We evaluate in a closed form the performance of a gradient flow algorithm using integro-differential PDEs as developed in physics of disordered systems for the Langevin dynamics. We analyze the performance of an approximate message passing algorithm estimating the maximum likelihood configuration via its state evolution. We conclude by comparing the above results: while we observe a drastic slow down of the gradient flow dynamics even in the region where the landscape is trivial, both the analyzed algorithms are shown to perform well even in the part of the region of parameters where spurious local minima are present.
In text/plain format

Archived Files and Locations

application/pdf  927.2 kB
file_xrsoann5zrajzhvaxfjb6i5kfe
arxiv.org (repository)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  article
Stage   submitted
Date   2019-07-01
Version   v2
Language   en ?
arXiv  1902.00139v2
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: e7db77f9-ae7c-4631-b389-13b3f9720834
API URL: JSON