Meta-learning for Multi-variable Non-convex Optimization Problems: Iterating Non-optimums Makes Optimum Possible
release_6zyyzkmju5bcbdjuk5aoxfwuwe
by
Jingyuan Xia, Shengxi Li, Jun-Jie Huang, Imad Jaimoukha, Xinwang Liu
2020
Abstract
In this paper, we aim to address the problem of solving a non-convex
optimization problem over an intersection of multiple variable sets. This kind
of problems is typically solved by using an alternating minimization (AM)
strategy which splits the overall problem into a set of sub-problems
corresponding to each variable, and then iteratively performs minimization over
each sub-problem using a fixed updating rule. However, due to the intrinsic
non-convexity of the overall problem, the optimization can usually be trapped
into bad local minimum even when each sub-problem can be globally optimized at
each iteration. To tackle this problem, we propose a meta-learning based Global
Scope Optimization (GSO) method. It adaptively generates optimizers for
sub-problems via meta-learners and constantly updates these meta-learners with
respect to the global loss information of the overall problem. Therefore, the
sub-problems are optimized with the objective of minimizing the global loss
specifically. We evaluate the proposed model on a number of simulations,
including solving bi-linear inverse problems: matrix completion, and non-linear
problems: Gaussian mixture models. The experimental results show that our
proposed approach outperforms AM-based methods in standard settings, and is
able to achieve effective optimization in some challenging cases while other
methods would typically fail.
In text/plain
format
Archived Files and Locations
application/pdf 4.7 MB
file_276xsa6wubhanac2g3exhfpoam
|
arxiv.org (repository) web.archive.org (webarchive) |
2009.04899v1
access all versions, variants, and formats of this works (eg, pre-prints)