kFW: A Frank-Wolfe style algorithm with stronger subproblem oracles
release_ke2wo3xzzbc4rg2wnkqf5jin24
by
Lijun Ding, Jicong Fan, Madeleine Udell
2020
Abstract
This paper proposes a new variant of Frank-Wolfe (FW), called kFW. Standard
FW suffers from slow convergence: iterates often zig-zag as update directions
oscillate around extreme points of the constraint set. The new variant, kFW,
overcomes this problem by using two stronger subproblem oracles in each
iteration. The first is a k linear optimization oracle (kLOO) that computes
the k best update directions (rather than just one). The second is a k
direction search (kDS) that minimizes the objective over a constraint set
represented by the k best update directions and the previous iterate. When
the problem solution admits a sparse representation, both oracles are easy to
compute, and kFW converges quickly for smooth convex objectives and several
interesting constraint sets: kFW achieves finite
4L_f^3D^4/γδ^2 convergence on polytopes and group norm
balls, and linear convergence on spectrahedra and nuclear norm balls. Numerical
experiments validate the effectiveness of kFW and demonstrate an
order-of-magnitude speedup over existing approaches.
In text/plain
format
Archived Files and Locations
application/pdf 1.4 MB
file_6mp5csu7fjdepjm4hceeirgkh4
|
arxiv.org (repository) web.archive.org (webarchive) |
2006.16142v1
access all versions, variants, and formats of this works (eg, pre-prints)