kFW: A Frank-Wolfe style algorithm with stronger subproblem oracles release_ke2wo3xzzbc4rg2wnkqf5jin24

by Lijun Ding, Jicong Fan, Madeleine Udell

Released as a article .

2020  

Abstract

This paper proposes a new variant of Frank-Wolfe (FW), called kFW. Standard FW suffers from slow convergence: iterates often zig-zag as update directions oscillate around extreme points of the constraint set. The new variant, kFW, overcomes this problem by using two stronger subproblem oracles in each iteration. The first is a k linear optimization oracle (kLOO) that computes the k best update directions (rather than just one). The second is a k direction search (kDS) that minimizes the objective over a constraint set represented by the k best update directions and the previous iterate. When the problem solution admits a sparse representation, both oracles are easy to compute, and kFW converges quickly for smooth convex objectives and several interesting constraint sets: kFW achieves finite 4L_f^3D^4/γδ^2 convergence on polytopes and group norm balls, and linear convergence on spectrahedra and nuclear norm balls. Numerical experiments validate the effectiveness of kFW and demonstrate an order-of-magnitude speedup over existing approaches.
In text/plain format

Archived Files and Locations

application/pdf  1.4 MB
file_6mp5csu7fjdepjm4hceeirgkh4
arxiv.org (repository)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  article
Stage   submitted
Date   2020-06-29
Version   v1
Language   en ?
arXiv  2006.16142v1
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: 786b18f5-7c7d-4e02-9674-f0c243c7c422
API URL: JSON