Randomized Block Cubic Newton Method release_qg467w63rrg5rkaiuz6amxsbkm

by Nikita Doikov, Peter Richtárik

Released as a article .

2018  

Abstract

We study the problem of minimizing the sum of three convex functions: a differentiable, twice-differentiable and a non-smooth term in a high dimensional setting. To this effect we propose and analyze a randomized block cubic Newton (RBCN) method, which in each iteration builds a model of the objective function formed as the sum of the natural models of its three components: a linear model with a quadratic regularizer for the differentiable term, a quadratic model with a cubic regularizer for the twice differentiable term, and perfect (proximal) model for the nonsmooth term. Our method in each iteration minimizes the model over a random subset of blocks of the search variable. RBCN is the first algorithm with these properties, generalizing several existing methods, matching the best known bounds in all special cases. We establish O(1/ϵ), O(1/√(ϵ)) and O( (1/ϵ)) rates under different assumptions on the component functions. Lastly, we show numerically that our method outperforms the state-of-the-art on a variety of machine learning problems, including cubically regularized least-squares, logistic regression with constraints, and Poisson regression.
In text/plain format

Archived Files and Locations

application/pdf  751.0 kB
file_lrxiuxe5u5hyzofty3gdsdzwha
arxiv.org (repository)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  article
Stage   submitted
Date   2018-02-12
Version   v1
Language   en ?
arXiv  1802.04084v1
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: 8fc63825-82cc-4bdd-889e-6452d6f8fb53
API URL: JSON