Reducing Runtime by Recycling Samples release_ljvahxcxbfdbfbkzpc7mb2x76q

by Jialei Wang, Hai Wang, Nathan Srebro

Released as a article .

2016  

Abstract

Contrary to the situation with stochastic gradient descent, we argue that when using stochastic methods with variance reduction, such as SDCA, SAG or SVRG, as well as their variants, it could be beneficial to reuse previously used samples instead of fresh samples, even when fresh samples are available. We demonstrate this empirically for SDCA, SAG and SVRG, studying the optimal sample size one should use, and also uncover be-havior that suggests running SDCA for an integer number of epochs could be wasteful.
In text/plain format

Archived Files and Locations

application/pdf  645.1 kB
file_3ccsghvwivcyfot4rvzt46ihra
arxiv.org (repository)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  article
Stage   submitted
Date   2016-02-05
Version   v1
Language   en ?
arXiv  1602.02136v1
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: d596d9db-515f-4744-96b2-32f6690f2294
API URL: JSON