Distributed estimation of principal eigenspaces release_bj6377y4uzahteewak4lr37eyq

by Jianqing Fan, Dong Wang, Kaizheng Wang, Ziwei Zhu

Published in Annals of Statistics by Institute of Mathematical Statistics.

2019   Volume 47, Issue 6, p3009-3031

Abstract

Principal component analysis (PCA) is fundamental to statistical machine learning. It extracts latent principal factors that contribute to the most variation of the data. When data are stored across multiple machines, however, communication cost can prohibit the computation of PCA in a central location and distributed algorithms for PCA are thus needed. This paper proposes and studies a distributed PCA algorithm: each node machine computes the top K eigenvectors and transmits them to the central server; the central server then aggregates the information from all the node machines and conducts a PCA based on the aggregated information. We investigate the bias and variance for the resulting distributed estimator of the top K eigenvectors. In particular, we show that for distributions with symmetric innovation, the empirical top eigenspaces are unbiased and hence the distributed PCA is "unbiased". We derive the rate of convergence for distributed PCA estimators, which depends explicitly on the effective rank of covariance, eigen-gap, and the number of machines. We show that when the number of machines is not unreasonably large, the distributed PCA performs as well as the whole sample PCA, even without full access of whole data. The theoretical results are verified by an extensive simulation study. We also extend our analysis to the heterogeneous case where the population covariance matrices are different across local machines but share similar top eigen-structures.
In text/plain format

Archived Files and Locations

application/pdf  977.5 kB
file_7l5vhszfabahllmtydwyzg4uyi
web.archive.org (webarchive)
europepmc.org (repository)
Read Archived PDF
Archived
Type  article-journal
Stage   published
Date   2019-10-31
Language   en ?
DOI  10.1214/18-aos1713
PubMed  31700197
PMC  PMC6836292
Container Metadata
Open Access Publication
Not in DOAJ
In Keepers Registery
ISSN-L:  0090-5364
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: 87ef42cb-46c9-4fee-a02c-f934bdd07246
API URL: JSON