Decentralized federated learning of deep neural networks on non-iid data
release_3erw3bohgjalvf4fgnm62msm2i
by
Noa Onoszko, Gustav Karlsson, Olof Mogren, Edvin Listo Zec
2021
Abstract
We tackle the non-convex problem of learning a personalized deep learning
model in a decentralized setting. More specifically, we study decentralized
federated learning, a peer-to-peer setting where data is distributed among many
clients and where there is no central server to orchestrate the training. In
real world scenarios, the data distributions are often heterogeneous between
clients. Therefore, in this work we study the problem of how to efficiently
learn a model in a peer-to-peer system with non-iid client data. We propose a
method named Performance-Based Neighbor Selection (PENS) where clients with
similar data distributions detect each other and cooperate by evaluating their
training losses on each other's data to learn a model suitable for the local
data distribution. Our experiments on benchmark datasets show that our proposed
method is able to achieve higher accuracies as compared to strong baselines.
In text/plain
format
Archived Files and Locations
application/pdf 1.2 MB
file_rc7c3uhwz5hg3lg267jaysfqt4
|
arxiv.org (repository) web.archive.org (webarchive) |
2107.08517v2
access all versions, variants, and formats of this works (eg, pre-prints)