Decentralized federated learning of deep neural networks on non-iid data release_3erw3bohgjalvf4fgnm62msm2i

by Noa Onoszko, Gustav Karlsson, Olof Mogren, Edvin Listo Zec

Released as a article .

2021  

Abstract

We tackle the non-convex problem of learning a personalized deep learning model in a decentralized setting. More specifically, we study decentralized federated learning, a peer-to-peer setting where data is distributed among many clients and where there is no central server to orchestrate the training. In real world scenarios, the data distributions are often heterogeneous between clients. Therefore, in this work we study the problem of how to efficiently learn a model in a peer-to-peer system with non-iid client data. We propose a method named Performance-Based Neighbor Selection (PENS) where clients with similar data distributions detect each other and cooperate by evaluating their training losses on each other's data to learn a model suitable for the local data distribution. Our experiments on benchmark datasets show that our proposed method is able to achieve higher accuracies as compared to strong baselines.
In text/plain format

Archived Files and Locations

application/pdf  1.2 MB
file_rc7c3uhwz5hg3lg267jaysfqt4
arxiv.org (repository)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  article
Stage   submitted
Date   2021-07-20
Version   v2
Language   en ?
arXiv  2107.08517v2
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: 013c7956-95aa-4514-bee9-9a289af57be3
API URL: JSON