Please enable JavaScript.
Coggle requires JavaScript to display documents.
Centralized Learning, Non-identical Client Distributions, Key Use Cases of…
Centralized Learning
Non-identical Client Distributions
(1) Feature Distribution Skew (covariate shift)
(3) Same Label, Different Features (concept drift)
(2) Label Distribution Skew (prior probability shift)
(4) Same Features, Different Label (concept shift)
(5) Quantity Skew or Unbalancedness
Key Use Cases of Federated Learning Applications
Financial services industry
Edge computing devices
IoT
Medical & Healthcare
Federated Learning System
Server: Cluster aggregator (or aggregator)
FL server module
System configuration handler module
Communication handler module
Model synthesis routine module
FL state manager
Model aggregation module
Clients: Distributed agent (or agent)
Database server (or database)
Federated Learning
Communication Efficient
Compression
Quantization, Sparsification
Gradient compression (upstream: clients ==> server)
Stateless algorithms
Top-K selection algorithm
QSGD algorithm
FedOpt
Propose a modification to improve the performance of the algorithm
better quantization method, compression - encode & decode operators
control the rate-distortion trade-off more efficient
take advantage of the knowledge in statistical property of client updates
Point out some limitations of one of the recent novel algorithms
Stateful algorithms
Model broadcast compression (downstream: clients <== server)
Local computation reduction (upstream: clients ==> server)?
Client selection (restrict number of participating clients)
Local updating (reduce communication rounds)
Decentralized Training & Peer-to-Peer Learning
Personalized Federated Learning
Security & Privacy
Data anonymity
Differential privacy
Secure multi-party
Homomorphic encryption
Main problems
Resource Heterogeneity
Computation ability
Communication ability
Data Heterogeneity
Dealing with non-iid data
data-based approach
Data Augmented
Vanilla method
Mixup method
Generative adversarial network (GAN) method
Data Sharing, ex: sharing a global dataset
algorithm-based approach
Local fine tuning
Personalization layers
Knowledge distillation
Multi-task learning
Structure adaptation
system-based approach
Client clustering
Edge device adaptation
Personalized Learning
unbalance data
Communication Overhead
Massively distributed participating clients
Machine Learning
Decentralized Learning
Federated Learning
Vertical FL
Horizontal FL