PFLlib

Privacy Evaluation

You can use the following privacy evaluation methods to assess the privacy-preserving capabilities of tFL/pFL algorithms in PFLlib. Please refer to ./system/flcore/servers/serveravg.py for an example. Note that most of these evaluations are not typically considered in the original papers. We encourage you to add more attacks and metrics for privacy evaluation.

Currently supported attacks:

• DLG (Deep Leakage from Gradients) attack

Currently supported metrics:

• PSNR (Peak Signal-to-Noise Ratio): an objective metric for image evaluation, defined as the logarithm of the ratio of the squared maximum value of RGB image fluctuations to the Mean Squared Error (MSE) between two images. A lower PSNR score indicates better privacy-preserving capabilities.

Systematical research support

To simulate Federated Learning (FL) under practical conditions, such as client dropout, slow trainers, slow senders, and network TTL (Time-To-Live), you can adjust the following parameters:

• -cdr: Dropout rate for clients. Clients are randomly dropped at each training round based on this rate.
• -tsr and -ssr: Slow trainer and slow sender rates, respectively. These parameters define the proportion of clients that will behave as slow trainers or slow senders. Once a client is selected as a "slow trainer" or "slow sender," it will consistently train/send slower than other clients.
• -tth: Threshold for network TTL in milliseconds.

Thanks to @Stonesjtu, this library can also record the GPU memory usage for the model.