Communication-Efficient Federated Self-Distillation Method for Medical Image Segmentation

Abstract

Translate Article

Federated learning is a distributed learning framework designed to facilitate collaborative training of deep learning models at the network edge, ensuring the privacy and security of users while effectively responding to the challenges posed by data silos within the healthcare domain. However, deploying intricate medical segmentation network models in realistic settings often leads to substantial communication overhead for federated learning clients, thereby severely undermining the operational efficiency of federated systems. In light of these issues, we propose an innovative federated learning approach known as FedSKD, which is intended to alleviate the significant communication overhead incurred during medical image segmentation tasks in federated learning. This method strikes a delicate balance between communication costs and model training accuracy. A key feature of the FedSKD framework is incorporating a self-knowledge distillation mechanism, which encompasses both prediction map distillation loss and feature map distillation loss. This integration plays a key role in enhancing model accuracy without the need for training an additional resource-intensive mentor model, thereby optimizing computational efficiency. Additionally, we introduce parameter difference dynamic compression technology to compress communications, resulting in a reduction of communication costs by more than 30% within the federated system. Furthermore, extensive experiments utilizing FedSKD were conducted on widely recognized public medical segmentation datasets such as KiTS19 and COVID-19. The experimental findings unequivocally demonstrate that our FedSKD not only significantly diminishes the expenses associated with federated communication but also enhances overall model accuracy.