Federated learning (FL) is an increasingly popular paradigm for protecting data privacy in machine learning systems. However, the data heterogeneity and high computation cost/latency are challenging barriers for employing FL in real-world applications with heterogeneous devices. In this paper, we propose a novel personalized FL framework named <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathtt {CompFL}$</tex-math></inline-formula> allowing cooperative training of models with varied structures to mitigate those issues. First, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathtt {CompFL}$</tex-math></inline-formula> initializes a set of expert models in varied sizes and allows each client to choose one or multiple expert models for training according to their capacities. Second, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathtt {CompFL}$</tex-math></inline-formula> combines the model decoupling strategy and local-global feature alignment to mitigate the adverse impact of label heterogeneity, where clients only share the feature extractor part for each model architecture. Third, to encourage mutual enhancement of various models, knowledge distillation in local training is further applied to improve the overall performance. To make our framework workable in real systems, we implement it in both centralized settings with server-coordinated parallel training, and decentralized settings with newly developed device-to-device training-forwarding schemes. Extensive experiments on benchmark datasets are conducted to verify the potential of our framework for personalized FL over heterogeneous devices.

Constructing a deepfake fingerprint detection model (D2FDM) to counteract spoofing attacks stemming from forged fingerprints is a formidable challenge, requiring massive fingerprint samples, computing resources, and profound professional expertise. Consequently, it is of paramount importance to formulate an effective strategy to protect the intellectual property (IP) of D2FDM. To achieve this, this paper proposes a practical ProIP-D2FDM framework to authenticate the D2FDM. Firstly, the identity of model ownership is hidden in the cross-material fingerprint using generative steganography. The generated invisible additive noise serves as a watermark, rendering the backdoor inconspicuous, and facilitating its embedding through collaborative training with the original fingerprint dataset. Then, the key trigger set is constructed by sampling the original fake fingerprints, and the target label of the trigger set is employed as the foundation for IP verification. Once authorized, the embedded identity can also be distributed. Finally, to prevent the functionality of D2FDM from being stolen via distillation attacks, this paper further designs a novel non-distillable strategy. In this strategy, the student model is deprived of any prior knowledge acquisition. Experimental results demonstrate that our method achieves a remarkable 100% watermark verification accuracy and 100% success rate in user identity authentication on D2FDM trained using two public fingerprint datasets, namely LivDet2017 and LivDet2019. When the non-distillable strategy devised in this paper is used, the performance of the student model, with the D2FDM as the teacher model, exhibits a 4% decline. Moreover, the framework presented in this paper shows remarkable robustness against several common attacks.

Black-box zero-th order optimizationis a central primitive for applications in fields as diverse as finance, physics, and engineering. In a common formulation of this problem, a designer sequentially attempts candidate solutions, receiving noisy feedback on the value of each attempt from the system. In this paper, we study scenarios in which feedback is also provided on the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">safety</i> of the attempted solution, and the optimizer is constrained to limit the number of unsafe solutions that are tried throughout the optimization process. Focusing on methods based on Bayesian optimization (BO), prior art has introduced an optimization scheme – referred to as <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SafeOpt</small> – that is guaranteed not to select <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">any</i> unsafe solution with a controllable probability over feedback noise as long as strict assumptions on the safety constraint function are met. In this paper, a novel BO-based approach is introduced that satisfies safety requirements irrespective of properties of the constraint function. This strong theoretical guarantee is obtained at the cost of allowing for an arbitrary, controllable but non-zero, rate of violation of the safety constraint. The proposed method, referred to as <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Safe-Bocp</small>, builds on online conformal prediction (CP) and is specialized to the cases in which feedback on the safety constraint is either noiseless or noisy. Experimental results on synthetic and real-world data validate the advantages and flexibility of the proposed <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Safe-Bocp</small>.

Due to the significant advancements in medical imaging, the application of artificial intelligence for early disease diagnosis has greatly contributed to reducing mortality caused by heart diseases. Cardiac MRI segmentation into the left ventricle (LV), right ventricle (RV) and myocardium (MYO) is a key step towards the early treatment and diagnosis of heart diseases. The proposed method consists of two main components: a localization part to extract the heart organ from other parts of the image, and a GAN model for segmentation. In the generator part of the generative adversarial network (GAN) model, a customized U-Net network is employed to segment the cardiac image. A combination of inception and residual blocks and utilizing an attention mechanism yield an improved version of U-Net. On the other hand, the discriminator part of the GAN model is designed to accurately distinguish between the ground truth image and the segmented image generated by the generator part. Our method is evaluated on two cardiac MRI datasets. The evaluation results on the ACDC 2017 challenge dataset show mean Dice scores of 0.947 for LV, 0.919 for RV, and 0.907 for MYO in cardiac MRI segmentation. The experimental results highlight that our proposed IRAU-Net method outperforms other state-of-the-art methods in terms of accuracy while significantly reducing computational costs.

Audio-Visual Target Speaker Extraction (AV-TSE) aims to mimic the human ability to enhance auditory perception using visual cues. Although numerous models have been proposed recently, most of them estimate target signals by primarily relying on local dependencies within acoustic features, underutilizing the human-like capacity to infer unclear parts of speech through contextual information. This limitation results in not only suboptimal performance but also inconsistent extraction quality across the utterance, with some segments exhibiting poor quality or inadequate suppression of interfering speakers. To close this gap, we propose a model-agnostic strategy called the Mask-And-Recover (MAR). It integrates both inter- and intra-modality contextual correlations to enable global inference within extraction modules. Additionally, to better target challenging parts within each sample, we introduce a Fine-grained Confidence Score (FCS) model to assess extraction quality and guide extraction modules to emphasize improvement on low-quality segments. To validate the effectiveness of our proposed model-agnostic training paradigm, six popular AV-TSE backbones were adopted for evaluation on the VoxCeleb2 dataset, demonstrating consistent performance improvements across various metrics.

Iris recognition has become a trend in the biometric discipline to identify an individual and is widely used compared to other biometrics and the traditional recognition system. Various factors, such as the obstruction of eyelids, eyelashes, shadows, specular reflection by eyeglasses, and other noises, can cause the occlusion of iris images. This study aims to propose a hybrid U-Net with ResNet segmentation method (HUREIS) in occlusion iris images for iris recognition. The HUREIS was compared with VGG-16 to determine its benchmark based on segmentation ac curacy and computational time. This study employed a MICHE I dataset that only considers occlusion images for eyelids and eyelashes for mobile devices. The findings showed that HUREIS performed the best dice score at 93.7% and Jaccard score at 88.3%, respectively. The performance of HUREIS is 5.4% better than VGG-16, with approximately 84% minimum computational time. The segmentation accuracy was compared with the state of-the-art methods based on the MICHE-I dataset and the U Net segmentation method to validate the effectiveness of the proposed iris segmentation method. Furthermore, this study evaluated the feature vector of HUREIS and VGG-16 to obtain the performance of iris classification using a support vector machine and decision tree C4.5 based on Euclidean distance and Cosine similarity. The Euclidean distance and cosine similarity were compared to determine the best image similarity score close to one. The results show that SVM for HUREIS achieved the optimal classification accuracy at 100% using Euclidean distance. Compared to decision tree C4.5, VGG-16 achieved 94.41% of accuracy using Euclidean distance.

Neural audio coding has emerged as a vivid research direction by promising good audio quality at very low bitrates unachievable by classical coding techniques. Here, end-to-end trainable autoencoder-like models represent the state of the art, where a discrete representation in the bottleneck of the autoencoder is learned. This allows for efficient transmission of the input audio signal. The learned discrete representation of neural codecs is typically generated by applying a quantizer to the output of the neural encoder. In almost all state-of-the-art neural audio coding approaches, this quantizer is realized as a Vector Quantizer (VQ) and a lot of effort has been spent to alleviate drawbacks of this quantization technique when used together with a neural audio coder. In this paper, we propose and analyze simple alternatives to VQ, which are based on projected Scalar Quantization (SQ). These quantization techniques do not need any additional losses, scheduling parameters or codebook storage thereby simplifying the training of neural audio codecs. For real-time speech communication applications, these neural codecs are required to operate at low complexity, low latency and at low bitrates. We address those challenges by proposing a new causal network architecture that is based on SQ and a Short-Time Fourier Transform (STFT) representation. The proposed method performs particularly well in the very low complexity and low bitrate regime.

This paper presents comprehensive optimizations of a neural audio coder built upon a variational autoencoder (VAE) system integrated with an arithmetic coder. Our optimizations focus on two primary aspects: a novel loss function design and advanced entropy modeling of bottleneck latent embeddings. The loss function design incorporates parameters from a psychoacoustic model (PAM) into the frame-wise distortion measure, providing excellent perceptual quality. In addition, a multi-time scale discriminator is utilized to minimize distortions across adjacent frames, reducing artifacts at frame edges. Also, the coder is optimized considering three sophisticated entropy models within the latent domain: the Factorized Entropy Model (FEM), the Hyperprior Model (HPM), and the Joint Hierarchical Model (JHM). Notably, the JHM enhances context modeling across frames to effectively predict components influenced by long-term dependencies. To verify the optimization performance, we conducted extensive experiments using a dataset consisting of commercial movie clips and two additional public datasets. Objective metrics consistently demonstrated that our optimized loss function and latent modeling achieved superior performance across all test datasets compared to traditional codecs such as LAME-MP3 and FDK-AAC. Subjective assessments also indicated that our system could offer comparable or superior auditory quality to FDK-AAC.