Asymmetric Error Control Under Imperfect Supervision: A Label-Noise-Adjusted Neyman–Pearson Umbrella Algorithm

The Neyman–Pearson (NP) classification paradigm addresses an important binary classification problem where users want to minimize type II error while controlling type I error under some specified level α, usually a small number. This problem is often faced in many genomic applications involving binary classification tasks. The terminology Neyman–Pearson classification paradigm arises from its connection to the Neyman–Pearson paradigm in hypothesis testing. The NP paradigm is applicable when one type of error (e.g., type I error) is far more important than the other type (e.g., type II error), and users have a specific target bound for the former. In this chapter, we review the NP classification literature, with a focus on the genomic applications as well as our contribution to the NP classification theory and algorithms. We also provide simulation examples and a genomic case study to demonstrate how to use the NP classification algorithm in practice.

Objective. Learning to classify cardiac abnormalities requires large and high-quality labeled datasets, which is a challenge in medical applications. Small datasets from various sources are often aggregated to meet this requirement, resulting in a final dataset prone to label noise due to inter- and intra-observer variability and different expertise. It is well known that label noise can affect the performance and generalizability of the trained models. In this work, we explore the impact of label noise and self-learning label correction on the classification of cardiac abnormalities on large heterogeneous datasets of electrocardiogram (ECG) signals. Approach. A state-of-the-art self-learning multi-class label correction method for image classification is adapted to learn a multi-label classifier for electrocardiogram signals. We evaluated our performance using 5-fold cross-validation on the publicly available PhysioNet/Computing in Cardiology (CinC) 2021 Challenge data, with full and reduced sets of leads. Due to the unknown label noise in the testing set, we tested our approach on the MNIST dataset. We investigated the performance under different levels of structured label noise for both datasets. Main results. Under high levels of noise, the cross-validation results of self-learning label correction show an improvement of approximately 3% in the challenge score for the PhysioNet/CinC 2021 Challenge dataset and an improvement in accuracy of 5% and reduction of the expected calibration error of 0.03 for the MNIST dataset. We demonstrate that self-learning label correction can be used to effectively deal with the presence of unknown label noise, also when using a reduced number of ECG leads.

This article addresses the challenges in classifying textual data obtained from open online platforms, which are vulnerable to distortion. Most existing classification methods minimize the overall classification error and may yield an undesirably large Type I error (relevant textual messages are classified as irrelevant), particularly when available data exhibit an asymmetry between relevant and irrelevant information. Data distortion exacerbates this situation and often leads to fallacious prediction. To deal with inestimable data distortion, we propose the use of the Neyman–Pearson (NP) classification paradigm, which minimizes Type II error under a user-specified Type I error constraint. Theoretically, we show that the NP oracle is unaffected by data distortion when the class conditional distributions remain the same. Empirically, we study a case of classifying posts about worker strikes obtained from a leading Chinese microblogging platform, which are frequently prone to extensive, unpredictable and inestimable censorship. We demonstrate that, even though the training and test data are susceptible to different distortion and therefore potentially follow different distributions, our proposed NP methods control the Type I error on test data at the targeted level. The methods and implementation pipeline proposed in our case study are applicable to many other problems involving data distortion. Supplementary materials for this article, including a standardized description of the materials available for reproducing the work, are available as an online supplement.

This paper proposes the method for cleaning up label noise in multivariate time-series outlier data. An image plotting method is proposed to reflect the tendency of original time-series data. The image data generated from the plotting method shows effectiveness, since the data can be smoothly utilized in label noise cleaning process and easy to analyze. To verify the availability of plotted data, label noise cleasing algorithm is combined to backbone network, which can estimate original noise ratio and select the data evaluated as true outlier. In addition, self-attention module is also combined to backbone network, to induce backbone network to concentrate on outlier data in plotted data. Finally, the performance of overall algorithm is evaluated through experiment based on open dataset. Using generated data through proposed plotting method, noise ratio estimation accuracy and label noise cleaning performance are evaluated. Additionally, the comparison between the proposed algorithm and the algorithm without self-attention module is also attended.

Fed-DR-Filter: Using global data representation to reduce the impact of noisy labels on the performance of federated learning

Due to the complex nature of medical image acquisition and annotation, medical datasets inevitably contain noise. This adversely affects the robustness and generalization of deep neural networks. Previous noise learning methods mainly considered noise arising from images being mislabeled, i.e., label noise, assuming all mislabeled images were of high quality. However, medical images can also suffer from severe data quality issues, i.e., data noise, where discriminative visual features for disease diagnosis are missing. In this paper, we propose QMix, a noise learning framework that learns a robust disease diagnosis model under mixed noise scenarios. QMix alternates between sample separation and quality-aware semi-supervised training in each epoch. The sample separation phase uses a joint uncertainty-loss criterion to effectively separate (1) correctly labeled images, (2) mislabeled high-quality images, and (3) mislabeled low-quality images. The semi-supervised training phase then learns a robust disease diagnosis model from the separated samples. Specifically, we propose a sample-reweighing loss to mitigate the effect of mislabeled low-quality images during training, and a contrastive enhancement loss to further distinguish them from correctly labeled images. QMix achieved state-of-the-art performance on six public retinal image datasets and exhibited significant improvements in robustness against mixed noise. Code will be available upon acceptance.

In a practical classification problem, there are cases where incorrect labels are included in training data due to label noise. We introduce a classification method in the presence of label noise that idealizes a classification method based on the expectation-maximization (EM) algorithm, and evaluate its performance theoretically. Its performance is asymptotically evaluated by assessing the risk function defined as the Kullback-Leibler divergence between predictive distribution and true distribution. The result of this performance evaluation enables a theoretical evaluation of the most successful performance that the EM-based classification method may achieve.

Atrial fibrillation (AF) is a common cardiac arrhythmia with serious health consequences if not detected and treated early. Detecting AF using wearable devices with photoplethysmography (PPG) sensors and deep neural networks has demonstrated some success using proprietary algorithms in commercial solutions. However, to improve continuous AF detection in ambulatory settings towards a population-wide screening use case, we face several challenges, one of which is the lack of large-scale labeled training data. To address this challenge, we propose to leverage AF alarms from bedside patient monitors to label concurrent PPG signals, resulting in the largest PPG-AF dataset so far (8.5 M 30-second records from 24,100 patients) and demonstrating a practical approach to build large labeled PPG datasets. Furthermore, we recognize that the AF labels thus obtained contain errors because of false AF alarms generated from imperfect built-in algorithms from bedside monitors. Dealing with label noise with unknown distribution characteristics in this case requires advanced algorithms. We, therefore, introduce and open-source a novel loss design, the cluster membership consistency (CMC) loss, to mitigate label errors. By comparing CMC with state-of-the-art methods selected from a noisy label competition, we demonstrate its superiority in handling label noise in PPG data, resilience to poor-quality signals, and computational efficiency.

We present a "learning to learn" approach for discovering white-box classification loss functions that are robust to label noise in the training data. We parameterise a flexible family of loss functions using Taylor polynomials, and apply evolutionary strategies to search for noise-robust losses in this space. To learn re-usable loss functions that can apply to new tasks, our fitness function scores their performance in aggregate across a range of training datasets and architectures. The resulting white-box loss provides a simple and fast "plug-and-play" module that enables effective label-noise-robust learning in diverse downstream tasks, without requiring a special training procedure or network architecture. The efficacy of our loss is demonstrated on a variety of datasets with both synthetic and real label noise, where we compare favourably to prior work.

Reliable fault detection is essential for ensuring the safe and efficient operation of electrochemical energy storage systems, including lithium-ion batteries and transformer. However, the performance of machine learning-based fault diagnosis models is often degraded in practice due to label noise in training data, caused by sensor inaccuracies, ambiguous fault transitions, and imperfect labeling processes. This paper proposes a lightweight and effective kernel-based data rectification framework to improve the robustness of fault detection under noisy label conditions. The method identifies and discards low-density data points that are statistically more likely to be mislabeled, using kernel density estimation and a tunable data discarding strategy. The approach is computationally efficient, classifier-agnostic, and easily applicable to existing fault diagnosis pipelines. We evaluate the proposed method on two datasets: simulated lithium-ion battery voltage data under various fault scenarios, and transformer winding oscillation wave data under multiple winding fault conditions. The results demonstrate that the rectification framework significantly improves classification accuracy across both Support Vector Machine (SVM) and Extreme Learning Machine (ELM) classifiers. Furthermore, the choice of discarding ratio is shown to be critical, with optimal performance achieved when the ratio is tuned close to the underlying noise level. These results highlight the potential of the proposed method to enhance the reliability of fault diagnosis in electrochemical energy storage systems. Future work will explore adaptive strategies to automatically optimize the rectification strength without requiring prior knowledge of the noise rate, and extend the framework to multi-sensor and multi-modal monitoring scenarios.

Noise-robust re-identification with triple-consistency perception

Animal species classification using deep neural networks with noise labels

Area under the curve may hide poor generalisation to external datasets

Standard damp heat (DH), temperature cycle (TC), and combined DH-TC tests were performed using monocrystalline Si 72-cell modules with a conventional ethylene vinyl acetate (EVA) encapsulant, and their module performance and electroluminescence images were investigated. During the DH test, a significant drop (~20%) in the maximum output power of the module was noticed, primarily because of the degradation of fill factor and an increase in series resistance at 5500 h of DH testing (DH5500), presumably due to the corrosion of metal electrodes by moisture ingress. Conversely, it was revealed that temperature cycling did not seriously degrade module performance until 1400 cycles. However, the combined DH5000-TC600 test suggested in this study, with a sequence of DH1000-TC200-DH1000-TC200-DH1000-TC200-DH2000, was confirmed to provide harsher conditions than the DH-only test by causing a 20% decrease in maximum output power (Pmax) after DH3000/TC400. Promisingly, we confirmed that the module with a polyolefin elastomer encapsulant showed better durability than the module with EVA even in the combined DH-TC test, showing a limited decrease in Pmax (~10%) even after the DH5500/TC600 test.

A dual-wavelength Erbium-doped fiber laser (EDFL) is demonstrated using a 1.5-m long EDF with a microfiber knot resonator (MKR) structure at the output end. The MKR is fabricated by heating and stretching a 3-cm long piece of EDF to form a microfiber, which is then manipulated to create a knot. The performance of the EDFL is investigated for two different MKR loop diameters. At a loop diameter of 4 mm, dual-wavelength laser is obtained at 1532.4 and 1556.0 nm with peak powers of −7.5 and −7.1 dBm and the corresponding optical signal-to-noise ratios of 22 and 32 dB, respectively. As the MKR diameter is reduced to 2 mm, the operating wavelength of laser shifts to a shorter wavelength, especially for the second lasing with a significant drop in output power due to the bending loss, which is more pronounced at smaller loop diameter. The slope efficiency of the EDFL drops from 0.55 to 0.09% as the MKR diameter decreases from 4 to 2 mm. © 2012 Wiley Periodicals, Inc. Microwave Opt Technol Lett 55:236–238, 2012; View this article online at wileyonlinelibrary.com. DOI 10.1002/mop.27292