Fairness in Semi-Supervised Learning: Unlabeled Data Help to Reduce Discrimination

Automatic analysis methods of electrocardiograms (ECGs) usually required large-scale annotated training data, but the annotation process is extremely time-consuming. While semi-supervised learning can leverage unlabeled data, its performance depends heavily on the quality of the initial labeled subset. Active learning has been used to identify the most informative samples for annotation, but conventional approaches face three critical limitations: (1) dependency on manual intervention for iterative query design, (2) prohibitive computational costs during sample selection, and (3) limited compatibility with semi-supervised learning frameworks. To address these limitations, we proposed an Unsupervised Active Feature-selective Semi-Supervised Learning (UAFSSL) framework for ECG analysis, including an unsupervised feature selection-based active learning module and a semi-supervised learning module. UAFSSL captures latent data distributions via unsupervised feature extraction, selects diverse and representative samples using pseudo-label clustering, and integrates seamlessly with semi-supervised learning to eliminate human intervention. We validated our algorithm on an ECG waveform segmentation task and an atrial fibrillation detection task. In the waveform segmentation task, our method improved the F1-score for P-wave delineation by 2.4% compared to random sampling, using only 5% of labeled samples. For the atrial fibrillation detection task, we evaluated our method on both the AFDB and a 24-hour dataset collected from 500 atrial fibrillation patients. Using only 200 labeled samples for model training, our method achieved AUC improvements of 2.5% and 2.2% over random sampling in five-fold cross validation. This is the first study to integrate unsupervised active learning with semi-supervised learning for automatic ECG analysis, offering a robust, automated solution to reduce annotation costs while enhancing clinical applicability.

Machine Learning (ML) involves the training of a model to make predictions based on data. A model might be described by several parameters, normally, weights and biases, where training is the process to determine the ideal parameters that comprise model, based on the dataset (a collection of raw data arranged in columns). ML can be divided in four main types, supervised learning, where training is based on labelled data; unsupervised learning, based on unlabelled data, where the goal is to find hidden patterns or structures within the data; semi-supervised learning, it uses a combination of labelled and unlabelled data; and finally, reinforcement learning, the learning is through trial and error, where the model receives reward by taking desirable actions and penalties for undesirable, the goal is to maximise the reward. There are a great variety of ML techniques for various tasks, such as dimensionality reduction, clustering, and regression, commonly applied for image recognition, speech recognition and predicting modelling. Scale inhibitor (SI) squeeze treatment is one of the most common techniques to inhibit the deposition of oilfield scale. It consists in bull-heading a scale inhibitor slug in producing wells, the chemical retains on the rock surface and is slowly released when the well is back in production, the well will be protected from scaling, if the produced chemical concentration is above a certain threshold, usually few ppm. Generally, process modelling is applied to predict the SI return concentration vs water produced, to determine the treatment design lifetime, based on the operating parameters, formation and inhibitor properties, particularly retention capability. Commonly, a historically matched field isotherm results in predictions with significant degree of accuracy, where the main disadvantage is that historical data for the producing well under consideration is necessary. The purpose of this manuscript is to develop a ML model, where the input is the treatment design, which consists of chemical slug volume, chemical concentration, and overflush volume, well formation characteristics (perforation heights, permeability and porosity), and the output the scale inhibitor return concentration vs. volume of water produced. The ML model was trained, tested and blind validated using squeeze treatment data for a field in the North Sea. The dataset consists of 141 field squeeze treatments, 9 for blind validation, and 131 for 80/20 training and testing split. The best ML model Random Decision Forrest (RDF), which resulted in significant accurate prediction over the 9 blind squeeze treatments. This model may be used as proxy for the process modelling of squeeze treatments.

Improving state-of-health estimation for lithium-ion batteries via unlabeled charging data

Semi-supervised learning (SSL) aims to train a machine learning (ML) model using both labeled and unlabeled data. While the unlabeled data have been used in various ways to improve the prediction accuracy, the reason why unlabeled data could help is not fully understood. One interesting and promising direction is to understand SSL from a causal perspective. In light of the independent causal mechanisms (ICM) principle, the unlabeled data can be helpful when the label causes the features but not vice versa. However, the causal relations between the features and labels can be complex in real world applications. In this article, we propose an SSL framework that works with general causal models in which the variables have flexible causal relations. More specifically, we explore the causal graph structures and design corresponding causal generative models which can be learned with the help of unlabeled data. The learned causal generative model can generate synthetic labeled data for training a more accurate predictive model. We verify the effectiveness of our proposed method by empirical studies on both simulated and real data.

We propose UnMixMatch, a semi-supervised learning framework which can learn effective representations from unconstrained unlabelled data in order to scale up performance. Most existing semi-supervised methods rely on the assumption that labelled and unlabelled samples are drawn from the same distribution, which limits the potential for improvement through the use of free-living unlabeled data. Consequently, the generalizability and scalability of semi-supervised learning are often hindered by this assumption. Our method aims to overcome these constraints and effectively utilize unconstrained unlabelled data in semi-supervised learning. UnMixMatch consists of three main components: a supervised learner with hard augmentations that provides strong regularization, a contrastive consistency regularizer to learn underlying representations from the unlabelled data, and a self-supervised loss to enhance the representations that are learnt from the unlabelled data. We perform extensive experiments on 4 commonly used datasets and demonstrate superior performance over existing semi-supervised methods with a performance boost of 4.79%. Extensive ablation and sensitivity studies show the effectiveness and impact of each of the proposed components of our method. The code for our work is publicly available.

Semi-supervised learning and continuous learning are fundamental paradigms for human-level intelligence. To deal with real-world problems where labels are rarely given and the opportunity to access the same data is limited, it is necessary to apply these two paradigms in a joined fashion. In this paper, we propose Label Propagation Adaptive Resonance Theory (LPART) for semi-supervised continuous learning. LPART uses an online label propagation mechanism to perform classification and gradually improves its accuracy as the observed data accumulates. We evaluated the proposed model on visual (MNIST, SVHN, CIFAR-10) and audio (NSynth) datasets by adjusting the ratio of the labeled and unlabeled data. The accuracies are much higher when both labeled and unlabeled data are used, demonstrating the significant advantage of LPART in environments where the data labels are scarce.

Gathering labeled data is one of the most time-consuming and expensive tasks in supervised machine learning. In practical applications, there are usually quite limited labeled training samples but abundant unlabeled data that is easy to collect. Semi-supervised learning and active learning are two important techniques for learning a discriminative classification model when labeled data is scarce. However, unlabeled data with significant noises and outliers cannot be well exploited and usually worsen the performance of semi-supervised learning and the performance of active learning also needs a powerful initial classifier learned from the quite limited labeled training data. In order to solve the above issues, in this paper we proposed a novel model of semi-supervised dictionary active learning (SSDAL), which aims to integrate semi-supervised learning and active learning to effectively use all the training data. In particular, two criterions based on estimated class possibility are designed to select the unlabeled data with confident class estimation for semi-supervised learning and the informative unlabeled data for active learning, respectively. Extensive experiments are conducted to show the superior performance of our method in classification applications, e.g., handwritten digit recognition, face recognition and large-scale image classification.

In real life, there is far more unprocessed data than labeled data, which brings a large amount of data that cannot be directly used for machine learning training. Based on the tweet dataset processed by Natural Language Processing (NLP), this paper uses a variety of machine learning models for training and comparison. Moreover, different performances are analyzed and discussed. Since labeled datasets are difficult to obtain, the use of supervised learning will be limited. However, the number of unlabeled datasets is very large, which can provide a continuous training set for machine learning. This paper conducted a comparative experiment on the effect of semi-supervised learning and obtained better results than supervised learning and unsupervised learning. The experiments in this paper prove that semi-supervised learning can effectively use unlabeled data and train machine learning models.

Data is one of the most important limiting factors of deep machine learning (ML) model in drilling applications. Though a big size of historical data can be available, high-quality cleaned and labeled data is usually limited. In this case study, we show that with limited labeled data, physics-based data augmentation combined with unsupervised learning significantly improves both stability and accuracy in bit wear ML model. It provides a pathway to overcome labeled data shortage and field data quality limitations. Labeled bit wear data is usually limited because only the final bit dull state can be labeled using dull photos for the entire drilling distance. To overcome this difficulty, an encoder-decoder unsupervised ML framework based on Bi-directional LSTM architecture is first applied to the data of entire drilling distance to extract and maximize data learnings. Then, a physics-based wear estimator is implemented in the learned latent space to augment labeled wear dataset, guided by true wear labels and offset-well data. Weighting factor in loss function is applied to optimize augmented and true datasets in final supervised learning step. The proposed real-time bit wear model is built on a combination of physics-based data augmentation and unsupervised data learning method. The model is applied on multiple field bit runs. Results show that by implementing the unsupervised data learnings only, the prediction accuracy is improved by 30% compared to the baseline ML model. By combining physics-based augmentation, the accuracy is further improved by 10%. More importantly, adding physics-based data augmentation significantly reduces prediction variance and unphysical wear outputs, therefore improving prediction stability by more than 30%. It should be noted that the prediction stability of AI/ML model is crucial in real-time application and decision making. The results show physics-based data augmentation not only increases the size of label dataset and prevents model overfitting, but also applies physics-based guidance to the ML model effectively. It is learned weighting factor plays a crucial role in balancing loss contributions from true wear labels and physics-based labels. While low weighting factor of physics-based labels diminishes the augmented data, high weighting factor disrespects the true wear labels leading to a high prediction bias. Overall, the bit wear model study shows physic-based data augmentation combined with unsupervised data learning can effectively improve model accuracy, stability, and overcome labeled data shortage difficulty. The proposed paper shows a case study for the bit wear ML model using a combination of physics-based data augmentation and unsupervised data learning. While labeled data is one of the major challenges of ML models in many drilling applications, this study provides a pathway to improve both accuracy and stability of deep ML models with limited labeled data.

Machine Learning is a modern and actively developing field of computer science, devoted to extracting and estimating dependencies from empirical data. It combines such fields as statistics, optimization theory and artificial intelligence. In practical tasks, the general aim of Machine Learning is to construct algorithms able to generalize and predict in previously unseen situations based on some set of examples. Given some finite information, Machine Learning provides ways to exract knowledge, describe, explain and predict from data. Kernel Methods are one of the most successful branches of Machine Learning. They allow applying linear algorithms with well-founded properties such as generalization ability, to non-linear real-life problems. Support Vector Machine is a well-known example of a kernel method, which has found a wide range of applications in data analysis nowadays. In many practical applications, some additional prior knowledge is often available. This can be the knowledge about the data domain, invariant transformations, inner geometrical structures in data, some properties of the underlying process, etc. If used smartly, this information can provide significant improvement to any data processing algorithm. Thus, it is important to develop methods for incorporating prior knowledge into data-dependent models. The main objective of this thesis is to investigate approaches towards learning with kernel methods using prior knowledge. Invariant learning with kernel methods is considered in more details. In the first part of the thesis, kernels are developed which incorporate prior knowledge on invariant transformations. They apply when the desired transformation produce an object around every example, assuming that all points in the given object share the same class. Different types of objects, including hard geometrical objects and distributions are considered. These kernels were then applied for images classification with Support Vector Machines. Next, algorithms which specifically include prior knowledge are considered. An algorithm which linearly classifies distributions by their domain was developed. It is constructed such that it allows to apply kernels to solve non-linear tasks. Thus, it combines the discriminative power of support vector machines and the well-developed framework of generative models. It can be applied to a number of real-life tasks which include data represented as distributions. In the last part of the thesis, the use of unlabelled data as a source of prior knowledge is considered. The technique of modelling the unlabelled data with a graph is taken as a baseline from semi-supervised manifold learning. For classification problems, we use this apporach for building graph models of invariant manifolds. For regression problems, we use unlabelled data to take into account the inner geometry of the input space. To conclude, in this thesis we developed a number of approaches for incorporating some prior knowledge into kernel methods. We proposed invariant kernels for existing algorithms, developed new algorithms and adapted a technique taken from semi-supervised learning for invariant learning. In all these cases, links with related state-of-the-art approaches were investigated. Several illustrative experiments were carried out on real data on optical character recognition, face image classification, brain-computer interfaces, and a number of benchmark and synthetic datasets.

Semi-supervised learning acts as an effective way to leverage massive unlabeled data. In this paper, we propose a novel training strategy, termed as Semi-supervised Contrastive Learning (SsCL), which combines the well-known contrastive loss in self-supervised learning with the cross entropy loss in semi-supervised learning, and jointly optimizes the two objectives in an end-to-end way. The highlight is that different from self-training based semi-supervised learning that conducts prediction and retraining over the same model weights, SsCL interchanges the predictions over the unlabeled data between the two branches, and thus formulates a co-calibration procedure, which we find is beneficial for better prediction and avoid being trapped in local minimum. Towards this goal, the contrastive loss branch models pairwise similarities among samples, using the pseudo labels generated from the cross entropy branch, and in turn calibrates the prediction distribution of the cross entropy branch with the contrastive similarity. We show that SsCL produces more discriminative representation and is beneficial to semi-supervised learning. Notably, on ImageNet with ResNet50 as the backbone, SsCL achieves 60.2% and 72.1% top-1 accuracy with 1% and 10% labeled samples, respectively, which significantly outperforms the baseline, and is better than previous semi-supervised and self-supervised methods.

Semi-supervised learning (SSL) has seen great strides when labeled data is scarce but unlabeled data is abundant. Critically, most recent work assume that such unlabeled data is drawn from the same distribution as the labeled data. In this work, we show that state-of-the-art SSL algorithms suffer a degradation in performance in the presence of unlabeled auxiliary data that does not necessarily possess the same class distribution as the labeled set. We term this problem as Auxiliary-SSL and propose AuxMix, an algorithm that leverages self-supervised learning tasks to learn generic features in order to mask auxiliary data that are not semantically similar to the labeled set. We also propose to regularize learning by maximizing the predicted entropy for dissimilar auxiliary samples. We show an improvement of 5% over existing baselines on a ResNet-50 model when trained on CIFAR10 dataset with 4k labeled samples and all unlabeled data is drawn from the Tiny-ImageNet dataset. We report competitive results on several datasets and conduct ablation studies.

Background: The air quality of any area depends upon the various PMs (particulate matter) and hazardous gases present in the air. Low-cost PM sensors and gas sensors are present in different target places to monitor the air quality, read the environmental data, and transmit it to local servers through the IoT device. The low-cost sensor is not reliable due to its low sensing capacity; therefore, the read data is calibrated with the meteorological data presented by the nearby meteorological Centre of that particular area. The calibrated reading data sent to the server could be analyzed through some Machine Learning [ML] models. The ML models help to predict the risk of asthma in a particular area. The risk of asthma is directly related to the air quality of the surroundings. It is observed that the air quality of the industrial area is much worse than the non-industrial belt. Air quality monitoring of industrial areas is always a challenging task due to the ununiformed pollution in some segregated places around the industry, emitting pollutants mostly from chimneys. The air quality of any area depends upon the PM (PM), i.e., PM2.5 and PM10.0, as well as the gasses like NO2(Nitrogen Dioxide), NH3 (Ammonia), SO2(Sulfur dioxide), CO(Carbon monoxide), O3(Ozone) and Benzene. These are the most hazardous gases generally emitted by common heavy industries like iron and steel. In this article, the researchers considered the industrial belt of the Asansol- Durgapur region of West Bengal, India, and predicted the risk of asthma attacks for the test dataset. The experiment was carried out on 10 different supervised machine learning [SML] models as well as semi-supervised machine learning (SSML) models. The SML models have been further refined through hyper-parameter tuning, and better results have been obtained in the case of some ML models. The result has been compared with the existing literature considering the same external environment from where the meteorological data was collected, and similar ML models have been used. The research outperformed the existing literature, which is depicted in the result and analysis section of the article. Methods: The study evaluated ML models, both supervised and semi-supervised, to assess pollution levels. Relevant features were selected while less relevant ones were discarded. Accuracy levels of different ML algorithms werecompared in the results. The most effective model for an IoT system was chosen to maximize accuracy. In semi-supervised learning, feature selection followed supervised learning, but testing was akin to unsupervised learning. Results were compared with supervised learning data, enhancing reliability. Results: The result employing various classifiers werepresented across tables after the independent parameter Ozone was removed. Following the output of several classifiers, the results were verified using the k-fold validation method, with k being set to 5 or 10, accordingly. Tables display the best outcome, which is indicated in bold characters. method: In this research work the researcher considered 9 different ML models and used them as supervised as well as semi supervised model to determine the pollution level of the certain area. In this research work the researcher also selected the most relevant features and discarded the less relevant features. In case of SML algorithm, the accuracy level of the different ML algorithm has been determined and depicted in the result analysis section. The most effective ML model has been chosen for the proposed embedded system so that accuracy could be achieved at most. In case of semi supervised algorithm the feature selection is done as per the supervised algorithm. In this case the training is done same as the SML algorithm, but the testing phase is done like unsupervised machine learning algorithm where the decision parameter is predicted and ultimately matched with the previously achieved data of SML algorithm. The reliability of this approach is much more effective than simple SML algorithm. Conclusion: This study focused on predicting asthma risk in the Asansol-Durgapur industrial belt, India, using low-cost PM and gas sensors. Data calibration with meteorological inputs enhanced accuracy. ML models predicted risk and were refined through hyper-parameter tuning. Comparative analysis showed superior performance, emphasizing the importance of precise air quality monitoring. While offering a robust framework for future research, the study’s limitation lies in its area-specific dataset.

A Semi-Supervised Machine Learning Model for Risk Analysis of Urinary Tract Infections in People with Dementia

Exploiting unlabeled data for battery state-of-health estimation using transformer-LSTM neural network with semi-supervised learning