Semi-Supervised Machine Learning and Adaptive Data Clustering Approach for Software Defect Prediction

Automatic classification of construction safety reports using semi-supervised YAKE-Guided LDA approach

Multireference (MR) diagnostics are common tools for identifying strongly correlated electronic structure that makes single-reference (SR) methods (e.g., density functional theory or DFT) insufficient for accurate property prediction. However, MR diagnostics typically require computationally demanding correlated wave function theory (WFT) calculations, and diagnostics often disagree or fail to predict MR effects on properties. To overcome these challenges, we introduce a semi-supervised machine learning (ML) approach with virtual adversarial training (VAT) of an MR classifier using 15 WFT and DFT MR diagnostics as inputs. In semi-supervised learning, only the most extreme SR or MR points are labeled, and the remaining point labels are learned. The resulting VAT model outperforms the alternatives, as quantified by the distinct property distributions of SR- and MR-classified molecules. To reduce the cost of generating inputs to the VAT model, we leverage the VAT model's robustness to noisy inputs by replacing WFT MR diagnostics with regression predictions in an MR decision engine workflow that preserves excellent performance. We demonstrate the transferability of our approach to larger molecules and those with distinct chemical composition from the training set. This MR decision engine demonstrates promise as a low-cost, high-accuracy approach to the automatic detection of strong correlation for predictive high-throughput screening.

Recent heightened enthusiasm towards Industrial Artificial Intelligence (IAI) and Industrial Internet of Things (IIoT) coupled with developments in smart sensor technologies have resulted in simultaneous incorporation of several advanced Condition Monitoring (CM) technologies within manufacturing and industrial sectors. Efficient utilization of CM data leads to enhanced safety, reliability and availability of manufacturing systems. In this regard, the paper proposes an efficient and novel hybrid Maintenance Decision Support System (MDSS) for fault diagnostic and prognostic considering CM data along with event- triggered data. The proposed MDSS model is a hybrid Machine Learning (ML)-based solution coupled with statistical techniques. In order to find an optimal maintenance policy, we concentrate the attention on a time-dependent Proportional Hazards Model (PHM) augmented with a semi-supervised ML approach. The developed hybrid model is capable of inferring and fusing High-Dimensional and Multi-modal Streaming (HDMS) data sources in an adaptive and autonomous fashion to recommend optimal maintenance decisions without human intervention. To illustrate the complete structure of the proposed MDSS, experimental evaluations are designed based on a dataset provided by NASA containing run-to-failure and CM data associated with aircraft engines. The effectiveness of the proposed model is demonstrated through a comprehensive set of comparisons with different ML algorithms.

Semi-supervised learning is drawing increasing attention in the era of big data, as the gap between the abundance of cheap, automatically collected unlabeled data and the scarcity of labeled data that are laborious and expensive to obtain is dramatically increasing. In this thesis, we first introduce a unified view of density-based clustering algorithms. Then, we build upon this view and bridge the areas of semi-supervised clustering and classification under a common umbrella of density-based techniques. We show that there are close relations between density-based clustering algorithms and the graph-based approach for transductive classification. These relations are then used as a basis for a new framework for semi-supervised classification based on building-blocks from density-based clustering. This framework is not only efficient and effective, but it is also statistically sound. We also generalize the core algorithm of the framework HDBSCAN* so that it can also perform semi-supervised clustering by directly taking advantage of any fraction of labeled data that may be available, rather than instance-level pairwise constraints. Experimental results on a large collection of datasets show the advantages of the proposed approach both for semi-supervised classification, as well as for semi-supervised clustering. In addition, we evaluate the semi-supervised learning algorithms to determine relationships between chemical structure and biological activity in datasets from Medicinal Chemistry. The datasets evaluated in this area are characterized by a low number of labeled examples, a high dimensionality, and in some cases, do not have a clear relationship between chemical structure and biological activity, which makes it difficult to use classification techniques and analyze chemical phenomena. We implement and validate semi-supervised classification approaches that are appropriate for data analysis in Medicinal Chemistry.

A semi-supervised deep learning approach for cropped image detection

A number of attempts to classify cancer samples using miRNA/gene expression profiles are known in literature. However, semi-supervised learning models have only been recently introduced to exploit the huge unlabeled expression profiles in enhancing sample classification. It is important to combine both miRNA and gene expression sets as that provides more information on the characteristics of cancer samples. The use of both of labeled and unlabeled miRNA and gene expression sets to enhance sample classification has not been explored yet. In this paper, two semi-supervised machine learning approaches, namely self-learning and co-training are adapted to enhance the quality of cancer sample classification. In self-learning, miRNA and gene based classifiers are enhanced independently. While in co-training, both miRNA and gene expression profiles are used simultaneously to provide different views of cancer samples. The approaches were evaluated using breast cancer, hepatocellular carcinoma (HCC) and lung cancer expression sets. Results show up to 20% improvement in F1-measure over Random Forests and SVM classifiers. Co-Training also outperforms Low Density Separation (LDS) approach by around 25% improvement in F1-measure in breast cancer.

In gas metal arc welding (GMAW) processes, including wire arc additive manufacturing (WAAM), machine learning (ML) is emerging as a powerful tool for monitoring both process and product anomalies. However, a significant challenge in real industrial environments is the reliance on large, balanced datasets for training supervised learning models. To address this issue, a shift toward unsupervised learning is gaining attention in this research field, offering the potential to work effectively with small and unbalanced datasets. However, different materials, sensors, and welding technologies have been used in the literature, making complex the comparison of the results. This work fills that gap by presenting a comprehensive comparison of both supervised and unsupervised learning methods. An experimental campaign was conducted on Invar 36 alloy—a material with limited WAAM research—where 15 wall structures were deposited with varying process parameters using the natural dip transfer process, aiming to identify the optimal parameters for this alloy. Data on welding current and voltage were captured, and during the qualification procedure, anomalies were detected, some of which led to product defects. Supervised, unsupervised, and semi-supervised ML approaches, along with a detailed frequency domain analysis of the collected signals, were applied to process the obtained unbalanced dataset. The results provide key insights: while supervised learning models can be applied to anomaly detection in small and unbalanced datasets, they are prone to overfitting, which limits their practical use due to the prevalence of normal cases over anomalies in the dataset, resulting in higher number of missed anomalies. In contrast, unsupervised models, with their lower generalization capability, tend to exhibit higher false alarm rates but better performance to identify anomalous data. This work not only compares in depth these data analytics methodologies but also offers guidance on selecting the appropriate ML algorithm based on specific industrial objectives and provides insights into the printability of Invar 36 for WAAM applications under natural dip transfer process.

A novel approach for software defect prediction through hybridizing gradual relational association rules with artificial neural networks

The problem of utilizing machine learning approach in Indonesian Named Entity Recognition (NER) system is the limited amount of labelled data for training process. However, unlike the limited availability of labelled data, unlabelled data is widely available from many sources. This enables a semi-supervised learning approach to solve this NER system problem. This research aims to design a semi-supervised learning model to solve NER system problem. A semi-supervised co-training learning is used to utilize unlabelled data in NER learning process to produce new labelled data that can be applied to enhance a new NER classification system. This research uses two kinds of data, Indonesian DBPedia data as labelled data and news article text from Indonesian news sites (kompas.com, cnnindonesia.com, tempo.co, merdeka.com and viva.co.id) as unlabelled data. The pre-processing steps applied to analyze unstructured text are sentence segmentation, tokenization, stemming, and PoS Tagging. The results of this pre-process are the NER and its context used as unlabelled data for the semi-supervised co-training process. The SVM algorithm is used as a classi□cation algorithm in this process. 10 Cross Fold Validation is used as the system testing approach. Based on the result of the NER testing system, the precision is 73.6%, the recall is 80.1% and f1 mean is 76.5%.

Using a semi-supervised machine learning approach we present a real-time anomaly detection system based on an autoencoder used for online data quality monitoring of the CMS electromagnetic calorimeter operating at the CERN LHC. We introduce a novel method that maximizes the anomaly detection performance making use of the time-dependence of anomalies and the spatial variations in the detector response. The autoencoder-based system efficiently detects anomalies in real time and maintains a very low false discovery rate. We validate the performance of this novel system with anomalies from LHC collision data taken in 2018 and 2022. In addition, results are presented after deploying the autoencoder-based system in the CMS online Data Quality Monitoring workflow at the beginning of LHC Run 3 resulting in the system to detect issues that were missed by the existing system.

Network Traffic Classification is the crucial phase of network monitoring. The network traffic once categorized per application it can be imposed with appropriate security policies to improve the performance of the network. The port and payload based traffic classification techniques used in the past decade relapsed owing to new techniques of encryption and tunneling emerging day-by-day. Recently, the statistical classification employing data mining techniques and analyzing the attributes for characterization of network traffic proved high efficiency. This proposed work characterized the network traffic based on semi-supervised machine learning approach. It initially clustered the auxiliary and content flow of application to precisely track the footprint of the application using improved k-means clustering which in turn is fed into C5.0 classifier to construct the classifier model. This system is designed to classify the network traffic accurately that resulted in high F-Score value of 0.993.

Ontology learning (OL) arises as an area to support semantic engineering because it enables to recover and to extract knowledge from the Web documents to improve the development of domain ontologies. One of the most fruitful fields of OL is artificial intelligence (AI), since it sustains new methods, techniques and tools, particularly related with Web- and text-mining. In this work, we are dealing with a meta-model incrementally developed, proposing an OL experiment with open source tools. To reach that, firstly, ontology about a university institution previously developed is increasable synthesized. Secondly, a particular methodology and strategy for OL is used to illustrate how could be included these text-mining and OL tools into a kind of (semi-) intelligent-agent, and how the overall ontological development process could be improved. Particularly, the OL "agent" test tool is applied to update/extend the university ontology by a semi-supervised machine learning approach.

High-dimensional semi-supervised learning via a fusion-refinement procedure

In this paper, the application of semi-supervised manifold learning techniques to the task of verifying hypothesized occurrences of spoken terms is investigated. These techniques are applied in a two stage spoken term detection framework where ASR lattices are first generated using a large vocabulary ASR system and hypothesized occurrences of spoken query terms in the lattices are verified in a second stage. The verification process is performed using a fixed dimensional feature representation derived from each hypothesized term occurrence. Two semi-supervised approaches namely, manifold regularized least squares (RLS) classification and spectral clustering, are investigated for distinguishing correct hypotheses from false alarms. It is shown that, exploiting unlabeled data in addition to labeled data using semi-supervised approaches, significantly improves the verification performance compared to the case where only the labeled data is used. This improvement in performance increases as the ratio of unlabeled to labeled data augments. It is also shown that, when training data is very limited, a comparable verification performance can be gained by exploiting only the acoustic similarity between the test samples using the spectral clustering approach.

As software continues to insinuate itself into nearly every aspect of our life, the quality of software has been an extremely important issue. Software Quality Assurance (SQA) is a process that ensures the development of high-quality software. It concerns the important problem of maintaining, monitoring, and developing quality software. Accurate detection of fault prone components in software projects is one of the most commonly practiced techniques that offer the path to high quality products without excessive assurance expenditures. This type of quality modeling requires the availability of software modules with known fault content developed in similar environment. However, collection of fault data at module level, particularly in new projects, is expensive and time-consuming. Semi-supervised learning and active learning offer solutions to this problem for learning from limited labeled data by utilizing inexpensive unlabeled data.;In this dissertation, we investigate semi-supervised learning and active learning approaches in the software fault prediction problem. The role of base learner in semi-supervised learning is discussed using several state-of-the-art supervised learners. Our results showed that semi-supervised learning with appropriate base learner leads to better performance in fault proneness prediction compared to supervised learning. In addition, incorporating pre-processing technique prior to semi-supervised learning provides a promising direction to further improving the prediction performance. Active learning, sharing the similar idea as semi-supervised learning in utilizing unlabeled data, requires human efforts for labeling fault proneness in its learning process. Empirical results showed that active learning supplemented by dimensionality reduction technique performs better than the supervised learning on release-based data sets.