Unknown network protocol classification method based on semi-supervised learning

BackgroundGenetic interaction profiles are highly informative and helpful for understanding the functional linkages between genes, and therefore have been extensively exploited for annotating gene functions and dissecting specific pathway structures. However, our understanding is rather limited to the relationship between double concurrent perturbation and various higher level phenotypic changes, e.g. those in cells, tissues or organs. Modifier screens, such as synthetic genetic arrays (SGA) can help us to understand the phenotype caused by combined gene mutations. Unfortunately, exhaustive tests on all possible combined mutations in any genome are vulnerable to combinatorial explosion and are infeasible either technically or financially. Therefore, an accurate computational approach to predict genetic interaction is highly desirable, and such methods have the potential of alleviating the bottleneck on experiment design.ResultsIn this work, we introduce a computational systems biology approach for the accurate prediction of pairwise synthetic genetic interactions (SGI). First, a high-coverage and high-precision functional gene network (FGN) is constructed by integrating protein-protein interaction (PPI), protein complex and gene expression data; then, a graph-based semi-supervised learning (SSL) classifier is utilized to identify SGI, where the topological properties of protein pairs in weighted FGN is used as input features of the classifier. We compare the proposed SSL method with the state-of-the-art supervised classifier, the support vector machines (SVM), on a benchmark dataset in S. cerevisiae to validate our method's ability to distinguish synthetic genetic interactions from non-interaction gene pairs. Experimental results show that the proposed method can accurately predict genetic interactions in S. cerevisiae (with a sensitivity of 92% and specificity of 91%). Noticeably, the SSL method is more efficient than SVM, especially for very small training sets and large test sets.ConclusionsWe developed a graph-based SSL classifier for predicting the SGI. The classifier employs topological properties of weighted FGN as input features and simultaneously employs information induced from labelled and unlabelled data. Our analysis indicates that the topological properties of weighted FGN can be employed to accurately predict SGI. Also, the graph-based SSL method outperforms the traditional standard supervised approach, especially when used with small training sets. The proposed method can alleviate experimental burden of exhaustive test and provide a useful guide for the biologist in narrowing down the candidate gene pairs with SGI. The data and source code implementing the method are available from the website: http://home.ustc.edu.cn/~yzh33108/GeneticInterPred.htm

Semi-supervised learning (SSL) relies on partial supervision information for prediction, where only a small set of samples are associated with labels. Performance of SSL is significantly degraded if the given labels are not reliable. Such problems arise in realistic applications such as web image search using noisy textual tags. This paper proposes a novel and efficient graph based SSL method with the unique capacity of pruning contradictory labels and inferring new labels through a bidirectional and alternating optimization process. The objective is to automatically identify the most suitable samples for manipulation, labeling or unlabeling, and meanwhile estimate a smooth classification function over a weighted graph. Different from other graph based SSL approaches, the proposed method employs a bivariate objective function and iteratively modifies label variables on both labeled and unlabeled samples. Starting from such a SSL setting, we present a relearning framework to improve the performance of base learner, particularly for the application of web image search. Besides the toy demonstration on artificial data, we evaluated the proposed method on Flickr image search with unreliable textual labels. Experimental results confirm the significant improvements of the method over the baseline text based search engine and the state-of-the-art SSL methods.

To evaluate a novel method of semisupervised learning (SSL) guided by automated sparse information from diagnostic reports to leverage additional data for deep learning-based malignancy detection in patients with clinically significant prostate cancer. This retrospective study included 7756 prostate MRI examinations (6380 patients) performed between January 2014 and December 2020 for model development. An SSL method, report-guided SSL (RG-SSL), was developed for detection of clinically significant prostate cancer using biparametric MRI. RG-SSL, supervised learning (SL), and state-of-the-art SSL methods were trained using 100, 300, 1000, or 3050 manually annotated examinations. Performance on detection of clinically significant prostate cancer by RG-SSL, SL, and SSL was compared on 300 unseen examinations from an external center with a histopathologically confirmed reference standard. Performance was evaluated using receiver operating characteristic (ROC) and free-response ROC analysis. P values for performance differences were generated with a permutation test. At 100 manually annotated examinations, mean examination-based diagnostic area under the ROC curve (AUC) values for RG-SSL, SL, and the best SSL were 0.86 ± 0.01 (SD), 0.78 ± 0.03, and 0.81 ± 0.02, respectively. Lesion-based detection partial AUCs were 0.62 ± 0.02, 0.44 ± 0.04, and 0.48 ± 0.09, respectively. Examination-based performance of SL with 3050 examinations was matched by RG-SSL with 169 manually annotated examinations, thus requiring 14 times fewer annotations. Lesion-based performance was matched with 431 manually annotated examinations, requiring six times fewer annotations. RG-SSL outperformed SSL in clinically significant prostate cancer detection and achieved performance similar to SL even at very low annotation budgets.Keywords: Annotation Efficiency, Computer-aided Detection and Diagnosis, MRI, Prostate Cancer, Semisupervised Deep Learning Supplemental material is available for this article. Published under a CC BY 4.0 license.

Semi-supervised kernel learning methods have been received much more attention in the past few years. Traditional semi-supervised Non-Parametric Kernel Learning (NPKL) methods usually formulate the learning task as a Semi-Definite Programming (SDP) problem, which is very time consuming. Although some fast semi-supervised NPKL methods have been proposed recently, they usually scale very poorly. Furthermore, many semi-supervised NPKL methods are developed based on the manifold assumption. But, such an assumption might be invalid when handling some high-dimensional and sparse data, which has severely negative effect on the performance of learning algorithms. In this paper, we propose a more efficient semi-supervised NPKL method, which can effectively learn a low-rank kernel matrix from must-link and cannot-link constraints. Specially, by virtue of the nonlinear spectral embedded technique based on extreme learning machine (ELM), the proposed method has the ability of coping with data points that do not have a clear manifold structure in a low dimensional space. The proposed method is formulated as a trace ratio optimization problem, which is combined with dimensionality reduction in ELM feature space and aims to find optimal low-rank kernel matrices. The proposed optimization problem can be solved much more efficiently than SDP solvers. Extensive experiments have validated the superior performance of the proposed method compared to state-of-the-art semi-supervised kernel learning methods.

Network security plays a very important role in today's web enabled world. In the 21st century, network traffic has increased because of the enormous growth in online users and their online communication. The number of security attacks is an increased with increase in internet users. The frequency and severity of such attacks have shown a great impact on network performance. Many classical machine learning algorithms have been proposed to solve the problem of intrusion detection with varying levels of success. Nowadays, availability of libelled data is a big issue. It is not only time consuming, but also expensive. Semi-supervised learning methods can make use of unlabelled examples in addition to the labelled ones. The developing field of semi-supervised learning, offers a promising direction for supplementary research. In this paper, we introduce a new semi-supervised mechanism for intrusion detection, which efficiently reduces false alarms, still maintaining a high detection rate. In our proposed semi-supervised learning approach, only a small quantity of labelled data and a large amount of unlabelled data has been used. This will improve the overall network security by reducing the security administrator's efforts and making the alert mechanism more practical.

Semi-supervised learning (SSL) is one of the dominant approaches to address the annotation bottleneck of supervised learning. Recent SSL methods can effectively leverage a large repository of unlabeled data to improve performance while relying on a small set of labeled data. One common assumption in most SSL methods is that the labeled and unlabeled data are from the same data distribution. However, this is hardly the case in many real-world scenarios, which limits their applicability. In this work, instead, we attempt to solve the challenging open-world SSL problem that does not make such an assumption. In the open-world SSL problem, the objective is to recognize samples of known classes, and simultaneously detect and cluster samples belonging to novel classes present in unlabeled data. This work introduces OpenLDN that utilizes a pairwise similarity loss to discover novel classes. Using a bi-level optimization rule this pairwise similarity loss exploits the information available in the labeled set to implicitly cluster novel class samples, while simultaneously recognizing samples from known classes. After discovering novel classes, OpenLDN transforms the open-world SSL problem into a standard SSL problem to achieve additional performance gains using existing SSL methods. Our extensive experiments demonstrate that OpenLDN outperforms the current state-of-the-art methods on multiple popular classification benchmarks while providing a better accuracy/training time trade-off. Code: https://github.com/nayeemrizve/OpenLDN.KeywordsOpen-worldSemi-supervised learningNovel classes

Using an ensemble of neural networks with consistency regularization is effective for improving performance and stability of deep learning, compared to the case of a single network. In this paper, we present a semi-supervised Deep Coupled Ensemble (DCE) model, which contributes to ensemble learning and classification landmark exploration for better locating the final decision boundaries in the learnt latent space. First, multiple complementary consistency regularizations are integrated into our DCE model to enable the ensemble members to learn from each other and themselves, such that training experience from different sources can be shared and utilized during training. Second, in view of the possibility of producing incorrect predictions on a number of difficult instances, we adopt class-wise mean feature matching to explore important unlabeled instances as classification landmarks, on which the model predictions are more reliable. Minimizing the weighted conditional entropy on unlabeled data is able to force the final decision boundaries to move away from important training data points, which facilitates semi-supervised learning. Ensemble members could eventually have similar performance due to consistency regularization, and thus only one of these members is needed during the test stage, such that the efficiency of our model is the same as the non-ensemble case. Extensive experimental results demonstrate the superiority of our proposed DCE model over existing state-of-the-art semi-supervised learning methods.

Automated diagnosis of infant hip dysplasia is heavily affected by the individual differences among infants and ultrasound machines. Hip sonographic images of 493 infants from various ultrasound machines were collected in the Department of Orthopedics in Yangzhou Maternal and Child Health Care Service Centre. Herein, we propose a semi-supervised learning method based on a feature pyramid network (FPN) and a contrastive learning scheme based on a Siamese architecture. A large amount of unlabeled data of ultrasound images was used via the Siamese network in the pre-training step, and then a small amount of annotated data for anatomical structures was adopted to train the model for landmark identification and standard plane recognition. The method was evaluated on our collected dataset. The method achieved a mean Dice similarity coefficient (DSC) of 0.7873 and a mean Hausdorff distance (HD) of 5.0102 in landmark identification, compared to the model without contrastive learning, which had a mean DSC of 0.7734 and a mean HD of 6.1586. The accuracy, precision, and recall of standard plane recognition were 95.4%, 91.64%, and 94.86%, respectively. The corresponding area under the curve (AUC) was 0.982. This study proposes a semi-supervised deep learning method following Graf's principle, which can better utilize a large volume of ultrasound images from various devices and infants. This method can identify the landmarks of infant hips more accurately than manual operators, thereby improving the efficiency of diagnosis of infant hip dysplasia.

Fisher discriminant analysis (FDA) is a classic supervised dimensionality reduction method in statistical pattern recognition. FDA can maximize the scatter between different classes, while minimizing the scatter within each class. As it only utilizes the labeled data and ignores the unlabeled data in the analysis process of FDA, it cannot be used to solve the unsupervised learning problems. Its performance is also very poor in dealing with semi-supervised learning problems in some cases. Recently, several semi-supervised learning methods as an extension of FDA have proposed. Most of these methods solve the semi-supervised problem by using a tradeoff parameter that evaluates the ratio of the supervised and unsupervised methods. In this paper, we propose a general semi-supervised dimensionality learning idea for the partially labeled data, namely the reconstruction probability class of labeled and unlabeled data. Based on the probability class optimizes Fisher criterion function, we propose a novel Semi-Supervised Local Fisher Discriminant Analysis (S2LFDA) method. Experimental results on real-world datasets demonstrate its effectiveness compared to the existing similar correlation methods.

In the literature, most existing graph-based semi-supervised learning methods only use the label information of observed samples in the label propagation stage, while ignoring such valuable information when learning the graph. In this paper, we argue that it is beneficial to consider the label information in the graph learning stage. Specifically, by enforcing the weight of edges between labeled samples of different classes to be zero, we explicitly incorporate the label information into the state-of-the-art graph learning methods, such as the low-rank representation (LRR), and propose a novel semi-supervised graph learning method called semi-supervised low-rank representation. This results in a convex optimization problem with linear constraints, which can be solved by the linearized alternating direction method. Though we take LRR as an example, our proposed method is in fact very general and can be applied to any self-representation graph learning methods. Experiment results on both synthetic and real data sets demonstrate that the proposed graph learning method can better capture the global geometric structure of the data, and therefore is more effective for semi-supervised learning tasks.

This paper investigates the use of semi-supervised learning (SSL) for predicting Alzheimers Disease (AD) conversion in Mild Cognitive Impairment (MCI) patients based on Magnetic Resonance Imaging (MRI). SSL methods differ from standard supervised learning methods in that they make use of unlabeled data - in this case data from MCI subjects whose final diagnosis is not yet known. We compare two widely used semi-supervised methods (low density separation (LDS) and semi-supervised discriminant analysis (SDA)) to the corresponding supervised methods using real and synthetic MRI data of MCI subjects. With simulated data, using SSL instead of supervised learning led to higher classification performance in certain cases, however, the applicability of semi-supervised methods depended strongly on the data distributions. With real MRI data, the SSL methods achieved significantly better classification performances over supervised methods. Moreover, even using a small number of unlabeled samples improved the AD conversion predictions.

Aiming at improving performance of visual classification in a cost-effective manner, this paper proposes an incremental semi-supervised learning paradigm called Deep Co-Space (DCS). Unlike many conventional semi-supervised learning methods usually performing within a fixed feature space, our DCS gradually propagates information from labeled samples to unlabeled ones along with deep feature learning. We regard deep feature learning as a series of steps pursuing feature transformation, i.e., projecting the samples from a previous space into a new one, which tends to select the reliable unlabeled samples with respect to this setting. Specifically, for each unlabeled image instance, we measure its reliability by calculating the category variations of feature transformation from two different neighborhood variation perspectives, and merged them into an unified sample mining criterion deriving from Hellinger distance. Then, those samples keeping stable correlation to their neighboring samples (i.e., having small category variation in distribution) across the successive feature space transformation, are automatically received labels and incorporated into the model for incrementally training in terms of classification. Our extensive experiments on standard image classification benchmarks (e.g., Caltech-256 and SUN-397) demonstrate that the proposed framework is capable of effectively mining from large-scale unlabeled images, which boosts image classification performance and achieves promising results compared to other semi-supervised learning methods.

Multitask learning in distributed environments is an important problem in the field of federated learning. In many practical problems, human or machine experts try to solve different, but highly correlated tasks, thus their performance can be improved through collaboration, but their private datasets and model parameters cannot be shared with other participants because of business and privacy issues. Semi-supervised learning is also actively researched nowadays, which is motivated by the utilization of the usually free, large set of unlabeled data in concept and representation learning. These unlabeled samples can be utilized in the knowledge sharing of the participants during their collaboration. We propose a method, called FedLinked, which combines the capabilities of these area (federated, multitask and semi-supervised learning) related algorithms in the form of a privacy preserving complex solution-block targeting semi-supervised multitask cross-silo problems. FedLinked is a collaborative learning method, which is implemented through the regularization of representation learning based on the cross-utility of participating clients. We evaluate FedLinked on image classification problems and compare its performance to FedAvg based and non-cooperating clients based solutions in multitask scenarios. Besides, a detailed analysis of our proposed method is provided.

The article examines the application of semi-supervised learning and computer vision methods for segmentation of spine MR images and diagnosis of deformation of intervertebral discs. Existing neural network architectures for spine MR image segmentation and semi-supervised learning methods used in medical image segmentation are reviewed. A system for diagnosis deformation of intervertebral discs is proposed, which consists of two modules: a segmentation module and a diagnostic module. The implementation of the proposed system using two convolutional neural networks is presented: U-Net for segmentation of spine MR images and ResNet for classification of the degree of deformation of each intervertebral disc based on the Pfirrmann classification of degenerative changes. The software implementation of the medical diagnosis system was developed in the Python programming language using the PyTorch library. Neural networks were trained on an open dataset of spine MR images using a variant of the Mean Teacher semi-supervised learning method. As a result of the verification, it was found that the system is capable of performing segmentation with high accuracy. It was found that the exact prediction of the grade of degenerative changes according to Pfirrmann remains a difficult task, but the introduction of another classification made it possible to increase the accuracy of diagnosis of deformation of intervertebral discs. The proposed medical system involves the addition of new diagnostic modules, which makes it possible to use it for the comprehensive analysis of various spine diseases

Training machine learning and deep learning models for medical image classification is a challenging task due to a lack of large, high-quality labeled datasets. As the labeling of medical images requires considerable time and effort from medical experts, models need to be specifically designed to train on low amounts of labeled data. Therefore, an application of semi-supervised learning (SSL) methods provides one potential solution. SSL methods use a combination of a small number of labeled datasets with a much larger number of unlabeled datasets to achieve successful predictions by leveraging the information gained through unsupervised learning to improve the supervised model. This paper provides a comprehensive survey of the latest SSL methods proposed for medical image classification tasks.