Posistive-Unlabeled Learning via Optimal Transport and Margin Distribution

Tax evasion detection has a crucial role in addressing tax revenue loss. In the real world, an accessed tax dataset only contains a small number of labeled taxpayers who evade tax (positive samples) and a large number of unlabeled taxpayers who either evade tax or do not evade tax. It is difficult to address this issue due to this nontraditional dataset. In addition, the basic features of taxpayers designed according to tax experts’ domain knowledge and experience are very limited to determining whether taxpayers evade tax. These limitations motivate the contribution of this work. In this paper, we argue that the tax evasion detection task in the real world should be formalized as a positive unlabeled (PU) learning problem. We propose a novel tax evasion detection method based on PU learning with Network Embedding features (PUNE). PUNE effectively detects tax evasion based on basic features and transaction network features that are extracted by a network embedding algorithm. Moreover, PUNE can work well even under label noise. To evaluate the effectiveness of PUNE, we conduct experimental tests on a real-world tax dataset. The results demonstrate that PUNE can significantly improve the performance of tax evasion detection.

Conventional supervised binary classification algorithms have been widely applied to address significant research questions using biological and biomedical data. This classification scheme requires two fully labeled classes of data (e.g. positive and negative samples) to train a classification model. However, in many bioinformatics applications, labeling data is laborious, and the negative samples might be potentially mislabeled due to the limited sensitivity of the experimental equipment. The positive unlabeled (PU) learning scheme was therefore proposed to enable the classifier to learn directly from limited positive samples and a large number of unlabeled samples (i.e. a mixture of positive or negative samples). To date, several PU learning algorithms have been developed to address various biological questions, such as sequence identification, functional site characterization and interaction prediction. In this paper, we revisit a collection of 29 state-of-the-art PU learning bioinformatic applications to address various biological questions. Various important aspects are extensively discussed, including PU learning methodology, biological application, classifier design and evaluation strategy. We also comment on the existing issues of PU learning and offer our perspectives for the future development of PU learning applications. We anticipate that our work serves as an instrumental guideline for a better understanding of the PU learning framework in bioinformatics and further developing next-generation PU learning frameworks for critical biological applications.

Positive-unlabeled learning for open set domain adaptation

Positive-unlabeled (PU) learning deals with the binary classification problem when only positive (P) and unlabeled (U) data are available, without negative (N) data. Existing PU methods perform well on the balanced dataset. However, in real applications such as financial fraud detection or medical diagnosis, data are always imbalanced. It remains unclear whether existing PU methods can perform well on imbalanced data. In this paper, we explore this problem and propose a general learning objective for PU learning targeting specially at imbalanced data. By this general learning objective, state-of-the-art PU methods based on optimizing a consistent risk can be adapted to conquer the imbalance. We theoretically show that in expectation, optimizing our learning objective is equivalent to learning a classifier on the oversampled balanced data with both P and N data available, and further provide an estimation error bound. Finally, experimental results validate the effectiveness of our proposal compared to state-of-the-art PU methods.

lncRNAs are involved in many biological processes, and their mutations and disorders are closely related to many diseases. Identification of LncRNA-Disease Associations (LDAs) helps us understand the pathogenesis of diseases and improve their diagnosis and treatment. However, experiments to determine LDAs are expensive, so it is essential to exploit effective computational methods to screen possible LDAs. In this study, we developed an LDA prediction model (LDA-DLPU) based on deep learning and positive-unlabeled (PU) learning. First, LDA-DLPU extracted features of lncRNAs and diseases based on singular value decomposition and regression model. Second, it selected negative LDAs based on PU learning and graph autoencoder. Finally, it classified unknown lncRNA-disease pairs based on deep neural network. LDA-DLPU obtained the best performance on two datasets. We predict that BCYRN1 and IFNG-AS1 may associate with leukemia.

Recently a promising research direction of statistical learning has been advocated, i.e., the optimal margin distribution learning, with the central idea of optimizing the margin distribution. As the most representative approach of this new learning paradigm, the optimal margin distribution machine (ODM) considers maximizing the margin mean and minimizing the margin variance simultaneously. The standard ODM exploits the l_2-norm penalty, which gives rise to a dense decision boundary. However, in some situations, the model with parsimonious representation is more preferred, due to the redundant noisy features or limited computing resources. In this paper, we propose the sparse optimal margin distribution machine (Sparse ODM), which aims to achieve better generalization performance with moderate model size. For optimization, we extends an efficient coordinate descent method to solve the final problem since the variables are decoupled. In each iteration, we propose a modified Newton method to solve the one-variable sub-problem. Experimental results on both synthetic and real data sets show the superiority of the proposed method.

Epilepsy is a chronic disorder of the brain. Intracranial electroencephalogram (iEEG) recorded from cortex is the most popular measurement for not only the diagnosis of epilepsy, but also the focus localization that is crucial for the surgery. In recent years, the machine learning methods have been rapidly developed and applied successfully to various real world problems. Given sufficient number of samples, the powerful deep learning methods can achieve high performance for epileptic focus localization. However, it is a challenging task to obtain large amount of labeled iEEG regarding focal/non-focal channels, since the annotations must be performed by multiple clinical experts through visual judgment on the long term iEEG signals. In order to reduce the necessary number of labeled training samples, we introduce the positive unlabeled (PU) learning method for classification of focal and non-focal epileptic iEEG signals. The proposed method enables us to learn a binary classifier by using small amount of labeled data containing only one class (i.e., focal signals) and unlabeled data containing two classes (i.e., focal and non-focal signals), which greatly reduces the workload of clinical experts for annotations. Experimental results on Bern dataset and iEEG recorded from Juntendo University Hospital demonstrate the effectiveness of our method.

C-PUGP: A cluster-based positive unlabeled learning method for disease gene prediction and prioritization

Support Vector Machine (SVM) has always been one of the most successful learning algorithms, with the central idea of maximizing the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">minimum margin</i> , i.e., the smallest distance from the instances to the classification boundary. However, recent theoretical results disclosed that maximizing the minimum margin does not necessarily lead to better generalization performance, and instead, the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">margin distribution</i> has been proven to be more crucial. Based on this idea, we propose the Optimal margin Distribution Machine (ODM), which can achieve a better generalization performance by optimizing the margin distribution explicitly. We characterize the margin distribution by the first- and second-order statistics, i.e., the margin mean and variance. The proposed method is a general learning approach which can be applied in any place where SVMs are used, and its superiority is verified both theoretically and empirically in this paper.

ε-Support vector regression (ε-SVR) is a powerful machine learning approach that focuses on minimizing the margin, which represents the tolerance range between predicted and actual values. However, recent theoretical studies have highlighted that simply minimizing structural risk does not necessarily result in well margin distribution. Instead, it has been shown that the distribution of margins plays a more crucial role in achieving better generalization performance. Furthermore, the kernel-free technique offers a significant advantage as it effectively reduces the overall running time and simplifies the parameter selection process compared to the kernel trick. Based on existing kernel-free regression methods, we present two efficient and robust approaches named quadratic hyper-surface kernel-free large margin distribution machine-based regression (QLDMR) and quadratic hyper-surface kernel-free least squares large margin distribution machine-based regression (QLSLDMR). The QLDMR optimizes the margin distribution by considering both ε-insensitive loss and quadratic loss function similar to the large-margin distribution machine-based regression (LDMR). QLSLDMR aims to reduce the cost of the computing process of QLDMR, which transforms inequality constraints into an equality constraint inspired by least squares support vector machines (LSSVR). Both models combined the spirit of optimal margin distribution with kernel-free technique and after simplification are convex so that they can be solved by some classical methods. Experimental results demonstrate the superiority of the optimal margin distribution combined with the kernel-free technique in robustness, generalization, and efficiency.

A kernel-free Laplacian quadratic surface optimal margin distribution machine with application to credit risk assessment

Deceptive reviews detection has attracted significant attention from both business and research communities. However, due to the difficulty of human labeling needed for supervised learning, the problem remains to be highly challenging. This paper proposed a novel angle to the problem by modeling PU (positive unlabeled) learning. A semi-supervised model, called mixing population and individual property PU learning (MPIPUL), is proposed. Firstly, some reliable negative examples are identified from the unlabeled dataset. Secondly, some representative positive examples and negative examples are generated based on LDA (Latent Dirichlet Allocation). Thirdly, for the remaining unlabeled examples (we call them spy examples), which can not be explicitly identified as positive and negative, two similarity weights are assigned, by which the probability of a spy example belonging to the positive class and the negative class are displayed. Finally, spy examples and their similarity weights are incorporated into SVM (Support Vector Machine) to build an accurate classifier. Experiments on gold-standard dataset demonstrate the effectiveness of MPIPUL which outperforms the state-of-the-art baselines.

A meta-learning framework using representation learning to predict drug-drug interaction

Intelligent fault detection methods based on deep learning have been developed rapidly in recent years. However, most of these methods are based on supervised learning which requires a fully labeled training set. It is difficult to obtain massive labeled samples in real applications incredibly accurately labeled fault samples from an operating system. The lack of labels and label noise becomes a great challenge for fault detection. To tackle this problem, in this article, we propose a positive-unlabeled learning based hybrid network (PUHN). It only needs part of the normal operating samples to be labeled. All other samples (including the rest of the normal samples and all fault samples) are unlabeled, which greatly reduces the labeling cost. PUHN consists of three modules: a nonnegative risk positive-unlabeled (PU) network for training the classifier, a feature extraction module, and a clustering layer for improving data separability and estimating the class priors of PU learning. The three are optimized as a whole and the corresponding optimization strategy is designed. The monitoring data of 24 wind turbines are used to verify the effectiveness and robustness of the proposed method. The experimental results indicate that the proposed method is superior to the benchmark methods, and the performance is significantly better than the supervised learning method when there exists label noise.

How can we classify graph-structured data only with positive labels? Graph-based positive-unlabeled (PU) learning is to train a binary classifier given only the positive labels when the relationship between examples is given as a graph. The problem is of great importance for various tasks such as detecting malicious accounts in a social network, which are difficult to be modeled by supervised learning when the true negative labels are absent. Previous works for graph-based PU learning assume that the prior distribution of positive nodes is known in advance, which is not true in many real-world cases. In this work, we propose GRAB (Graph-based Risk minimization with iterAtive Belief propagation), a novel end-to-end approach for graph-based PU learning that requires no class prior. GRAB models a given graph as a Markov network and runs the marginalization and update steps iteratively. The marginalization step estimates the marginals of latent variables, while the update step trains a classifier network utilizing the computed priors in the objective function. Extensive experiments on five datasets show that GRAB achieves state-of-the-art accuracy, even compared with previous methods that are given the true prior.