Fully Bayesian analysis of the relevance vector machine classification for imbalanced data problem

This research has three parts addressing classification, especially the imbalanced data problem, which is one of the most popular and essential issues in the domain of classification. The first part is to study the Adaptive Boosting (AdaBoost) algorithm. AdaBoost is an effective solution for classification, but it still needs improvement in the imbalanced data problem. This part proposes a method to improve the AdaBoost algorithm using the new weighted vote parameters for the weak classifiers. Our proposed weighted vote parameters are determined not only by the global error rate but also by the classification accuracy rate of the positive class, which is our primary interest. The imbalanced index of the data is also a factor in constructing our algorithms. The numeric studies show that our proposed algorithms outperform the traditional ones, especially regarding the evaluation criterion of the F--1 Measure. Theoretic proofs of the advantages of our proposed algorithms are presented. The second part treats the Relevance Vector Machine (RVM), which is a supervised learning algorithm extended from the Support Vector Machine (SVM) based on the Bayesian sparsity model. Compared with the regression problem, RVM classification is challenging to conduct because there is no closed-form solution for the weight parameter posterior. The original RVM classification algorithm uses Newton's method in optimization to obtain the mode of weight parameter posterior, then approximates it by a Gaussian distribution in Laplace's method. This original model would work, but it just applies the frequency methods in a Bayesian framework. This part first proposes a Generic Bayesian RVM classification, which is a pure Bayesian model. We conjecture that our algorithm achieves convergent estimates of the quantities of interest compared with the nonconvergent estimates of the original RVM classification algorithm. Furthermore, a fully Bayesian approach with the hierarchical hyperprior structure for RVM classification is proposed, which improves the classification performance, especially in the imbalanced data problem. The third part is an extended work of the second one. The original RVM classification model uses the logistic link function to build the likelihood, which makes the model hard to conduct since the posterior of the weight parameter has no closed-form solution. This part proposes the probit link function approach instead of the logistic one for the likelihood function in RVM classification, namely PRVM (RVM with the Probit link function). We show that the posterior of the weight parameter in our model follows the multivariate normal distribution and achieves a closed-form solution. A latent variable is needed in our algorithm to simplify the Bayesian computation greatly, and its conditional posterior follows a truncated normal distribution. Compared with the original RVM classification model, our proposed one is another pure Bayesian approach and it has a more efficient computation process. For the prior structure, we first consider the Normal-Gamma independent prior to propose a Generic Bayesian PRVM algorithm. Furthermore, the Fully Bayesian PRVM algorithm with a hierarchical hyperprior structure is proposed, which improves the classification performance, especially in the imbalanced data problem.

Artificial intelligence (AI)-based rotating machinery fault diagnosis has extreme importance in the industrial automation and control systems since rotating machinery constitutes approximately 40% of the overall machinery in the industry. However, the majority of AI-based solutions in rotor faults diagnosis are in an experimental stage due to: 1) inadequate and unrealistic faulty data; 2) lack of opportunities to utilize domain-specific fault features; and 3) limitations in employing deep learning and advanced learning strategies. Moreover, structural rotor fault (SRF) is one of the critical but least addressed faults in rotor faults diagnosis even though it is the root cause of the majority of rotating machinery issues. Hence, we develop an SRF diagnosis framework, which addresses the issues of industrial data acquisition by creating a subsampled data set incorporating distinctive frequency components (DFC). The data scarcity and imbalance problems are handled through an augmentation method using soft-dynamic time warping (soft-DTW), enhanced by fault information content (FIC)-based weighing scheme. At the fault classification phase, we proposed an early classification (EC) approach for SRF that predicts the faults with an acceptable tradeoff between earliness and accuracy. In this line, first, a sequential deep learning classifier is developed by considering accuracy only as an objective. Then, early decision policy is defined by taking accuracy and earliness into account. The model demonstrated exceptional performance over state-of-the-art methods in SRF diagnosis and achieved 99.5% accuracy with 14% earliness on the Meggitt testbed data set and 98.32% accuracy with 55.68% earliness on the MaFaulDa data set.

To address the imbalanced data problem in molecular docking-based virtual screening methods, this paper proposes a virtual screening method for drug proteins based on imbalanced data mining, which introduces machine learning technology into the virtual screening technology for drug proteins to deal with the imbalanced data problem in the virtual screening process and improve the accuracy of the virtual screening. First, to address the data imbalance problem caused by the large difference between the number of active compounds and the number of inactive compounds in the docking conformation generated by the actual virtual screening process, this paper proposes a way to improve the data imbalance problem using SMOTE combined with genetic algorithm to synthesize new active compounds artificially by upsampling active compounds. Then, in order to improve the accuracy in the virtual screening process of drug proteins, the idea of integrated learning is introduced, and the random forest (RF) extended from Bagging integrated learning technique is combined with the support vector machine (SVM) technique, and the virtual screening of molecular docking conformations using RF-SVM technique is proposed to improve the prediction accuracy of active compounds in docking conformations. To verify the effectiveness of the proposed technique, first, HIV-1 protease and SRC kinase were used as test data for the experiments, and then, CA II was used to validate the model of the test data. The virtual screening of drug proteins using the proposed method in this paper showed an improvement in both enrichment factor (EF) and AUC compared with the use of the traditional virtual screening, for the test dataset. Therefore, it can be shown that the proposed method can effectively improve the accuracy of drug virtual screening.

3DACN: 3D Augmented convolutional network for time series data

An unsolved problem in the digital mapping of categorical soil variables and soil types is the imbalanced number of observations, which leads to reduced accuracy and the loss of the minority class (the class with a significantly lower number of observations compared to other classes) in the final map. So far, synthetic over‐ and under‐sampling techniques have been explored in soil science; however, more efficient approaches that do not have the drawbacks of these techniques and guarantee retention of the minority classes in the produced map are essentially required. Such approaches suggested in the present study for digital mapping of soil classes include machine learning models of ensemble gradient boosting, cost‐sensitive learning and one‐class classification (OCC) of the minority class combined with multi‐class classification. In this regard, extreme gradient boosting (XGB) as an ensemble gradient learner, a cost‐sensitive decision tree (CSDT) within the C5.0 algorithm, and a one‐class support vector machine combined with multi‐class classification (OCCM) were investigated to map eight soil great groups with a naturally imbalanced frequency of observations in northwest Iran. A total of 453 profile data points were used for mapping the soil great groups of the study area. A data split was done manually for each class separately, which resulted in an overall 70% of the data for calibration and 30% for validation. The bootstrapping approach of calibration (with 10 runs) was performed to produce multiple maps for each model. The 10 bootstraps were evaluated against the hold‐out validation dataset. The average values of accuracy measures, including Kappa (K), overall accuracy (OA), producer's accuracy (PA) and user's accuracy (UA), were explored. In addition, the results of this study were compared with a previous study in the same area, in which resampling techniques were used to deal with imbalanced data for digital soil class mapping. The findings show that all three suggested methods can deal well with the imbalanced classification problem, with OCCM showing the highest K (= 0.76) and OA (= 82) in the validation stage. Also, this model can guarantee the retention of the minority classes in the final map. Comparing the present approaches with the previous study approach demonstrates that the three newly suggested methods can remarkably increase both overall and individual class accuracy for mapping.

Multi-classification of arrhythmias using ResNet with CBAM on CWGAN-GP augmented ECG Gramian Angular Summation Field

The imbalanced data problem occurs in many application domains and is considered to be a challenging problem in machine learning and data mining. Most resampling methods for synthetic data focus on minority class without considering the data distribution of major classes. In contrast to previous works, the proposed method considers both majority classes and minority classes to learn feature embeddings and utilizes appropriate loss functions to make feature embedding as discriminative as possible. The proposed method is a comprehensive framework and different deep learning feature extractors can be utilized for different domains. We conduct experiments utilizing seven numerical datasets and one image dataset based on multiclass classification tasks. The experimental results indicate that the proposed method provides accurate and stable results.

Evaluating the impact of learning from weather data, in addition to bridge data, on the performance of bridge deterioration prediction is critical for identifying the right data needed for better prediction for enhanced bridge maintenance decision making. However, the majority of the studies in the bridge domain did not consider such evaluation. For those that conducted the evaluation, their evaluation results usually varied. There is, thus, a need for re-evaluating whether the use of weather data could improve the prediction performance. However, conducting the evaluation is challenging because of class imbalance problems in the bridge domain. Therefore, prior to the evaluation, conducting data sampling to alleviate/eliminate such problems is necessary. To address these needs, this paper offers a pilot evaluation study for better evaluating the impact of learning from weather data on bridge deterioration prediction. To conduct the evaluation, a sampling method was used to deal with the data imbalance problems, and a deep neural network model was developed to predict the condition ratings of decks, superstructures, and substructures. A number of alternative sampling methods were tested and the prediction performances—with and without weather data—were compared. The preliminary experimental results indicated that: (1) the random over-sampling method outperformed the other alternatives; and (2) the change in the prediction performance after further learning from the weather data was only marginal.

Conserved motifs in biological sequences are closely related to their structure and functions. Recently, discriminative motif discovery methods have attracted more and more attention. However, little attention has been devoted to the data imbalance problem, which is one of the main reasons affecting the performance of the discriminative models. In this article, a simulated evolution method is applied to solve the multi-class imbalance problem at the stage of data preprocessing, and at the stage of Hidden Markov Models (HMMs) training, a random under-sampling method is introduced for the imbalance between the positive and negative datasets. It is shown that, in the task of discovering targeting motifs of nine subcellular compartments, the motifs found by our method are more conserved than the methods without considering data imbalance problem and recover the most known targeting motifs from Minimotif Miner and InterPro. Meanwhile, we use the found motifs to predict protein subcellular localization and achieve higher prediction precision and recall for the minority classes.

Fine-grained attribute analysis and data imbalance have always been research hotspots in the field of computer vision. Due to the complexity and diversity of fine-grained attribute images, traditional image classification methods have shortcomings in paying attention to fine-grained attributes of images and perform poorly when dealing with imbalanced data sets. To overcome these problems, this study proposes a fine-grained image threshold classification algorithm based on deep metric learning. By introducing a metric learning method, the focus on fine-grained attributes of images is enhanced. At the same time, by applying pairwise loss and proxy loss, the classification accuracy of the model is improved and the model convergence speed is accelerated. In order to deal with the problem of data imbalance, a classifier based on threshold analysis is designed. The classifier uses threshold analysis technology to achieve multi-level classification of fine-grained images, thereby improving the problem of low classification accuracy of a few categories in imbalanced data sets. Experimental results show that the proposed fine-grained image threshold classification algorithm based on deep metric learning is significantly superior to other methods in terms of classification accuracy.

Autonomous driving technology includes four links: perception, cognition, decision-making, and behaviour. The cognition plays a vital role as a bridge between perception and decision-making. In the cognition link, accurate identification of the intention and behaviour of surrounding vehicles is a necessary prerequisite for autonomous vehicles to make correct behaviour decisions, and it is conducive to reducing the driving risk of own vehicle. Different from driving style, driving tendency has the characteristics of dynamic and behavioural decision-making preference. Being able to identify the driving tendency of surrounding vehicles in real time will be more conducive to making reasonable behavioural decisions for own vehicle. Therefore, this paper takes the lane change trajectory data of passenger cars in the NGSIM open-source database as the research sample. Firstly, the problem of data imbalance is considered, and the imbalanced samples are sampled using the method of uniformly dividing the data set. Secondly, from the perspective of own vehicle, the characteristic indicators that can represent the tendency of the surrounding vehicles to change lanes are mined. Finally, the GRU deep learning algorithm is used to establish a real-time identification model for aggressive lane change behaviour. Compared with the traditional method, the method proposed in this paper can effectively identify the tendency of lane change behaviour of surrounding vehicles in the early stage of lane-changing behaviour and has higher recognition accuracy. The research results show that the real-time identification of the tendency of the lane change behaviour of the surrounding vehicles is beneficial to the assessment and quantification of the driving risk of own vehicle, and also provides a theoretical basis for the decision-making of own vehicle, which is of great significance to the safe driving of own vehicle.

Although imbalanced data have been studied for many years, the problem of data imbalance is still a major problem in the development of machine learning and artificial intelligence. The development of deep learning and artificial intelligence has further expanded the impact of imbalanced data, so studying imbalanced data classification is of practical significance. We propose an image oversampling algorithm based on the influence function and sample weights. Our scheme not only synthesizes high-quality minority class samples but also preserves the original features and information of minority class images. To address the lack of visually reasonable features in SMOTE when synthesizing images, we improve the pre-training model by removing the pooling layer and the fully connected layer in the model, extracting the important features of the image by convolving the image, executing SMOTE interpolation operation on the extracted important features to derive the synthesized image features, and inputting the features into a DCGAN network generator, which maps these features into the high-dimensional image space to generate a realistic image. To verify that our scheme can synthesize high-quality images and thus improve classification accuracy, we conduct experiments on the processed CIFAR10, CIFAR100, and ImageNet-LT datasets.

BackgroundDiabetes Mellitus (DM) has become the third chronic non-communicable disease that hits patients after tumors, cardiovascular and cerebrovascular diseases, and has become one of the major public health problems in the world. Therefore, it is of great importance to identify individuals at high risk for DM in order to establish prevention strategies for DM.MethodsAiming at the problem of high-dimensional feature space and high feature redundancy of medical data, as well as the problem of data imbalance often faced. This study explored different supervised classifiers, combined with SVM-SMOTE and two feature dimensionality reduction methods (Logistic stepwise regression and LAASO) to classify the diabetes survey sample data with unbalanced categories and complex related factors. Analysis and discussion of the classification results of 4 supervised classifiers based on 4 data processing methods. Five indicators including Accuracy, Precision, Recall, F1-Score and AUC are selected as the key indicators to evaluate the performance of the classification model.ResultsAccording to the result, Random Forest Classifier combining SVM-SMOTE resampling technology and LASSO feature screening method (Accuracy = 0.890, Precision = 0.869, Recall = 0.919, F1-Score = 0.893, AUC = 0.948) proved the best way to tell those at high risk of DM. Besides, the combined algorithm helps enhance the classification performance for prediction of high-risk people of DM. Also, age, region, heart rate, hypertension, hyperlipidemia and BMI are the top six most critical characteristic variables affecting diabetes.ConclusionsThe Random Forest Classifier combining with SVM-SMOTE and LASSO feature reduction method perform best in identifying high-risk people of DM from individuals. And the combined method proposed in the study would be a good tool for early screening of DM.

In recent years, credit evaluation has become an issue of increasing concern for financial institutions. However, since most research focuses on the risk classification process, the problem of data imbalance is ignored. In real data sets, there are often far more users with good credit than users with bad credit, and the imbalance of data often easily leads to a decline in the classification performance of the model. Therefore, previous research is very limited in practical application scenarios. In this paper, we establish a new integration method for credit evaluation, which is classified into three steps: First, data preprocessing. Before inputting samples into the model, we take a series of preprocessing steps, such as missing data processing, data dimensionality reduction. Secondly, in view of the imbalance problem, the data is divided into multiple clusters using an unsupervised clustering algorithm, and the SMOTE method is used to generate minority samples in the clusters whose ratio exceeds the threshold. Finally, the GBDT2NN and Factorization Machine methods are integrated to classify the samples. In order to verify the effectiveness of this method, we use the Kaggle competition data set for verification. The results show that this method is better than other algorithms in the field of credit evaluation in terms of recall rate and AUC value.

Currently, huge sizes of indeterminate data are effortlessly collected or created at a very high pace in numerous real-life applications. Classifying this indefinite big data, is computationally intensive as large amount of data is related with existential probability of undefined or undetermined values of raw data. In this study, we propose a data mining approach for the classification of big dataset based on death toll by epidemic outbreak of Ebola virus and comparing its relevance with other epidemic diseases and generalizing error and intra class separability using relevance vector machine classifier.