Topics in imbalanced data classification : AdaBoost and Bayesian relevance vector machine

Fully Bayesian analysis of the relevance vector machine classification for imbalanced data problem

Mental fatigue (MF) is a common phenomenon in our daily lives. In the workplace, mental fatigue can increase the risk of human errors. Moreover, if it is not detected and handled correctly, it can cause various problems. Electroencephalogram (EEG) is one of many modalities that is widely used by researchers for detecting mental fatigue. However, due to the difficulty in provoking fatigue condition during the EEG measurement, this study proposed two experimental designs for detecting mental fatigue, the first design is by using physical induction (PI) and the 2nd design is by using mental induction (MI). We obtained the EEG signals from 20 healthy participants, from 14 channels of wireless EEG headset. Each participant was given four types of cognitive tests in the form of a computer game including Trail, Span, Stroop, and Arithmetic. The subjective measurement of fatigue condition was measured by using the Swedish Occupational Fatigue Inventory (SOFI) questionnaires. The EEG signal was extracted by using power percentage (PP) features of alpha ( $\alpha$ ), beta ( $\beta$ ), and theta ( $\theta$ ) bands of frequency from all channels. As for classification, we introduced the use Relevance Vector Machine (RVM) approach which is claimed to have better performance than SVM as the precedence method. According to the results, all of the cognitive tests has increased average value of response time (RT) and decreased correction score (CS) on post-induction compared to pre-induction. Another result showed that the best mental fatigue detection is obtained from mental induction rather than physical induction. The classification results demonstrate an accuracy of 93.7% (MI) and 92.6% (PI) are achieved by RVM approach. Furthermore, RVM has also less execution time compared to SVM.

Microcalcification (MC) clusters in mammograms can be important early signs of breast cancer in women. Accurate detection of MC clusters is an important but challenging problem. In this paper, we propose the use of a recently developed machine learning technique -- relevance vector machine (RVM) -- for automatic detection of MCs in digitized mammograms. RVM is based on Bayesian estimation theory, and as a feature it can yield a decision function that depends on only a very small number of so-called <i>relevance vectors</i>. We formulate MC detection as a supervised-learning problem, and use RVM to classify if an MC object is present or not at each location in a mammogram image. MC clusters are then identified by grouping the detected MC objects. The proposed method is tested using a database of 141 clinical mammograms, and compared with a support vector machine (SVM) classifier which we developed previously. The detection performance is evaluated using the free-response receiver operating characteristic (FROC) curves. It is demonstrated that the RVM classifier matches closely with the SVM classifier in detection performance, and does so with a much sparser kernel representation than the SVM classifier. Consequently, the RVM classifier greatly reduces the computational complexity, making it more suitable for real-time processing of MC clusters in mammograms.

ABSTRACTAddressing the challenges of low feature extraction dimensions and insufficient distinct information for gesture differentiation for smart home appliances, this article proposed an innovative gesture recognition algorithm, integrating fractional Fourier transform (FrFT) with relevance vector machine (RVM). The process involves using FrFT to transform raw gesture data into the fractional domain, thereby expanding the dimensions of information extraction. Subsequently, high‐dimensional feature vectors are created from fractional domain, and RVM classifiers are employed for joint optimization of feature selection and classification decision functions, achieving optimal classification performance. A dataset was constructed using five different types of gestures recorded on the TI millimeter‐wave radar platform to validate the effectiveness of this method. The experimental results demonstrate that the RVM selected the optimal FrFT order of 0.6, with the best feature set comprising fractional spectral entropy, peak factor, and second‐order central moment. Recognition rates for each gesture exceeded 96.2%, with an average rate of 98.5%. This performance surpasses three comparative methods in both recognition accuracy and real‐time processing, indicating high potential for future applications.

Currently, huge sizes of indeterminate data are effortlessly collected or created at a very high pace in numerous real-life applications. Classifying this indefinite big data, is computationally intensive as large amount of data is related with existential probability of undefined or undetermined values of raw data. In this study, we propose a data mining approach for the classification of big dataset based on death toll by epidemic outbreak of Ebola virus and comparing its relevance with other epidemic diseases and generalizing error and intra class separability using relevance vector machine classifier.

Abstract—Existing electroencephalography (EEG) based Alzheimer’s disease (AD) diagnostic systems typically rely on experts to visually inspect and segment the collected signals into artefact-free epochs and on support vector machine (SVM) based classifiers. The manual selection process, however, introduces biases and errors into the diagnostic procedure, renders it “semi-automated, ” and makes the procedure costly and labour-intensive. In this paper, we overcome these limitations by proposing the use of an automated artefact removal (AAR) algorithm to remove artefacts from the EEG signal without the need for human intervention. We investigate the effects of the so-called wavelet-enhanced independent component analysis (wICA) AAR on three classes of EEG features, namely spectral power, coherence, and amplitude modulation, and ultimately, on diagnostic accuracy, specificity and sensitivity. Furthermore, we propose to replace the binary SVM classifier with a soft-decision relevance vec-tor machine (RVM) classifier. Experimental results show the proposed RVM-based system outperforming the SVM trained on features extracted from both manually-selected and wICA-processed epochs. Moreover, the class membership information output by the RVM is shown to provide clinicians with a richer pool of information to assist with AD assessment. Keywords—Alzheimer’s disease, electroencephalography, support vector machine (SVM), relevance vector machine (RVM), wICA. I.

For thermal-based applications, images are obtained by an Infrared camera through Plank’s law. Thermal sensitivity is the smallest temperature difference detected by the camera. Thermal-images were captured through the heat emitted from plant leaves. Initially, the Gaussian noise in the medical Infrared (IR) images is pre-processed by the median filter. Then the features from the preprocessed images are obtained through the principal component analysis algorithm. From the extracted features, the optimal features are selected using the Scale Invariant Differential Evolution-based Feature (SIDEF) algorithm. Through the exploitation of the selected features, the hybrid genetic algorithm with Relevance Vector Machine (HGRVMA) classifier classifies the features into diseases and non-diseases. To validate the performance of the proposed algorithm, it is compared with the existing algorithms in terms of metrics such as sensitivity, accuracy, precision, and recall. The validation results prove that the proposed HGRVM algorithm is optimal than the existing algorithms for all the metrics.

Hughes phenomenon is a serious problem in supervised classification of hyperspectral images in particular for small land-cover patches. A solution for this problem through integrating the capabilities of a nonlinear discriminating analysis with relevance vector machine (RVM) is proposed in this paper. It first transforms the hyperdimensional data to a new space with a better class separability. Then, a multiclass RVM classifier processes the transformed data for precise labeling of the classes. The results show that the proposed approach outperforms both RVM as well as support vector machine (SVM), when they are applied to the original hyperdimensional data space. Indeed, it is an advantage for key information detection in the classification context.

Although imbalanced data have been studied for many years, the problem of data imbalance is still a major problem in the development of machine learning and artificial intelligence. The development of deep learning and artificial intelligence has further expanded the impact of imbalanced data, so studying imbalanced data classification is of practical significance. We propose an image oversampling algorithm based on the influence function and sample weights. Our scheme not only synthesizes high-quality minority class samples but also preserves the original features and information of minority class images. To address the lack of visually reasonable features in SMOTE when synthesizing images, we improve the pre-training model by removing the pooling layer and the fully connected layer in the model, extracting the important features of the image by convolving the image, executing SMOTE interpolation operation on the extracted important features to derive the synthesized image features, and inputting the features into a DCGAN network generator, which maps these features into the high-dimensional image space to generate a realistic image. To verify that our scheme can synthesize high-quality images and thus improve classification accuracy, we conduct experiments on the processed CIFAR10, CIFAR100, and ImageNet-LT datasets.

In machine learning, classification and regression are two of the most noteworthy key topics since they occur extensively in numerous real-world applications. However, real life data is hardly ever found balanced, rather skewed data is the common occurrence. This poses some serious challenges to the standard techniques of classification and regression. The performance and effectiveness of these techniques are substantially affected by overfitting, creating a bias towards the majority class. In recent years, quite a few number of methods have been introduced for classification of imbalanced data. But most of them are designed for binary classes and it is difficult or inefficient to extend them for multiple classes. Moreover, data imbalance problem occurs frequently in regression analysis too, with only a handful of algorithms robust enough to tackle to this problem. In this paper, we propose an effective recursive method for multi-class classification with imbalanced data. Our proposed algorithm partitions and balances the data, and is applied recursively coupled with ensemble techniques. Furthermore, we also extend our proposed method to solve the data imbalance problem in regression analysis. Experimental results demonstrate that the proposed recursive technique is effective and improves the performance when compared to existing methods for classification and regression with imbalanced distribution.

Accurate identification of protein-DNA binding sites is significant for both understanding protein function and drug design. Machine-learning-based methods have been extensively used for the prediction of protein-DNA binding sites. However, the data imbalance problem, in which the number of nonbinding residues (negative-class samples) is far larger than that of binding residues (positive-class samples), seriously restricts the performance improvements of machine-learning-based predictors. In this work, we designed a two-stage imbalanced learning algorithm, called ensembled hyperplane-distance-based support vector machines (E-HDSVM), to improve the prediction performance of protein-DNA binding sites. The first stage of E-HDSVM designs a new iterative sampling algorithm, called hyperplane-distance-based under-sampling (HD-US), to extract multiple subsets from the original imbalanced data set, each of which is used to train a support vector machine (SVM). Unlike traditional sampling algorithms, HD-US selects samples by calculating the distances between the samples and the separating hyperplane of the SVM. The second stage of E-HDSVM proposes an enhanced AdaBoost (EAdaBoost) algorithm to ensemble multiple trained SVMs. As an enhanced version of the original AdaBoost algorithm, EAdaBoost overcomes the overfitting problem. Stringent cross-validation and independent tests on benchmark data sets demonstrated the superiority of E-HDSVM over several popular imbalanced learning algorithms. Based on the proposed E-HDSVM algorithm, we further implemented a sequence-based protein-DNA binding site predictor, called DNAPred, which is freely available at http://csbio.njust.edu.cn/bioinf/dnapred/ for academic use. The computational experimental results showed that our predictor achieved an average overall accuracy of 91.7% and a Mathew's correlation coefficient of 0.395 on five benchmark data sets and outperformed several state-of-the-art sequence-based protein-DNA binding site predictors.

This paper presents an improved semi-supervised learning approach for defect prediction involving class imbalanced and limited labeled data problem. This approach employs random under-sampling tech...

The imbalance data problem in classification is a significant research area and has attracted a lot attention in recent years. Rebalancing class distribution techniques such as over-sampling or under-sampling are the most common approaches to deal with this problem. This paper presents a new method so called Diversity and Separable Metrics in Over-Sampling Technique (DSMOTE) to handle the imbalanced learning problems. The main idea of the DSMOTE is to use a diversity and separable measure which shows a positive impact on the minority class. This improvement is achieved by reduce overfitting by using a diversity measure. Moreover by using the separable measure the risk of generating new samples in decision boundaries with hard-to-learn samples is decreased. The proposed method improves the learning accuracy in three stages including; (1) removal of abnormal samples from minority class, (2) selecting the top three samples of minority class according to desired criteria and (3) generating new sample using selected samples. The experiments are conducted on five real world datasets which is taken from Iran University of Medical Science and also six different UCI datasets. Moreover, three different classifiers, four resampling algorithms and six performance evaluation measures are used to evaluate the proposed method. The reported results indicate that the proposed approach demonstrates a better or at least comparable performance compared to those of the state-of-the-art methods.

To address the imbalanced data problem in molecular docking-based virtual screening methods, this paper proposes a virtual screening method for drug proteins based on imbalanced data mining, which introduces machine learning technology into the virtual screening technology for drug proteins to deal with the imbalanced data problem in the virtual screening process and improve the accuracy of the virtual screening. First, to address the data imbalance problem caused by the large difference between the number of active compounds and the number of inactive compounds in the docking conformation generated by the actual virtual screening process, this paper proposes a way to improve the data imbalance problem using SMOTE combined with genetic algorithm to synthesize new active compounds artificially by upsampling active compounds. Then, in order to improve the accuracy in the virtual screening process of drug proteins, the idea of integrated learning is introduced, and the random forest (RF) extended from Bagging integrated learning technique is combined with the support vector machine (SVM) technique, and the virtual screening of molecular docking conformations using RF-SVM technique is proposed to improve the prediction accuracy of active compounds in docking conformations. To verify the effectiveness of the proposed technique, first, HIV-1 protease and SRC kinase were used as test data for the experiments, and then, CA II was used to validate the model of the test data. The virtual screening of drug proteins using the proposed method in this paper showed an improvement in both enrichment factor (EF) and AUC compared with the use of the traditional virtual screening, for the test dataset. Therefore, it can be shown that the proposed method can effectively improve the accuracy of drug virtual screening.

3DACN: 3D Augmented convolutional network for time series data