A weighted multiple classifier framework based on random projection

In this paper, we present a novel ensemble method random projection random discretization ensembles (RPRDE) to create ensembles of linear multivariate decision trees by using a univariate decision tree algorithm. The present method combines the better computational complexity of a univariate decision tree algorithm with the better representational power of linear multivariate decision trees. We develop random discretization (RD) method that creates random discretized features from continuous features. Random projection (RP) is used to create new features that are linear combinations of original features. A new dataset is created by augmenting discretized features (created by using RD) with features created by using RP. Each decision tree of a RPRD ensemble is trained on one dataset from the pool of these datasets by using a univariate decision tree algorithm. As these multivariate decision trees (because of features created by RP) have more representational power than univariate decision trees, we expect accurate decision trees in the ensemble. Diverse training datasets ensure diverse decision trees in the ensemble. We study the performance of RPRDE against other popular ensemble techniques using C4.5 tree as the base classifier. RPRDE matches or outperforms other popular ensemble methods. Experiments results also suggest that the proposed method is quite robust to the class noise.

One of the major developments in machine learning in the past decade is the ensemble method, which finds highly accurate classifier by combining many moderately accurate component classifiers. In this research work, new ensemble classification methods are proposed with homogeneous ensemble classifier using bagging and heterogeneous ensemble classifier using arcing and their performances are analyzed in terms of accuracy. A Classifier ensemble is designed using Radial Basis Function RBF and Support Vector Machine SVM as base classifiers. The feasibility and the benefits of the proposed approaches are demonstrated by the means of standard datasets of direct marketing. The main originality of the proposed approach is based on three main parts: preprocessing phase, classification phase and combining phase. A wide range of comparative experiments are conducted for standard datasets of direct marketing. The performance of the proposed homogeneous and heterogeneous ensemble classifiers are compared to the performance of other standard homogeneous and heterogeneous ensemble methods. The standard homogeneous ensemble methods include Error correcting output codes, dagging and heterogeneous ensemble methods include majority voting, stacking. The proposed ensemble methods provide significant improvement of accuracy compared to individual classifiers and the proposed bagged RBF and SVM performs significantly better than ECOC and Dagging and the proposed hybrid RBF-SVM performs significantly better than voting and stacking. Also heterogeneous models exhibit better results than homogeneous models for standard datasets of direct marketing.

Speller brain-computer interface (BCI) systems can help neuromuscular disorders patients write their thoughts by using the electroencephalogram (EEG) signals by just focusing on the speller tasks. For practical speller-based BCI systems, the P300 event-related brain potential is measured by using the EEG signal. In this paper, we design a robust machine-learning algorithm for P300 target detection. The novel spatial-temporal linear feature learning (STLFL) algorithm is proposed to extract high-level P300 features. The STLFL method is a modified linear discriminant analysis technique focusing on the spatial-temporal aspects of information extraction. A new P300 detection structure is then proposed based on the combination of the novel STLFL feature extraction and discriminative restricted Boltzmann machine (DRBM) for the classification approach (STLFL + DRBM). The effectiveness of the proposed technique is evaluated using two state-of-the-art P300 BCI datasets. Across the two available databases, we show that in terms of average target recognition accuracy and standard deviation values, the proposed STLFL + DRBM method outperforms traditional methods by 33.5, 78.5, 93.5, and 98.5% for 1, 5, 10, and 15 repetitions, respectively, in BCI competition III datasets II and by 71.3, 100, 100, and 100% for 1, 5, 10, and 15 repetitions, respectively, in BCI competition II datasets II and by 67.5 ± 4, 84.2 ± 2.5, 93.5 ± 1, 96.3 ± 1, and 98.4 ± 0.5% for rapid serial visual presentation (RSVP) based dataset in repetitions 1–5. The method has some advantages over the existing variants including its efficiency, robustness with a small number of training samples, and a high ability to create discriminative features between classes.

This paper disseminates an extra dimension of substantial analysis demonstrating the trade-offs between the performance of Parametric (P) and Non-Parametric (NP) classification algorithms when applied to classify faults occurring in pneumatic actuators. Owing to the criticality of the actuator failures, classifying faults accurately may lead to robust fault tolerant models. In most cases, when applying machine learning, the choice of existing classifier algorithms for an application is random. This work, addresses the issue and quantitatively supports the selection of appropriate algorithm for non-parametric datasets. For the case study, popular parametric classification algorithms namely: Naïve Bayes (NB), Logistic Regression (LR), Linear Discriminant Analysis (LDA), Perceptron (PER) and non-parametric algorithms namely: Multi-Layer Perceptron (MLP), k Nearest Neighbor (kNN), Support Vector Machine (SVM), Decision Tree (DT) and Random Forest (RF) are implemented over a non-parametric, imbalanced synthetic dataset of a benchmark actuator process. Upon using parametric classifiers, severe adultery in results is witnessed which misleads the interpretation towards the accuracy of the model. Experimentally, about 20% improvement in accuracy is obtained on using non-parametric classifiers over the parametric ones. The robustness of the models is evaluated by inducing label noise varying between 5% to 20%. Triptych analysis is applied to discuss the interpretability of each machine learning model. The trade-offs in choice and performance of algorithms and the evaluating metrics for each estimator are analyzed both quantitatively and qualitatively. For a more cogent reasoning through validation, the results obtained for the synthetic dataset are compared against the industrial dataset of the pneumatic actuator of the sugar refinery, Development and Application of Methods for Actuator Diagnosis in Industrial Control Systems (DAMADICS). The efficiency of non-parametric classifiers for the pneumatic actuator dataset is well proved.

Exploring sex classification from earprints - A comparison of supervised machine learning algorithms and conventional linear discriminant analysis.

In this paper we propose and experimentally analyze ensemble methods based on random projections (as feature extraction method) and SVM with polynomial kernels (as learning algorithm). We show that, under suitable conditions, polynomial kernels are approximately preserved by random projections, with a degradation related to the square of the degree of the polynomial. Experimental results with Random Subspace and Random Projection ensembles of polynomial SVMs, support the hypothesis the low degree polynomial kernels, introducing with high probability lower distortions in the projected data, are better suited to the classification of high dimensional DNA microarray data.

ABSTRACTObjectiveIn spinal surgery, precise identification of high‐speed bur milling states is crucial for patient safety. This study investigates whether integrating tactile and auditory perception can enhance the accuracy of milling state detection in robot‐assisted cervical laminectomy.MethodsBased on the mathematical and physical model of vibration and sound in high‐speed bur milling bone, the feasibility of employing vibration and sound characteristics to identify the milling states of high‐speed bur is studied systematically. Cervical laminectomy was performed on the cervical spine of the sheep. During the signal acquisition process, acceleration sensors and microphones were installed to collect vibration and sound signals, respectively. Seven milling states were set up in the experiment: (1) Milling depths of cortical bone (CTB): 0.5, 1.0, and 1.5 mm; (2) Milling depths of milling of cancellous bone (CCB): 0.5, 1.0, and 1.5 mm; (3) Boundary conditions: high‐speed bur idling or complete penetration of bone (PT). The milling speed was set at 0.5 mm/s, the milling angle was 45°, and the bur diameter was 4 mm. The vibration or sound was extracted by Fast Fourier Transform (FFT) in the frequency domain of the first nine harmonics to generate the feature vector in 9 dimensions (9‐D) space. These vibration and sound features were combined to form an 18‐D multi‐perception spatial vector for subsequent analysis, including five machine learning algorithms: Support Vector Machine (SVM), K Nearest Neighbors (KNN), Naive Bayes (NB), Linear Discriminant Analysis (LDA), and Decision Tree (DT), and deep learning models: Long Short‐Term Memory networks (LSTM).ResultsBased on the 18‐D features of tactile and auditory multisensory fusion, the LSTM model is trained using 6600 sets of high‐speed bur milling data. In order to achieve the best performance, a layer‐by‐layer parameter optimization strategy was used to determine the optimal parameter configuration, and finally, a single‐layer LSTM with 12 memory units was constructed. In terms of accuracy and stability, the model is significantly superior to the machine learning algorithms (SVM, KNN, NB, LDA, and DT), and the accuracy of LSTM is 99.32% in the milling states identification of cervical lamina milling with high‐speed bur.ConclusionThrough theoretical analysis and experimental verification, the study built a multi‐perception fusion framework based on tactile and auditory perception and accurately identified the cervical vertebra milling states through the LSTM model, which can provide perception means for operational spinal surgery robots in the future.

Learning in the feed-forward random neural network: A critical review

A polarity analysis framework for Twitter messages

Data dimension reduction (DDR) is all about mapping data from high dimensions to low dimensions, various techniques of DDR are being used for image dimension reduction like Random Projections, Principal Component Analysis (PCA), the Variance approach, LSA-Transform, the Combined and Direct approaches, and the New Random Approach. Auto-encoders (AE) are used to learn end-to-end mapping. In this paper, we demonstrate that pre-processing not only speeds up the algorithms but also improves accuracy in both supervised and unsupervised learning. In pre-processing of DDR, first PCA based DDR is used for supervised learning, then we explore AE based DDR for unsupervised learning. In PCA based DDR, we first compare supervised learning algorithms accuracy and time before and after applying PCA. Similarly, in AE based DDR, we compare unsupervised learning algorithm accuracy and time before and after AE representation learning. Supervised learning algorithms including support-vector machines (SVM), Decision Tree with GINI index, Decision Tree with entropy and Stochastic Gradient Descent classifier (SGDC) and unsupervised learning algorithm including K-means clustering, are used for classification purpose. We used two datasets MNIST and FashionMNIST Our experiment shows that there is massive improvement in accuracy and time reduction after pre-processing in both supervised and unsupervised learning.

Since 1980, obesity has increasingly become a global health issue—it is increasingly prevalent among children with serious implications for their health and their later well-being as adults. The World Health Organization considers obesity a non-communicable disease which may lead to severe diseases such as diabetes, cardiovascular diseases, and various kinds of cancers. Detrimental dietary patterns and the absence of physical activity are the root causes of obesity. Data mining is a quickly developing field with numerous applications in the field of medical services. One of its several utilizations is the forecasting of sicknesses. The proposed work intends to examine varied machine learning techniques: K- Nearest Neighbor, Decision Tree, Ensemble Random Forest, and Deep Neural Network to prognosticate obesity. The inference analysis shows that the ensemble random forest classification with XG Boosting is the most precise procedure to prognosis obesity.KeywordsObesityNon-communicable diseaseData miningEnsemble random forestDeep neural network

Background: Linear Discriminant Analysis (LDA) is a powerful and widely used technique for classification with correlated variables. Principal Components (PCs) group these variables into linear combinations and produce independent variables. The LDA on these PC’s may provide better classification accuracy in clinical diagnostics than on usual measurements. Methodology: Two datasets were utilized for demonstration: one from a Sudden Sensorineural Hearing Loss (SSNHL) case-control study and the other from a Gall Bladder (GB) case-control study. Linear Discriminant Analysis (LDA) was conducted on the actual correlated measured variables for group classification, as well as on the derived principal component variables, to compare their classification accuracies. Performance metrics including Sensitivity, Specificity, Positive Predictive Value (PPV), Negative Predictive Value (NPV), Classification Accuracy, and F1 Score were assessed. For validation, a third simulated dataset was employed. Additionally, LDA was performed on each dataset using eigenvectors of the control group applied to the cases and vice versa, revealing a strong agreement in classification as measured by the kappa statistic. Results: When LDA was applied to the actual lipid measurements in the SSNHL dataset, the classification accuracy was 57.2%, and the F1 score was 39.7%. However, when LDA was performed using principal components (PCs), the classification accuracy markedly improved to 99.2%, with an F1 score of 98.5%. Similarly, for the GB cancer dataset, the classification accuracy and F1 score were initially 77.2% and 77.3%, respectively. Upon applying LDA with the PCs, these metrics were significantly enhanced to 98.4% and 98.3%, respectively. For the simulated dataset, both the classification accuracy and F1 score were 99.1%. The study also demonstrated that the classification accuracy and F1 score remained consistent regardless of whether the eigenvectors from the cases or controls were used to classify new subjects (Kappa Statistic = 0.962, P < 0.001). Conclusion: In group separation, utilizing principal components significantly improves classification accuracy and overall performance metrics, outperforming the use of the original correlated predictors.

Dynamic ensemble selection (or dynamic pruning) is a learning paradigm that reduces the size of an ensemble of classifiers for each test pattern as input. The purpose of this approach is to improve the performances of the ensemble method by eliminating dynamically all the weak learners for a specific input. As of now, most of the dynamic pruning methods use information about the validation data and intermediate performances of the ensemble's classifiers on it, but none of them take the advantage of the intrinsic information given by those classifiers. In this paper we propose a new dynamic ensemble selection framework based on a tree internal supervised measure. This implies that our approach works on homogeneous ensemble methods with trees as base learners. This approach is motivated by the fact that in high dimensional datasets, usual metrics like euclidean distance suffer from the curse of dimensionality. Moreover, a similarity measure based on decision tree structure can be considered as supervised, which is well suited for supervised learning. Our framework is called ST-DES for Similarity Tree Dynamic Ensemble Selection. In this work, we use one of the best ensemble algorithm, Random Forest, as the base ensemble generator. We then compare the performance in terms of accuracy of ST-DES against the best known approaches in literature on different UCI datasets. We show that our approach outperforms the existing ones especially for high dimensional datasets. Besides, we show that this approach is not disturbed by random noise in features.

The random neural network (RNN) is a mathematical model for an “integrate and fire” spiking network that closely resembles the stochastic behavior of neurons in mammalian brains. Since its proposal in 1989, there have been numerous investigations into the RNN's applications and learning algorithms. Deep learning (DL) has achieved great success in machine learning. Recently, the properties of the RNN for DL have been investigated, in order to combine their power. Recent results demonstrate that the gap between RNNs and DL can be bridged and the DL tools based on the RNN are faster and can potentially be used with less energy expenditure than existing methods.

Ensemble classification based on generalized additive models