A New Incremental Learning Algorithm with Probabilistic Weights Using Extended Data Expression

An incremental learning algorithm, Learn++, which allows supervised classification algorithms to learn from new data without forgetting previously acquired knowledge, is introduced. Learn++ is based on generating multiple classifiers using strategically chosen distributions of the training data and combining these classifiers through weighted majority voting. Learn++ shares various notions with psycho-physiological models of learning. The Learn++ algorithm, simulation results, and how the algorithm is related to various concepts in psycho-physiological learning models are discussed. The algorithm was tested on a variety of real world and synthetic datasets. Two sets of results are presented for optical handwritten digit recognition and gas sensing.

A-iLearn: An adaptive incremental learning model for spoof fingerprint detection

We describe an incremental learning algorithm designed to learn in challenging non-stationary environments, where the underlying data distribution that governs the classification problem changes at an unknown rate. The algorithm is based on a multiple classifier system that generates a new classifier every time a new dataset becomes available from the changing environment. We consider the particularly challenging form of this problem, where we assume that the previously generated data points are no longer available, even if some of those points may still be relevant in the new environment. The algorithm employs a strategic weighting mechanism to determine the error of each classifier on the current data distribution, and then combines the classifiers using a dynamically weighted majority voting. We describe the implementation details of algorithm, and track its performance as a function of the environment's rate of change. We show that the algorithm is able to track the changing environment, even when the environment changes drastically over a short period of time.

This paper proves the problem of losing incremental samples' information of the present SVM incremental learning algorithm from both theoretic and experimental aspects, and proposes a new incremental learning algorithm with support vector machine based on hyperplane-distance. According to the geometric character of support vector, the algorithm uses Hyperplane-Distance to extract the samples, selects samples which are most likely to become support vector to form the vector set of edge, and conducts the support vector machine training on the vector set. This method reduces the number of training samples and effectively improves training speed of incremental learning. The results of experiment performed on Chinese webpage classification show that this algorithm can reduce the number of training samples effectively and accumulate historical information. The HD-SVM algorithm has higher training speed and better precision of classification.

Semi-supervised learning is especially important in data mining applications because it can make use of plentiful unlabeled data to train the high-quality learning models. Semi-Supervised Support Vector Machine (S3VM) is a powerful semi-supervised learning model. However, the high computational cost and non-convexity severely impede the S3VM method in large-scale applications. Although several learning algorithms were proposed for S3VM, scaling up S3VM is still an open problem. To address this challenging problem, in this paper, we propose a new incremental learning algorithm to scale up S3VM (IL-S3VM) based on the path following technique in the framework of Difference of Convex (DC) programming. The traditional DC programming based algorithms need multiple outer loops and are not suitable for incremental learning, and traditional path following algorithms are limited to convex problems. Our new IL-S3VM algorithm based on the path-following technique can directly update the solution of S3VM to converge to a local minimum within one outer loop so that the efficient incremental learning can be achieved. More importantly, we provide the finite convergence analysis for our new algorithm. To the best of our knowledge, our new IL-S3VM algorithm is the first efficient path following algorithm for a non-convex problem (i.e., S3VM) with local minimum convergence guarantee. Experimental results on a variety of benchmark datasets not only confirm the finite convergence of IL-S3VM, but also show a huge reduction of computational time compared with existing batch and incremental learning algorithms, while retaining the similar generalization performance.

The Space Time Bilinear (STBL) model is a special form of a multiple bilinear time series which can be used to model time series which exhibit bilinear behavior on a spatial neighborhood structure. The STBL model and its identification have been proposed and discussed by Dai and Billard (1998). In this paper, we compare the STBL model with STARMA and single ARMA model. All problems are addressed by setting up the model in state space form and applying the Kalman filter. An application of the STBL model to epidemic surveillance data is given and the results com pared with those from other models.

Brain Computer Interfaces allow the interaction between a person and their environment using signals extracted directly from the brain. One of the most common non-invasive methods of brain signal acquisition is the electroencephalography (EEG). An EEG based BCI system generally involves four steps: preprocessing, feature extraction, feature selection, and classification. In order to design a real applicable BCI system, it is important to provide: good classification performance, adequate computational cost, robustness to variations of the signal between trials and between subjects, and a classifiers model that cope with highly dimensional data. Ensemble of classifiers is a learning model that, with a proper design, can satisfy those conditions, which make them a good match for a BCI application. In this paper, some ensemble of classifiers designs are evaluated and compared with other BCI approaches on three different subjects. In the proposed model, Genetic Algorithm is employed as feature selection method and Wavelet Packet Decomposition as preprocessing procedure. Four fusion methods were applied in the ensembles design, including: Majority Voting, Weighted Majority Voting, Genetic Algorithm for classifier selection and Genetic Algorithm to compute the weights for the Weighted Majority fusion method. The best results were obtained with the Weighted Majority Voting fusion method based on Genetic Algorithm.

Detecting harmful content or hate speech on social media is a significant challenge due to the high throughput and large volume of content production on these platforms. Identifying hate speech in a timely manner is crucial in preventing its dissemination. We propose a novel stacked ensemble approach for detecting hate speech in English tweets. The proposed architecture employs an ensemble of three classifiers, namely support vector machine (SVM), logistic regression (LR), and XGBoost classifier (XGB), trained using word2vec and universal encoding features. The meta classifier, LR, combines the outputs of the three base classifiers and the features employed by the base classifiers to produce the final output. It is shown that the proposed architecture improves the performance of the widely used single classifiers as well as the standard stacking and classifier ensemble using majority voting. We also present results on the use of various combinations of machine learning classifiers as base classifiers. The experimental results from the proposed architecture indicated an improvement in the performance on all four datasets compared with the standard stacking, base classifiers, and majority voting. Furthermore, on three of these datasets, the proposed architecture outperformed all state-of-the-art systems.

An integrated TOPSIS crow search based classifier ensemble: In application to stock index price movement prediction

Proposing a classifier ensemble framework based on classifier selection and decision tree

Sentiment analysis extracts the emotions expressed in text and has been employed in many fields including politics, elections, movies, retail businesses and in recent years microblogs to understand, track and control the human sentiments or reactions toward products events or ideas. Nevertheless challenges such as different styles of writing, use of negation and sarcasm, existence of spelling mistakes, invention of new words etc. provide obstacle in the correct classification of sentiments. This paper provides an ensemble of classifiers framework for sentiment analysis. The proposed weighted majority voting ensemble method combines six models including Naïve Bayes, Logistic Regression, Stochastic Gradient Descent, Random Forest, Decision Tree and Support Vector Machine to form a single classifier. Weights of the individual classifiers of the ensemble are chosen as accuracy or Fl-score by optimizing their performance. This approach combines models based on the simple majority voting as opposed to the one based on weighted majority voting. Additionally, a comparison is drawn among these six individual classifiers to evaluate their performance. The proposed ensemble model is tested on some existing sentiment datasets, including SemEval 2017 Task 4A, 4B and 4C. The results demonstrate that the Logistic Regression classifier is optimal as compared to other individual classifiers. Furthermore, the proposed ensemble weighted majority voting classifier with the six individual classifiers performs better compared to the simple majority voting and all independent classifiers.

An incremental one-class learning algorithm is proposed for the purpose of outlier detection. Outliers are identified by estimating - and thresholding - the probability distribution of the training data. In the early stages of training a non-parametric estimate of the training data distribution is obtained using kernel density estimation. Once the number of training examples reaches the maximum computationally feasible limit for kernel density estimation, we treat the kernel density estimate as a maximally-complex Gaussian mixture model, and keep the model complexity constant by merging a pair of components for each new kernel added. This method is shown to outperform a current state-of-the-art incremental one-class learning algorithm (Incremental SVDD [5]) on a variety of datasets, while requiring only an upper limit on model complexity to be specified.KeywordsTraining DataGaussian Mixture ModelOutlier DetectionKernel Density EstimationNormal ClassThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Incremental learning for [formula omitted]-Support Vector Regression

Boosting is recognized as one of the most successful techniques for generating classifier ensembles. Typically, the classifier outputs are combined by the weighted majority vote. The purpose of this study is to demonstrate the advantages of some fuzzy combination methods for ensembles of classifiers designed by Boosting. We ran two-fold cross-validation experiments on six benchmark data sets to compare the fuzzy and nonfuzzy combination methods. On the "fuzzy side" we used the fuzzy integral and the decision templates with different similarity measures. On the "nonfuzzy side" we tried the weighted majority vote as well as simple combiners such as the majority vote, minimum, maximum, average, product, and the Naive-Bayes combination. In our experiments, the fuzzy combination methods performed consistently better than the nonfuzzy methods. The weighted majority vote showed a stable performance, though slightly inferior to the performance of the fuzzy combiners.

Sample property drift is an essential issue for interval type-2 fuzzy neural networks (IT2FNNs). When the samples with fresh properties appear, IT2FNN invariably suffers from catastrophic forgetting due to the modification of its numerous parameters. To solve this problem, an antiforgetting incremental learning algorithm is proposed to update IT2FNN. First, a double-displacement indicator (DDI) is designed to detect when catastrophic forgetting occurs caused by property drift. It integrates the indicators from the feature and target spaces to avoid missing detection of property breakpoints. Second, a multilevel learning objective is developed to perceive catastrophic forgetting. The convergence, diversity, and stability criteria of fuzzy rules are embedded into the objective to improve the compatibility of IT2FNN for different properties. Third, an adaptive hierarchical update strategy (AHUS) is proposed to update the parameters of IT2FNN. With AHUS, the parameters are shared among samples with different properties, which can alleviate catastrophic forgetting. Finally, some experiments have verified that the performance of the presented method is superior to other methods in dynamic system identification.