A New Semi-supervised Learning Based Ensemble Classifier for Recurring Data Stream

Recent advances in storage and processing have provided the possibility of automatic gathering of information, which in turn leads to fast and continuous flows of data. The data which are produced and stored in this way are called data streams. Data streams are produced in large size, and much dynamism and have some unique properties which make them applicable to model many real data mining applications. The main challenge of streaming data is the occurrence of concept drift. In addition, regarding the costs of labeling of instances, it is often assumed that only a small fraction of instances are labeled. In this paper, we propose an ensemble algorithm to classify instances of non-stationary data streams in a semi-supervised environment. Furthermore, this method is intended to recognize recurring concept drifts of data streams. In the proposed algorithm, a pool of classifiers is maintained by the algorithm with each classifier being representative of one single concept. At first, a batch of instances is classified by the algorithm. Thereafter, some of these instances are labeled and this partially labeled batch is used to update the classifiers in the pool. This process repeats for consecutive batches of the streams. The main advantage of the algorithm is that it uses unlabeled instances as well as labeled ones in the learning task. Experimental results show the effectiveness of the proposed algorithm over the state-of-the-art methods, in different aspects.

Ensemble classification technique is great advantage over conventional classifier such as statistical, binary and neural network classifier. Ensemble technique improves the performance of other classifier with some valid constraints. In the improvement of ensemble classifier cluster play important role for data grouping before classification. Cluster oriented ensemble classifier maintain and control the diversity of data during classification process. The diversity of cluster oriented ensemble classifier implied in stream data classification. Stream data classification is critical task due to diversity of data. The critical task of diversity such as infinite population, data drift and feature evaluation technique is overcome through ensemble classifier. Various author proposed a method for stream data classification using ensemble technique. In this paper we give the review of ensemble technique used in stream data classification with clustering technique.

Today, there is a tremendous growth in the amount of data being generated from various fields (such as smartphones, social networks, emails, customer click streams, different types of sensors and Internet of Things) that show Big Data attributes. Recently efforts have been made towards developing models for knowledge discovery from such data under the research area of stream mining or data stream classification in particular. Ensemble learners have become the popular approach in data stream classification because of their stability-elasticity property, which enables handling data stream challenges such as concept drift, recurrent concepts, novel class detection, and class imbalance. In this paper, we compare ten ensemble classifiers with respect to concept drift and class imbalance using Prequential AUC. In addition, Friedman nonparametric statistical test and Nemenyi post-hoc test were used to identify the best approach among them. This work to some extent can serve as part of a review of existing ensemble classifier algorithms for non-stationary data streams.

Decision tree has become one of the most accepted tools for mining data streams after Hoeffding tree was anticipated in the literature. The most vital point of constructing the decision tree is to find out the best attribute to split the considered node. Numerous methods to resolve this problem were presented so far, however, there are some shortcomings such that they are either mathematically not justified or time-consuming. In this paper, a new decision tree algorithm named Adaptive Gaussian Decision Tree (AGDT) is proposed that employs a statistical method (Gaussian Bound) for determining the best attribute in a node. This statistical method ensures that the finest attribute chosen in the considered node using a limited data sample is the same as it would be in the case of the complete data stream hence solving the storage constraints associated with the data stream. In order to handle the concept drift problems AGDT uses fixed-size window to determine which nodes are aging and may need updating. During implementation it is observed that the proposed decision tree method provides a greater classification accuracy, compared with the existing algorithm with the same probability for concept drift data streams.

Most stream classifiers need to detect and react to concept drifts, as traditional machine learning goes to big data machine learning. The most popular ways to adaptive to concept drifts are incrementally learning and classifier dynamic ensemble. Recent years, ensemble classifiers have become an established research line in this field, mainly due to their modularity which offers a natural way of adapting to changes. However, many ensembles which process instances in blocks do not react to sudden changes sufficiently quickly, and which process streams incrementally do not offer accurate reactions to gradual and incremental changes. Fortunately, an Online Accuracy Updated Ensemble (OAUE) algorithm was presented by Brzezinski and Stefanowski. OAUE algorithm has been proven to be an effective ensemble to deal with drifting data stream. But, it has a potentially weakness to adaptive to sudden changes as it uses a fixed window. Therefore, we put forward a Window-Adaptive Online Accuracy Updated Ensemble (WAOAUE) algorithm, which is based on OAUE, and a change detector is added to the ensemble for deciding the window size of each candidate classifier. The proposed algorithm was experimentally compared with four state-of-the-art online ensembles, include OAUE, and provided best practice for big data stream mining.

Genetic Programming is an evolutionary soft computing approach. Data streams are the order of the day input sources. In general, data streams exhibit a peculiar behavior of drifting the concepts as time passes by. Here is a study of GP Classifier on Concept Drifting Data Streams. GP classifier performance is compared to that of other state-of-the-art data mining and stream classification approaches. Boosting is a machine learning meta-algorithm for performing supervised learning. A weak learner is defined to be a classifier which is only slightly correlated with the true classification. In contrast, a strong learner is a classifier that is arbitrarily well-correlated with the true classification. Boosting combines a set of weak learners to create a strong learner. It is observed that the Boosting GP approach is beating Boosting Naive Bayes classification on Concept Drifting Data Streams. Hence it is found that GP is a competent algorithm for Concept Drifting Data Stream classification.

New challenges have emerged in data mining as the traditional techniques have floundered with real-time data streams. The traditional technique needs refurbishing so as to acclimatize with concept drifting data streams. Thus dealing with the concept changes is the most imperative task of stream data mining. Ensemble classifiers have the ability to automatically adapt with the incoming drifts and, therefore, it is the most interesting research area in data stream mining. Bagging, Boosting and Random forest generation are the common ensemble techniques and are the most popular machine learning approaches in the current scenario for static data (Gomes HM, Bifet A, Read J, Barddal JP, Enembreck F, Pfharinger B, Abdessalem T (2017) Adaptive random forests for evolving data stream classification. Mach Learn 106(9–10):469–1495, [1]). A large number of base classifiers in an ensemble can cause computational overhead. Data mining classifiers for real-time data streams, therefore, need to be updated constantly and retrained with the labeled instances of the newly arrived novel classes in data streams and to cope with concept drift; otherwise, the mining models will become less and less accurate as time passes by. However, for data streams, adaptive random forest algorithms have been widely used for ensemble generation due to its competence to handle different types of drifts. This paper proposes a modified adaptive random forest with meta level learner algorithm and concept adaptive very fast decision tree to overcome the concept drift problem in real-time data streams. The proposed algorithm is experimentally compared with state-of-the-art adaptive random forest algorithm on several real synthetic datasets. Results indicate its efficiency in terms of accuracy and processing time.

In order to solve the problem of low accuracy of time dimension feature extraction and classification of high-dimensional large data streams, this paper proposes a time dimension feature extraction and classification algorithm of high-dimensional large data streams based on unsupervised learning. Analyze the trend of high-dimensional data flow changes under machine learning, and achieve dimensionality reduction of high-dimensional large traffic time dimensional data through local save projection. Analyze the spatial relationship between feature attributes and feature space, segment and fit high-dimensional big data streams and time dimensional feature data streams, further segment time dimensional sequences using sliding windows, and complete feature extraction through discrete dyadic wavelet transform. According to the clustering algorithm, cluster the time dimension feature data stream, calculate the cosine similarity of the feature data, model the time dimension feature stream of training samples, use the feature classification function to minimize the classification loss, and use unsupervised learning to achieve the final classification task. The test results show that this method can improve the temporal feature extraction and classification accuracy streams.

Massive volumes of data streams can be found in numerous applications such as network intrusion detection, financial transaction flows, telephone call records, sensor streams, and meteorological data. In recent years, there are increasing demands for mining data streams. Unlike the finite, statically stored data sets, stream data are massive, continuous, temporally ordered, dynamically changing, and potentially infinite [5]. For example, Cortes et al. report that AT&T long distance call records consist of 300 million records per day for 100 million customers. For the stream data applications, the volume of data is usually too huge to be stored or to be scanned for more than once. Further, in data streams, the data points can only be sequentially accessed. Random access to data is not allowed. Extensive research has been done for mining data streams, including those on the stream data classification [3, 20], mining frequent patterns [9, 17, 18], and clustering stream data [1, 2, 8, 9, 10, 11, 12, 13, 14, 16, 19]. In this paper, we study the clustering of multiple and parallel data streams. Our study should be differentiated from some previous studies on clustering stream data [19, 1]. Our goal is to group multiple streams with similar behavior and trend together, instead of to cluster the data records within one data stream. There are various applications where it is desirable to cluster the streams themselves rather than the individual data records within them. For example, the price of a stock may rise and fall from time to time. To reduce the financial risk, an investor may prefer to spread his investment over a number of stocks which may exhibit different behaviors. As another application, in meteorological study and disaster prediction, it is useful to cluster meteorological data streams from different geographical regions of similar curvature trends in order to identify regions with similar meteorological behaviors. Yet another example is that a super market may record sales on different merchandizes. There may be some relationship among the sales of different merchandizes and thus the merchant can make use of the correlation to manipulate the prices to maximize the profit. Clustering refers to partition a data set into clusters such that members within the same cluster are similar in a certain sense and members of different clusters are dissimilar. Current clustering techniques can be broadly classified into several categories: partitioning methods (e.g., k-means and k-medoids), hierarchical methods (e.g. BIRCH [22]), densitybased methods (e.g. DBSCAN [15]), and grid-based methods (e.g. CLIQUE [4]). However, these methods are designed only for static data sets and can not be directly applied to data streams. O pe n A cc es s D at ab as e w w w .in te ch w eb .o rg

Combining block-based and online methods in learning ensembles from concept drifting data streams

The work in this paper is based on an industrial debutanizer column in a petroleum refinery located in Malaysia, which produces LPG (liquefied petroleum gas) as the top stream and light naphtha as the bottom stream. The controlled outputs, which are the critical product quality to be measured, are the concentrations of the top and the bottom streams. It is required to maintain the product quality for profitability, and recent trends in soft-computing have seen a plethora of methods developed for this purpose. However, due to practical constraints experienced by the plant that limit extensive plant step tests, data available for the development of the soft sensor for this industrial debutanizer column is scarce. Hence, in this paper, a neuro-fuzzy technique (ANFIS) is investigated and compared against the widely used ANN model for the special case of limited training data samples of the industrial debutanizer column. It is observed that in comparison to ANN, ANFIS has better generalization performance and can work well with limited training data samples. From the results, it was observed that the prediction performance of the ANFIS model generally improved the RMSE by approximately 10 times in comparison to that of ANN.

Today's ever‐increasing generation of streaming data demands novel data mining approaches tailored to mining dynamic data streams. Data streams are non‐static in nature, continuously generated, and endless. They often suffer from class imbalance and undergo temporal drift. To address the classification of consecutive data instances within imbalanced data streams, this research introduces a new ensemble classification algorithm called Rarity Updated Ensemble with Oversampling (RUEO). The RUEO approach is specifically designed to exhibit robustness against class imbalance by incorporating an imbalance‐specific criterion to assess the efficacy of the base classifiers and employing an oversampling technique to reduce the imbalance in the training data. The RUEO algorithm was evaluated on a set of 20 data streams and compared against 14 baseline algorithms. On average, the proposed RUEO algorithm achieves an average‐accuracy of 0.69 on the real‐world data streams, while the chunk‐based algorithms AWE, AUE, and KUE achieve average‐accuracies of 0.48, 0.65, and 0.66, respectively. The statistical analysis, conducted using the Wilcoxon test, reveals a statistically significant improvement in average‐accuracy for the proposed RUEO algorithm when compared to 12 out of the 14 baseline algorithms. The source code and experimental results of this research work will be publicly available at https://github.com/vkiani/RUEO.

Nowadays, many physical world appliances found data streams like telecommunication system, multimedia data, medical data streams. Traditional data stream mining allows storage of data and multiple scan of dataset. But it is next to impossible to save or scan it more than one or two times, because of its mountainous size. It is essential to develop the processing systems which scans once and examines the methods. Because of this, data stream mining becomes an emerging topic for research in knowledge discovery. Effective classification of such data streams finds many stream mining provocations like immeasurable length, increment learning, concept drift. So, we have to either update existing mining classifiers or generate a new technique for data stream classification. In this paper, we point out three different classification methods of decision tree called Hoeffding tree, VFDT, and CVFDT, which focuses on these classification problems.

Convolutional Neural Networks have been widely used in remote sensing scene classification. Since this kind of model needs a large number of training samples containing data category information, a Generative Adversarial Network (GAN) is usually used to address the problem of lack of samples. However, GAN mainly generates scene data samples that do not contain category information. To address this problem, a novel supervised adversarial Lie Group feature learning network is proposed. In the case of limited data samples, the model can effectively generate data samples with category information. There are two main differences between our method and the traditional GAN. First, our model takes category information and data samples as the input of the model and optimizes the constraint of category information in the loss function, so that data samples containing category information can be generated. Secondly, the object scale sample generation strategy is introduced, which can generate data samples of different scales and ensure that the generated data samples contain richer feature information. After large-scale experiments on two publicly available and challenging datasets, it is found that our method can achieve better scene classification accuracy even with limited data samples.

Mining massive data streams in real-time is one of the contemporary challenges for machine learning systems. Such a domain encompass many of difficulties hidden beneath the term of Big Data. We deal with massive, incoming information that must be processed on-the-fly, with lowest possible response delay. We are forced to take into account time, memory and quality constraints. Our models must be able to quickly process large collection of data and swiftly adapt themselves to occurring changes (shifts and drifts) in data streams. In this paper, we propose a novel version of simple, yet effective Naïve Bayes classifier for mining streams. We add a weighting module, that automatically assigns an importance factor to each object extracted from the stream. The higher the weight, the bigger influence given object exerts on the classifier training procedure. We assume, that our model works in the non-stationary environment with the presence of concept drift phenomenon. To allow our classifier to quickly adapt its properties to evolving data, we imbue it with forgetting principle implemented as weight decay. With each passing iteration, the level of importance of previous objects is decreased until they are discarded from the data collection. We propose an efficient sigmoidal function for modeling the forgetting rate. Experimental analysis, carried out on a number of large data streams with concept drift prove that our weighted Naïve Bayes classifier displays highly satisfactory performance in comparison with state-of-the-art stream classifiers.