GPU-based State Adaptive Random Forest for Evolving Data Streams

Long short-term memory self-adapting online random forests for evolving data stream regression

New challenges have emerged in data mining as the traditional techniques have floundered with real-time data streams. The traditional technique needs refurbishing so as to acclimatize with concept drifting data streams. Thus dealing with the concept changes is the most imperative task of stream data mining. Ensemble classifiers have the ability to automatically adapt with the incoming drifts and, therefore, it is the most interesting research area in data stream mining. Bagging, Boosting and Random forest generation are the common ensemble techniques and are the most popular machine learning approaches in the current scenario for static data (Gomes HM, Bifet A, Read J, Barddal JP, Enembreck F, Pfharinger B, Abdessalem T (2017) Adaptive random forests for evolving data stream classification. Mach Learn 106(9–10):469–1495, [1]). A large number of base classifiers in an ensemble can cause computational overhead. Data mining classifiers for real-time data streams, therefore, need to be updated constantly and retrained with the labeled instances of the newly arrived novel classes in data streams and to cope with concept drift; otherwise, the mining models will become less and less accurate as time passes by. However, for data streams, adaptive random forest algorithms have been widely used for ensemble generation due to its competence to handle different types of drifts. This paper proposes a modified adaptive random forest with meta level learner algorithm and concept adaptive very fast decision tree to overcome the concept drift problem in real-time data streams. The proposed algorithm is experimentally compared with state-of-the-art adaptive random forest algorithm on several real synthetic datasets. Results indicate its efficiency in terms of accuracy and processing time.

Summary form only given. Data streams are continuous flows of data. Examples of data streams include network traffic, sensor data, call center records and so on. Their sheer volume and speed pose a great challenge for the data mining community to mine them. Data streams demonstrate several unique properties: infinite length, concept-drift, concept-evolution, and feature-evolution. Concept-drift occurs in data streams when the underlying concept of data changes over time. Concept-evolution occurs when new classes evolve in streams. Feature-evolution occurs when feature set varies with time in data streams. Each of these properties adds a challenge to data stream mining. This invited talk will present an organized picture on how to handle various data mining techniques in data streams: in particular, how to handle classification in evolving data streams by addressing these challenges.

Using Ensemble StackingC Method and Base Classifiers to Ameliorate Prediction Accuracy of Pedagogical Data

Corrigendum to ‘Deep learning improves taphonomic resolution: high accuracy in differentiating tooth marks made by lions and jaguars'

In recent years, the Edge Computing (EC) paradigm has emerged as an enabling factor for developing technologies like the Internet of Things (IoT) and 5G networks, bridging the gap between Cloud Computing services and end-users, supporting low latency, mobility, and location awareness to delay-sensitive applications. An increasing number of solutions in EC have employed machine learning (ML) methods to perform data classification and other information processing tasks on continuous and evolving data streams. Usually, such solutions have to cope with vast amounts of data that come as data streams while balancing energy consumption, latency, and the predictive performance of the algorithms. Ensemble methods achieve remarkable predictive performance when applied to evolving data streams due to several models and the possibility of selective resets. This work investigates a strategy that introduces short intervals to defer the processing of mini-batches. Well balanced, our strategy can improve the performance (i.e., delay, throughput) and reduce the energy consumption of bagging ensembles to classify data streams. The experimental evaluation involved six state-of-art ensemble algorithms (OzaBag, OzaBag Adaptive Size Hoeffding Tree, Online Bagging ADWIN, Leveraging Bagging, Adaptive RandomForest, and Streaming Random Patches) applying five widely used machine learning benchmark datasets with varied characteristics on three computer platforms. As a result, our strategy can significantly reduce energy consumption in 96% of the experimental scenarios evaluated. Despite the trade-offs, it is possible to balance them to avoid significant loss in predictive performance.

Most machine learning algorithms rely on a set of hyperparameters, which are tuned either by domain experts or using automated methods. In the context of data streams, the complexity of this task is compounded by the need for tuning every time a concept drift occurs. The problem of automated tuning may lead to classification performance improvements. Current methods optimise the hyperparameters from scratch every time concept drift is detected. Additionally, meta-learning techniques developed for static machine learning tasks allow pre-built knowledge to adapt faster to new tasks. We propose a meta-learning approach for automated tuning in evolving data streams, specifically for tuning parameters in Adaptive Random Forest (ARF) and parameters for several drift detectors. We compare our approach against non-adaptive methods on both synthetic and real-world datasets. Empirical results show that our approach makes a compromises between predictive performance and resource consumption for ARF, and between true positives and false positives for drift detectors.

Hypertension is a chronic disease and major risk factor for cardiovascular and cerebrovascular diseases that often leads to damage to target organs. The prevention and treatment of hypertension is crucially important for human health. In this paper, a novel ensemble method based on a flexible neural tree (FNT) is proposed to identify hypertension-related active compounds. In the ensemble method, the base classifiers are Multi-Grained Cascade Forest (gcForest), support vector machines (SVM), random forest (RF), AdaBoost, decision tree (DT), Gradient Boosting Decision Tree (GBDT), KNN, logical regression, and naïve Bayes (NB). The classification results of nine classifiers are utilized as the input vector of FNT, which is utilized as a nonlinear ensemble method to identify hypertension-related drug compounds. The experiment data are extracted from hypertension-unrelated and hypertension-related compounds collected from the up-to-date literature. The results reveal that our proposed ensemble method performs better than other single classifiers in terms of ROC curve, AUC, TPR, FRP, Precision, Specificity, and F1. Our proposed method is also compared with the averaged and voting ensemble methods. The results reveal that our method could identify hypertension-related compounds more accurately than two classical ensemble methods.

Microorganisms are fundamental to ecosystems, human health, and various industrial processes. The rapid and accurate identification of microorganisms is crucial for diagnosing diseases, monitoring environmental changes, and advancing biotechnological applications. Traditional methods of microbial identification, including culturing and microscopy, are often labor-intensive and time-consuming, necessitating the development of computational techniques to automate and improve accuracy. This project presents a comprehensive approach to microbe prediction by leveraging advanced machine learning algorithms applied to a dataset containing genomic and morphological features of ten different microorganism species. The study investigates the performance of four classification algorithms — K-Nearest Neighbors (KNN), Naive Bayes, Random Forest, and Decision Trees — to classify microorganisms based on multiple quantitative features extracted from images and genetic data. The methodology involves extensive data preprocessing steps such as cleaning, normalization, polynomial feature expansion, and dimensionality reduction using Principal Component Analysis (PCA) to enhance model performance. Rigorous experimentation and comparative analysis reveal that ensemble methods, particularly Random Forest, provide superior classification accuracy, robustness, and generalizability across unseen data. In addition to model development, a user-friendly web interface built using Streamlit facilitates real-time microorganism prediction, making the system accessible to researchers and practitioners without deep technical expertise. The platform enables the input of feature values and instantly returns predicted microorganism classes along with relevant biological descriptions and preventive measures, thereby bridging the gap between computational predictions and practical microbiological insights. This project contributes to the emerging intersection of microbiology and machine learning by demonstrating an effective pipeline from raw data to deployable predictive tool. Its implications span environmental monitoring, medical diagnostics, and industrial microbiology, emphasizing the potential for machine learning to revolutionize microbial identification and understanding. Key Words – K-Nearest Neighbors (KNN), Naive Bayes, Random Forest, and Decision Trees, Stream lit user interface software.

The data stream domain has become increasingly important in recent years because of its applicability to a wide variety of applications. Problems such as data mining and privacy preservation which have been studied for traditional data sets cannot be easily solved for the data stream domain. This is because the large volume of data arriving in a stream renders most algorithms to inefficient as most mining and privacy preservation algorithms require multiple scans of data which is unrealistic for stream data. More importantly, the characteristics of the data stream can change over time and the evolving pattern needs to be captured. In this talk, I’ll discuss the issued and focus on how to mine evolving data streams and preserve privacy.

Processing data streams gained much importance in recent years. Standard machine learning algorithms do not cope well with non-stationary streaming data, where decision models evolve and generate so-called concept drift. Online adaptive algorithms emerged to solve these issues. They learn incrementally and generally require explicit forgetting mechanisms to adapt to concept drift. In this paper, we propose the application of Kalman filtering to handle evolving data streams. This novel approach addresses data stream mining and concept drift management challenges from a new perspective, directly modelling a representation suitable for the data streams. First, we study a Kalman filter based learning a pproach and investigate its integration into the Naïve Bayes algorithm, namely KalmanNB. Additionally, we propose the Hoeffding Kalman Tree, a combination of the Hoeffding Tree with KalmanNB. Empirical results demonstrate that the Kalman filter based approach inherently manages concept drifts, and it adapts to the emerging concept more rapidly than the state-of-the-art algorithms. Moreover, it is an accurate and robust approach and requires less storage while still being faster.

In recent years, the progress in hardware technology has made it possible for organizations to store and record large streams of transactional data. This results in databases which grow without limit at a rapid rate. This data can often show important changes in trends over time. In such cases, it is useful to understand, visualize and diagnose the evolution of these trends. When the data streams are fast and continuous, it becomes important to analyze and predict the trends quickly in online fashion. In this paper, we discuss the concept of velocity density estimation, a technique used to understand, visualize and determine trends in the evolution of fast data streams. We show how to use velocity density estimation in order to create both temporal velocity profiles and spatial velocity profiles at periodic instants in time. These profiles are then used in order to predict three kinds of data evolution: dissolution, coagulation and shift. Methods are proposed to visualize the changing data trends in a single online scan of the data stream, and a computational requirement which is linear in the number of data points. In addition, batch processing techniques are proposed in order to identify combinations of dimensions which show the greatest amount of global evolution. The techniques discussed in this paper can be easily extended to spatio-temporal data, changes in data snapshots at fixed instances in time, or any other data which has a temporal component during its evolution.

The increasing use of technology in diverse field has caused generation of huge volumes of information streams. Data streams contains bulk of data points generated at high speed continuously from various applications like log records, web clicks etc. With recent advancement in technology need for analysis of such unbounded streams is increasing day by day. Data mining process helps to excavate useful knowledge from rapidly generated raw data streams. In context with the continuously generated data, mining data streams is emerging challenging task in which several issues like limited space, limited time, accuracy, handling evolving data need to be considered. This paper provides an overview of various approaches for handling changing and evolving data streams.

Classification and high dimensional data are the challenging problems in machine learning. In this paper, we carry out an empirical study of model selection for high dimensional data. We select three single classifiers, i.e., decision tree, support vector machine, and naive bayes, and two ensemble methods, i.e., bagging and random forest, to compare the performance of these five methods through real high dimensional data. Six real gene datasets with high correlation are used in this empirical study. Experimental results have illustrated that ensemble methods are better than single classifiers in terms of accuracy metric. In which, decision tree model is the weakest one, and bagging is the best model. The performance of random forest is much more efficient in terms of computing time metrics compared with bagging. From this empirical study, we can conclude that ensemble methods are an effective solution for high dimensional data.KeywordsHigh dimensional dataSupervised learningClassification

In evolving data stream, when its concept undergoes a change it is known as concept drift. Detecting concept drift and handling it is a challenging task in data stream mining. If an algorithm is not adapted to concept drift, then it directly affects its performance. A number of algorithms have been developed to handle concept drift, but they are not suited for both sudden concept drift and gradual concept drift. Thus, there is a demand for an algorithm that can react to both the types of concept drift as well as incur less computational cost. A new approach hybrid early drift detection method (HEDDM) has been proposed for drift detection, which works with an ensemble method to improve the performance.