Incremental Learning from Positive Data

Most machine learning models used in network intrusion detection system (IDS) studies are batch models which require all targeted intrusions to be present in the training data. This approach is slow because computer networks produce massive amounts of data. Furthermore, new network intrusion variants continuously emerge. Retraining the model using these extensive and evolving data takes time and resources. This study proposes AB-HT: an ensemble incremental learning algorithm for IDSs. AB-HT utilizes incremental Adaptive Boosting (AdaBoost) and Hoeffding Tree algorithms. AB-HT model could detect new intrusions without retraining the model using old training data. Thus, it could reduce the computational resources needed to retrain the model while maintaining the model's performance. We compared it to an AdaBoost-Decision Tree model, a batch learning model, to analyze the effectiveness of the incremental learning approach. Then we compared it to other incremental learning models, the Hoeffding Tree (HT) and Hoeffding Anytime Tree (HATT) models. The experimental results showed that the proposed incremental model had shorter training times than the AdaBoost-Decision Tree model in the long run. Also, on average, the AB-HT model has 18% higher F1-score values than the HT and HATT models. These advantages show that the AB-HT algorithm has promising potential to be used in the IDS field.

Semi-supervised learning is especially important in data mining applications because it can make use of plentiful unlabeled data to train the high-quality learning models. Semi-Supervised Support Vector Machine (S3VM) is a powerful semi-supervised learning model. However, the high computational cost and non-convexity severely impede the S3VM method in large-scale applications. Although several learning algorithms were proposed for S3VM, scaling up S3VM is still an open problem. To address this challenging problem, in this paper, we propose a new incremental learning algorithm to scale up S3VM (IL-S3VM) based on the path following technique in the framework of Difference of Convex (DC) programming. The traditional DC programming based algorithms need multiple outer loops and are not suitable for incremental learning, and traditional path following algorithms are limited to convex problems. Our new IL-S3VM algorithm based on the path-following technique can directly update the solution of S3VM to converge to a local minimum within one outer loop so that the efficient incremental learning can be achieved. More importantly, we provide the finite convergence analysis for our new algorithm. To the best of our knowledge, our new IL-S3VM algorithm is the first efficient path following algorithm for a non-convex problem (i.e., S3VM) with local minimum convergence guarantee. Experimental results on a variety of benchmark datasets not only confirm the finite convergence of IL-S3VM, but also show a huge reduction of computational time compared with existing batch and incremental learning algorithms, while retaining the similar generalization performance.

A-iLearn: An adaptive incremental learning model for spoof fingerprint detection

The development of efficient arrhythmia detection systems is crucial for physiological measurements and computer-aided diagnosis. Existing systems rely mainly on offline learning methods and lack the ability to assimilate new data or recognize emerging categories. To address these challenges, this study introduces an arrhythmia detection model that is resistant to data imbalance and has continuous learning capabilities, specifically for incremental learning of new ECG data categories. The system incorporates constraints derived from the new class data and implements a dynamic mechanism for updating connection weights, facilitating the incremental continual learning of classes. Confronted with the problem of models forgetting the original data and overfitting with the added data in continuous learning, we introduce a data balancing method by regularization to balance the model's memory and learning of the two types of data. Facing the data imbalance problem in continuous learning, we introduce a posteriori probability weighting strategy. This strategy assigns greater importance to high-value samples based on the model's posterior residual kernel density estimates. Comprehensive testing of the model using various datasets from the MIT-BIH database indicates superior performance in incremental learning tasks. The results reveal that the model not only excels in class incremental learning but also ensures effective balancing across different data classes.

This paper proposed a diagnostic supporting tool - i <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">+</sup> DiaKAW (intelligent and interactive knowledge acquisition workbench), which automatically extracts useful knowledge from massive medical data by applying various data mining techniques for supporting real medical diagnosis. To fulfill this effort, i <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">+</sup> DiaKAW has been developed to provide a novel dynamic on-line decision tree learning scheme - i <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">+</sup> Learning (intelligent and incremental learning) methodology that is able to incrementally incorporate the new incoming data with an existing decision tree without relearning the entire tree from scratch. i <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">+</sup> Learning strategy makes up for the traditional incremental decision tree learning algorithms by concerning the new available features in addition to the new coming instances. Such theory is a new attempt and is designed specifically for bridging the current gaps. The empirical results demonstrate a comparison of performance of various classification algorithms on several real-life datasets from UCI repository.

Most facial expression recognition (FER) systems used facial expression data created during a short period of time and this data is used for learning/training of FER systems. There are many facial expression patterns (i.e. a particular expression can be represented in many different patterns) which cannot be generated and used as learning/training data in a short time. Therefore, in order to maintain its high accuracy and robustness for a long time of a facial expression recognition system, the classifier should be evolved adaptively over time and space. We proposed a facial expression recognition system that has the aptitude of incrementally learning and thus can learn all possible patterns of expressions that may be generated in feature. After extraction of region of interest (face), the system extracts Speeded-Up Robust Features (SURF). A novel SURF descriptor template based nearest neighbor classifier is proposed for classification. This classifier is used as base/weak classifier for incremental learning algorithm Learn++. A vast range of experimentation is performed on five different databases that demonstrate the incremental learning capability of the proposed system. The experiments using the incrementally learning classification demonstrate promising results.

Incremental learning for [formula omitted]-Support Vector Regression

This paper proves the problem of losing incremental samples' information of the present SVM incremental learning algorithm from both theoretic and experimental aspects, and proposes a new incremental learning algorithm with support vector machine based on hyperplane-distance. According to the geometric character of support vector, the algorithm uses Hyperplane-Distance to extract the samples, selects samples which are most likely to become support vector to form the vector set of edge, and conducts the support vector machine training on the vector set. This method reduces the number of training samples and effectively improves training speed of incremental learning. The results of experiment performed on Chinese webpage classification show that this algorithm can reduce the number of training samples effectively and accumulate historical information. The HD-SVM algorithm has higher training speed and better precision of classification.

An incremental one-class learning algorithm is proposed for the purpose of outlier detection. Outliers are identified by estimating - and thresholding - the probability distribution of the training data. In the early stages of training a non-parametric estimate of the training data distribution is obtained using kernel density estimation. Once the number of training examples reaches the maximum computationally feasible limit for kernel density estimation, we treat the kernel density estimate as a maximally-complex Gaussian mixture model, and keep the model complexity constant by merging a pair of components for each new kernel added. This method is shown to outperform a current state-of-the-art incremental one-class learning algorithm (Incremental SVDD [5]) on a variety of datasets, while requiring only an upper limit on model complexity to be specified.KeywordsTraining DataGaussian Mixture ModelOutlier DetectionKernel Density EstimationNormal ClassThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Detecting spamming activities in a campus network using incremental learning

Bayesian network utilizes graphical model to describe dependencies among variables in probabilistic way, it is one of the most important model for uncertainty processing in Artificial Intelligence. Incremental learning of Bayesian networks has been received more attentions in recent years, in this paper a novel method is proposed to learn Bayesian network from incremental data. In this method, a novel incremental scoring function is designed to adaptively adjust the tendency of matching new and old data in the process of incremental learning. We propose an improved adaptive incremental structure learning algorithm for Bayesian network. Theoretical analysis and experimental results both demonstrate the proposed method outperforms other state-of-the-art methods.

The coronavirus pandemic has seriously affected public health and social order. Prediction methods based on machine learning can identify the infectivity phenotype and pandemic risk of coronavirus. Currently, six types of coronaviruses that infect humans have been discovered, with significant differences in viral genome sequences. Continuous genetic variation of the virus will lead to reduced performance of machine learning models and potential learning forgetting. To solve this challenge, we propose an incremental learning and knowledge distillation framework (ILKD). First, we employ Dna2Vec to extract virus features and encode the virus sequence into virus feature vector. Second, we use hierarchical clustering to continuously identify new coronavirus groups. Third, ILKD employ a combined strategy of incremental learning and knowledge distillation to transform the Back Propagation (BP) neural network to continuously learn and predict the phenotypes of human-to-human coronavirus infection. Experimental results show that ILKD can effectively alleviate the learning forgetting phenomenon. Further analysis reveals ILKD has better performance than other incremental learning models, and has important public health application value.

Incremental human action recognition with dual memory

The Internet of Things (IoT) has become an integral part of everyday life, and plays a significant role in various sectors by enabling device-to-device communication without human intervention. However, the constant connectivity of these devices to the Internet exposes them to numerous cyberattacks, potentially leading to data breaches, device malfunctions, and unauthorized network access. The diverse range of threats targeting IoT systems highlights the need for robust security solutions, such as machine learning-based intrusion detection systems, which have attracted growing research interest. This study proposes a novel adaptive approach for intrusion detection in IoT environments by combining batch and incremental learning models. Our work overcomes the limitations of traditional machine learning-based intrusion detection methods, emphasizing the potential of incremental learning to develop more effective and resilient solutions. Unlike most existing approaches that rely solely on batch learning and lack adaptability to integrate new data for long-term security, our method offers a more flexible alternative. Specifically, we leverage the predictive power of LightGBM in batch mode to improve the performance of a Passive-Aggressive classifier as a meta-learner in incremental mode. This dual learning approach aims to enhance intrusion detection in two key ways: effectively capturing diverse attack patterns with batch learning and adapting to emerging attack patterns with incremental learning. Experiments conducted on the CICIoT2023 dataset demonstrate that the proposed approach successfully balances performance and adaptability. It maintains accurate detection capabilities across various classification tasks while remaining responsive to evolving attack patterns.

A defect detection system for wire arc additive manufacturing using incremental learning