RETRACTED: Bayesian approach to incremental batch learning on forest cover sensor data for multiclass classification

The ability to learn more concepts from incrementally arriving data over time is essential for the development of a lifelong learning system. However, deep neural networks often suffer from forgetting previously learned concepts when continually learning new concepts, which is known as the catastrophic forgetting problem. The main reason for catastrophic forgetting is that past concept data are not available, and neural weights are changed during incrementally learning new concepts. In this article, we propose an incremental concept learning framework that includes two components, namely, ICLNet and RecallNet. ICLNet, which consists of a trainable feature extractor and a dynamic concept memory matrix, aims to learn new concepts incrementally. We propose a concept-contrastive loss to alleviate the magnitude of neural weight changes and mitigate the catastrophic forgetting problems. RecallNet aims to consolidate old concepts memory and recall pseudo samples, whereas ICLNet learns new concepts. We propose a balanced online memory recall strategy to reduce the information loss of old concept memory. We evaluate the proposed approach on the MNIST, Fashion-MNIST, and SVHN data sets and compare it with other pseudorehearsal-based approaches. Extensive experiments demonstrate the effectiveness of our approach.

The development of efficient arrhythmia detection systems is crucial for physiological measurements and computer-aided diagnosis. Existing systems rely mainly on offline learning methods and lack the ability to assimilate new data or recognize emerging categories. To address these challenges, this study introduces an arrhythmia detection model that is resistant to data imbalance and has continuous learning capabilities, specifically for incremental learning of new ECG data categories. The system incorporates constraints derived from the new class data and implements a dynamic mechanism for updating connection weights, facilitating the incremental continual learning of classes. Confronted with the problem of models forgetting the original data and overfitting with the added data in continuous learning, we introduce a data balancing method by regularization to balance the model's memory and learning of the two types of data. Facing the data imbalance problem in continuous learning, we introduce a posteriori probability weighting strategy. This strategy assigns greater importance to high-value samples based on the model's posterior residual kernel density estimates. Comprehensive testing of the model using various datasets from the MIT-BIH database indicates superior performance in incremental learning tasks. The results reveal that the model not only excels in class incremental learning but also ensures effective balancing across different data classes.

Early warning of severe weather caused by intense convective weather systems is challenging. To help such activities, meteorological satellites with high temporal and spatial resolution have been utilized for the monitoring of instability trends along with water vapor variation. The current study proposes a retrieval algorithm based on an artificial neural network (ANN) model to quickly and efficiently derive total precipitable water (TPW) and convective available potential energy (CAPE) from Korea’s second geostationary satellite imagery measurements (GEO-KOMPSAT-2A/Advanced Meteorological Imager (AMI)). To overcome the limitations of the traditional static (ST) learning method such as exhaustive learning, impractical, and not matching in a sequence data, we applied an ANN model with incremental (INC) learning. The INC ANN uses a dynamic dataset that begins with the existing weight information transferred from a previously learned model when new samples emerge. To prevent sudden changes in the distribution of learning data, this method uses a sliding window that moves along the data with a window of a fixed size. Through an empirical test, the update cycle and the window size of the model are set to be one day and ten days, respectively. For the preparation of learning datasets, nine infrared brightness temperatures of AMI, six dual channel differences, temporal and geographic information, and a satellite zenith angle are used as input variables, and the TPW and CAPE from ECMWF model reanalysis (ERA5) data are used as the corresponding target values over the clear-sky conditions in the Northeast Asia region for about one year. Through the accuracy tests with radiosonde observation for one year, the INC NN results demonstrate improved performance (the accuracy of TPW and CAPE decreased by approximately 26% and 26% for bias and about 13% and 12% for RMSE, respectively) when compared to the ST learning. Evaluation results using ERA5 data also reveal more stable error statistics over time and overall reduced error distribution compared with ST ANN.

An improved self-organizing incremental neural network model for short-term time-series load prediction

Incremental learning for crop growth parameters estimation and nitrogen diagnosis from hyperspectral data

With the exponential growth in the speed and volume of remote sensing data, deep learning models are expected to adapt and continually learn over time. Unfortunately, the domain shift between multi-source remote sensing data from various sensors and regions poses a significant challenge. Segmentation models face difficulty in adapting to incremental domains due to catastrophic forgetting, which can be addressed via incremental learning methods. However, current incremental learning methods mainly focus on class-incremental learning, wherein classes belong to the same remote sensing domain, and neglect investigations into incremental domains in remote sensing. To solve this problem, we propose a domain-incremental learning method for semantic segmentation in multi-source remote sensing data. Specifically, our model aims to incrementally learn a new domain while preserving its performance on previous domains without accessing previous domain data. To achieve this, our model has a unique parameter learning structure that reparametrizes domain-agnostic and domain-specific parameters. We use different optimization strategies to adapt to domain shift in incremental domain learning. Additionally, we adopt multi-level knowledge distillation loss to mitigate the impact of label space shift among domains. The experiments demonstrate that our method achieves excellent performance in domain-incremental settings, outperforming existing methods with only a few parameters.

A defect detection system for wire arc additive manufacturing using incremental learning

Detecting fraud accurately in credit cards is critical as this financial sector incurs significant losses for cardholders. Nonetheless, most studies adopted standard machine learning and few incremental learning, which are inadequate for addressing credit card challenges, such as rapid data arrival, unlimited data, data sensitivity, and performance decline over time. For this purpose, we propose a chunk-based incremental feature learning approach that optimises the fraud model topology for each new chunk and keeps track of one chunk each time. The model consists of several connected sub-models, where a new sub-model is optimally created for each new chunk. To avoid the network growing indefinitely, we limit the number of sub-models. To this end, we retain the most relevant sub-models to the current chunk’s data distribution and re-combine them to create the optimal model. We evaluate our approach using two credit card datasets: the first of medium scale contains 2-day payments in 2013, and the second of considerable scale possesses 6-month payments in 2019. We split these datasets into multiple chunks to learn and test incrementally. We compare our approach with static learning methods trained with different scenarios. Moreover, we vary the number of historical sub-models to check their impact on the predictive performance.

Since the beginning of planet Earth, volcanic activity has generated a series of events of great destructive capacity. Many populations located in areas close to volcanoes coexist with a complex combination of benefits and several risks. In order to provide early warning of an eventuality, different approaches have been developed to alert the inhabitants of these areas. Computer scientists have built and tested many automated tools based on Supervised Learning (SL) to solve this problem. However, despite the high potential of SL algorithms, current research does not consider the dynamic nature of volcanoes, which constantly change their baseline over time through interactions in the Earth's crust. In this study, a documentary review was developed around SL in volcanology. This study shows that Incremental Learning (IL) offers greater advantages than non-incremental learning for this type of application domains. One of them is that incremental learning considers dynamic systems and updates the objective function in real time without retraining the classifiers. Based on the above, this paper presents an updated review of the current literature examining different types of learning methods (incremental and non-incremental learning) in the volcano detecting task.

Novel deep neural network based pattern field classification architectures

Catastrophic forgetting is a major problem that affects neural networks during progressive learning. In it, the previously learned representation vanishes as the network learns new information. The extreme learning machine is one of the variants of the neural network. It is used in many domains due to fast training and good generalization ability. However, like other neural networks, it suffers from catastrophic forgetting and negative forward and backward transfer during the progression of neurons in incremental learning. The study hypothesizes that it is due to overlapping in hidden neurons and output weights. The global representation by an activation function further supports this hypothesis. To address this, the study proposes a neuron clustering approach to mitigate it in an adaptive class incremental extreme learning machine. The neuron clustering method activates k nearest neurons during learning and testing. It helps to partition the network to select overlapping subnetwork. Experimental results on four food datasets show that the proposed approach reduces negative forward and backward transfer when neurons are added incrementally during progressive learning.

Integrating regression formulas and kernel functions into locally adaptive knowledge-based neural networks: A case study on renal function evaluation

Traditionally, the performance of deep convolutional neural networks relies on a large number of labeled datasets in advance. However, in real-world applications, training data are not collected at once, so an algorithm which can deal with continuous incoming data is needed. This learning method is called incremental learning, whose main problem is the catastrophic forgetting. Neural network will perform badly on the old classes after training for the new classes. To solve this problem, this paper proposes an integrated incremental network approach based on the PEDCC-loss, and the cosine distance between the sample’s output feature and the PEDCC (predefined evenly distributed class centroids). Old and new knowledge learned by neural networks are stored separately. During the training of each network, PEDCC-Loss is used to constrain the cosine distance between the output feature and their corresponding pre-defined class center. Meanwhile, the old knowledge is retained by different learning rates across the network, and the retention mode of the old samples is discussed. In test phase, the final prediction is determined by the cosine distances between PEDCC and output features across all networks. Our experiments on EMNIST, CIFAR100 and TinyImageNet datasets show that our approach can learn classes incrementally without quickly failing of performance. Compared with some existing algorithms, such as Hou, iCaRL and finetune, our approach has better performance.

In this study, the deep multi-layered group method of data handling (GMDH)-type neural network algorithm using revised heuristic self-organization method is proposed and applied to medical image diagnosis of liver cancer. The deep GMDH-type neural network can automatically organize the deep neural network architecture which has many hidden layers. The structural parameters such as the number of hidden layers, the number of neurons in hidden layers and useful input variables are automatically selected to minimize prediction error criterion defined as Akaike’s information criterion (AIC) or prediction sum of squares (PSS). The architecture of the deep neural network is automatically organized using the revised heuristic self-organization method which is a type of the evolutionary computation. This new neural network algorithm is applied to the medical image diagnosis of the liver cancer and the recognition results are compared with the conventional 3-layered sigmoid function neural network.

Speech signal has various features of speakers. This feature is extracted from speech signal processing. The speaker is identified by the speaker identification system. In this paper, we propose the speaker identification system that uses the incremental learning based on neural network. Recorded speech signal through the microphone is blocked to the frame of 1024 speech samples. Energy is divided speech signal to voiced signal and unvoiced signal. The extracted 12 orders LPC cpestrum coefficients are used with input data for neural network. The speakers are identified with the speaker identification system using the neural network. The neural network has the structure of MLP which consists of 12 input nodes, 8 hidden nodes, and 4 output nodes. The number of output node means the identified speakers. The first output node is excited to the first speaker. Incremental learning begins when the new speaker is identified. Incremental learning is the learning algorithm that already learned weights are remembered and only the new weights that are created as adding new speaker are trained. It is learning algorithm that overcomes the fault of neural network. The neural network repeats the learning when the new speaker is entered to it. The architecture of neural network is extended with the number of speakers. Therefore, this system can learn without the restricted number of speakers.