- Research Article
10
- 10.1016/j.neunet.2024.106436
Self-balancing Incremental Broad Learning System with privacy protection
- Jun 06, 2024
- Neural Networks
- Weiwen Zhang + 5 more +5
Self-balancing Incremental Broad Learning System with privacy protection
In recent years, deep learning (DL) has been widely used in vehicle misbehavior detection and has attracted great attention due to its powerful nonlinear mapping ability. However, because of the large number of network parameters, the training processes of these methods are time consuming. Besides, the existing detection methods lack scalability; thus, they are not suitable for Internet of Vehicles (IoV) where new data are constantly generated. In this article, the concept of the broad learning system (BLS) is innovatively introduced into vehicle misbehavior detection. In order to make better use of vehicle information, key features are first extracted from the collected raw data. Then, a BLS is established, which is able to calculate the connection weight of the network efficiently and effectively by ridge regression approximation. Finally, the system can be updated and refined by an incremental learning algorithm based on the newly generated data in IoV. The experimental results show that the proposed method performs much better than DL or traditional classifiers, and could update and optimize the old model fastly and progressively while improving the system's misbehavior detection accuracy.
Self-balancing Incremental Broad Learning System with privacy protection
Self-balancing Incremental Broad Learning System with privacy protection
Broad Multitask Learning System With Group Sparse Regularization.
The broad learning system (BLS) featuring lightweight, incremental extension, and strong generalization capabilities has been successful in its applications. Despite these advantages, BLS struggles in multitask learning (MTL) scenarios with its limited ability to simultaneously unravel multiple complex tasks where existing BLS models cannot adequately capture and leverage essential information across tasks, decreasing their effectiveness and efficacy in MTL scenarios. To address these limitations, we proposed an innovative MTL framework explicitly designed for BLS, named group sparse regularization for broad multitask learning system using related task-wise (BMtLS-RG). This framework combines a task-related BLS learning mechanism with a group sparse optimization strategy, significantly boosting BLS's ability to generalize in MTL environments. The task-related learning component harnesses task correlations to enable shared learning and optimize parameters efficiently. Meanwhile, the group sparse optimization approach helps minimize the effects of irrelevant or noisy data, thus enhancing the robustness and stability of BLS in navigating complex learning scenarios. To address the varied requirements of MTL challenges, we presented two additional variants of BMtLS-RG: BMtLS-RG with sharing parameters of feature mapped nodes (BMtLS-RGf), which integrates a shared feature mapping layer, and BMtLS-RGf and enhanced nodes (BMtLS-RGfe), which further includes an enhanced node layer atop the shared feature mapping structure. These adaptations provide customized solutions tailored to the diverse landscape of MTL problems. We compared BMtLS-RG with state-of-the-art (SOTA) MTL and BLS algorithms through comprehensive experimental evaluation across multiple practical MTL and UCI datasets. BMtLS-RG outperformed SOTA methods in 97.81% of classification tasks and achieved optimal performance in 96.00% of regression tasks, demonstrating its superior accuracy and robustness. Furthermore, BMtLS-RG exhibited satisfactory training efficiency, outperforming existing MTL algorithms by 8.04-42.85 times.
Read moreBroad Minimax Probability Learning System and its Application in Regression Modeling
Broad learning system (BLS), a novel incremental learning algorithm, has attracted more and more attention and application. BLS achieves a perfect balance between learning performance and modeling efficiency based on the mean square error (MSE) criterion. However, MSE assumes that a random error between an observed value and real value is a Gaussian distribution, which may be inconsistent to the real-world situation. Moreover, the distributional assumptions may cast doubt on the generalization and validity of the built BLS model. In this article, we propose a novel distribution-free BLS for regression analysis, called broad minimax probability learning system (BMPLS). Learning from the idea of minimax probability machine regression, BMPLS builds a regression model by maximizing the worst probability of the regression function being within the allowed error range. It adequately utilizes the mean and covariance information of the hidden-space feature without making any distributional assumptions of the random error. In order to further improve the learning performance of the model, a regularized BMPLS (RBMPLS) is proposed by applying elastic-net regularization term to BMPLS. The corresponding optimization algorithm is also designed for calculating the output weights of the model. Multiple experiments based on public datasets were carried out to demonstrate the feasibility of the proposed algorithms.
Read moreNetwork Intrusion Detection Based on Deep Belief Network Broad Equalization Learning System
Network intrusion detection systems are an important defense technology to guarantee information security and protect a network from attacks. In recent years, the broad learning system has attracted much attention and has been introduced into intrusion detection systems with some success. However, since the traditional broad learning system is a simple linear structure, when dealing with imbalanced datasets, it often ignores the feature learning of minority class samples, leading to a poorer recognition rate of minority class samples. Secondly, the high dimensionality and redundant features in intrusion detection datasets also seriously affect the training time and detection performance of the traditional broad learning system. To address the above problems, we propose a deep belief network broad equalization learning system. The model fully learns the large-scale high-dimensional dataset via a deep belief network and represents it as an optimal low-dimensional dataset, and then introduces the equalization loss v2 reweighing idea into the broad learning system and learns to classify the low-dimensional dataset via a broad equalization learning system. The model was experimentally tested using the CICIDS2017 dataset and fully validated using the CICIDS2018 dataset. Compared with other algorithms in the same field, the model shortens the training time and has a high detection rate and a low false alarm rate.
Read moreDiscriminative projective dictionary pair based broad metric learning system: algorithm and its applications in pattern classification
Pattern classification plays a pivotal role in a wide range of domains, including computer vision and healthcare. The Broad Learning System (BLS) has attracted considerable attention for its competitive classification performance and computational efficiency. However, its reliance on randomly initialized parameters and lack of iterative updates often lead to performance instability. Directly applying backpropagation to refine these parameters may further result in overfitting. To address these limitations, this research propose a novel framework called the Discriminative Projective Dictionary Pair-based Broad Metric Learning System (D-BMLS). The foundation of this system is the Broad Metric Learning System (BMLS), which integrates a metric subsystem that employs iterative learning to reduce sensitivity to random initialization while leveraging the structural advantages of metric learning to suppress overfitting. Although this improves robustness, it can also introduce computational overhead and still struggle with nonlinear data modeling due to the dual-mapping structure of BLS. To overcome these challenges, D-BMLS incorporates Discriminative Projective Dictionary Pair Learning, which encodes input data into a low-dimensional, linearly separable space. This reduces the number of learnable parameters and enhances the model’s capacity to capture nonlinear relationships through linear transformations. Extensive experiments on five different tasks including image classification, signal recognition, and high-dimensional feature analysis demonstrate the superior performance of D-BMLS. Ablation studies on three benchmark datasets verify the contributions of each component, and results on a synthetic dataset highlight the metric subsystem’s effectiveness in mitigating overfitting.
Read moreA Broad Ensemble Learning System for Drifting Stream Classification
In a data stream environment, classification models must effectively and efficiently handle concept drift. Ensemble methods are widely used for this purpose; however, the ones available in the literature either use a large data chunk to update the model or learn the data one by one. In the former, the model may miss the changes in the data distribution, while in the latter, the model may suffer from inefficiency and instability. To address these issues, we introduce a novel ensemble approach based on the Broad Learning System (BLS), where mini chunks are used at each update. BLS is an effective lightweight neural architecture recently developed for incremental learning. Although it is fast, it requires huge data chunks for effective updates and is unable to handle dynamic changes observed in data streams. Our proposed approach, named Broad Ensemble Learning System (BELS), uses a novel updating method that significantly improves best-in-class model accuracy. It employs an ensemble of output layers to address the limitations of BLS and handle drifts. Our model tracks the changes in the accuracy of the ensemble components and reacts to these changes. We present the mathematical derivation of BELS, perform comprehensive experiments with 35 datasets that demonstrate the adaptability of our model to various drift types, and provide its hyperparameter, ablation, and imbalanced dataset performance analysis. The experimental results show that the proposed approach outperforms 10 state-of-the-art baselines, and supplies an overall improvement of 18.59% in terms of average prequential accuracy.
Read moreDownsizing and enhancing broad learning systems by feature augmentation and residuals boosting
Recently, a broad learning system (BLS) has been theoretically and experimentally confirmed to be an efficient incremental learning system. To get rid of deep architecture, BLS shares the same architecture and learning mechanism of the well-known functional link neural networks (FLNN), but works in broad learning way on both the randomly mapped features of original features of data and their randomly generated enhancement nodes. As such, BLS often requires a huge heap of hidden nodes to achieve the prescribed or satisfactory performance, which may inevitably cause both overwhelming storage requirement and overfitting phenomenon. In this study, a stacked architecture of broad learning systems called D&BLS is proposed to achieve enhanced performance and simultaneously downsize the system architecture. By boosting the residuals between previous and current layers and simultaneously augmenting the original input space with the outputs of the previous layer as the inputs of current layer, D&BLS stacks several lightweight BLS sub-systems to guarantee stronger feature representation capability and better classification/regression performance. Three fast incremental learning algorithms of D&BLS are also developed, without the need for the whole re-training. Experimental results on some popular datasets demonstrate the effectiveness of D&BLS in the sense of both enhanced performance and reduced system architecture.
Read moreBroad learning system based on fractional order optimization.
Broad learning system based on fractional order optimization.
Incremental Multilayer Broad Learning System With Stochastic Configuration Algorithm for Regression
Broad learning system (BLS) is a novel randomized learning framework which has a faster modeling efficiency. Although BLS with incremental learning has a better extendibility for updating model rapidly, the incremental mode of BLS lacks self-supervision mechanism which cannot adjust the structure adaptively. Learning from the idea of stochastic configuration network (SCN), a novel incremental multi-layer broad learning system based on stochastic configuration (SC) algorithm is proposed for regression, termed as IMLBLS-SC. Firstly, to improve the feature learning ability, SC algorithm is adopted to configure the parameters of enhancement nodes instead of random weights. Secondly, the multi-layer model with enhancement nodes can be added gradually according to supervision mechanism without human intervention. Thirdly, all the enhancement nodes and feature nodes are fully connected with output nodes. Finally, 2 function approximation problems and 8 classical datasets are selected to verify the regression performance of IMLBLS-SC, experimental results demonstrate that IMLBLS-SC outperforms the RVFLN, SCN, BLS and BSCN.
Read moreA federated bidirectional connection broad learning scheme for secure data sharing in Internet of Vehicles
Data sharing in Internet of Vehicles (IoV) makes it possible to provide personalized services for users by service providers in Intelligent Transportation Systems (ITS). As IoV is a multi-user mobile scenario, the reliability and efficiency of data sharing need to be further enhanced. Federated learning allows the server to exchange parameters without obtaining private data from clients so that the privacy is protected. Broad learning system is a novel artificial intelligence technology that can improve training efficiency of data set. Thus, we propose a federated bidirectional connection broad learning scheme (FeBBLS) to solve the data sharing issues. Firstly, we adopt the bidirectional connection broad learning system (BiBLS) model to train data set in vehicular nodes. The server aggregates the collected parameters of BiBLS from vehicular nodes through the federated broad learning system (FedBLS) algorithm. Moreover, we propose a clustering FedBLS algorithm to offload the data sharing into clusters for improving the aggregation capability of the model. Some simulation results show our scheme can improve the efficiency and prediction accuracy of data sharing and protect the privacy of data sharing.
Read moreIncremental Bayesian broad learning system and its industrial application
Broad learning system (BLS) is viewed as a class of neural networks with a broad structure, which exhibits an efficient training process through incremental learning. An incremental Bayesian framework broad learning system is proposed in this study, where the posterior mean and covariance over the output weights are both derived and updated in an incremental manner for the increment of feature nodes, enhancement nodes, and input data, respectively, and the hyper-parameters are simultaneously updated by maximizing the evidence function. In such a way, the scale of matrix operations is capable of being effectively reduced. To verify the performance of this proposed approach, a number of experiments by using four benchmark datasets and an industrial case are carried out. The experimental results demonstrate that the proposed method can not only achieve a better outcome compared to the classical BLS and other comparative algorithms but also incrementally remodel the system.
Read moreA novel simultaneous monitoring method for surface roughness and tool wear in milling process
Quality prediction and condition monitoring are crucial for realizing zero-defect intelligent manufacturing. Surface roughness is an important technical indicator to measure the surface quality of parts. In the cutting process, tool wear is directly related to the dynamic change of the surface roughness and the machining efficiency. To enhance data utilization and reduce computational costs, this paper develops a novel two-task simultaneous monitoring method for surface roughness and tool wear. First, the enhancement layer corresponding to each sub-task in the broad learning system is replaced with a reservoir with echo state characteristics, and through information sharing between sub-tasks and the capture of their respective dynamic characteristics, a broad echo state two-task learning system with incremental learning is constructed. Then, an end-face milling machining experiment was conducted on a vertical machining center, and the feasibility of the developed method for two-task simultaneous monitoring of surface roughness and tool wear was verified through feature extraction and nonlinear dimensionality reduction of the monitoring signals during the machining process. The experimental results indicate that the developed two-task simultaneous monitoring method is superior to other two-task learning methods in terms of comprehensive performance, and it lays a solid foundation for the high-quality and efficient machining of CNC machine tools.
Read moreMulti-Attn BLS: Multi-head attention mechanism with broad learning system for chaotic time series prediction
Multi-Attn BLS: Multi-head attention mechanism with broad learning system for chaotic time series prediction
Bidimensionally partitioned online sequential broad learning system for large-scale data stream modeling
Incremental broad learning system (IBLS) is an effective and efficient incremental learning method based on broad learning paradigm. Owing to its streamlined network architecture and flexible dynamic update scheme, IBLS can achieve rapid incremental reconstruction on the basis of the previous model without the entire retraining from scratch, which enables it adept at handling streaming data. However, two prominent deficiencies still persist in IBLS and constrain its further promotion in large-scale data stream scenarios. Firstly, IBLS needs to retain all historical data and perform associated calculations in the incremental learning process, which causes its computational overhead and storage burden to increase over time and as such puts the efficacy of the algorithm at risk for massive or unlimited data streams. Additionally, due to the random generation rule of hidden nodes, IBLS generally necessitates a large network size to guarantee approximation accuracy, and the resulting high-dimensional matrix calculation poses a greater challenge to the updating efficiency of the model. To address these issues, we propose a novel bidimensionally partitioned online sequential broad learning system (BPOSBLS) in this paper. The core idea of BPOSBLS is to partition the high-dimensional broad feature matrix bidimensionally from the aspects of instance dimension and feature dimension, and consequently decompose a large least squares problem into multiple smaller ones, which can then be solved individually. By doing so, the scale and computational complexity of the original high-order model are substantially diminished, thus significantly improving its learning efficiency and usability for large-scale complex learning tasks. Meanwhile, an ingenious recursive computation method called partitioned recursive least squares is devised to solve the BPOSBLS. This method exclusively utilizes the current online samples for iterative updating, while disregarding the previous historical samples, thereby rendering BPOSBLS a lightweight online sequential learning algorithm with consistently low computational costs and storage requirements. Theoretical analyses and simulation experiments demonstrate the effectiveness and superiority of the proposed algorithm.
Read moreAn active learning-based incremental deep-broad learning algorithm for unbalanced time series prediction
An active learning-based incremental deep-broad learning algorithm for unbalanced time series prediction