- Research Article
18
- 10.1016/j.neucom.2021.07.065
Graph-based broad learning system for classification
- Jul 30, 2021
- Neurocomputing
- Zheng Liu + 3 more +3
Graph-based broad learning system for classification
Broad learning system for semi-supervised learning
Graph-based broad learning system for classification
Graph-based broad learning system for classification
Broad Multitask Learning System With Group Sparse Regularization.
The broad learning system (BLS) featuring lightweight, incremental extension, and strong generalization capabilities has been successful in its applications. Despite these advantages, BLS struggles in multitask learning (MTL) scenarios with its limited ability to simultaneously unravel multiple complex tasks where existing BLS models cannot adequately capture and leverage essential information across tasks, decreasing their effectiveness and efficacy in MTL scenarios. To address these limitations, we proposed an innovative MTL framework explicitly designed for BLS, named group sparse regularization for broad multitask learning system using related task-wise (BMtLS-RG). This framework combines a task-related BLS learning mechanism with a group sparse optimization strategy, significantly boosting BLS's ability to generalize in MTL environments. The task-related learning component harnesses task correlations to enable shared learning and optimize parameters efficiently. Meanwhile, the group sparse optimization approach helps minimize the effects of irrelevant or noisy data, thus enhancing the robustness and stability of BLS in navigating complex learning scenarios. To address the varied requirements of MTL challenges, we presented two additional variants of BMtLS-RG: BMtLS-RG with sharing parameters of feature mapped nodes (BMtLS-RGf), which integrates a shared feature mapping layer, and BMtLS-RGf and enhanced nodes (BMtLS-RGfe), which further includes an enhanced node layer atop the shared feature mapping structure. These adaptations provide customized solutions tailored to the diverse landscape of MTL problems. We compared BMtLS-RG with state-of-the-art (SOTA) MTL and BLS algorithms through comprehensive experimental evaluation across multiple practical MTL and UCI datasets. BMtLS-RG outperformed SOTA methods in 97.81% of classification tasks and achieved optimal performance in 96.00% of regression tasks, demonstrating its superior accuracy and robustness. Furthermore, BMtLS-RG exhibited satisfactory training efficiency, outperforming existing MTL algorithms by 8.04-42.85 times.
Read moreBroad Minimax Probability Learning System and its Application in Regression Modeling
Broad learning system (BLS), a novel incremental learning algorithm, has attracted more and more attention and application. BLS achieves a perfect balance between learning performance and modeling efficiency based on the mean square error (MSE) criterion. However, MSE assumes that a random error between an observed value and real value is a Gaussian distribution, which may be inconsistent to the real-world situation. Moreover, the distributional assumptions may cast doubt on the generalization and validity of the built BLS model. In this article, we propose a novel distribution-free BLS for regression analysis, called broad minimax probability learning system (BMPLS). Learning from the idea of minimax probability machine regression, BMPLS builds a regression model by maximizing the worst probability of the regression function being within the allowed error range. It adequately utilizes the mean and covariance information of the hidden-space feature without making any distributional assumptions of the random error. In order to further improve the learning performance of the model, a regularized BMPLS (RBMPLS) is proposed by applying elastic-net regularization term to BMPLS. The corresponding optimization algorithm is also designed for calculating the output weights of the model. Multiple experiments based on public datasets were carried out to demonstrate the feasibility of the proposed algorithms.
Read moreNetwork Intrusion Detection Based on Deep Belief Network Broad Equalization Learning System
Network intrusion detection systems are an important defense technology to guarantee information security and protect a network from attacks. In recent years, the broad learning system has attracted much attention and has been introduced into intrusion detection systems with some success. However, since the traditional broad learning system is a simple linear structure, when dealing with imbalanced datasets, it often ignores the feature learning of minority class samples, leading to a poorer recognition rate of minority class samples. Secondly, the high dimensionality and redundant features in intrusion detection datasets also seriously affect the training time and detection performance of the traditional broad learning system. To address the above problems, we propose a deep belief network broad equalization learning system. The model fully learns the large-scale high-dimensional dataset via a deep belief network and represents it as an optimal low-dimensional dataset, and then introduces the equalization loss v2 reweighing idea into the broad learning system and learns to classify the low-dimensional dataset via a broad equalization learning system. The model was experimentally tested using the CICIDS2017 dataset and fully validated using the CICIDS2018 dataset. Compared with other algorithms in the same field, the model shortens the training time and has a high detection rate and a low false alarm rate.
Read moreDiscriminative projective dictionary pair based broad metric learning system: algorithm and its applications in pattern classification
Pattern classification plays a pivotal role in a wide range of domains, including computer vision and healthcare. The Broad Learning System (BLS) has attracted considerable attention for its competitive classification performance and computational efficiency. However, its reliance on randomly initialized parameters and lack of iterative updates often lead to performance instability. Directly applying backpropagation to refine these parameters may further result in overfitting. To address these limitations, this research propose a novel framework called the Discriminative Projective Dictionary Pair-based Broad Metric Learning System (D-BMLS). The foundation of this system is the Broad Metric Learning System (BMLS), which integrates a metric subsystem that employs iterative learning to reduce sensitivity to random initialization while leveraging the structural advantages of metric learning to suppress overfitting. Although this improves robustness, it can also introduce computational overhead and still struggle with nonlinear data modeling due to the dual-mapping structure of BLS. To overcome these challenges, D-BMLS incorporates Discriminative Projective Dictionary Pair Learning, which encodes input data into a low-dimensional, linearly separable space. This reduces the number of learnable parameters and enhances the model’s capacity to capture nonlinear relationships through linear transformations. Extensive experiments on five different tasks including image classification, signal recognition, and high-dimensional feature analysis demonstrate the superior performance of D-BMLS. Ablation studies on three benchmark datasets verify the contributions of each component, and results on a synthetic dataset highlight the metric subsystem’s effectiveness in mitigating overfitting.
Read moreA Broad Ensemble Learning System for Drifting Stream Classification
In a data stream environment, classification models must effectively and efficiently handle concept drift. Ensemble methods are widely used for this purpose; however, the ones available in the literature either use a large data chunk to update the model or learn the data one by one. In the former, the model may miss the changes in the data distribution, while in the latter, the model may suffer from inefficiency and instability. To address these issues, we introduce a novel ensemble approach based on the Broad Learning System (BLS), where mini chunks are used at each update. BLS is an effective lightweight neural architecture recently developed for incremental learning. Although it is fast, it requires huge data chunks for effective updates and is unable to handle dynamic changes observed in data streams. Our proposed approach, named Broad Ensemble Learning System (BELS), uses a novel updating method that significantly improves best-in-class model accuracy. It employs an ensemble of output layers to address the limitations of BLS and handle drifts. Our model tracks the changes in the accuracy of the ensemble components and reacts to these changes. We present the mathematical derivation of BELS, perform comprehensive experiments with 35 datasets that demonstrate the adaptability of our model to various drift types, and provide its hyperparameter, ablation, and imbalanced dataset performance analysis. The experimental results show that the proposed approach outperforms 10 state-of-the-art baselines, and supplies an overall improvement of 18.59% in terms of average prequential accuracy.
Read moreSemi-Supervised Broad Learning System Based on Manifold Regularization and Broad Network
Broad Learning System (BLS) are widely used in many fields because of its strong feature extraction ability and high computational efficiency. However, the BLS is mainly used in supervised learning, which greatly limits the applicability of the BLS. And the obtained data is less labeled data, but is a large number of unlabeled data. Therefore, the BLS is extended based on the semi-supervised learning of manifold regularization framework to propose a semi-supervised broad learning system (SS-BLS). Firstly, the features are extracted from labeled and unlabeled data by building feature nodes and enhancement nodes. Then the manifold regularization framework is used to construct Laplacian matrix. Next, the feature nodes, enhancement nodes and Laplacian matrix are combined to construct the objective function, which is effectively solved by ridge regression in order to obtain the output coefficients. Finally, the validity of the SS-BLS is verified by three different complex data of G50C, MNIST, and NORB, respectively. The experiment result show that the SS-BLS can achieve higher classification accuracy for different complex data, takes on fast operation speed and strong generalization ability.
Read moreChebyshev Polynomial Broad Learning System
The broad learning system (BLS) has been attracting more and more attention due to its excellent property in the field of machine learning. A great deal of variants and hybrid structures of BLS have also been designed and developed for better performance in some specialized tasks. In this paper, the Chebyshev polynomials are introduced into the BLS to take advantage of their powerful approximation capability, where the feature windows are replaced by a set of Chebyshev polynomials. This new variant, named Chebyshev polynomial BLS (CPBLS), has a light structure with a reduction in computational complexity since the sparse autoencoder is removed. Instead, the dimension of each input sample is expended by n + 1 Chebyshev polynomials, mapping the original feature into a new feature space with higher dimension, which helps to classify the patterns in training. The proposed CPBLS is evaluated by some popular datasets from UCI and KEEL repositories, and it outperforms some representative neural networks and neuro-fuzzy models in terms of classification accuracy. The CPBLS also show some advantages over the recent developed compact fuzzy BLS (CFBLS) which indicates its great potential in future research and real-world applications.
Read moreBroad Learning System with Proportional-Integral-Differential Gradient Descent
Broad learning system (BLS) has attracted much attention in recent years due to its fast training speed and good generalization ability. Most of the existing BLS-based algorithms use the least square method to calculate its output weights. As the size of the training data set increases, this approach will cause the training efficiency of the model to be seriously reduced, and the solution of the model will also be unstable. To solve this problem, we have designed a new gradient descent method (GD) based on the proportional-integral-differential technique (PID) to replace the least square operation in the existing BLS algorithms, which is called PID-GD-BLS. Extensive experimental results on four benchmark data sets show that PID-GD can achieve faster convergence rate than traditional optimization algorithms such as Adam and AdaMod, and the generalization performance and stability of the PID-GD-BLS are much better than that of BLS and its variants. This study provides a new direction for BLS optimization and a better solution for BLS-based data mining.
Read moreMulti-Attn BLS: Multi-head attention mechanism with broad learning system for chaotic time series prediction
Multi-Attn BLS: Multi-head attention mechanism with broad learning system for chaotic time series prediction
An Incremental-Self-Training-Guided Semi-Supervised Broad Learning System.
The broad learning system (BLS) has recently been applied in numerous fields. However, it is mainly a supervised learning system and thus not suitable for specific practical applications with a mixture of labeled and unlabeled data. Despite a manifold regularization-based semi-supervised BLS, its performance still requires improvement, because its assumption is not always applicable. Therefore, this article proposes an incremental-self-training-guided semi-supervised BLS (ISTSS-BLS). Distinctive to traditional self-training, where all unlabeled data are labeled simultaneously, incremental self-training (IST) obtains unlabeled data incrementally from an established sorted list based on the distance between the data and their cluster center. During iterative learning, a small portion of labeled data is first used to train BLS. The system recursively self-updates its structure and meta-parameters using: 1) the double-restricted mechanism and 2) the dynamic neuron-incremental mechanism. The double-restricted mechanism is beneficial to preventing the introduction of incorrect pseudo-labeled samples, and the dynamic neuron-incremental mechanism guides the self-updating of the network structure effectively based on the training accuracy of the labeled data. These strategies guarantee a parsimonious model during the update. Besides, a novel metric, the accuracy-time ratio (A/T), is proposed to evaluate the model's performance comprehensively regarding time and accuracy. In experimental verifications, ISTSS-BLS performs outstandingly on 11 datasets. Specifically, the IST is compared with the traditional one on three scales data, saving up to 52.02% learning time. In addition, ISTSS-BLS is compared with different state-of-the-art alternatives, and all results indicate that it possesses significant advantages in performance.
Read moreEfficient Ensemble Broad Learning System Based on Dropout and DropConnect
Broad Learning System is an emerged efficient algorithm for training single hidden layer feedforward neural network (SLFN) with fast speed and good generalization ability. However, it is very difficult to determine the appropriate broad learning system structure and broad learning system may perform overfitting due to the dependence between nodes in processing fully connected network. In order to deal with these problems, efficient ensemble broad learning system based on Dropout and DropConnect are proposed in this paper. The proposed Dropout Ensemble Broad Learning System randomly discards hidden nodes to improve diversity between individuals and reduce the synergy between nodes to improve prediction stability. The DropConnect Ensemble Broad Learning System randomly drops connect weights to generate more complementary models by adding input attribute disturbance. The experimental results and statistical analysis on UCI data sets confirm that the proposed method can solve the problem of model overfitting caused by the strong dependence between the nodes of ensemble broad learning system and also show that the proposed approaches outperform the original BLS on prediction stability and classification accuracy.
Read moreBroad stochastic configuration network for regression
Broad stochastic configuration network for regression
Short-Term Load Forecasting Using Broad Learning System
Short-term load forecasting (STLF) is of great value in power system operation. This paper proposes an STLF method based on the broad learning system (BLS). First, factors that have major impacts on load demands are selected as input variables of STLF. Then, the construction of BLS is discussed. The original input is mapped into feature nodes using the sparse autoencoder, and the network expands in width through the enhancement nodes which are generated from the mapped features. Both the feature and enhancement nodes are directly connected to the output layer. Such a flat structure is exploited to implement incremental learning, which updates BLS dynamically without retraining. Finally, the proposed model is tested on the ISO New England dataset. Test results indicate that BLS is accurate and efficient compared with the existing machine learning techniques. Moreover, the incremental learning capability improves the efficiency of the update of STLF models remarkably.
Read moreEquivalent Modeling of Microgrids Based on Optimized Broad Learning System
The DC microgrid is an important structure of microgrids. Aiming at the problem of the grid-connected DC microgrid modeling, a grid-connected DC microgrid equivalent modeling method based on the optimized Broad Learning System (BLS) is proposed. Taking the electrical parameter data of the grid-connected DC microgrid access point as the training data set of BLS, the equivalent model of the grid-connected equivalent model is constructed. In order to further improve the accuracy and generalization performance of the model, the shark smell optimization (SSO) algorithm is used to optimize the input weights and thresholds of the BLS. Furthermore, the shark smell optimization-Broad Learning System (SSO-BLS) algorithm is proposed. SSO-BLS is compared with RBF, BLS, BFO-ELM, and other algorithms. The results show that the grid-connected DC microgrid model based on SSO-BLS has good accuracy and generalization characteristics.
Read more