A CNN-Based Broad Learning System

Network intrusion detection systems are an important defense technology to guarantee information security and protect a network from attacks. In recent years, the broad learning system has attracted much attention and has been introduced into intrusion detection systems with some success. However, since the traditional broad learning system is a simple linear structure, when dealing with imbalanced datasets, it often ignores the feature learning of minority class samples, leading to a poorer recognition rate of minority class samples. Secondly, the high dimensionality and redundant features in intrusion detection datasets also seriously affect the training time and detection performance of the traditional broad learning system. To address the above problems, we propose a deep belief network broad equalization learning system. The model fully learns the large-scale high-dimensional dataset via a deep belief network and represents it as an optimal low-dimensional dataset, and then introduces the equalization loss v2 reweighing idea into the broad learning system and learns to classify the low-dimensional dataset via a broad equalization learning system. The model was experimentally tested using the CICIDS2017 dataset and fully validated using the CICIDS2018 dataset. Compared with other algorithms in the same field, the model shortens the training time and has a high detection rate and a low false alarm rate.

In this paper, we propose a new deep learning structure named deep-broad network (DBNet) that is efficient for classification task. By modifying the decision-making mechanism of the deep structure, the proposed method can improve the testing efficiency while maintaining the testing accuracy. Specifically, the modified convolutional neural networks (CNNs) are first pre-trained and used to extract high-quality features. And then the dimension of extracted features is reduced by linear mapping. Finally, the broad learning system (BLS) uses processed features to make decisions. Compared with the previous deep structure, the efficiency of the proposed model is improved. Compared with the BLS, the DBNet has better performance. The proposed model is evaluated by using the CIFAR-10, CIFAR-100 and MNIST datasets. And experimental results show that the DBNet is effective and efficient. Code and models are available at https://github.com/YHDang/DBNet.

Accurate short-term load forecasting (STLF) is critical to the operation and control of the power grid. In the literature, deep learning techniques have shown its superiority for accurate load forecasting over other methods due to its strong regression ability. However, the training process of the deep learning systems is time-consuming providing that they contain a large number of parameters due to the complex structure. This paper presents a novel broad learning system (BLS) approach for load forecasting. The BLS, which is established as a flat network, is an alternative to the deep learning system by expanding the structure in a wide sense. The expanded broad structure of BLS ensures its strong approximation capability of nonlinear mapping. Moreover, the BLS has a much faster training process compared to deep learning due to its simple structure. The proposed method is tested with an open dataset at low-voltage level from Australia and the testing results demonstrate the effectiveness of the proposed method.

The broad learning system (BLS) is designed based on the technology of compressed sensing and pseudo-inverse theory, and consists of feature nodes and enhancement nodes, has been proposed recently. Compared with the popular deep learning structures, such as deep neural networks, BLS has the ability of rapid incremental learning and can remodel the system without the usual tedious retraining process. However, given that BLS is still in its infancy, it still needs analysis, improvements, and verification. In this article, we first analyze the principle of fast incremental learning ability of BLS in depth. Second, in order to provide an in-depth analysis of the BLS structure, according to the novel structure design concept of deep neural networks, we present four brand-new BLS variant networks and their incremental realizations. Third, based on our analysis of the effect of feature nodes and enhancement nodes, a new BLS structure with a semantic feature extraction layer has been proposed, which is called SFEBLS. The experimental results show that SFEBLS and its variants can increase the accuracy rate on the NORB dataset 6.18%, Fashion-MNIST dataset by 3.15%, ORL data by 5.00%, street view house number dataset by 12.88%, and CIFAR-10 dataset by 18.42%, respectively, and the four brand-new BLS variant networks also obviously outperform the original BLS.

In this paper, traditional DeepWalk method and broad learning system (BLS) are used to classify network nodes in graph embedding, and results of classification are compared. When categorizing, DeepWalk adopts one vs rest (OvR) logistic regression method, and BLS is applied after the production of vector representations. In order to obviously compare results of the two classification methods, Support Vector Machine (SVM) and Convolutional Neural Network (CNN) are employed to carry out the experiment on multi-label classification of BlogCatalog. The experimental result shows that F1 score of BLS is obviously higher than DeepWalk and other methods, and training time of BLS is much less than other methods. These performances make our method suitable to graph embedding.

In recent years, electroencephalogram (EEG) e-motion recognition has been becoming an emerging field in artificial intelligence area, which can reflect the relation between emotional states and brain activity. In this paper, we designed a novel architecture, i.e., broad dynamical graph learning system (BDGLS), to deal with EEG signals. By integrating the advantage of dynamical graph convolution neural networks (DGCNN) and broad learning system (BLS), BDGLS has the ability of extracting features on non-Euclidean domain and randomly generating nodes to find the desired connection weights simultaneously. We evaluated our system on SJTU emotion EEG dataset (SEED), and used differential entropy (DE) features as input data. In the experiments, BDGLS achieved the best result, compared with the state-of-the-art methods, e.g., support vector machine (SVM), deep belief networks (DBN), graph convolutional neural networks (DCNN) and DGCNN. Especially the performance on all-frequency band of DE features, BDGLS reached the highest average recognition accuracy of 93.66% with the standard deviation of 6.11%. The result demonstrated the excellent classification ability of BDGLS in EEG emotion recognition.

In recent years, research on facial expression recognition (FER) has become an increasingly active research topic. Deep learning is a new area, which gives a new way to classify images of human faces into emotion categories. However, it faces many difficulties caused by poor robustness and real-time performance. This paper designs a new architecture network based on Broad Learning System (BLS) for facial expressions recognition. It is established as a flat network. The original inputs are transferred and placed as mapped features in feature nodes, while the structure is expanded in wide sense in the enhancement nodes. To evaluate our architecture we tested the proposed method with the Extended Cohn-Kanade Dataset (CK+). The experimental results show that the BLS approach is very effective in facial expression recognition to compare with convolutional neural networks.

Chinese herbal medicine is an important part of traditional Chinese medicine (TCM). With developing of traditional Chinese medicine, the usage of Chinese herbal medicine is growing rapidly. It is essential to identify Chinese herbal medicine correctly since Chinese herbal medicine is used to treat disease. However, identifying Chinese herbal medicine is a hard task because lots of Chinese herbal medicine with different properties displays similar appearance, such as Radix StephaniaeTetrandrae and Radix Paeoniae Alba. Traditional methods of classifying Chinese herbal medicine are low-efficiency and rely on professional medical knowledge. Machine learning methods can reduce the need for professional knowledge in some fields due to its self-learning ability. In this study, a framework, called CNN & BLS, combining the convolutional neural network (CNN) with broad learning system (BLS) for identifying the Chinese herbal medicine, is proposed. Experimental results show the CNN & BLS displays the promising performance for identifying Chinese herbal medicine.

In an intelligent manufacturing context, the smooth operations of mechanical equipment in the production process of enterprises and timely fault diagnosis during operations have become increasingly important. However, the effect of traditional fault diagnosis depends on the feature extraction quality and experts’ empirical knowledge, which is inefficient and costly, and cannot match the needs of mechanical equipment fault diagnosis in intelligent manufacturing. The TSK fuzzy system has a strong approximation capability and the ability to interpret expert knowledge. The broad learning system (BLS) has strong feature extraction and fast computation capabilities. In this paper, we present a new model—the TSK fuzzy broad learning system (TSK-BLS). The model integrates the advantages of the BLS and the fuzzy system at the same time, which can be calculated quickly and accurately by pseudo-inverse and symmetry methods. On the other hand, the model is an embedded model-building mechanism, which extends the application scope of BLS theory. The model was tested on a bearing fault data set, provided by Case Western Reserve University, and the model’s fault diagnosis accuracy was as high as 0.9967. The results were compared with those of the convolutional neural network (CNN) and the BLS models, whose fault diagnosis accuracies are 0.6833 and 0.9133, respectively. Comparison showed that the proposed fault diagnosis model—TSK-BLS, achieved significant improvements.

The newly proposed Broad Learning System (BLS) offers an alternative way to deep learning which saves a time-consuming training process and powerful computing resources. However, the randomly generated feature nodes and lots of enhancement nodes in BLS may reduce the performance of the final classifier. Aiming at the problems in randomly generated feature nodes which suffer from unpredictability and need guidance, this paper proposes an adaptive feature nodes evolutionary algorithm (AFNE) to extract better features; While broad learning neural network often requires a large number of enhancement nodes parameters to ensure its performance, which easily leads to the redundancy or dependency between features, as well as the performance degradation of the final model. Therefore, this article also proposes a new criterion based on node sensitivity to prune the network enhancement layer nodes to remove redundant nodes, reduce the network scale, and increase the generalization ability. The proposed algorithm ADP-BLS can improve the accuracy and generalization performance of the final classifier through the evolution of feature nodes and the pruning of enhancement nodes. Extensive comparative experiments on real world data sets verify the effectiveness of the proposed ADP-BLS. At the same time, when verifying the module validity of the innovative algorithms added in this article, experiments also show that the AFNE and pruning method integrated into BLS can improve the model to a certain extent.

A method for capacity estimation of lithium-ion batteries based on adaptive time-shifting broad learning system

Broad Learning System [1] proposed recently demonstrates efficient and effective learning capability. This model is also proved to be suitable for incremental learning algorithms by taking the advantages of random vector flat neural networks. In this paper, a modified BLS structure based on the K-means feature extraction is developed. Compared with the original broad learning system, acceptable performance on more complicated data set, such as CIFAR-10, is achieved. Furthermore, it is proved that the proposed model in [1] is flexible and potential in various applications.

In recent years, the latest maintenance modelling techniques that adopt the data-based method, such as machine learning (ML), have brought about a broad range of useful applications. One of the major challenges in the automotive industry is the early detection of component failure for quick response, proper action, and minimizing maintenance costs. A vital component of an automobile system is an air pressure system (APS). Failure of APS without adequate and quick responses may lead to high maintenance costs, loss of lives, and component damages. This paper addresses classification problem where we detect whether a fault does or does not belong to APS. If a failure occurs in APS, it is classified as positive class; otherwise, it is classified as negative class. Hence, in this paper, we propose broad embedded logistic regression (BELR). The proposed BELR is applied to predict APS failure. It combines a broad learning system (BLS) and logistic regression (LogR) classifier as a fusion model. The proposed approach capitalizes on the strength of BLS and LogR for a better APS failure prediction. Additionally, we employ the BLS’s feature-mapped nodes for extracting features from the input data. Additionally, we use the enhancement nodes of the BLS to enhance the features from feature-mapped nodes. Hence, we have features that can assist LogR for better classification performances, even when the data is skewed to the positive class or negative class. Furthermore, to prevent the curse of dimensionality, a common problem with high-dimensional data sets, we utilize principal component analysis (PCA) to reduce the data dimension. We validate the proposed BELR using the APS data set and compare the results with the other robust machine learning classifiers. The commonly used evaluation metrics, namely Recall, Precision, an F1-score, to evaluate the model performance. From the results, we validate that performance of the proposed BELR.

Recently, broad learning system (BLS) have demonstrated excellent performance in hyperspectral images (HSI) classification. However, due to the complex geometric structure and spatial layout of HSI, the linear sparse features in broad learning system are difficult to fully represent hyperspectral data. In addition, the features learned by broad learning system lack more effective discriminative ability, which leads to the limited expressive ability of features. To address the issues, we propose a graph convolutional enhanced discriminative broad learning system (GCDBLS) for HSI Classification. GCDBLS aggregates the node information in the adjacency graph through graph convolution, and then learns the context relationship, so as to obtain rich nonlinear spatial spectral features in hyperspectral images; In order to extract more discriminative hyperspectral image features, GCDBLS introduces the concept of local intra-class scatter and local inter-class scatter. By minimizing the local intra-class feature distance and maximizing the local inter-class feature distance, GCDBLS can improve the discrimination ability of BLS extracted features. On three HSI datasets, the experiments compared with the latest classification methods show that the proposed method achieves good results, and improves the classification performance of hyperspectral images.

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.