Deep Belief Networks Oriented Clustering

With the rapid development of artificial intelligence technology of deep learning method, it has been applied to many fields, especially to life science. In this paper, a novel approach for the task of voiceprint recognition was proposed. The combination of deep belief network (DBN) and support vector machine (SVM) was used to identify the voiceprint of 10 different individuals. Based on a 24-dimension Mel Frequency Cepstrum Coefficient (MFCC), the authors extracted 256-dimension deep voiceprint features via DBN model that is developed by stacking three layers of Restricted Boltzmann Machine (RBM), and conducted voiceprint recognition by approach of SVM. According to the recognition results, the new approach can significantly improve both of accuracy and efficiency when it was compared with traditional voiceprint features and recognition models. The low dimensional features of voiceprint was extracted into higher dimensions by DBN model, while SVM can avoid the elevation of computation complexity caused by the increases of feature dimension. The combined strengths have been fully expressed by the experimental results.

Measuring brain activity through Electroencephalogram (EEG) analysis for eye state prediction has attracted attention from machine learning researchers. There have been many methods for EEG analysis using supervised and unsupervised machine learning techniques. The tradeoff between the accuracy and computation time of these methods in performing the analysis is an important issue that is rarely investigated in the previous research. This paper accordingly proposes a new method for EEG signal analysis through Self-Organizing Map (SOM) clustering and Deep Belief Network (DBN) approaches to efficiently improve the computation and accuracy of the previous methods. The method is developed using SOM clustering and DBN, which is a deep layer neural network with multiple layers of Restricted Boltzmann Machines (RBMs). The results on a dataset with 14980 instances and 15 attributes representing the values of the electrodes demonstrated that the method is efficient for EEG analysis. In addition, compared with the other supervised methods, the proposed method was able to significantly improve the accuracy of the EEG prediction.

The localization accuracy of a fingerprint-based localization system is dependent on several factors, one of which is the accuracy and efficiency at which the fingerprint database is clustered. Most highly efficient and accurate clustering algorithms have high time-dependent computational complexity (CC), which tends to limit their practical applicability. A technique that has yet to be explored is the sequential hybridization of multiple low-time CC clustering algorithms to produce a single moderate-time CC clustering algorithm with high localization accuracy. As a result, this paper proposes a clustering algorithm with a moderate time CC that is based on the sequential hybridization of the closest access point (CAP) and improved k-means clustering algorithms. The performance of the proposed sequential hybrid clustering algorithm is determined and compared to the modified affinity propagation clustering (m-APC), fuzzy c-mean (FCM), and 2-CAP algorithms presented in earlier research works using four experimentally generated and publicly available fingerprint databases. The performance metrics considered for the comparisons are the position root mean square error (RMSE) and clustering time based on big O notation. The simulation results show that the proposed sequential hybrid clustering algorithm has improved localization accuracy with position RMSEs of about 54%, 77%, and 52%, respectively, higher than those of the m-APC, FCM, and 2-CAP algorithms. In terms of clustering time, it is 99% and 79% faster than the m-APC and FCM algorithms, respectively, but 90% slower than the 2-CAP algorithm. The results have shown that it is possible to develop a clustering algorithm that has a moderate clustering time with very high localization accuracy through sequential hybridization of multiple clustering algorithms that have a low clustering time with poor localization accuracy. Doi: 10.28991/ESJ-2024-08-02-02 Full Text: PDF

PurposeThe purpose of this paper is to propose an approach to incorporate contextual information into collaborative filtering (CF) based on the restricted Boltzmann machine (RBM) and deep belief networks (DBNs). Traditionally, neither the RBM nor its derivative model has been applied to modeling contextual information. In this work, the authors analyze the RBM and explore how to utilize a user’s occupation information to enhance recommendation accuracy.Design/methodology/approachThe proposed approach is based on the RBM. The authors employ user occupation information as a context to design a context-aware RBM and stack the context-aware RBM to construct DBNs for recommendations.FindingsThe experiments on the MovieLens data sets show that the user occupation-aware RBM outperforms other CF models, and combinations of different context-aware models by mutual information can obtain better accuracy. Moreover, the context-aware DBNs model is superior to baseline methods, indicating that deep networks have more qualifications for extracting preference features.Originality/valueTo improve recommendation accuracy through modeling contextual information, the authors propose context-aware CF approaches based on the RBM. Additionally, the authors attempt to introduce hybrid weights based on information entropy to combine context-aware models. Furthermore, the authors stack the RBM to construct a context-aware multilayer network model. The results of the experiments not only convey that the context-aware RBM has potential in terms of contextual information but also demonstrate that the combination method, the hybrid recommendation and the multilayer neural network extension have significant benefits for the recommendation quality.

Deep belief network based electricity load forecasting: An analysis of Macedonian case

Fuzzy c-means (FCM) clustering algorithm has been widely used in automated image segmentation. However, the standard FCM algorithm is sensitive to noise because of taking no into account the gray and spatial information of pixel. The paper proposes an improved FCM algorithm for image segmentation. We use the degree of gray similarity and distribution statistics of the neighbor pixels to form a new membership function for clustering. Not only it is effective to remove the noise spots and reduce the spurious blobs, but also it is ease to correct the misclassified pixels. Experimental results on three types of image indicate that the propose algorithm is more accurate and robust than the standard FCM algorithm

Metaheuristic algorithms have been hybridized with the standard K-means to address the latter’s challenges in finding a solution to automatic clustering problems. However, the distance calculations required in the standard K-means phase of the hybrid clustering algorithms increase as the number of clusters increases, and the associated computational cost rises in proportion to the dataset dimensionality. The use of the standard K-means algorithm in the metaheuristic-based K-means hybrid algorithm for the automatic clustering of high-dimensional real-world datasets poses a great challenge to the clustering performance of the resultant hybrid algorithms in terms of computational cost. Reducing the computation time required in the K-means phase of the hybrid algorithm for the automatic clustering of high-dimensional datasets will inevitably reduce the algorithm’s complexity. In this paper, a preprocessing phase is introduced into the K-means phase of an improved firefly-based K-means hybrid algorithm using the concept of the central limit theorem to partition the high-dimensional dataset into subgroups of randomly formed subsets on which the K-means algorithm is applied to obtain representative cluster centers for the final clustering procedure. The enhanced firefly algorithm (FA) is hybridized with the CLT-based K-means algorithm to automatically determine the optimum number of cluster centroids and generate corresponding optimum initial cluster centroids for the K-means algorithm to achieve optimal global convergence. Twenty high-dimensional datasets from the UCI machine learning repository are used to investigate the performance of the proposed algorithm. The empirical results indicate that the hybrid FA-K-means clustering method demonstrates statistically significant superiority in the employed performance measures and reducing computation time cost for clustering high-dimensional dataset problems, compared to other advanced hybrid search variants.

Accurate and robust image segmentation was identified as one of the most challenging issues facing PET quantification in oncological imaging. This difficulty is compounded by the low spatial resolution and high noise characteristics of PET images. The fuzzy C-means (FCM) clustering algorithm was largely used in various medical image segmentation approaches. However, the algorithm is sensitive to both noise and intensity heterogeneity since it does not take into account spatial contextual information. To overcome this limitation, a new fuzzy segmentation technique adapted to typical noisy and low resolution oncological PET data is proposed. PET images smoothed using a nonlinear anisotropic diffusion filter are added as a second input to the proposed FCM algorithm to incorporate spatial information (FCM-S). In addition, a methodology was developed to integrate the a trous wavelet transform in the standard FCM algorithm (FCM-SW) to allow handling of heterogeneous lesions' uptake. The algorithm was applied to the simulated data of the NCAT phantom, incorporating heterogeneous lesions in the lung and clinical PET/CT images of 21 patients presenting with histologically proven nonsmall-cell lung cancer (NSCLC) and 7 patients presenting with laryngeal squamous cell carcinoma (LSCC) to assess its performance for segmenting tumors with arbitrary size, shape, and tracer uptake. For NSCLC patients, the maximal tumor diameters measured from the macroscopic examination of the surgical specimen served as the ground truth for comparison with the maximum diameter estimated by the segmentation technique, whereas for LSCC patients, the 3D macroscopic tumor volume was considered as the ground truth for comparison with the corresponding PET-based volume. The proposed algorithm was also compared to the classical FCM segmentation technique. There is a good correlation (R2 = 0.942) between the actual maximal diameter of primary NSCLC tumors estimated using the proposed PET segmentation procedure and those measured from the macroscopic examination, and the regression line agreed well with the line of identity (slope = 1.08) for the group analysis of the clinical data. The standard FCM algorithm seems to underestimate actual maximal diameters of the clinical data, resulting in a mean error of -4.6 mm (relative error of -10.8 +/- 23.1%) for all data sets. The mean error of maximal diameter estimation was reduced to 0.1 mm (0.9 +/- 14.4%) using the proposed FCM-SW algorithm. Likewise, the mean relative error on the estimated volume for LSCC patients was reduced from 21.7 +/- 22.0% for FCM to 8.6 +/- 28.3% using the proposed FCM-SW technique. A novel unsupervised PET image segmentation technique that allows the quantification of lesions in the presence of heterogeneity of tracer uptake was developed and evaluated. The technique is being further refined and assessed in clinical setting to delineate treatment volumes for the purpose of PET-guided radiation therapy treatment planning but could find other applications in clinical oncology such as the assessment of response to treatment.

There is a current trend towards wearable electrocardiogram (ECG) measurement systems, which enables measurement while the subject performs their normal activities of daily living (e.g., walking, driving, eating). This type of measurement is susceptible to higher levels of contaminants, compared to bedside measurements, due to subject movement and a measurement environment that is not well-controlled. Contaminants in the measured signal (e.g., motion artifact, power line interference) can cause incorrect interpretations, including misdiagnoses. Therefore, prior to ECG interpretations, it is important to have an algorithm capable of automatically classifying the measured ECG based on their signal quality. ECG of low signal quality can undergo additional pre-processing to mitigate the contaminants or the signal can be discarded. This can reduce misdiagnoses, including false-alarms which is a top medical technology hazard. In this paper, we propose an algorithm based on Deep Belief Networks (DBN) which can differentiate between noisy and clean signal measurements. Our algorithm is designed based on a three layer Restricted Boltzmann Machine (RBM) in which the first two RBMs are trained to extract the features and apply them to the third layer of RBM to classify the data. Results, using the MIT-BIH Arrhythmia database, demonstrate that our algorithm can successfully recognize a noisy ECG signal from a clean signal, with a classification accuracy between 75% and 99.5% depending on the level of contaminants. Our algorithm also correctly identifies clean arrhythmic signals and does not misidentify them as noisy. The proposed algorithm is applicable to any ECG measurement systems including wearable and bedside.

Supply chain flexibility is the processing ability of the enterprize to deal with the uncertain environment of supply and demand. In this paper, we consider the supply side (node interrupt) and demand side (demand fluctuations) under uncertain environment. By using the deep belief network (DBN), which is composed of multilayer Restricted Boltzmann Machine (RBM), it establishes the supply chain flexibility network with optimization of the transfer node and flow. The deep belief network is trained by the data of large manufacturing enterprize, compared with the traditional neural network (MLR, BP and GA). The results show that the deep belief network model overcomes the shortcomings of the traditional neural networks, such as easy to fall into local optimum, long training time and low function fitting degree, and it has higher prediction accuracy. This network model based on the deep belief network can promote the supply chain flexibility more, when supply and demand fluctuations occur.

Big data is technology that has the ability to manage very large amounts of data, in very fast time to allow real-time analysis and reactions. Several clustering methods which are used to group data are Fuzzy C-Means (FCM) and K-Means Clustering. K-Means Clustering algorithm is a method of partitioning existing data into two or more group. This research goal was to compare the performance of K-Means and Fuzzy C-Means algorithms in clustering data using big data technology. In this research, Hadoop and Hive were chosen the big data technology. The knowledge of Shia history on student and lecturer of Syarif Hidayatullah State Islamic University Jakarta were the data which used in this research, The testing was done by constructing application K-Means Fuzzy C-Means using Java language, Hadoop and Hive and then test the performance of K-Means and Fuzzy C-Means algorithms in data clustering. It compares both algorithms in terms of accuracy, execution time, and time complexity of the algorithms. In the application K-Means Fuzzy C-Means, evaluation were performed with data filter and the average accuracy difference result of K-Means and Fuzzy C-Means is 8.03% with the better accuracy owned by K-Means. The average execution time difference is 718.58 ms, which K-Means was faster than is Fuzzy C-Means. The time complexities of both algorithms have the same value O(n2) and the Big O equation resulted in an average difference of 93,568 with the smallest value on K-Means. Thus, K-Means algorithm is better than the Fuzzy C-Means in terms of accuracy, execution time, and the time complexity.

High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning

Accurate and real-time traffic flow prediction nowadays shows more and more dependence on big transportation data. Deep learning, a powerful method for feature learning, has turned out to be an effective tool to cope with these explosive data. Recently, deep models, especially unsupervised models like deep belief networks (DBN) and stacked autoencoder (SAE), are being employed into the field of traffic research and have shown great prospect. However, there is still a vacancy in the exploration on comparing the performances of different kinds of deep architectures to find an optimal solution. In this paper, we set up two deep-learning-based traffic flow prediction models for feature extraction and performances comparison: One is a deep belief networks (DBN) based on restricted Boltzmann machines (RBMs) that have Gaussian visible units and binary hidden units, and the other is a DBN based on RBMs with all units being binary. A conclusion is drawn where the former one performs better in traffic flow prediction after a series of experiments.

The clustering methods based on Fuzzy C-Means (FCM) are frequently used in image-segmentation. But the standard FCM algorithm has some defects, especially ignoring the pixel spatial information's influence on the classification result. For the sake of a more reasonable objective function, an improved FCM algorithm is proposed in this paper, which uses spatial information and S-function to determine the weight coefficients of the objective function. Experimental results show that the proposed algorithm has better performance than the standard FCM algorithm.

Network intrusion detection systems are an important defense technology to guarantee information security and protect a network from attacks. In recent years, the broad learning system has attracted much attention and has been introduced into intrusion detection systems with some success. However, since the traditional broad learning system is a simple linear structure, when dealing with imbalanced datasets, it often ignores the feature learning of minority class samples, leading to a poorer recognition rate of minority class samples. Secondly, the high dimensionality and redundant features in intrusion detection datasets also seriously affect the training time and detection performance of the traditional broad learning system. To address the above problems, we propose a deep belief network broad equalization learning system. The model fully learns the large-scale high-dimensional dataset via a deep belief network and represents it as an optimal low-dimensional dataset, and then introduces the equalization loss v2 reweighing idea into the broad learning system and learns to classify the low-dimensional dataset via a broad equalization learning system. The model was experimentally tested using the CICIDS2017 dataset and fully validated using the CICIDS2018 dataset. Compared with other algorithms in the same field, the model shortens the training time and has a high detection rate and a low false alarm rate.