Text document clustering using Spectral Clustering algorithm with Particle Swarm Optimization

Simulated annealing spectral clustering algorithm for image segmentation

In order to overcome the drawbacks of falling into local extremum and lower optimization precision of standard particle swarm optimization (PSO) algorithm, multipopulation strategy, adaptive dynamic adjustment strategy and co-evolution mode are introduced into the standard PSO algorithm in order to propose an improved coevolutionary PSO(MPACEPSO) algorithm based on multi-strategy evolution mode and multi-population co-evolutionary mechanism. In the evolutionary process of MPACEPSO algorithm, the multi-population strategy is used to divide the population into several subpopulations, which use different co-evolutionary model to evolve. These sub-populations are influenced and promoted each other in order to realize the exchange of information and co-evolution among the sub-populations, improve the convergence speed and search precision of MPACEPSO algorithm, and effectively suppress the appearance of the local optimum. The adaptive dynamic adjustment strategy of inertia weight is used to keep the diversity of population, reduce the probability of falling into the local extremum. Finally, the ZDT functions are selected to test the optimization performance of proposed MPACEPSO algorithm. The experimental results show that the proposed MPACEPSO algorithm has faster convergence speed, stronger global search ability, higher solving precision and better dynamic optimization performance. The experimental result analysis shows that the proposed MPACEPSO algorithm is insensitive to parameters and easy to be used in solving the complex optimization problems.

Spectral clustering has become a typical and efficient clustering method used in a variety of applications. The critical step of spectral clustering is the similarity measurement, which largely determines the performance of the spectral clustering method. In this paper, we propose a novel spectral clustering algorithm based on the local similarity measure of shared neighbors. This similarity measurement exploits the local density information between data points based on the weight of the shared neighbors in a directed k $$ k $$ -nearest neighbor graph with only one parameter k $$ k $$ , that is, the number of nearest neighbors. Numerical experiments on synthetic and real-world datasets demonstrate that our proposed algorithm outperforms other existing spectral clustering algorithms in terms of the clustering performance measured via the normalized mutual information, clustering accuracy, and F $$ F $$ -measure. As an example, the proposed method can provide an improvement of 15.82% in the clustering performance for the Soybean dataset.

For the spectral clustering algorithm in the large-scale samples, there are bottlenecks in the storage space and computing time, and the paper analyzes the current common solutions, namely based on the sparse t-nearest neighbor spectral clustering. In order to improve the accuracy of the spectral clustering algorithms, the Euclidean distance based on the attribute weight of information entropy is proposed to calculate the similarity between the samples. First, calculate the weight of each attribute of the sample and then calculate the similarity between the samples. The degree matrix is applied to the spectral clustering of sparse t-nearest neighbors in the last number of numbers. The experimental results show that the clustering accuracy of the method on some data sets is higher than that of the original spectral clustering algorithm.

In this paper, nonlinear functions generated by randomly initialized multilayer perceptrons (MLPs) and simultaneous recurrent neural networks (SRNs) are learned by MLPs and SRNs. Training SRNs is a challenging task and a new learning algorithm - DEPSO is introduced. DEPSO is a standard particle swarm optimization (PSO) algorithm with the addition of a differential evolution step to aid in swarm convergence. The results from DEPSO are compared with the standard backpropagation (BP) and PSO algorithms. It is further verified that functions generated by SRNs are harder to learn than those generated by MLPs but DEPSO provides better learning capabilities for the functions generated by MLPs and SRNs as compared to BP and PSO. These three algorithms are also trained on several benchmark functions to confirm results.

Particle swarm optimisation (PSO) was developed by Kennedy and Eberhart in 1995 (Kennedy & Eberhart, 1995) inspired by the collective behaviour of natural birds or fish. PSO is a stochastic optimisation technique that uses a behaviour of population composed by many search points called particle. In spite of easy implementation in computer algorithms, it is well known as a powerful numerical optimizer. In the typical PSO algorithms, a set of particles searches the optimal solution in the problem space efficiently, by sharing the common attractor called global best. There are many modified versions of PSO by improving convergence property to a certain problem. While, a standard PSO is defined by Bratton and Kennedy (Bratton & Kennedy, 2007) to give a real standard for PSO studies. PSO seems as one of the evolutionary computations (ECs), and it has been shown that PSO is comparable to a genetic algorithm (Angeline, 1998). Thus, a lot of studies have demonstrated the effectiveness of PSO family in optimizing various continuous and discrete optimization problems. And a plenty of applications of PSO, such as the neural network training, PID controller tuning, electric system optimisation have been studied and achieved well results (Kennedy, 1997). However, PSO is often failed in searching the global optimal solution in the case of the objective function has a large number of dimensions. The reason of this phenomenon is not only existence of the local optimal solutions, the velocities of the particles sometimes lapsed into the degeneracy, so that the successive range is restricted in the sub-plain of the whole search hyper-plain. The sub-plane that is defined by finite number of particle velocities is a partial space in the whole search space. The issue of local optima in PSO has been studied and proposed several modifications on the basic particle driven equation (Parsopoulos et al., 2001; Hendtlass, 2005; Liang et al., 2006). There used a kind of adaptation technique or randomized method (e.g. mutation in evolutionary computations) to keep particles velocities or to accelerate them. Although such improvements work well and have ability to avoid fall in the local optima, the problem of early convergence by the degeneracy of some dimensions is still remaining, even if there are no local optima. Hence the PSO algorithm does not always work well for the high-dimensional function.

Due to its complexity and non-deterministic polynomial-time hard characteristic, multirobot task allocation problem remains a challenging issue in the field of cooperative robotics. Thanks to its easy implementation and promising convergence speed, the particle swarm optimization method has recently aroused increasing research interest in the area of multirobot task allocation problem. However, the efficiency of the standard particle swarm optimization is hindered by several deficiencies such as the inefficient capabilities in balancing exploration and exploitation, as well as the high likelihood of plunging into stagnation. Aiming at enhancing the performance of particle swarm optimization via remedying these two drawbacks, this paper proposes an improved particle swarm optimization method, which integrates standard particle swarm optimization 2011 with evolutionary game theory. To prevent particles being locked into stagnation, particles in the proposed particle swarm optimization first adopt the updating rules of standard particle swarm optimization 2011 to undertake their movements. Subsequently, attempting to well trade off the exploration and exploitation capabilities of particles, a novel self-adaptive strategy, which is determined by the evolutionary stable strategies of evolutionary game theory and the iteration number of particle swarm optimization, is presented to adaptively adjust the main control parameters of particles in the proposed particle swarm optimization. Since the convergence of particle swarm optimization remains paramount and dramatically affects the performance of particle swarm optimization, this paper also analytically investigates the convergence of the proposed particle swarm optimization and provides a convergence-guaranteed parameter selection principle for the proposed method. Finally, leveraging the development of the proposed particle swarm optimization, this paper completes the design of a new particle swarm optimization–based multirobot task allocation method. The performance of the new particle swarm optimization–based multirobot task allocation method is tested through three different allocation cases against four well-known evolutionary methods. Experimental results confirm that the proposed method generally outperforms its contenders in terms of the solution quality. Moreover, the proposed method performs slightly better than the majority of its peers as far as the computation time is concerned.

Neural Network system for SoC is one topic in the world. Recently, SoC began to combine AI technique by Python. When developers consider Intelligent system on SoC, the co-design software and hardware are significant. In this case, small devices and embedded system are target system. For IoT, those techniques will grow up in the world. Those cases, avoiding premature convergence while keeping performance is a challenge in training the neural network, especially in a case of large NNs or a large number of training data. In this case, one problem is a huge memory and processing power. For tiny memory and processing system as SoC, we propose an improved particle swarm optimization (PSO) algorithm called the PSOseed2 algorithm for training NN. The PSOseed2 algorithm solves the premature convergence of the standard PSO (SPSO) algorithm by slightly modifying the velocity update function without adding many computational tasks to the SPSO algorithm. We evaluated this algorithm on field programmable gate array (FPGA)-based NN and software-based NN and trained these NNs with different PSO algorithms that are SPSO, PSOseed, PSOseed2, and dissipative PSO. Experimental results with different datasets confirmed that the NNs trained by the proposed PSOseed2 algorithm had better recognition rates and lower global learning errors than the NN trained by other PSO algorithms. In this talk, I would like to talk those relational research in the world and our results and discussion.

SummaryCluster ensemble combines multiple base clustering results in a suitable way to improve the accuracy of the clustering result. In the conventional cluster ensemble frameworks, pairwise constraints and constraint projections have not been used together, and spectral clustering algorithm is rarely adopted to serve as the consensus function. In this paper, we design a constraint projections for semi‐supervised spectral clustering ensemble (CPSSSCE) model. It takes advantages of spectral clustering algorithm and executes semi‐supervised learning twice. Compared to traditional cluster ensemble approaches, CPSSSCE is characterized by several properties. First, the original data are transformed to lower‐dimensional representations by constraint projection before base clustering. Second, a similarity matrix is constructed using the base clustering results and modified using pairwise constraints. Third, the spectral clustering algorithm is applied to process the similarity matrix to obtain a consensus cluster result. Extensive experiments on standard University of California Irvine Machine Learning Repository (UCI) and Microsoft datasets demonstrated that the CPSSSCE is superior to other cluster ensemble algorithms including a semi‐supervised spectral clustering ensemble.

Document clustering plays a vital role in text mining fields such as information retrieval, sentiment analysis, and text organizing. Document clustering aims to automatically divide a collection of documents based on some aspects of similarity into groups that are meaningful, useful or both. This paper aims to improve the clustering task for the Arabic documents. Recent studies show that partitioning clustering algorithms are more suitable for clustering process. However, k-means is the most common algorithm that is being used for clustering process because of its simplicity and speed. It can only generate an arbitrary solution because the results depend on the initial centers for the desired clusters “the seeds”. In this paper, a new modified k-means algorithm called PSO K-means, supported by Particle Swarm Optimization (PSO) is applied to enhance the Arabic document clustering process. Then, an intensive comparative study between the proposed model and the standard k-means algorithm is applied. Also, the stemming algorithms those are being used in Arabic language processing were assessed. Through the experiments, an evaluation for the new algorithm is done with three different Arabic data sets. The results demonstrate that the proposed model can produce more accurate results compared to the standard k-means algorithm for Arabic language documents. On the other hand, Arabic light stemmer is more suitable for the stemming step.

The spectral clustering algorithm's space complexity is O(n <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ), while time complexity is O(n <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> ). When dealing with large amounts of data, the memory will overflow and run-time is too long. For the general problem of spectral clustering, if the clustering data of sub-problem between the original problem has the same probability distribution, it can be applied to divide and conquer strategy for the problem of spectral clustering, by the spectral clustering results of sub-problems to get the spectral clustering results of original problem. To spectral clustering image segmentation as a research object, we will discuss the divide and conquer strategy for spectral clustering in this paper. Experiments show that the application of divide and conquer method for spectral clustering image segmentation, we can get a perfect performance in image segmentation.

Inertia weight is one of the most important adjustable parameter of particle swarm optimization (PSO). The proper selection of inertia weight can prove a right balance between global search and local search. In this paper, two novel PSOs with non-linear inertia weight based on the tangent function and the arc tangent function are provided, respectively. The performance of the proposed PSO model is compared with standard PSO with linearly-decrease inertia weight. The experimental results demonstrated that our proposed PSO model is better than standard PSO in terms of convergence rate and solution precision.

The primary user emulation attack (puea) is considered to be one of the common threats in cognitive radio networks (crns). in this problem, an attacker emulates the primary user (pu) signal to deceive other secondary users (sus) and forcing them to leave the white spaces (free spaces) in the spectrum assigned before to the PU. The PUEA is detected and localized using the time-difference-of-arrival (TDOA) localization technique based on stochastic optimization algorithms. Particle swarm optimization (PSO) algorithms are proposed to minimize the cost function provided by the TDOA measurements and to increase the accuracy of the detection. The PSO variants are evolved by changing the parameters of the standard PSO such as inertia weight and acceleration constants. These approaches are presented and compared with the standard PSO in terms of convergence speed and processing time. This paper presents the first study of designing a PSO algorithm suitable for the localization problem and it will be considered as a good guidance for applying the optimization algorithms in wireless positioning techniques. Mean square error (MSE) and cumulative distribution function (CDF) are used as the evaluation metrics to measure the accuracy of the proposed algorithms. Simulation results show that the proposed PSO approaches provide higher accuracy and faster convergence than the standard PSO, social spider optimization (SSO), cuckoo search (CS), firefly optimization (FA) and Taylor series estimation (TSE) methods.

In order to improve the stability and sensitivity of particle swarm optimization (PSO) algorithm and to solve the problem of premature convergence, in this re-search, a computer PSO algorithm based on thermodynamic motion mechanism is proposed based on the principle of thermodynamic motion mechanism. Firstly, the thermodynamic motion phenomenon, the diffusion law in kinematics and the standard PSO algorithm are introduced. Then, according to the basic idea of thermodynamic motion mechanism, the standardized PSO algorithm is optimized and its optimization process is introduced. Finally, the experimental results are analysed by setting the test function. The results show that among the five test functions, the computer PSO algorithm based on thermodynamic motion mechanism has a higher probability of jumping out of the local optimal solution. Its robustness and stability are much better than standard PSO algorithms. The evolution ability of the computer PSO algorithm based on thermodynamic motion mechanism is better than that of the standard PSO algorithm. The standard PSO algorithm is superior because it is based on thermodynamic motion mechanism. The research in this paper can provide good guidance for improving the performance of PSO algorithm.

Evrimsel Algoritmalar (EA’lar), ağ tasarımı problemleri, yol bulma problemleri, sosyal ve ekonomik planlama gibi karmaşık optimizasyon problemlerini çözmek için yaygın olarak kullanılan algoritmalardır. Kullandıkları akıllı yöntemler sayesinde EA’lar, yeni çözümler üretmek için ilk çözümü, yinelemeli olarak, iyileştirirler. Sıklıkla kullanılan EA'lardan biri Parçacık Sürü Optimizasyonu (PSO) algoritmasıdır. PSO algoritması, sürünün sunduğu en iyi sonuç ile sürüdeki her bir bireyin en iyi sonuçlarını temel alarak, verilen bir fonksiyonunun optimum değerine ulaşır. PSO başarılı bir algoritma olmasına rağmen, sadece en iyiyi temel alan yapısı sebebiyle, belli bir düzene sahip olmayan, aldatıcı fonksiyon türlerinde, lokal değere sıkışıp optimum değere ulaşmayabilmektedir. Bu makalede, daha önce yapmış olduğumuz bir çalışmadan ilham alarak, PSO’ya bipolar davranış eklenerek, yeni bir algoritma olan Bipolar Parçacık Sürü Optimizasyonu (BPSO) sunulmuştur. BPSO algoritmasında, PSO’da olduğu gibi sadece en iyi bireylerin değil, kötü bireylerin de algoritmanın işleyişine katılmasına olanak sağlanmıştır. BPSO algoritmasının performansını, standart PSO algoritmasının performansıyla ile kıyaslamak için on test fonksiyonu kullanılmıştır. Test sonuçlarına göre, BPSO, standart PSO’ya göre daha başarılı sonuçlar sunmuştur.