Population statistics for particle swarm optimization: Resampling methods in noisy optimization problems

We propose a new particle swarm optimization algorithm for problems where objective functions are subject to zero-mean, independent, and identically distributed stochastic noise. While particle swarm optimization has been successfully applied to solve many complex deterministic nonlinear optimization problems, straightforward applications of particle swarm optimization to noisy optimization problems are subject to failure because the noise in objective function values can lead the algorithm to incorrectly identify positions as the global/personal best positions. Instead of having the entire swarm follow a global best position based on the sample average of objective function values, the proposed new algorithm works with a set of statistically global best positions that include one or more positions with objective function values that are statistically equivalent, which is achieved using a combination of statistical subset selection and clustering analysis. The new PSO algorithm can be seamlessly integrated with adaptive resampling procedures to enhance the capability of PSO to cope with noisy objective functions. Numerical experiments demonstrate that the new algorithm is able to consistently find better solutions than the canonical particle swarm optimization algorithm in the presence of stochastic noise in objective function values with different resampling procedures.

Particle Swarm Optimization (PSO) is a population-based algorithm designed to find good solutions to optimization problems. Its characteristics have encouraged its adoption to tackle a variety of problems in different fields. However, when such problems are subject to noise, the performance of PSO suffers an immediate deterioration which demands the incorporation of noise handling mechanisms. One such mechanism comprises resampling methods, which re-evaluate the solutions multiple times in order to estimate their true objective values. The state-of-the-art integration with which the best results have been obtained utilizes the resampling method named Optimal Computing Budget Allocation (OCBA). This resampling method starts by estimating the objective values of all the solutions via Equal Resampling (ER), and then sequentially allocating further re-evaluations to the estimated best solutions. However, after having a first estimate via ER, we question the importance of the additional efforts to correctly select the true best solutions when a good-enough and accurate one can be selected. In this paper, we propose a new PSO algorithm based on ER in which the additional evaluations are allocated at once to the estimated best solutions, thus skipping the complexity of using OCBA. Experiments on 20 large-scale benchmark functions subject to different levels of noise show that the proposed algorithm produces similar results to PSO with OCBA in most cases.

A dynamic adaptive particle swarm optimization and genetic algorithm is presented to solve constrained engineering optimization problems. A dynamic adaptive inertia factor is introduced in the basic particle swarm optimization algorithm to balance the convergence rate and global optima search ability by adaptively adjusting searching velocity during search process. Genetic algorithm–related operators including a selection operator with time-varying selection probability, crossover operator, and n-point random mutation operator are incorporated in the particle swarm optimization algorithm to further exploit optimal solutions generated by the particle swarm optimization algorithm. These operators are used to diversify the swarm and prevent premature convergence. Tests on nine constrained mechanical engineering design optimization problems with different kinds of objective functions, constraints, and design variables in nature demonstrate the superiority of the dynamic adaptive particle swarm optimization and genetic algorithm against several other meta-heuristic algorithms in terms of solution quality, robustness, and convergence rate in most cases.

Particle swarm optimization (PSO) algorithm has shown promising performances on various benchmark functions and engineering optimization problems. However, it is still difficult to achieve a satisfying trade-off between exploration and exploitation for all the optimization problems and different evolving stages. Furthermore, control parameters of some related mechanisms need pre-experience by the requirement of trial-and-error scheme. This paper presents a novel PSO algorithm, which adaptively adopts various search strategies, called Self-adapting Hybrid Strategy PSO (SaHSPS). Unlike some other peer PSO variants, this method dynamically changes the probabilities of different strategies according to their previous successful searching memories, without any additional control parameters. The probabilities of different strategies would be re-initialized according to a proposed dynamic probabilistic model to diverse the population. Besides, particles are updated by probabilistically selected strategies after niching PSO with Ring topology. Moreover, a dynamic updating mechanism by niching PSO is proposed to guarantee the parallel searching capability during the whole evolution process. Thus, this proposed algorithm might be problem-independent and search-stage-independent, yielding more satisfying solutions on various optimization problems. A comprehensive experimental study is conducted on 28 benchmark functions of CEC 2013 special session on real-parameter optimization, including shifted, rotated, multi-modal, high conditioned, expanded and composition problems, compared with several state-of-the-art variants of PSO and differential evolution (DE) algorithms. Comparison results show that SaHSPS obtains outstanding performances on the majority of the test problems. Moreover, a practical engineering problem, real power loss minimization of IEEE 30-bus power system, is used to further evaluate SaHSPS. The numerical results, compared with other stochastic search algorithms, show that SaHSPS could find high-quality solutions with higher probability.

Fuzzy systems (FSs) have been extensively applied to automatic control, pattern recognition, and decision analysis. However, a common bottleneck is encountered in the derivation of fuzzy rules, which is often difficult, time consuming, and relies on expert knowledge. To automate the design of FSs, many metaheuristic learning algorithms have been proposed. One major optimization category uses Swarm Intelligence (SI) model (Kennedy et al., 2001). The SI technique studies collective behavior in decentralized systems. Its development was based on mimicking the social behavior of animals or insects in an effort to find the optima in the problem space. SI models are initialized with a population of random solutions. One well-known SI model is particle swarm optimization (PSO) (Kennedy & Eberhart, 1995). Many modified PSO models have been proposed and successfully applied to different optimization problems (Clerc & Kennedy, 2002; Bergh & Engelbrecht, 2004; Ratnaweera et al., 2004; Juang, 2004; Kennedy & Mendes, 2006; Parrott & Li, 2006; Chen & Li, 2007). FS design using PSO has also been proposed in several studies (Juang, 2004; Chatterjee et al., 2005; Juang et al., 2007; Araujo & Coelho, 2008; Sharma et al., 2009). Another well-known SI is ant colony optimization (ACO) (Dorigo & Stutzle, 2004). The ACO technique is inspired by real ant colony observations. It is a multi-agent approach that was originally proposed to solve difficult discrete combinatorial optimization problems, such as the traveling salesman problem (TSP) (Dorigo et al., 1996; Dorigo & Gambardella, 1997). In the original ACO meta-heuristic, artificial ant colonies cooperate to find good solutions for difficult discrete optimization problems. Different ACO models have been applied to FS design problems (Cassillas et al., 2000; Cassillas et al., 2005; Mucientes & Casillas; 2007; Juang & Lo, 2007; Juang et al., 2008; Juang & Lu; 2009). In (Cassillas et al.,2000; Mucientes & Casillas; 2007; Juang et al., 2008; Juang & Lu; 2009), the FS input space was partitioned in grid type with antecedent part parameters of an FS manually assigned in advance. In (Juang & Lo, 2007), the FS input space was flexibly partitioned using a fuzzy clustering-like algorithm in order to reduce the total number of rules. For all of these studies, the consequent part parameters were optimized in discrete space using ACO. Since only the consequent part parameters are optimized, and the optimization space is restricted to be discrete, the designed FSs are unsuitable for problems where high accuracy is a major concern. Source: Fuzzy Systems, Book edited by: Ahmad Taher Azar, ISBN 978-953-7619-92-3, pp. 216, February 2010, INTECH, Croatia, downloaded from SCIYO.COM

In this paper, a new method to deal with equality or inequality constraints in constrained optimization and a new neighborhood structure for Particle Swarm Optimization (PSO) called Grouped Directed Structure (GDS) are proposed. We use the new method and GDS together with the PSO algorithm to solve the well-known 24 benchmark constrained optimization problems in the literature. The PSO algorithm is well known for its fast convergence to the possible optimal position. However, in constrained optimization, the performance of PSO is not as good as it is in unconstrained optimization, partly because PSO is not good at finding the feasible region. Due to the motivation by this weakness of PSO, we develop a method called Numerical Gradient (NG) to find the feasible region. By means of the information that NG can provide, we utilize the PSO algorithm with GDS to find the optimal position of the problem. We call this new PSO variant Numerical Gradient Particle Swarm Optimization (NGPSO). A detailed description of the mechanism for NGPSO is provided and some numerical results are presented compared with the results from the existing PSO variants dealing with constraint optimization problems.

Portfolio is the set of assets either real or financial owned by investor. One asset used by investor is stock because the stock has various price along the period time. Nowadays, stock investments have been done by investors. Stock price can be either profit or loss so that it is required portfolio selection. From the various price of stock, we can compute the return, expected return, and the risk of stock. In this research, we will make the optimal portfolio which can result maximum return and minimum risk of stock with the available investation. Goal programming is the mathematical programming techniques to solve the objectives subject to some constraints where the there are priorities for each objective. An application of goal programming is portfolio selection. Portfolio selection is determining some stocks optimizing maximum return and minimum risk of stock with the available investation. The novelty of this research is Goal Programming as portfolio selection method will be optimized by Particle Swarm Optimization (PSO) so that it is called Particle Swarm Optimization-Goal Programming (PSOGP). In PSO, there is initialization of particle i.e. proportion of investation on each stock as decision variable. Particle must be constructed so that it can satisfy the constraint investation of all stocks. In optimization process, the new particle is also modified so that satisfying the constraints. In multiobjective optimization, sort the fitness on the value of the first priority objective. If first fitness have the same value of the objective, then sort them on the second priority objective, and so forth. Based on simulation, PSOGP can be applied on the portfolio selection and can optimize priority of some objectives.

Each university has different regulations in making lecture schedule according to the conditions of respective institution. In this study, regulation on lecture scheduling used the maximum limit of credits of lecturer in a day, in which if exceeds the maximum limit, then the schedule of course will be moved on another day. This study attempts to optimize the schedule preparation in each semester by considering the maximum limit of credits of a lecturer in a day. Lecture scheduling is a combination of space, time, and resources. It is categorized into combinatorial optimization group. There are two methods for solving the combinatorial optimization problems, i.e. exact and approximation method. The approximation method consists of two types, heuristics and metaheuristics. The algorithm metaheuristics categories include genetic algorithm (GA), particle swarm optimization (PSO), ant colony optimization (ACO), bee colony optimization (BCO), simulated annealing, and so on. This paper employed metaheuristics approach. Research on the preparation of lecture scheduling using ACO algorithm has been conducted and proven to be able to prepare the lecture scheduling. The ACO algorithm has numbers of parameters that in solving issues, one must manually set a number of parameters by using the Design of Experiments (DoE) tool, which takes time to solve the optimization problem. Hence, to solve the issue, an automated parameter optimization is required. PSO algorithm has fewer parameters compared to the ACO. Therefore, the purpose of this paper is a hybrid between PSO and ACO in preparing lecture scheduling.

A new hybrid optimization algorithm, a hybridization of cuckoo search and particle swarm optimization (CSPSO), is proposed in this paper for the optimization of continuous functions and engineering design problems. This algorithm can be regarded as some modifications of the recently developed cuckoo search (CS). These modifications involve the construction of initial population, the dynamic adjustment of the parameter of the cuckoo search, and the incorporation of the particle swarm optimization (PSO). To cover search space with balance dispersion and neat comparability, the initial positions of cuckoo nests are constructed by using the principle of orthogonal Lation squares. To reduce the influence of fixed step size of the CS, the step size is dynamically adjusted according to the evolutionary generations. To increase the diversity of the solutions, PSO is incorporated into CS using a hybrid strategy. The proposed algorithm is tested on 20 standard benchmarking functions and 2 engineering optimization problems. The performance of the CSPSO is compared with that of several meta-heuristic algorithms based on the best solution, worst solution, average solution, standard deviation, and convergence rate. Results show that in most cases, the proposed hybrid optimization algorithm performs better than, or as well as CS, PSO, and some other exiting meta-heuristic algorithms. That means that the proposed hybrid optimization algorithm is competitive to other optimization algorithms.

Particle swarm optimization (PSO) is a promising evolutionary approach related to a particle moves over the search space with velocity, which is adjusted according to the flying experiences of the particle and its neighbors, and flies towards the better and better search area over the course of search process. Although the PSO is effective in solving the global optimization problems, there are some crucial user-input parameters, such as cognitive and social learning rates, affect the performance of algorithm since the search process of a PSO algorithm is nonlinear and complex. Consequently, a PSO with well-selected parameter settings may result in good performance. This work develops an evolving PSO based on the Clerc’s PSO to evaluate the fitness of objective function and a genetic algorithm (GA) to evolve the optimal design parameters to provide the usage of PSO. The crucial design parameters studied herein include the cognitive and social learning rates as well as constriction factor for the Clerc’s PSO. Several benchmarking cases are experimented to generalize a set of optimal parameters via the evolving PSO. Furthermore, the better parameters are applied to the engineering optimization of a pressure vessel design.

Particle swarm optimisation (PSO) was developed by Kennedy and Eberhart in 1995 (Kennedy & Eberhart, 1995) inspired by the collective behaviour of natural birds or fish. PSO is a stochastic optimisation technique that uses a behaviour of population composed by many search points called particle. In spite of easy implementation in computer algorithms, it is well known as a powerful numerical optimizer. In the typical PSO algorithms, a set of particles searches the optimal solution in the problem space efficiently, by sharing the common attractor called global best. There are many modified versions of PSO by improving convergence property to a certain problem. While, a standard PSO is defined by Bratton and Kennedy (Bratton & Kennedy, 2007) to give a real standard for PSO studies. PSO seems as one of the evolutionary computations (ECs), and it has been shown that PSO is comparable to a genetic algorithm (Angeline, 1998). Thus, a lot of studies have demonstrated the effectiveness of PSO family in optimizing various continuous and discrete optimization problems. And a plenty of applications of PSO, such as the neural network training, PID controller tuning, electric system optimisation have been studied and achieved well results (Kennedy, 1997). However, PSO is often failed in searching the global optimal solution in the case of the objective function has a large number of dimensions. The reason of this phenomenon is not only existence of the local optimal solutions, the velocities of the particles sometimes lapsed into the degeneracy, so that the successive range is restricted in the sub-plain of the whole search hyper-plain. The sub-plane that is defined by finite number of particle velocities is a partial space in the whole search space. The issue of local optima in PSO has been studied and proposed several modifications on the basic particle driven equation (Parsopoulos et al., 2001; Hendtlass, 2005; Liang et al., 2006). There used a kind of adaptation technique or randomized method (e.g. mutation in evolutionary computations) to keep particles velocities or to accelerate them. Although such improvements work well and have ability to avoid fall in the local optima, the problem of early convergence by the degeneracy of some dimensions is still remaining, even if there are no local optima. Hence the PSO algorithm does not always work well for the high-dimensional function.

Experimental study of a new hybrid PSO with mutation for economic dispatch with non-smooth cost function

The particle swarm optimization (PSO) algorithm presents a new way for finding optimal solutions of complex optimization problems. In this paper a modified particle swarm optimization algorithm is presented. We modify the PSO algorithm in some aspects. Firstly, a contractive factor is introduced to the position update equation, and the particles are limited in search region. A new strategy for updating velocity is then adopted, in which the velocity is weakened linearly. Thirdly, using an idea of intersecting two modified PSO algorithms. Finally, adding an item of integral control in the modified algorithm can improve its global search ability. Based on these strategies, we proposed a new PSO algorithm named crossed PSO algorithm. Simulation results show that the crossed PSO is superior to the original PSO algorithm and can get overall promising performance over a wide range of problems.

Individualism of particles in particle swarm optimization

The performance of Particle Swarm Optimization (PSO) brings attention to the field of algorithms when deals with different optimization problems. Due to her simple implementation, small consumption, and very effective in finding a solution in many problems, (PSO) becomes well known to the field of algorithms. In addition, the late proposed algorithms mostly are compared to the well-known algorithm such as PSO. Thus, the Global African Buffalo Optimization (GABO) was proposed lately and yet not been compared to the old well-known algorithms in terms of accuracy and time consumption. However, in this paper, a comparison between Particle Swarm Optimization (PSO) and Global African Buffalo Optimization (GABO) algorithms was performed. Five different nonlinear equations with their upper and lower boundaries values were selected as the test optimization functions problem in addition to PSO was applied to real case study. The experimental results illustrated the differences in the performances of both algorithms toward the optimum solution. At the end of the experiments, the PSO algorithm quickly convergence towards the optimum solution using a few particles and iterations rather than GABO. However, the experimental result showed that PSO achieved good results in all the test cases within a short time. In many cases, PSO and GABO are promising optimization methods.