Evolutionary algorithms, simulated annealing and tabu search: a comparative study

Supervised Neural Network (NN) learning is a process in which input patterns and known targets are presented to a NN while it learns to recognize (classify, map, fit, etc.) them as desired. The learning is mathematically defined as an optimisation problem, i.e., an error function representing the differences between the desired and actual output, is being minimized (Bishop, 1995; Haykin, 1999). Because the most popular supervised learning techniques are gradient based (Backpropagation BP), they suffer from the so-called Local Minima Problem (Bishop, 1995). This has motivated the employment of Global Optimisation (GO) methods for the supervised NN learning. Stochastic and heuristic GO approaches including Evolutionary Algorithms (EA) demonstrated promising performance over the last decades (Smagt, 1994; Sexton et al., 1998; Jordanov & Georgieva, 2007; etc.). EA appeared more powerful than BP and its modifications (Sexton et al., 1998; Alba & Chicone 2004), but hybrid methods that combine the advantages of one or more GO techniques and local searches were proven to be even better (Yao, 1999; Rocha et al., 2003; Alba & Chicano, 2004; Ludemir et al., 2006). Hybrid methods were promoted over local searches and simple population based techniques in Alba & Chicone (2004). The authors compared five methods: two BP implementations (gradient descent and Levenberg-Marquardt), Genetic Algorithms (GA), and two hybrid methods, combining GA with different local methods. The methods were used for NN learning applied to problems arising in medicine. Ludemir et al. (2006) optimized simultaneously NN weights and topology with a hybrid method combining Simulated Annealing (SA), Tabu Search (TS) and BP. A set of new solutions was generated on each iteration by TS rules, but the best solution was only accepted according to the probability distribution as in conventional SA. Meanwhile, the topology of the NN was also optimized and the best solution was kept. Finally, BP was used to train the best NN topology found in the previous stages. The new methodology compared favorably with SA and TS on four classification and one prediction problems. Plaginakos et al. (2001) performed several experiments to evaluate various training methods – six Differential Evolution (DE) implementations (with different mutation operators), BP, BPD (BP with deflection), SA, hybridization of BP and SA (BPSA), and GA. They reported O pe n A cc es s D at ab as e w w w .in te ch w eb .o rg

The design of hybrid metaheuristics with ideas from the simulated annealing and evolutionary algorithms fields is a fruitful research line. In this paper, we present a new hybrid algorithm based on a genetic algorithm, whose search process simulates several parallel simulated annealing processes. An empirical study comparing the new model with classic simulated annealing, hybrid metaheuristics based on evolutionary algorithms and simulated annealing, and two evolutionary algorithms, concludes that the alternative scheme for combining ideas from simulated annealing and evolutionary algorithms introduced by the proposal may improve the performance of this kind of hybrid algorithms.

Neural networks and fuzzy systems are two soft‐computing paradigms for system modelling. Adapting a neural or fuzzy system requires to solve two optimization problems: structural optimization and parametric optimization. Structural optimization is a discrete optimization problem which is very hard to solve using conventional optimization techniques. Parametric optimization can be solved using conventional optimization techniques, but the solution may be easily trapped at a bad local optimum. Evolutionary computation is a general‐purpose stochastic global optimization approach under the universally accepted neo‐Darwinian paradigm, which is a combination of the classical Darwinian evolutionary theory, the selectionism of Weismann, and the genetics of Mendel. Evolutionary algorithms are a major approach to adaptation and optimization. In this paper, we first introduce evolutionary algorithms with emphasis on genetic algorithms and evolutionary strategies. Other evolutionary algorithms such as genetic programming, evolutionary programming, particle swarm optimization, immune algorithm, and ant colony optimization are also described. Some topics pertaining to evolutionary algorithms are also discussed, and a comparison between evolutionary algorithms and simulated annealing is made. Finally, the application of EAs to the learning of neural networks as well as to the structural and parametric adaptations of fuzzy systems is also detailed.

In this paper, we study the resource management problem in Direct Sequence-Wideband Code Division Multiple Access (DS-WCDMA) systems. The control variables, transmission power, and transmission rate for resource management are considered. Three meta-heuristic techniques, Genetic Algorithms (GA), Simulated Annealing (SA), and Tabu Search (TS), are utilized to solve this optimization problem. Also, a nonlinear programming technique, Generalized Reduced Gradient (GRG) method, is adopted to compare with the three meta-heuristic techniques. Two approaches, the single objective approach and the multiobjective approach, are used in the simulation. The results obtained by GA, SA, TS, and GRG are compared in a single objective approach. In a multiobjective approach, Multiobjective Genetic Algorithm (MOGA) is employed to compare with the other two well-known multiobjective evolutionary algorithms (MOEAs), Pareto Archived Evolution Strategy (PAES) and Micro-Genetic Algorithms (MICRO-GA). Two scenarios, scenario (a): 25 users and scenario (b): 50 users, are considered for both approaches. The simulation results of a single objective approach show that GA outperforms SA, TS, and GRG in the two scenarios. Also, the simulation results of a multiobjective approach show that MOGA outperforms PAES and MICRO-GA, and obtains nondominated trade-off solutions with better convergence and diversity between trade minimization of total power and maximization of total rate in two scenarios.

Evaluation of stochastic global optimization methods for modeling vapor–liquid equilibrium data

Aiming at the disadvantage of premature convergence of basic genetic algorithm, an adaptive simulated annealing genetic tabu search algorithm is proposed. This algorithm fully combines the global convergence and adaptability of simulated annealing algorithm and the strong climbing ability and high efficiency of tabu search strategy. It has strong convergence and adaptability. The simulation results of the adaptive simulated annealing genetic tabu search algorithm are given and compared with the basic genetic algorithm and simulated annealing algorithm. The simulation results show that the algorithm has better convergence and optimization performance, and can better solve the combinatorial optimization problem.

Scheduling problems in real systems often require sequence-dependent setup times. The topic of sequence-dependent setup times has not been addressed adequately, and improved competitiveness is thus not achieved. This study proposes a hybrid approach that takes advantage of simulated annealing (SA) and tabu search (TS) to solve single-machine tardiness problems with sequence-dependent setup times. To verify the proposed approach, experiments were conducted on benchmark problem sets that included both the weighted and un-weighted tardiness problems. The results show that the performance of the hybrid approach is superior to that of the SA, genetic algorithm, TS and ant colony optimization approaches, and is comparable with the Tabu-VNS approach. And the proposed approach found new upper bound values for many benchmark problems with an acceptable computational time.

The performance of an artificial neural network (ANN) depends upon the selection of proper connection weights, network architecture, and cost function during network training. This paper presents a hybrid approach (GaTSa) to optimize the performance of the ANN in terms of architecture and weights. GaTSa is an extension of a previous method (TSa) proposed by the authors. GaTSa is based on the integration of the heuristic simulated annealing (SA), tabu search (TS), genetic algorithms (GA), and backpropagation, whereas TSa does not use GA. The main advantages of GaTSa are the following: a constructive process to add new nodes in the architecture based on GA, the ability to escape from local minima with uphill moves (SA feature), and faster convergence by the evaluation of a set of solutions (TS feature). The performance of GaTSa is investigated through an empirical evaluation of 11 public-domain data sets using different cost functions in the simultaneous optimization of the multilayer perceptron ANN architecture and weights. Experiments demonstrated that GaTSa can also be used for relevant feature selection. GaTSa presented statistically relevant results in comparison with other global and local optimization techniques.

This article presents a novel approach to the modeling and reduction of electromagnetic interference (EMI) caused by radiated emissions of integrated circuits (ICs) inside rectangular metallic enclosures of telecommunications devices. This type of analysis applies for several types of modern telecommunications equipment found in high‐speed networks as well as in mobile communications. A generic model of such a device is created. The ICs are modeled as small electric dipoles and their interaction with the enclosure walls is studied by using the dyadic Green's functions. The electric field on the enclosure walls is computed and its reduction is studied as optimization problem using evolutionary algorithms. Two algorithms are employed: Genetic algorithms (GAs) and parallel recombinative simulated annealing (PRSA). PRSA is a hybrid evolutionary strategy that inherits properties from both GAs and simulated annealing. Monte Carlo simulation is subsequently applied to the optimization results to derive the electric field on the metallic walls and also to perform a worst‐case analysis. The applications of the above approach in early PCB design process are discussed. © 2007 Wiley Periodicals, Inc. Microwave Opt Technol Lett 49: 3049–3055, 2007; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/mop.22893

A comparative study of various meta-heuristic techniques applied to the multilevel thresholding problem

Metaheuristics for optimization problems in computer communications

With the increasing complexity of modern VLSI circuits, achieving high quality built-in self-test requires monitoring an increasingly large number of internal nodes. Due to the limitations in observing large numbers of nodes, it has become increasingly necessary to compact the output from a large number of lines to a small number of lines in a process known as space compaction. In the past, space compaction has usually been performed by circuit-independent compactors, such as the popular parity function. Recently, a class of circuit-specific space compactors, known as programmable space compactors (PSCs), has been proposed. A PSC is a highly effective space compactor tailored for a specific circuit under test (CUT) subjected to a specific set of test stimuli. Circuit-specific information such as the fault-free and expected faulty behaviour of a circuit can be used to choose compactors which have better fault coverage and/or lower area costs than the parity function. The drawback of PSCs is the difficulty involved in finding optimal PSCs for a circuit. The space of possible PSC functions is extremely large and grows exponentially with the number of lines to be compacted. Although searching for optimal PSC functions is a difficult problem, it is not impossible. This thesis presents a method for searching for combinational PSCs based upon genetic algorithms (GAs), a randomized search technique. Randomized search techniques, such as simulated annealing and genetic algorithms, have become increasingly popular in recent years as methods for solving search and optimization problems beyond the capabilities of classical techniques. This thesis develops a procedure for using circuit-specific information in the application of (GAs) to the search for optimal PSCs for a given CUT and a given set of test stimuli. A measure of the effectiveness, or fitness, of PSCs is developed as a means for guiding the genetic search. The factors used to assess the effectiveness of a PSC are its

Three heuristic search methods, Genetic Algorithm, Simulated Annealing and Tabu Search are implemented to invert Rayleigh wave phase velocity for shallow S-wave velocity profiling in seismic surface wave surveying. Unlike linearized least-squares inversion, they do not require derivative calculation, or an initial model, only the forward modeling calculation. In this study, the performances of the three heuristic techniques are compared with numerical experiments. With common paramerization and search limits, the three algorithms can accurately reconstruct shallow S-wave profiles with and without a velocity reversal from the synthetic data without any specific initial models. The Genetic Algorithms and the Simulated Annealing show fast convergence. However, the Simulated Annealing can find models with the smallest misfits. It is also found that the decrease of the misfits is the most gradual in the Tabu Search.

Withdrawn.

2차원 저장소 적재는 NP-hard 문제로서 그 문제의 정확한 해를 구하는 것이 어려운 것으로 알려져 있으며, 이의 더 좋은 해를 얻기 위해 유전자(genetic) 알고리즘, 시뮬레이티드 어닐링(simulated annealing), 타부서치(tabu search)등과 같은 근사적 접근법이 제안되어 왔다. 하지만 분지한계(branch-and-bound)나 타부서치 기법들을 이용한 기존의 대표적인 근사 알고리즘들은 휴리스틱 알고리즘의 해에 기반을 둠으로 효율성이 낮고 반복수행에 의한 계산시간이 길다. 따라서 본 논문에서는 이러한 근사 알고리즘의 복잡성을 간소화하고, 알고리즘의 효율성을 높이기 위해 적재가능성을 판단하는 적합성 함수(fitness function)를 정의하고 이를 이용하여 어떤 특정 개체의 적재영역을 판단하는데 영향을 주는 적재영역의 수를 계산한다. 또한, 이들을 이용한 새로운 휴리스틱 알고리즘을 제안하였다. 끝으로 기존의 휴리스틱 또는 메타휴리스틱 기법과의 비교실험을 통해 기존의 휴리스틱 알고리즘인 FFF와 FBS에 비해 97%의 결과가 같거나 우수하였으며, 타부서치 알고리즘에 비해 86%의 결과가 같거나 우수한 것으로 나타났다. The two-dimensional bin packing problem(2D-BPP) has been known to be NP-hard, and it is difficult to solve the problem exactly. Many approximation methods, such as genetic algorithm, simulated annealing and tabu search etc, have been also proposed to gain better solutions. However, the existing approximation algorithms, such as branch-and-bound and tabu search, have shown the low efficiency and the long execution time due to a large of iterations. To solve these problems, we first define the fitness function to simplify and increase the utility of algorithm. The function decides whether an item is packed into a given area, and as an important information for a packing strategy, the number of subarea that can accommodate a given item is obtained from the variant of the fitness function. Then we present a heuristic algorithm BF for 2D bin packing, constructed by the fitness function and subarea. Finally, the effectiveness of the proposed algorithm will be expressed by the comparison experiments with the heuristic and the metaheuristic of the literatures. As comparing with existing heuristic algorithms and metaheuristic algorithms, it has been found that the packing rate of algorithm BP is the same as 97% as existing heuristic algorithms, FFF and FBS, or better than them. Also, it has been shown the same as 86% as tabu search algorithm or better.