From simulated annealing to stochastic continuation: a new trend in combinatorial optimization

Simulated annealing (SA) is a generic optimization method whose popularity stems from its simplicity and its global convergence properties; it emulates the physical process of annealing whereby a solid is heated and then cooled down to eventually reach a minimum energy configuration. Although successfully applied to many difficult problems, SA is widely reported to converge very slowly, and it is common practice to relax some of its convergence conditions as well as to allow extra freedom in its design. However, variations on the theme of annealing usually come without optimal convergence guarantees.In this paper, we review the fundamentals of SA and we focus on acceleration techniques that come with a rigorous mathematical justification. We discuss the design of the candidate-solution generation mechanism, the issue of finite-time cooling, and the technique of acceleration by concave distortion of the objective function. We also investigate a recently introduced generalization of SA—stochastic continuation—which significantly increases the design flexibility by allowing the candidate-solution generation mechanism and the objective function to vary with temperature.KeywordsSimulated AnnealingEnergy LandscapeCommunication MechanismGlobal Convergence PropertyCommunication MatrixThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Tabu search (TS) is a metaheuristic that guides a local heuristic search procedure to explore the solution space beyond local optimality. It has achieved widespread successes in solving practical optimization problems. This paper proposes the stochastic TS strategy for discrete optimization and makes an investigation of its global convergence. The strategy considered introduces the Metropolis criterion and simulated annealing process into a general framework of TS. It has been proved that the strategy converges asymptotically to global optimal solutions, and satisfies the necessary and sufficient conditions for global asymptotic convergence. Furthermore, it produces a higher convergent rate than the simulated annealing algorithm.

Simulated annealing (SA) and its deterministic analog, namely continuation, are well-known approaches to global optimization. SA is asymptotically optimal but converges very slowly, whereas deterministic continuation is much less computationally demanding but is suboptimal. In this paper, we introduce a new class of hybrid algorithms which combines the theoretical advantages of SA with the practical advantages of deterministic continuation. We call this class of algorithms stochastic continuation (SC). We show that SC inherits the convergence properties of generalized SA under fairly mild assumptions and that it can be successfully applied to piecewise constant signal reconstruction. Our numerical experiments indicate that SC substantially outperforms SA.

We address the problem of reconstructing a piecewise constant 3-D signal from a few noisy 2-D line-integral projections. More generally, the theory developed here readily applies to the recovery of an ideal n-D signal (n ges 1) from indirect measurements corrupted by noise. Stabilization of this class of ill-conditioned inverse problems is achieved with the Potts prior model, which leads to the minimization of a discontinuous, highly multimodal cost function. To carry out this challenging optimization task, we introduce a new class of annealing-type algorithms we call stochastic continuation (SC). We show that SC inherits the desirable finite-time convergence properties of generalized simulated annealing (under mild assumptions) and that it can be successfully applied to signal reconstruction. Our numerical experiments indicate that SC outperforms standard simulated annealing

Large-scale renewable energy bases increasingly employ automatic voltage control (AVC) to coordinate heterogeneous reactive-power resources. The resulting voltage regulation process inherently involves sampling, communication delay, and nonlinear device characteristics, which may induce nontraditional voltage oscillations and stability degradation that cannot be adequately captured by conventional continuous-time or small-signal analysis. This paper proposes a discrete-time nonlinear voltage stability analysis framework for renewable energy bases with multi-reactive-power-resource coupling under AVC-based coordinated control. The voltage regulation dynamics are formulated as a discrete-time nonlinear closed-loop system by incorporating sampled AVC actions, delayed voltage feedback, and nonlinear voltage–reactive-power coupling. An incremental system representation is constructed, and a strong-contraction-based stability criterion is derived using sector-bounded nonlinearity descriptions and linear matrix inequalities, providing a sufficient condition for global voltage convergence without local linearization. Extensive numerical studies are conducted on a representative renewable energy base with parallel and series coupling topologies. A total of 2916 randomized configurations are evaluated. The proposed criterion achieves consistency rates exceeding 96% for the parallel topology and 99% for the series topology when compared with time-domain simulations, while the probability of dangerous misjudgment remains below 1%. Scenario-based simulations further demonstrate that coupling topology plays a critical role in shaping voltage stability behaviors, and state-space analysis further supports the observed stability behaviors. These results indicate that nonlinear strong contraction offers an effective and practical stability notion for AVC-based voltage regulation in renewable energy bases.

Intergenerational mobility and macroeconomic history dependence

This paper represents how typical advanced engineering design can be structured using a set of parameters and objective functions corresponding to the nature of the problem. The set of parameters can be in different types, including integer, real, cyclic, combinatorial, interval, etc. Similarly, the objective function can be presented in various types including integer (discrete), float, and interval. The simulated annealing with crystallization heuristic can deal with all these combinations of parameters and objective functions when the crystallization heuristic presents a sensibility for real parameters. Herein, simulated annealing with the crystallization heuristic is enhanced by combining Bates and Gaussian distributions and by incorporating feedback strategies to emphasize exploration or refinement, or a combination of the two. The problems that are studied include solving an electrical impedance tomography problem with float parameters and a partially evaluated objective function represented by an interval requiring the solution of 32 sparse linear systems defined by the finite element method, as well as an airplane design problem with several parameters and constraints used to reduce the explored domain. The combination of the proposed feedback strategies and simulated annealing with the crystallization heuristic is compared with existing simulated annealing algorithms and their benchmark results are shown. The enhanced simulated annealing approach proposed herein showed better results for the majority of the studied cases.

Optimization is an important subject in solving scientific and engineering problems. One conventional searching technique for finding the global minimum is to use gradient descent, which finds the direction for the next iteration from the gradient of the objective function. For complicated problems, the gradient descent technique often gets stuck at a local minimum where the objective function has surrounding barriers. In addition, the complexity of most combinatorial optimization problems increases dramatically with the problem size and makes it very difficult to obtain the globally optimal solution in a reasonably computational time. Complicated optimization problems could also be solved by using branch-and-bound searching method [1], or by relaxing some constraints and solving the simplified problems [2]. Several attractive methods have been reported to help the solutions of the optimization problems to escape from local minima [3 4, 5, 6, 7, 8, 9, 10]. Simulated annealing is one of the popular approaches which is widely applicable to the combinatorial optimization problems [3, 4]. Simulated annealing is a generalization of iterative improvement methods in that it accepts, with a non-zero but gradually decreasing probability, deterioration in the cost function of an optimization problem. A solution from simulated annealing is close to the global minimum within a polynomial upper bound for the computational time and is independent of the initial condition. Mean field annealing is a deterministic approximation to simulated annealing [7]. It analytically approximates the relevant Boltzmann distribution and can execute at 50 times faster than simulated annealing for some problems. The quality of the solutions produced by mean field annealing is highly dependent on the choice of the initial temperature, the cooling speed and the final temperature. So far, simulated annealing has been successfully reported in layout generation of VLSI circuits [11, 12], test pattern generation [13], integrated-circuit delay reduction [14], image segmentation [15] and noise filtering in image processing [16].

SUMMARY The seismic waveform inversion problem is usually cast into the framework of Bayesian statistics in which prior information on the model parameters is combined with the data and physics of the forward problem to estimate the a posteriori probability density (PPD) in model space. The PPD is a function of an objective or fitness function computed from the observed and synthetic data. In general, the PPD or the fitness function is multimodal and its shape is unknown. Global optimization methods such as simulated annealing (SA) and genetic algorithms (GA) do not require that the shape of the fitness function be known. In this paper, we investigate GA to rapidly sample the most significant portion or portions of the PPD, when very little prior information is available. First, we use a simple three operator (selection, crossover and mutation) GA acting on a randomly chosen finite population of haploid binary coded models. We use plane wave transformed synthetic seismic data and a normalized cross-correlation function [E(m)] in the frequency domain as a fitness function. A moderate value of crossover probability, a low value of mutation probability, a high value of update probability and a proper population size are required to reach very close to the global maximum of the fitness function. Next, with an attempt to accelerate convergence we show that the concepts from simulated annealing can be used in stretching of the fitness function, i.e., we use exp [E(m)/T] rather than E(m) as the fitness function, where T is a control parameter analogous to temperature in simulated annealing. By a schemata analysis, we show that at low temperatures, schemata with above average fitness values are reproduced in large numbers causing a much more rapid convergence of the algorithm. A high value of temperature T assigns nearly equal selection probability to most of the schemata and thus retains diversity among the members of the population. Thus a GA with a step function type cooling schedule (very high temperature in the beginning followed by rapid cooling to a very low temperature) improves the performance dramatically: high values of the fitness function are obtained rapidly using only half as many models as would be required by a conventional GA. Similar performance could also be achieved by first using a high mutation probability and then decreasing the mutation probability to a very low value, while retaining the same low temperature throughout. We also address the problem of ‘genetic drift’ which causes the finite GAs to converge to one peak or the other when the algorithm is applied to a highly multimodal fitness function with several peaks of nearly the same height. A parallel genetic algorithm based on the concept of ‘punctuated equilibria’ is implemented to circumvent the problem. We run several GAs each with a finite subpopulation in parallel and collect many good models from each one of these runs. These are then used to grasp the most significant portion(s) of the PPD in model space. We then compute the weighted mean model and use the derived good models to estimate uncertainty in the derived model parameters.

The aim of the present work is to generalize the contraction theory for the analysis of the convergence of fractional order systems for both continuous-time and discrete-time systems. Contraction theory is a methodology for assessing the stability of trajectories of a dynamical system with respect to one another. The result of this study is a generalization of the Lyapunov matrix equation and linear eigenvalue analysis. The proposed approach gives a necessary and sufficient condition for exponential and global convergence of nonlinear fractional order systems. The examples elucidate that the theory is very straightforward and exact.

As the number of electric vehicles (EVs) continues to grow and the demand for charging infrastructure is also increasing, how to improve the charging infrastructure has become a bottleneck restricting the development of EVs. In other words, reasonably planning the location and capacity of charging stations is important for development of the EV industry and the safe and stable operation of the power system. Considering the construction and maintenance of the charging station, the distribution network loss of the charging station, and the economic loss on the user side of the EV, this paper takes the node and capacity of charging station planning as control variables and the minimum cost of system comprehensive planning as objective function, and thus proposes a location and capacity planning model for the EV charging station. Based on the problems of low efficiency and insufficient global optimization ability of the current algorithm, the simulated annealing immune particle swarm optimization algorithm (SA-IPSO) is adopted in this paper. The simulated annealing algorithm is used in the global update of the particle swarm optimization (PSO), and the immune mechanism is introduced to participate in the iterative update of the particles, so as to improve the speed and efficiency of PSO. Voronoi diagram is used to divide service area of the charging station, and a joint solution process of Voronoi diagram and SA-IPSO is proposed. By example analysis, the results show that the optimal solution corresponding to the optimisation method proposed in this paper has a low overall cost, while the average charging waiting time is only 1.8 min and the charging pile utilisation rate is 75.5%. The simulation comparison verifies that the improved algorithm improves the operational efficiency by 18.1% and basically does not fall into local convergence.

In image restoration, the goal is often to bring back a high-quality version of an image from a lower-quality copy of it. In this article, we will investigate one kind of recovery issue, namely recovering photos that have been blurred by noise in digital photographs (sometimes known as "salt and pepper" noise). When subjected to noise at varying frequencies and intensities (30,50,70,90). In this paper, we used the conjugate gradient algorithm to Restorative images and remove noise from them, we developed the conjugate gradient algorithm with three limits using the conjugate condition of Dai and Liao, where the new algorithm achieved the conditions for descent and global convergence under some assumptions. According to the results of the numerical analysis, the recently created approach is unequivocally superior to both the fletcher and reeves (FR) method and the fletcher and reeves three-term (TTFR) metod. Use the structural similarity index measure (SSIM), which is used to measure image quality and the higher its value, the better the result. The original image was compared with all the noisy images and each according to the percentage of noise as well as the images processed with the four methods.

In this paper, we consider continuous-time semidecentralized dynamics for the equilibrium computation in a class of aggregative games. Specifically, we propose a scheme where decentralized projected-gradient dynamics are driven by an integral control law. To prove global exponential convergence of the proposed dynamics to an aggregative equilibrium, we adopt a quadratic Lyapunov function argument. We derive a sufficient condition for global convergence that we position within the recent literature on aggregative games, and in particular we show that it improves on established results.

This contribution discusses the global convergence behavior of iterative algorithms solving nonlinear approximation problems via a sequence of linear approximation problems. Essentially there are two sufficient conditions for global convergence: 1) The algorithm should have only critical points as accumulation points. 2) The parametrization should imply that the parameters of refined approximations have accumulation points. For some important nonlinear families of approximating functions it is proved that the above conditions hold, provided that the parametrization is restricted to suitable subfamilies.

This correspondence first shows that the global convergence analysis of the method proposed in the above paper is incomplete. Then we provide a counter example to show that the sufficient condition for global convergence is incorrect