Simulation-based optimization using simulated annealing with ranking and selection

This paper develops a variant of simulated annealing (SA) algorithm for solving discrete stochastic optimization problems where the objective function is stochastic and can be evaluated only through Monte Carlo simulations. In the proposed variant of SA, the Metropolis criterion depends on whether the objective function values indicate statistically significant difference at each iteration. The differences between objective function values are considered to be statistically significant based on confidence intervals associated with these values. Unlike the original SA, our method uses a constant temperature. We show that the configuration that has been visited most often in the first m iterations converges almost surely to a global optimizer.

Population-based approaches have given us new search strategies and ideas in order to solve optimization problems. Usually, these methods are based on the performance carried out by a finite number of agents, which by the interaction between them they evolve and work all over the search space. Also, it is well-known that the correct employment of parameter values in this kind of method can positively impact their performance and behavior. In this context, the present work focuses on the design of a hybrid architecture which smartly balances the population size on run-time. In order to smartly balance and control the population size, a modular approach, named Linear Modular Population Balancer (LMPB), is proposed. The main ideas behind the designed architecture include the solving strategy behind a population-based metaheuristic, the influence of learning components based on multiple statistical modeling methods which transform the dynamic data generated into knowledge, and the possibilities to tackle both discrete and continuous optimization problems. In this regard, three modules are proposed for LMPB, which concern tasks such as the management of the population-based algorithm, parameter setting, probabilities, learning methods, and selection mechanism for the population size to employ. In order to test the viability and effectiveness of our proposed approach, we solve a set of well-known benchmark functions and the multidimensional knapsack problem (MKP). Additionally, we illustrate promising solving results, compare them against state-of-the-art methods which have proved to be good options for solving optimization problems, and give solid arguments for future work in the necessity to keep evolving this type of proposed architecture.

A Survey of Lagrangean Techniques for Discrete Optimization

A modification of the simulated annealing (SA) algorithm for solving discrete stochastic optimization problems where the objective function is stochastic and can be evaluated only through Monte Carlo simulation is proposed. In this modification, the Metropolis criterion depends on whether the objective function values indicate a statistically significant difference at each iteration. The differences between objective function values are considered to be statistically significant based on confidence intervals associated with these values. Unlike the original SA, the proposed method uses a constant temperature. It is shown that the configuration that has been visited most often in the first k iterations converges almost surely to a global optimizer. Computational results and comparisons with other SA algorithms are presented to demonstrate the performance of the proposed SA algorithm.

An approach for the concurrent homogenization-based microstructure type and topology optimization problem

Solution quality of random search methods for discrete stochastic optimization

CHAPTER 2.8 - Convexity and Discrete Optimization

We consider in this paper the adaptation of heuristics used for programming nondeterministic games to the problems of discrete optimization. In particular, we use some “game” heuristic methods of decision-making in various discrete optimization problems. The object of each of these problems is programming anytime algorithms. Among the problems described in this paper, there are the classical traveling salesman problem and some connected problems of minimization for nondeterministic finite automata. The first of the considered methods is the geometrical approach to some discrete optimization problems. For this approach, we define some special characteristics relating to some initial particular case of considered discrete optimization problem. For instance, one of such statistical characteristics for the traveling salesman problem is a significant development of the so-called “distance functions” up to the geometric variant such problem. And using this distance, we choose the corresponding specific algorithms for solving the problem. Besides, other considered methods for solving these problems are constructed on the basis of special combination of some heuristics, which belong to some different areas of the theory of artificial intelligence. More precisely, we shall use some modifications of unfinished branchand-bound method; for the selecting immediate step using some heuristics, we apply dynamic risk functions; simultaneously for the selection of coefficients of the averaging-out, we also use genetic algorithms; and the reductive self-learning by the same genetic methods is also used for the start of unfinished branch-and-bound method again. This combination of heuristics represents a special approach to construction of anytime-algorithms for the discrete optimization problems. This approach can be considered as an alternative to application of methods of linear programming, and to methods of multi-agent optimization, and also to neural networks.

In this paper we present two recent metaheuristics, particle swarm optimization and differential evolution algorithms, to solve the single machine total weighted tardiness problem, which is a typical discrete combinatorial optimization problem. Most of the literature on both algorithms is concerned with continuous optimization problems, while a few deal with discrete combinatorial optimization problems. A heuristic rule, the smallest position value (SPV) rule, borrowed from the random key representation in genetic algorithms, is developed to enable the continuous particle swarm optimization and differential evolution algorithms to be applied to all permutation types of discrete combinatorial optimization problems. The performance of these two recent population based algorithms is evaluated on widely used benchmarks from the OR library. The computational results show that both algorithms show promise in solving permutation problems. In addition, a simple but very efficient local search method based on the variable neighbourhood search (VNS) is embedded in both algorithms to improve the solution quality and the computational efficiency. Ultimately, all the best known or optimal solutions of instances are found by the VNS version of both algorithms.

We discuss the choice of the estimation of the optimal solution when random search methods are applied to solve discrete stochastic optimization problems. At the present time, such optimization methods usually estimate the optimal solution using either the feasible solution the method is currently exploring or the feasible solution visited most often so far by the method. We propose using all the observed objective function values generated as the random search method moves around the feasible region seeking an optimal solution to obtain increasingly more precise estimates of the objective function values at the different points in the feasible region. At any given time, the feasible solution that has the best estimated objective function value (largest one for maximization problems; the smallest one for minimization problems) is used as the estimate of the optimal solution. We discuss the advantages of using this approach for estimating the optimal solution and present numerical results showing that modifying an existing random search method to use tnhis approach for estimating the optimal soluation appears to yield improved performance. We also present sereval rate of convergence results for random search methods using our approach for estimating the optimal solution. One these random search methods is a new variant of the stochastic comparison method; in addition to specifying the rate of convergence of this method, we prove that it is guaranteed to converge almost surely to the set of global optimal solutions and present a result that demonstrates that this method is likely to perform well in practice.

In this paper, the matrix game based on triangular intuitionistic fuzzy payoff is put forward. Then, we get a conclusion that the equilibrium solution of this game model is equivalent to the solution of a pair of the primal–dual single objective intuitionistic fuzzy linear optimization problems ( I F L O P 1 ) and ( I F L O D 1 ) . Furthermore, by applying the accuracy function, which is linear, we transform the primal–dual single objective intuitionistic fuzzy linear optimization problems ( I F L O P 1 ) and ( I F L O D 1 ) into the primal–dual discrete linear optimization problems ( G L O P 1 ) and ( G L O D 1 ) . The above primal–dual pair ( G L O P 1 ) – ( G L O D 1 ) is symmetric in the sense the dual of ( G L O D 1 ) is ( G L O P 1 ) . Thus the primal–dual discrete linear optimization problems ( G L O P 1 ) and ( G L O D 1 ) are called the symmetric primal–dual discrete linear optimization problems. Finally, the technique is illustrated by an example.

We present a modification of the simulated annealing algorithm designed for solving discrete stochastic optimization problems. Like the original simulated annealing algorithm, our method has the hi...

Simultaneous generalized hill climbing (SGHC) algorithms provide a framework for using heuristics to simultaneously address sets of intractable discrete optimization problems where information is shared between the problems during the algorithm execution. Many well-known heuristics can be embedded within the SGHC algorithm framework. This paper shows that the solutions generated by an SGHC algorithm are a stochastic process that satisfies the Markov property. This allows problem probability mass functions to be formulated for particular sets of problems based on the long-term behavior of the algorithm. Such results can be used to determine the proportion of iterations that an SGHC algorithm will spend optimizing over each discrete optimization problem. Sufficient conditions that guarantee that the algorithm spends an equal number of iterations in each discrete optimization problem are provided. SGHC algorithms can also be formulated such that the overall performance of the algorithm is independent of the initial discrete optimization problem chosen. Sufficient conditions are obtained guaranteeing that an SGHC algorithm will visit the globally optimal solution for each discrete optimization problem. Lastly, rates of convergence for SGHC algorithms are reported that show that given a rate of convergence for the embedded GHC algorithm, the SGHC algorithm can be designed to preserve this rate.

Fuzzy systems (FSs) have been extensively applied to automatic control, pattern recognition, and decision analysis. However, a common bottleneck is encountered in the derivation of fuzzy rules, which is often difficult, time consuming, and relies on expert knowledge. To automate the design of FSs, many metaheuristic learning algorithms have been proposed. One major optimization category uses Swarm Intelligence (SI) model (Kennedy et al., 2001). The SI technique studies collective behavior in decentralized systems. Its development was based on mimicking the social behavior of animals or insects in an effort to find the optima in the problem space. SI models are initialized with a population of random solutions. One well-known SI model is particle swarm optimization (PSO) (Kennedy & Eberhart, 1995). Many modified PSO models have been proposed and successfully applied to different optimization problems (Clerc & Kennedy, 2002; Bergh & Engelbrecht, 2004; Ratnaweera et al., 2004; Juang, 2004; Kennedy & Mendes, 2006; Parrott & Li, 2006; Chen & Li, 2007). FS design using PSO has also been proposed in several studies (Juang, 2004; Chatterjee et al., 2005; Juang et al., 2007; Araujo & Coelho, 2008; Sharma et al., 2009). Another well-known SI is ant colony optimization (ACO) (Dorigo & Stutzle, 2004). The ACO technique is inspired by real ant colony observations. It is a multi-agent approach that was originally proposed to solve difficult discrete combinatorial optimization problems, such as the traveling salesman problem (TSP) (Dorigo et al., 1996; Dorigo & Gambardella, 1997). In the original ACO meta-heuristic, artificial ant colonies cooperate to find good solutions for difficult discrete optimization problems. Different ACO models have been applied to FS design problems (Cassillas et al., 2000; Cassillas et al., 2005; Mucientes & Casillas; 2007; Juang & Lo, 2007; Juang et al., 2008; Juang & Lu; 2009). In (Cassillas et al.,2000; Mucientes & Casillas; 2007; Juang et al., 2008; Juang & Lu; 2009), the FS input space was partitioned in grid type with antecedent part parameters of an FS manually assigned in advance. In (Juang & Lo, 2007), the FS input space was flexibly partitioned using a fuzzy clustering-like algorithm in order to reduce the total number of rules. For all of these studies, the consequent part parameters were optimized in discrete space using ACO. Since only the consequent part parameters are optimized, and the optimization space is restricted to be discrete, the designed FSs are unsuitable for problems where high accuracy is a major concern. Source: Fuzzy Systems, Book edited by: Ahmad Taher Azar, ISBN 978-953-7619-92-3, pp. 216, February 2010, INTECH, Croatia, downloaded from SCIYO.COM

It is known that global optimality conditions for nonconvex optimization problems are, in general, difficult to obtain. Moreover, zero duality gap property does not, in general, hold for a nonconvex optimization problem and there are examples where a nonzero duality gap exists for a discrete linear program when the conventional (linear) Lagrangian dual formulation is applied. In this paper, a nonlinear Lagrangian function is introduced and applied to establish zero duality gaps between discrete optimization problems and their nonlinear Lagrangian dual problems. An example of a discrete linear program is given to illustrate the results. An application of the results is also presented to a discrete network optimization problem.KeywordsDiscrete optimizationnonlinear Lagrangian functionnetwork optimizationzero duality gap