Paralleled Hardware Annealing for Optimal Solutions

Hopfield neural networks (HNN) and simulated annealing (SA) are two recent approaches to solving many combinatorial optimization problems. But, the disadvantages of HNN are that it can often converge at local minimum and the quality of solutions depends on the initial state because of sensitivity of parameterization. On the other hand, SA is designed to cope with the problem of convergence to a local minimum. SA, however, requires unacceptably large computing time. So, a new algorithm called mean field annealing (MFA) which can be interpreted as a generalization of HNN, is applied to optimization problems efficiently. In this paper, we propose a new approach based on MFA to solving of the minimization problem. Also we show that our approach can be applied to multi-layer as well as two-layer routing, and optimal solutions can be obtained by properly adjusting the parameter in the given energy function.

This paper presents an optimal design method by means of mean field annealing (MFA). The mean field theory (MFT) is introduced, and a certain MFA application extension based on Peierls inequality is explained in detail, the critical temperature that is important for classes of problem while applying MFA is discussed. With MFA, the optimal size design of electromagnetic device is carried out. The results are compared with two existing algorithms, the modified simulated annealing (MSA) and one-variable stochastic simulated annealing based MFA (S-MFA), in both operating cost and final quantities. To modify the final results to reach the global optimum, Hooke-Jeeves pattern search method is adopted. Favorable results show the effectiveness of the proposed scheme.

Multi-constrained quality-of-service routing (QoSR) is to find a feasible path that satisfies multiple constraints simultaneously, as an NPC problem, which is also a big challenge for wireless mesh networks. In this paper, we propose a novel Routing Scheme based on Mean Field Annealing (MFA_RS) to solve this problem. MFA_RS first uses a function of the wireless link’s delay and capacity as the cost function, both of which are QoS parameters, and then seeks to find a feasible path by mean field annealing. Because MFA_RS uses a set of deterministic equations to replace the stochastic process in Simulated Annealing (SA), and uses saddle point approximation in the calculation of the stationary probability distribution at equilibrium, thus the convergence time is much less than the algorithm based on SA. Simulation results demonstrate that MFA_RS is an effective algorithm and is very fit for wireless mesh networks where the topology may be changing constantly.

We suggest solution method for optimization problems using correlated noises. The correlated noises are introduced to neural networks to discuss mechanism of synfire chain. Kawamura and Okada have introduced correlated noises to associative memory models and have analyzed those dynamics. In the associative memory models, memory patterns are memorized as attractors in the minimum of the system. They found the correlated noise can make the state transit between the attractors. However, the mechanism of the state transition has not been known enough yet. One the other hand, for combinational optimization problems, the energy function of a problem can be defined. Therefore, finding a optimum solution is finding a minima of the energy function. The steepest descent method searches one of the solutions by going down along the gradient direction. By this method, however, the state is usually trapped in a local minimum of the system. In order to escape from the local minimum, the simulated annealing, i.e. Metropolis method, or chaotic disturbance is introduced. These methods can be represented by adding thermal noises or chaotic inputs to the dynamic equation. In this paper, we show that correlated noises introduced to neural networks can be applied to solve the optimization problems. We solve the TSP that is a typical combinational optimization problem of NP-hard, and evaluated solutions obtained by using the steepest descent method, the simulated annealing and the proposed method with the correlated noises. As results, in the case of ten cities, the proposed method with correlated noises can obtain more optimum solutions than the steepest descent method and the simulated annealing. In the cases of large numbers of cites, where it is hard to find one of the optimum solutions, our method can obtain solutions at least as same level as the simulated annealing.

Optimization problems are approached using mean field annealing (MFA), which is a deterministic approximation, using mean field theory and based on Peierls's inequality, to simulated annealing. The MFA mathematics are applied to three different objective function examples. In each case, MFA produces a minimization algorithm that is a type of graduated nonconvexity. When applied to the ;weak-membrane' objective, MFA results in an algorithm qualitatively identical to the published GNC algorithm. One of the examples, MFA applied to a piecewise-constant objective function, is then compared experimentally with the corresponding GNC weak-membrane algorithm. The mathematics of MFA are shown to provide a powerful and general tool for deriving optimization algorithms.

This paper presents an unsupervised segmentation approach applying the mean field annealing (MFA) heuristic with the modified cost function. The idea is to cast a clustering problem as a minimization problem where the criteria for the optimum segmentation is chosen as the minimization of the Euclidean distance between samples to cluster centers. To resolve the optimal problem using a Hopfield or simulated annealing neural network, the penalty terms are combined into a weighted sum using several coefficients determined by user. Using the MFA network to medical image segmentation, the need for finding weighting factors in the energy function can be eliminated and the rate of convergence is much faster than that of simulated annealing. The experimental results show that good and valid solutions can be obtained using the MFA neural network.

Combinatorial optimization problems become computationally intractable as these NP-hard problems scale. We previously proposed extraction-type majority voting logic (E-MVL), a quantum-inspired algorithm using digital logic circuits. E-MVL mimics the thermal spin dynamics of simulated annealing (SA) through controlled sparsification of spin interactions for efficient ground-state search. This study investigates the performance potential of E-MVL through systematic optimization and comprehensive benchmarking against SA. The target problem is the Sherrington-Kirkpatrick (SK) model with bimodal and Gaussian coupling distributions. Through equilibrium state analysis, we demonstrate that the sparsity control mechanism provides a consistent search of the solution space regardless of the problem's coupling distribution (bimodal, Gaussian) or size. E-MVL not only achieves the best performance among all tested algorithms-solving exact solutions up to 1600 spins where the best SA baseline is limited to 400 spins-but also provides insights that significantly improve SA's own temperature scheduling. These results establish E-MVL's dual contribution as both an efficient optimizer and a practical methodology for enhancing SA performance. Moreover, FPGA implementation achieved an approximately 6-fold faster solution speed than SA.

We describe a global optimization method called mean field annealing (MFA) and its application to two basic problems in seismic data processing: Seismic deconvolution and surface related multiple attenuation. MFA replaces the stochastic nature of the simulated annealing method with a set of deterministic update rules that act on the average value of the variables rather than on the variables themselves, based on the mean field approximation. As the temperature is lowered, the MFA rules update the variables in terms of their values at a previous temperature. By minimizing averages, it is possible to converge to an equilibrium state considerably faster than a standard simulated annealing method. The update rules are dependent on the form of the cost function and are obtained easily when the cost function resembles the energy function of a Hopfield network. The mapping of a problem onto a Hopfield network is not a precondition for using MFA, but it makes analytic calculations simpler. The seismic deconvolution problem can be mapped onto a Hopfield network by parameterizing the source and the reflectivity in terms of binary neurons. In this context, the solution of the problem is obtained when the neurons of the network reach their stable states. By minimizing the cost function of the network with MFA and using an appropriate cooling schedule, it is possible to escape local minima. A similar idea can also be applied to design an operator that attenuates surface related multiple reflections from plane‐wave transformed seismograms assuming a 1-D earth. The cost function for the multiple elimination problem is based on the criterion of minimum energy of the multiple suppressed data.

The impact of neighbourhood size on the process of simulated annealing: Computational experiments on the flowshop scheduling problem

Supervised Neural Network (NN) learning is a process in which input patterns and known targets are presented to a NN while it learns to recognize (classify, map, fit, etc.) them as desired. The learning is mathematically defined as an optimisation problem, i.e., an error function representing the differences between the desired and actual output, is being minimized (Bishop, 1995; Haykin, 1999). Because the most popular supervised learning techniques are gradient based (Backpropagation BP), they suffer from the so-called Local Minima Problem (Bishop, 1995). This has motivated the employment of Global Optimisation (GO) methods for the supervised NN learning. Stochastic and heuristic GO approaches including Evolutionary Algorithms (EA) demonstrated promising performance over the last decades (Smagt, 1994; Sexton et al., 1998; Jordanov & Georgieva, 2007; etc.). EA appeared more powerful than BP and its modifications (Sexton et al., 1998; Alba & Chicone 2004), but hybrid methods that combine the advantages of one or more GO techniques and local searches were proven to be even better (Yao, 1999; Rocha et al., 2003; Alba & Chicano, 2004; Ludemir et al., 2006). Hybrid methods were promoted over local searches and simple population based techniques in Alba & Chicone (2004). The authors compared five methods: two BP implementations (gradient descent and Levenberg-Marquardt), Genetic Algorithms (GA), and two hybrid methods, combining GA with different local methods. The methods were used for NN learning applied to problems arising in medicine. Ludemir et al. (2006) optimized simultaneously NN weights and topology with a hybrid method combining Simulated Annealing (SA), Tabu Search (TS) and BP. A set of new solutions was generated on each iteration by TS rules, but the best solution was only accepted according to the probability distribution as in conventional SA. Meanwhile, the topology of the NN was also optimized and the best solution was kept. Finally, BP was used to train the best NN topology found in the previous stages. The new methodology compared favorably with SA and TS on four classification and one prediction problems. Plaginakos et al. (2001) performed several experiments to evaluate various training methods – six Differential Evolution (DE) implementations (with different mutation operators), BP, BPD (BP with deflection), SA, hybridization of BP and SA (BPSA), and GA. They reported O pe n A cc es s D at ab as e w w w .in te ch w eb .o rg

Global optimization for track finding

The N-Queen problem is a classical combinatorial optimization and constraint satisfaction problem that involves placing N queens on an N × N chessboard such that no two queens threaten each other. Due to its exponential search space and increasing computational complexity with larger board sizes, the problem serves as an important benchmark for evaluating the efficiency of heuristic and metaheuristic optimization techniques. This study presents a comprehensive performance analysis of the Simulated Annealing (SA) optimization approach for solving the N-Queen problem across varying board dimensions. The proposed methodology models the N-Queen problem as an optimization task by defining an objective function that minimizes the number of conflicting queen pairs. Simulated Annealing is employed to iteratively explore the solution space, allowing probabilistic acceptance of suboptimal solutions to escape local minima. Key parameters, including initial temperature, cooling schedule, and termination criteria, are systematically tuned to evaluate their impact on convergence speed and solution quality. Experimental results demonstrate that the SA-based approach effectively finds valid solutions for large values of N with high success rates and reduced computational effort compared to exhaustive and deterministic methods. Performance metrics such as solution accuracy, convergence time, number of iterations, and conflict reduction rate are analyzed and discussed. The results indicate that Simulated Annealing offers a robust and scalable solution for the N-Queen problem, particularly for higher-dimensional instances where traditional methods become impractical. Overall, this study highlights the suitability of Simulated Annealing as an efficient metaheuristic for solving complex combinatorial optimization problems and provides insights into its performance characteristics when applied to the N-Queen problem.

Recently, an interesting analogy between problems in combinatorial optimization and statistical mechanics has been developed and has proven useful in solving certain traditional optimization problems such as computer design, partitioning, component placement, wiring, and traveling salesman problems. The analogy has resulted in a methodology, termed "simulated annealing," which, in the process of iterating to an optimum, uses Monte Carlo sampling to occasionally accept solutions to discrete optimization problems which increase rather than decrease the objective function value. This process is counter to the normal 'steepest-descent' algorithmic approach. However, it is argued in the analogy that by taking such controlled uphill steps, the optimizing algorithm need not get "stuck" on inferior solutions. This paper presents an application of the simulated annealing method to solve the quadratic assignment problem (QAP). Performance is tested on a set of "standard" problems, as well as some newly generated larger problems (n = 50 and n = 100). The results are compared to those from other traditional heuristics, e.g., CRAFT, biased sampling, and a revised Hillier procedure. It is shown that under certain conditions simulated annealing can yield higher quality (lower cost) solutions at comparable CPU times. However, the simulated annealing algorithm is sensitive to a number of parameters, some of whose effects are investigated and reported herein through the analysis of an experimental design.

This paper presents our work on the static task scheduling model using the mean-field annealing (MFA) technique. Mean-field annealing is a technique of thermostatic annealing that takes the statistical properties of particles as its learning paradigm. It combines good features from the Hopfield neural network and simulated annealing, to overcome their weaknesses and improve on their performances. Our MFA model for task scheduling is derived from its prototype, namely, the graph partitioning problem. MFA is deterministic in nature and this gives the advantage of faster convergence to the equilibrium temperature, compared to simulated annealing. Our experimental work verifies this finding on various network and task graph sizes. Our work also includes the simulation of the MFA model on several network topologies and parameters.Keywordstask schedulingmean-field annealing and graph partitioning

Multi-constrained quality of service (QoS) routing aims at finding an optimal path that satisfies a set of QoS parameters, as an NP complete problem, which is also a big challenge for wireless mesh networks (WMNs). Heuristic algorithms with polynomial and pseudo-polynomial-time complexities are often used to deal with this problem. However, existing solutions, most of which suffered either from excessive computational complexities or from low performance, were proposed only for wired networks and cannot be used directly in wireless mesh networks. In this paper, we propose a novel routing scheme based on mean field annealing (MFA-RS) to solve this problem. MFA-RS first uses a function of two QoS parameters, wireless link’s delay and transmission success rate as the cost function, and then seeks to find a feasible path by MFA. Because MFA-RS uses a set of deterministic equations to replace the stochastic process in simulated annealing (SA) and uses saddle point approximation in the calculation of the stationary probability distribution at equilibrium, the convergence time is much less than the routing scheme based on SA (SA-RS). Simulation results demonstrate that MFA-RS is an effective algorithm and is very fit for WMNs.