Mean field annealing: a formalism for constructing GNC-like algorithms

This paper presents an optimal design method by means of mean field annealing (MFA). The mean field theory (MFT) is introduced, and a certain MFA application extension based on Peierls inequality is explained in detail, the critical temperature that is important for classes of problem while applying MFA is discussed. With MFA, the optimal size design of electromagnetic device is carried out. The results are compared with two existing algorithms, the modified simulated annealing (MSA) and one-variable stochastic simulated annealing based MFA (S-MFA), in both operating cost and final quantities. To modify the final results to reach the global optimum, Hooke-Jeeves pattern search method is adopted. Favorable results show the effectiveness of the proposed scheme.

Optimization is an important subject in solving scientific and engineering problems. One conventional searching technique for finding the global minimum is to use gradient descent, which finds the direction for the next iteration from the gradient of the objective function. For complicated problems, the gradient descent technique often gets stuck at a local minimum where the objective function has surrounding barriers. In addition, the complexity of most combinatorial optimization problems increases dramatically with the problem size and makes it very difficult to obtain the globally optimal solution in a reasonably computational time. Complicated optimization problems could also be solved by using branch-and-bound searching method [1], or by relaxing some constraints and solving the simplified problems [2]. Several attractive methods have been reported to help the solutions of the optimization problems to escape from local minima [3 4, 5, 6, 7, 8, 9, 10]. Simulated annealing is one of the popular approaches which is widely applicable to the combinatorial optimization problems [3, 4]. Simulated annealing is a generalization of iterative improvement methods in that it accepts, with a non-zero but gradually decreasing probability, deterioration in the cost function of an optimization problem. A solution from simulated annealing is close to the global minimum within a polynomial upper bound for the computational time and is independent of the initial condition. Mean field annealing is a deterministic approximation to simulated annealing [7]. It analytically approximates the relevant Boltzmann distribution and can execute at 50 times faster than simulated annealing for some problems. The quality of the solutions produced by mean field annealing is highly dependent on the choice of the initial temperature, the cooling speed and the final temperature. So far, simulated annealing has been successfully reported in layout generation of VLSI circuits [11, 12], test pattern generation [13], integrated-circuit delay reduction [14], image segmentation [15] and noise filtering in image processing [16].

We describe a global optimization method called mean field annealing (MFA) and its application to two basic problems in seismic data processing: Seismic deconvolution and surface related multiple attenuation. MFA replaces the stochastic nature of the simulated annealing method with a set of deterministic update rules that act on the average value of the variables rather than on the variables themselves, based on the mean field approximation. As the temperature is lowered, the MFA rules update the variables in terms of their values at a previous temperature. By minimizing averages, it is possible to converge to an equilibrium state considerably faster than a standard simulated annealing method. The update rules are dependent on the form of the cost function and are obtained easily when the cost function resembles the energy function of a Hopfield network. The mapping of a problem onto a Hopfield network is not a precondition for using MFA, but it makes analytic calculations simpler. The seismic deconvolution problem can be mapped onto a Hopfield network by parameterizing the source and the reflectivity in terms of binary neurons. In this context, the solution of the problem is obtained when the neurons of the network reach their stable states. By minimizing the cost function of the network with MFA and using an appropriate cooling schedule, it is possible to escape local minima. A similar idea can also be applied to design an operator that attenuates surface related multiple reflections from plane‐wave transformed seismograms assuming a 1-D earth. The cost function for the multiple elimination problem is based on the criterion of minimum energy of the multiple suppressed data.

Reports a new method for optimizing satellite broadcasting schedules based on the Hopfield neural model in combination with the mean field annealing theory. A clamping technique is used with an associative matrix, thus reducing the dimensions of the solution space. A formula for estimating the critical temperature for the mean field annealing procedure is derived, hence enabling the updating of the mean field theory equations to be more economical. Several factors on the numerical implementation of the mean field equations using a straightforward iteration method that may cause divergence are discussed; methods to avoid this kind of divergence are also proposed. Excellent results are consistently found for problems of various sizes.

Multi-constrained quality-of-service routing (QoSR) is to find a feasible path that satisfies multiple constraints simultaneously, as an NPC problem, which is also a big challenge for wireless mesh networks. In this paper, we propose a novel Routing Scheme based on Mean Field Annealing (MFA_RS) to solve this problem. MFA_RS first uses a function of the wireless link’s delay and capacity as the cost function, both of which are QoS parameters, and then seeks to find a feasible path by mean field annealing. Because MFA_RS uses a set of deterministic equations to replace the stochastic process in Simulated Annealing (SA), and uses saddle point approximation in the calculation of the stationary probability distribution at equilibrium, thus the convergence time is much less than the algorithm based on SA. Simulation results demonstrate that MFA_RS is an effective algorithm and is very fit for wireless mesh networks where the topology may be changing constantly.

Hopfield neural networks (HNN) and simulated annealing (SA) are two recent approaches to solving many combinatorial optimization problems. But, the disadvantages of HNN are that it can often converge at local minimum and the quality of solutions depends on the initial state because of sensitivity of parameterization. On the other hand, SA is designed to cope with the problem of convergence to a local minimum. SA, however, requires unacceptably large computing time. So, a new algorithm called mean field annealing (MFA) which can be interpreted as a generalization of HNN, is applied to optimization problems efficiently. In this paper, we propose a new approach based on MFA to solving of the minimization problem. Also we show that our approach can be applied to multi-layer as well as two-layer routing, and optimal solutions can be obtained by properly adjusting the parameter in the given energy function.

In this paper we propose an identification and restoration algorithm for image degraded by noise as well as blurring. In our proposed method, the original images are modeled as Markov Random Fields(MRF) and the blurred image is presented by the gaussian blurred model. To estimate the parameters involved in the image model and to restore the degraded images, we use Generalized Maximum Likelihood(GML) estimation. The GML consists of the two procedures which is a Maximum Likelihood(ML) estimation and a Maximum A Posteriori(MAP) estimation. These procedures are implemented by using the Expectation-Maximization(EM) algorithm and mean field annealing(MFA) based on the mean field theory.

This paper presents an unsupervised segmentation approach applying the mean field annealing (MFA) heuristic with the modified cost function. The idea is to cast a clustering problem as a minimization problem where the criteria for the optimum segmentation is chosen as the minimization of the Euclidean distance between samples to cluster centers. To resolve the optimal problem using a Hopfield or simulated annealing neural network, the penalty terms are combined into a weighted sum using several coefficients determined by user. Using the MFA network to medical image segmentation, the need for finding weighting factors in the energy function can be eliminated and the rate of convergence is much faster than that of simulated annealing. The experimental results show that good and valid solutions can be obtained using the MFA neural network.

In this paper, we introduce a new solution for the load balancing problem, which is an important issue in parallel processing. Our novel load balancing technique called MGA is a hybrid algorithm of Mean Field Annealing (MFA) and Simulated annealing-like Genetic Algorithm (SGA). SGA uses the Metropolis criteria for state transition as in simulated annealing to keep the convergence property in MFA. The MGA combines the benefit of both the rapid convergence property of MFA and various and effective genetic operations of SGA. We compare the proposed MGA with MFA and GA. Our experimental results show that our new technique improves performance in terms of communication cost, load imbalance and maximum execution time.KeywordsCommunication CostTask AllocationGenetic AlgorithmMapping ProblemGenetic OperationThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This paper presents some new applications of Mean Field Annealing (MFA). a technique which usually finds global minima in nonconvex optimization problems. to the restoration of nuclear medicine and magnetic resonance images. The image restoration technique presented in this paper preserves the edges while reducing noise. By selection of proper parameters, it can be used for segmentation. Nonuniform temperature anneahg is introduced for the restoration of images with signal dependent noise. guod for 3-D images. By the use of successive over-relaxation (SOR) algorithm in minimization, These MFA image restoration techniques are fast enw& for practical use in processing 3-D images.

Most of the existing video coding algorithms produce highly visible artifacts in the reconstructed images as the bit-rate is lowered. These artifacts are due to the information loss caused by the quantization process. Since these algorithms treat decoding as simply the inverse process of encoding, these artifacts are inevitable. In this paper, we propose an encoder/decoder paradigm in which both the encoder and decoder solve an estimation problem based on the available bitstream and prior knowledge about the source image and video. The proposed technique makes use of a priori information about the original image through a nonstationary Gauss-Markov model. Utilizing this mode, a maximum a posteriori (MAP) estimate is obtained iteratively using mean field annealing. The fidelity to the data is preserved by projecting the image onto a constraint set defined by the quantizer at each iteration. The performance of the proposed algorithm is demonstrated on an H.261-type video codec. It is shown to be effective in improving the reconstructed image quality considerably while reducing the bit-rate.

This paper presents an application of mean field annealing (MFA) theory for motion estimation in medical image sequences. More precisely, we present the adaptation of the MFA algorithm to solve the motion estimation problem in NMR (nuclear magnetic resonance) image sequences. The proposed algorithm ensures a satisfying trade-off between precision and computational time, in other words, it ensures a good efficiency in comparison with stochastic algorithms. The results are compared in term of precision and computational time with some basic algorithms, such as the basic block-matching and Horn & Schunck. The results are illustrated on both synthetic images (obtained using the free form deformation principle) and real NMR medical image sequences.

This paper describes a new framework for constructing mean-field energy functions for use in relational matching. The starting point is the Bayesian relational consistency model of Wilson and Hancock (1995). Hitherto, the optimisation of the consistency measure has been effected by the deterministic hill climbing process known as discrete relaxation which is prone to local convergence if local maxima are present. By applying ideas from statistical physics to the configurational matching probabilities we determine the effective potentials of an equivalent Boltzmann distribution. Formally, these potentials are weighted sums of the Hamming distances between matched neighbourhoods in the data graph and their counterparts in the model graph. Adopting a simple softening ansatz we derive mean-field equations for minimising the global graph matching potential. This provides an efficient means of locating the global optima of the Bayesian consistency measure.

A novel method for obtaining dense displacement fields from a sequence of frames is presented. The algorithm is based on modeling the displacement field as a Markov random field, hence making it possible to use the equivalence of a Markov random field and a Gibbs distribution. The Gibbs distribution is used to define a mean field probability density function, and the displacement field is found using a mean field annealing algorithm. Mean field annealing is a technique which finds the global or near global minima in nonconvex optimization problems. Analysis of the algorithm and experimental results are presented. >

In this work, we look at mean field annealing (MFA) from two different perspectives: information theory and statistical mechanics. An iterative, deterministic algorithm is developed to obtain the mean field solution for disparity calculation in stereo images.KeywordsGibbs SamplerRelative EntropyInternal FieldDisparity FunctionField SolutionThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.