Multiprocessor scheduling using mean-field annealing

Hopfield neural networks (HNN) and simulated annealing (SA) are two recent approaches to solving many combinatorial optimization problems. But, the disadvantages of HNN are that it can often converge at local minimum and the quality of solutions depends on the initial state because of sensitivity of parameterization. On the other hand, SA is designed to cope with the problem of convergence to a local minimum. SA, however, requires unacceptably large computing time. So, a new algorithm called mean field annealing (MFA) which can be interpreted as a generalization of HNN, is applied to optimization problems efficiently. In this paper, we propose a new approach based on MFA to solving of the minimization problem. Also we show that our approach can be applied to multi-layer as well as two-layer routing, and optimal solutions can be obtained by properly adjusting the parameter in the given energy function.

We describe a global optimization method called mean field annealing (MFA) and its application to two basic problems in seismic data processing: Seismic deconvolution and surface related multiple attenuation. MFA replaces the stochastic nature of the simulated annealing method with a set of deterministic update rules that act on the average value of the variables rather than on the variables themselves, based on the mean field approximation. As the temperature is lowered, the MFA rules update the variables in terms of their values at a previous temperature. By minimizing averages, it is possible to converge to an equilibrium state considerably faster than a standard simulated annealing method. The update rules are dependent on the form of the cost function and are obtained easily when the cost function resembles the energy function of a Hopfield network. The mapping of a problem onto a Hopfield network is not a precondition for using MFA, but it makes analytic calculations simpler. The seismic deconvolution problem can be mapped onto a Hopfield network by parameterizing the source and the reflectivity in terms of binary neurons. In this context, the solution of the problem is obtained when the neurons of the network reach their stable states. By minimizing the cost function of the network with MFA and using an appropriate cooling schedule, it is possible to escape local minima. A similar idea can also be applied to design an operator that attenuates surface related multiple reflections from plane‐wave transformed seismograms assuming a 1-D earth. The cost function for the multiple elimination problem is based on the criterion of minimum energy of the multiple suppressed data.

This paper presents an unsupervised segmentation approach applying the mean field annealing (MFA) heuristic with the modified cost function. The idea is to cast a clustering problem as a minimization problem where the criteria for the optimum segmentation is chosen as the minimization of the Euclidean distance between samples to cluster centers. To resolve the optimal problem using a Hopfield or simulated annealing neural network, the penalty terms are combined into a weighted sum using several coefficients determined by user. Using the MFA network to medical image segmentation, the need for finding weighting factors in the energy function can be eliminated and the rate of convergence is much faster than that of simulated annealing. The experimental results show that good and valid solutions can be obtained using the MFA neural network.

This paper presents an optimal design method by means of mean field annealing (MFA). The mean field theory (MFT) is introduced, and a certain MFA application extension based on Peierls inequality is explained in detail, the critical temperature that is important for classes of problem while applying MFA is discussed. With MFA, the optimal size design of electromagnetic device is carried out. The results are compared with two existing algorithms, the modified simulated annealing (MSA) and one-variable stochastic simulated annealing based MFA (S-MFA), in both operating cost and final quantities. To modify the final results to reach the global optimum, Hooke-Jeeves pattern search method is adopted. Favorable results show the effectiveness of the proposed scheme.

Multi-constrained quality-of-service routing (QoSR) is to find a feasible path that satisfies multiple constraints simultaneously, as an NPC problem, which is also a big challenge for wireless mesh networks. In this paper, we propose a novel Routing Scheme based on Mean Field Annealing (MFA_RS) to solve this problem. MFA_RS first uses a function of the wireless link’s delay and capacity as the cost function, both of which are QoS parameters, and then seeks to find a feasible path by mean field annealing. Because MFA_RS uses a set of deterministic equations to replace the stochastic process in Simulated Annealing (SA), and uses saddle point approximation in the calculation of the stationary probability distribution at equilibrium, thus the convergence time is much less than the algorithm based on SA. Simulation results demonstrate that MFA_RS is an effective algorithm and is very fit for wireless mesh networks where the topology may be changing constantly.

Optimization is an important subject in solving scientific and engineering problems. One conventional searching technique for finding the global minimum is to use gradient descent, which finds the direction for the next iteration from the gradient of the objective function. For complicated problems, the gradient descent technique often gets stuck at a local minimum where the objective function has surrounding barriers. In addition, the complexity of most combinatorial optimization problems increases dramatically with the problem size and makes it very difficult to obtain the globally optimal solution in a reasonably computational time. Complicated optimization problems could also be solved by using branch-and-bound searching method [1], or by relaxing some constraints and solving the simplified problems [2]. Several attractive methods have been reported to help the solutions of the optimization problems to escape from local minima [3 4, 5, 6, 7, 8, 9, 10]. Simulated annealing is one of the popular approaches which is widely applicable to the combinatorial optimization problems [3, 4]. Simulated annealing is a generalization of iterative improvement methods in that it accepts, with a non-zero but gradually decreasing probability, deterioration in the cost function of an optimization problem. A solution from simulated annealing is close to the global minimum within a polynomial upper bound for the computational time and is independent of the initial condition. Mean field annealing is a deterministic approximation to simulated annealing [7]. It analytically approximates the relevant Boltzmann distribution and can execute at 50 times faster than simulated annealing for some problems. The quality of the solutions produced by mean field annealing is highly dependent on the choice of the initial temperature, the cooling speed and the final temperature. So far, simulated annealing has been successfully reported in layout generation of VLSI circuits [11, 12], test pattern generation [13], integrated-circuit delay reduction [14], image segmentation [15] and noise filtering in image processing [16].

Optimization problems are approached using mean field annealing (MFA), which is a deterministic approximation, using mean field theory and based on Peierls's inequality, to simulated annealing. The MFA mathematics are applied to three different objective function examples. In each case, MFA produces a minimization algorithm that is a type of graduated nonconvexity. When applied to the ;weak-membrane' objective, MFA results in an algorithm qualitatively identical to the published GNC algorithm. One of the examples, MFA applied to a piecewise-constant objective function, is then compared experimentally with the corresponding GNC weak-membrane algorithm. The mathematics of MFA are shown to provide a powerful and general tool for deriving optimization algorithms.

In a graph, a clique is a set of vertices such that every pair is connected by an edge. MAX-CLIQUE is the optimization problem of finding the largest clique in a given graph and is NP-hard, even to approximate well. Several real-world and theory problems can be modeled as MAX-CLIQUE. In this paper, we efficiently approximate MAX-CLIQUE in a special case of the Hopfield network whose stable states are maximal cliques. We present several energy-descent optimizing dynamics; both discrete (deterministic and stochastic) and continuous. One of these emulates, as special cases, two well-known greedy algorithms for approximating MAX-CLIQUE. We report on detailed empirical comparisons on random graphs and on harder ones. Mean-field annealing, an efficient approximation to simulated annealing, and a stochastic dynamics are the narrow but clear winners. All dynamics approximate much better than one which emulates a "naive" greedy heuristic.

Multi-constrained quality of service (QoS) routing aims at finding an optimal path that satisfies a set of QoS parameters, as an NP complete problem, which is also a big challenge for wireless mesh networks (WMNs). Heuristic algorithms with polynomial and pseudo-polynomial-time complexities are often used to deal with this problem. However, existing solutions, most of which suffered either from excessive computational complexities or from low performance, were proposed only for wired networks and cannot be used directly in wireless mesh networks. In this paper, we propose a novel routing scheme based on mean field annealing (MFA-RS) to solve this problem. MFA-RS first uses a function of two QoS parameters, wireless link’s delay and transmission success rate as the cost function, and then seeks to find a feasible path by MFA. Because MFA-RS uses a set of deterministic equations to replace the stochastic process in simulated annealing (SA) and uses saddle point approximation in the calculation of the stationary probability distribution at equilibrium, the convergence time is much less than the routing scheme based on SA (SA-RS). Simulation results demonstrate that MFA-RS is an effective algorithm and is very fit for WMNs.

Reports a new method for optimizing satellite broadcasting schedules based on the Hopfield neural model in combination with the mean field annealing theory. A clamping technique is used with an associative matrix, thus reducing the dimensions of the solution space. A formula for estimating the critical temperature for the mean field annealing procedure is derived, hence enabling the updating of the mean field theory equations to be more economical. Several factors on the numerical implementation of the mean field equations using a straightforward iteration method that may cause divergence are discussed; methods to avoid this kind of divergence are also proposed. Excellent results are consistently found for problems of various sizes.

Global optimization for track finding

An neural network (ENN) is presented for the max cut problem of an undirected graph G(V, E) in this paper. The goal of the NP-hard problem is to find a partition of V into two disjoint subsets such that the cut size be maximized. The cut size is the sum of weights on edges in E whose endpoints belong to different subsets. The ENN combines the evolutionary initialization scheme of the neural state into the energy minimization criteria of the binary neural network. The performance of ENN is evaluated through simulations in randomly weighted complete graphs and unweighted random graphs with up to 1000 vertices. The results show that the evolutionary initialization scheme drastically improves the solution quality. ENN can always find better solutions than the maximum neural network, the mean field annealing, the simulated annealing, and the greedy algorithm.

Aiming at edge computing nodes' limited computing and storage capacity, a two-layer task scheduling model based on blockchain and task cache was proposed. The high-similarity task results were cached in the edge cache pool, and the blockchain-assisted task caching model was combined to enhance system security. The genetic evolution algorithm was used to solve the minimum cost that the optimal scheduling model can obtain. The genetic algorithm’s initialization and mutation operations were adjusted to improve the convergence rate. Compared with algorithms without cache pooling and blockchain, the proposed joint blockchain and task caching task scheduling model reduced the cost by 9.4% and 14.3%, respectively. As the capacity space of the cache pool increased, the system cost gradually decreased. Compared with the capacity space of 3GB, the system cost of 10Gbit capacity space was reduced by 10.6%. The system cost decreased as the computing power of edge nodes increased. Compared with edge nodes with a computing frequency of 8GHz, the nodes cost at 18GHz was reduced by 36.4%. Therefore, the proposed edge computing task scheduling model ensures the security of task scheduling based on reducing delay and control costs, providing a foundation for modern industrial task scheduling.

My research in optimization at the Department of Systems and Computer Engineering at Carleton University has brought me into contact with researchers in the Department of Electronics, our sister department in the general field of Electrical and Computer Engineering (ECE), which has a major research thrust in computer-aided design (CAD) of very large scale integrated (VLSI) circuits. I have come to realize that the design of VLSI systems consists in large part of solving incredibly massive mixed-integer nonlinear optimization problems, together with enormous circuit simulations. This is impossible without the use of CAD techniques. Over time, ECE-CAD researchers have both borrowed useful standard techniques from operations research (OR) and have invented their own, often to deal with the sheer scale and complexity of the design problems they face. Interaction between the OR and ECE-CAD communities would seem to be a natural development. Coincidentally, the INFORMS Journal on Computing had an area entitled “High Performance Computation” that covered (i) the solution of OR problems using new computing technologies, (ii) the application of OR techniques in the design and use of highperformance computing and communication systems, and (iii) solution methods for ultra-large-scale OR applications. The viability of this area was debated during 2003 as some of its aspects migrated to other areas (e.g., the new “Telecommunications and Electronic Commerce” area), and the number of submissions declined. The area was eventually closed, although I argued that the overlap between OR and ECE-CAD, essentially covering items (ii) and (iii), was a very active research area. This Special Cluster of papers was conceived as a way to test that argument. I was recruited at the same time to prepare a tutorial on ECE-CAD for the 2004 INFORMS Annual Meeting (see John W. Chinneck, Michel Nakhla, and Q. J. Zhang 2004. Computer-aided design for electrical and computer engineering. H. J. Greenberg, ed. Tutorials on Emerging Methodologies and Applications in Operations Research. Springer, New York, 6-1 to 6-44). The preparation of the article reinforced my observation about the research overlap between OR and ECE-CAD. A search of the electrical engineering literature for 2000 through early 2004 turned up 46,725 papers mentioning “simulate” or “simulation” as keywords in the abstract, 14,216 papers mentioning “optimization” or “optimize,” and relatively smaller numbers for specific techniques such as “neural network” (6,251), “genetic algorithm” (2,603), “linear programming” (576), “simulated annealing” (448), and “branch and bound” (199). One surprise was the relatively small number of papers using the generic keywords “mathematical programming” (68) or “operations research” (37). The general conclusion of the tutorial article is that the OR and the ECE-CAD communities have much to offer each other. This is certainly the case for the papers gathered in this Special Cluster. We see known OR techniques adapted for use in ECECAD. In “Integer Linear Programming Models for Global Routing,” Behjat et al. apply integer linear programming in a heuristic to solve enormous NPhard connection routing problems for VLSI circuits. In “Task Scheduling in a Finite-Resource, Reconfigurable Hardware/Software Codesign Environment,” Loo and Wells use simulated annealing, genetic algorithms, and random search techniques to solve scheduling problems in hardware-software co-design. We also see new techniques specifically developed by the ECE-CAD community to deal with problems of extreme scale. In “A Projection-Based Reduction Approach to Computing Sensitivity of Steady-State Response of Nonlinear Circuits,” Pai et al. develop methods for sensitivity analysis in extremely large nonlinear programs, especially those in which the objective function is very costly to evaluate. The methods have special relevance for simulation-based

Multi-agent patrolling problem has received growing attention from many researchers due to its wide range of potential applications. In realistic environment, e.g., security patrolling, each location has different visitation requirement according to the required security level. Therefore, a patrolling system with non-uniform visiting frequency is preferable. The difference in visiting frequency generally causes imbalanced workload amongst agents leading to inefficiency. This paper, thus, aims at partitioning a given area to balance agents’ workload by considering that different visiting frequency and then generating route inside each sub-area. We formulate the problem of frequency-based multi-agent patrolling and propose its semi-optimal solution method, whose overall process consists of two steps – graph partitioning and sub-graph patrolling. Our work improve traditional k-means clustering algorithm by formulating a new objective function and combine it with simulated annealing – a useful tool for operations research. Experimental results illustrated the effectiveness and reasonable computational efficiency of our approach.