Local Search in Solution of Constraint Satisfaction Problems Represented by Non-Numerical Matrices

Constraint satisfaction and search problems generally fall into the class of problems for which a direct algorithmic solution does not exist. The solution of these problems requires the examination of state spaces. A problem solver (inference engine) alone is not able to organize and maintain state space consistently. For this purpose a so‐called truth maintenance system is required. Our truth maintenance system (MEKON) organizes data within a data abstraction called a context. Each context corresponds to one problem state and contains currently believed data. The truth maintenance system provides believed data retrieval, belief revision, contradiction handling and non‐monotonicity handling, the features that help a problem solver to examine state spaces. MEKON represents an ATMS‐like system implemented within BEST (Blackboard‐based Expert System Toolkit). However, some special MEKON features such as state variables, context sensitive generation of assumptions and explicit context generation, that are not present in standard ATMSs, facilitate not only the solution of constraint satisfaction problems, but also the solution of search problems (not provided by standard ATMSs). Being deeply integrated with Prolog/Rex, BEST’s knowledge representation language, and BEST’s inference engine, MEKON provides a simple and efficient means for the examination of state spaces. Facts, hypotheses (assumptions), and concepts (frames) are used to describe a problem state, contexts are used to represent points in the state spaces, while rules are used to perform state transitions. MEKON is the only truth maintenacne system that provides truth maintenance capabilities on local blackboards thus enabling the solution of complex problems which include different kinds of real constraint satisfaction and search problems concerning diagnosis, allocation tasks, classification tasks, planning or scenario making.

In-memory computing (IMC) is gaining momentum for low-power acceleration of data intensive problems, particularly in the frame of artificial intelligence (AI). Analogue matrix-vector multiplication (MVM) in crosspoint arrays allows to dramatically improve energy efficiency in neural networks for inference and training [1]. Recently, it has been shown that a feedback configuration of MVM can accelerate the solution of generic linear matrix problems, such as linear systems, matrix inversion and eigenvector calculation, with a single computational step [2,3]. This is achieved by the high parallelism of MVM in the memory array, combined with the physical iteration executed naturally in the feedback loop. Eigenvector extraction allows to address general problems, such as PageRank of internet webpages, with complexity O(1), i.e., the time to solve a ranking problem remains fixed irrespective of the size of the coefficient matrix [4]. In this work, we address two specific problems of AI acceleration, namely regression and optimization.Linear regression can be viewed as one of the most basic problems of machine learning: given a sequence of points, one must predict the value of the next point in the sequence. This problem can be solved in just one computational step by using the closed-loop crosspoint array of Fig. 1, where the independent variables x_i are physically written within the two crosspoint arrays, while the dependent variables y_i are applied externally as input currents. The crosspoint arrays are connected within a closed-loop configuration by using operational amplifiers (OAs) in the rows and the columns of the arrays. The OA output yields the coefficients of the best fitting line along the sequence, i.e., the coefficients a and b in the line equation y= ax + b providing the least mean squares (LMS) regression of the sequence. This problem can be extended to any arbitrary dimension, i.e., not only data on x-y plane, but also data on x-y-z space and any hyperspace of n dimensions. The problem can also be extended to logistic regression, which enables classification in one computational step. In particular, the crosspoint regression circuit allows the training of a 2-layer perceptron in just one step, by assuming random weights in the first layer according to the extreme learning machine (ELM) approach. In-memory computing thus appears as a promising option for accelerating neural network training with complexity O(1).The continuous-time, analogue closed-loop approach can be extended to a discrete-time, digital recurrent neural network (RNN) for addressing optimization problems [5]. The RNN allows the efficient solution of constraint satisfaction problems (CSPs), such as Sudoku, by taking advantage of analogue MVM in the crosspoint array. In the RNN, stochastic iterations can allow the minimization of the cost function, hence the number of violated constraints, thus leading to the global minimum. Noise can be injected as spike jitter by phase change memory devices as integrate-and-fire neurons. The convergence to the solution can be efficiently accelerated by a 2-layer RNN, where the first layer is programmed to prevent the violation of constraints in the CSP. In-memory solution of CSPs can dramatically improve the energy efficiency and speed of data intensive optimization in transport, industry, and society in general.

The paper presents two types of matrix-like structures, the C-systems and the D-systems, which are proposed to be used in representation and handling the constraints over finite domains. The modifications of the well known constraints propagation algorithms have been developed by the author. The proposed approach allows representing the n-ary relations in a compressed form, and accelerating searching the solution by means of analyzing the specific features of the constraints matrices.

Multi-core processors have been broadly available to the public in the last five years. Parallelism has become a common design feature for computational intensive algorithms. In this paper we present a parallel implementation of an algorithm called interval constraint propagation for solution of constraint satisfaction problems over real numbers. Unlike existing implementations of this algorithm, our implementation scales well to many CPU cores with shared memory for sparse constraint satisfaction problems. We present scalability data for a quad-core processor on a number of benchmarks for non-linear constraint solvers.

MEMPSODE: A global optimization software based on hybridization of population-based algorithms and local searches

Getting All Solutions of Constraint Satisfaction Problems in Polynomial Time

Phase transition of constraint satisfaction problem plays an important role in computational complexity. In this paper, we use the first moment and the second moment to get phase transition of counting solutions of constraint satisfaction problems. We apply the method to counting the solutions of random k-SAT with moderately growing k and the model d-k-CSP. The threshold points of the two models are precisely located.

Constraint Programming typically uses the technique of depth-first branch and bound as the method of solving optimization problems. Although this method can give the optimal solution, for large problems, the time needed to find the optimal can be prohibitive. This paper introduces a method for using local search techniques within a Constraint Programming framework, and applies this technique to vehicle routing problems. We introduce a Constraint Programming model for vehicle routing, and a system for integrating Constraint Programming and local search techniques. We then describe how the method can be accelerated by handling core constraints using fast local checks, while other more complex constraints are left to the constraint propagation system. We have coupled our local search method with a meta-heuristic to avoid the search being trapped in local minima. Several meta-heuristics are investigated ranging from a simple Tabu Search method to Guided Local Search. An empirical study over benchmark problems shows the relative merits of these techniques. Investigations indicate that the specific long-term memory technique used by Guided Local Search can be used as a diversification method for Tabu Search, resulting in significant benefit. Several new best solutions on the Solomon problems are found in relatively few iterations of our algorithm.

Performing a local search around solutions found by an evolutionary algorithm is a common practice. Local search is well known to significantly improve the solutions, in particular in the case of combinatorial problems. In this paper a new local search procedure is proposed that uses a locally established utopia point. In the tests in which several instances of the Travelling Salesman Problem (TSP) were solved using an evolutionary algorithm the proposed local search procedure outperformed a local search procedure based on Pareto dominance. Because the local search is focused on improving individual solutions and the multiobjective evolutionary algorithm can improve diversity, various strategies of sharing computational resources between the evolutionary algorithm and the local search are used in this paper. The results attained by the tested methods are compared with respect to computation time, which allows a fair comparison between strategies that distribute computational resources between the evolutionary optimization and the local search in various proportions.

Constraint programming and local search are two well known optimization technologies. In recent years, methods for combining these two technologies have been put forward, one of which advocates the use of constraint programming for searching the local neighborhood of the current solution. We present a search technique which improves on the performance of this constraint programming-based local search method and perform experiments on a variety of both simple and more complex combinatorial problems. We also demonstrate the benefit of combining local and complete search methods.

An important implementation issue in the design of hybrid evolutionary multiobjective optimization algorithms with local search (i.e., multiobjective memetic algorithms) is how to strike a balance between local search and global search. If local search is applied to all individuals at every generation, almost all computation time is spent by local search. As a result, global search ability of memetic algorithms is not well utilized. We can use three ideas for decreasing the computation load of local search. One idea is to apply local search to only a small number of individuals. This idea can be implemented by introducing a local search probability, which is used to choose only a small number of initial solutions for local search from the current population. Another idea is a periodical (i.e., intermittent) use of local search. This idea can be implemented by introducing a local search interval (e.g., every 10 generations), which is used to specify when local search is applied. The other idea is an early termination of local search. Local search for each initial solution is terminated after a small number of neighbors are examined. This idea can be implemented by introducing a local search length, which is the number of examined neighbors in a series of iterated local search from a single initial solution. In this paper, we discuss the use of these three ideas to strike a local-global search balance. Through computational experiments on a two-objective 500-item knapsack problem, we compare various settings of local search such as short local search from all individuals at every generation, long local search from only a few individuals at every generation, and periodical long local search from all individuals. Global search in this paper means genetic search by crossover and mutation in multiobjective memetic algorithms.

The goal of this paper is twofold. First, we introduce a class of local search procedures for solving optimization and constraint problems. These procedures are based on various heuristics for choosing variables and values in order to examine a general neighborhood. Second, four combinations of heuristics are empirically evaluated by using the graph-coloring problem and a real world application — the frequency assignment problem. The results are also compared with those obtained with other approaches including simulated annealing, Tabu search, constraint programming and heuristic graph coloring algorithms. Empirical evidence shows the benefits of this class of local search procedures for solving large and hard instances.

Using constraint programming (CP) to explore a local-search neighbourhood was first tried in the mid 1990s. The advantage is that constraint propagation can quickly rule out uninteresting neighbours, sometimes greatly reducing the number actually probed. However, a CP model of the neighbourhood has to be handcrafted from the model of the problem: this can be difficult and tedious. That research direction appears abandoned since large-neighbourhood search (LNS) and constraint-based local search (CBLS) arose as alternatives that seem easier to use. Recently, the notion of declarative neighbourhood was added to the technology-independent modelling language MiniZinc, for use by any backend to MiniZinc, but currently only used by a CBLS backend. We demonstrate that declarative neighbourhoods are indeed technology-independent by using the old idea of CP-based neighbourhood exploration: we explain how to encode automatically a declarative neighbourhood into a CP model of the neighbourhood. This enables us to lift any CP solver into a local-search backend to MiniZinc. Our prototype is competitive with CP, CBLS, and LNS backends to MiniZinc.

In the liner shipping business, shipping ports represent the main nodes in the maritime transportation network. These ports have a collection of terminals where container vessels can load and discharge containers. However, the logistics and planning of operations differ depending on the vessel size. Large container vessels visit a single terminal, whereas smaller container vessels, or feeder vessels, visit several terminals to transport all containers within the multiple terminals of the port. In this paper, we study the Port Scheduling Problem, the problem of scheduling the operations of feeder vessels in multi-terminal ports. The resulting problem can be identified as a version of the General Shop Scheduling Problem. We consider a Constraint Programming formulation of the problem, and we propose a matheuristic solution approach for solving large instances. The proposed matheuristic is a hybrid solution method that combines Constraint Programming with a local search heuristic. The solution approach benefits from the fast search capabilities of local search algorithms to explore the solution space using an Adaptive Large Neighbourhood Search heuristic. During the search, we further use the Constraint Programming model as an intensification technique, every time a new best-known solution is found. We conduct detailed computational experiments on the PortLib instances, showing that the incorporation of Constraint Programming within the heuristic search can result in significant benefits. The high instability in solution quality obtained by local search heuristics can be lowered by a simple combination of both methods.

Handbook of Constraint Programming