Combining Search and Inference: Trading Space for Time

A method is given for constructing a max register, a linearizable, wait-free concurrent data structure that supports a write operation and a read operation that returns the largest value previously written. For fixed m, an m-valued max register can be constructed from one-bit multi-writer multi-reader registers at a cost of at most [lg m] atomic register operations per write or read. The construction takes the form of a binary search tree: applying classic techniques for building unbalanced search trees gives an unbounded max register with cost O(min(log v, n)) to read or write a value v, where n is the number of processes.It is also shown how a max register can be used to transform any monotone circuit into a wait-free concurrent data structure that provides write operations setting the inputs to the circuit and a read operation that returns the value of the circuit on the largest input values previously supplied. The cost of a write is bounded by O(Sd min([lg m], n), where m is the size of the alphabet for the circuit, S is the number of gates whose value changes as the result of the write, and d is the number of inputs to each gate; the cost of a read is min([lg m], O(n)). While the resulting data structure is not linearizable in general, it satisfies a weaker but natural consistency condition. As an application, we obtain a simple, linearizable, wait-free counter implementation with a cost of O(min(log n log v, n)) to perform an increment and O(min(log v, n)) to perform a read, where v is the current value of the counter. For polynomially-many increments, this becomes O(log2n), an exponential improvement on the best previously known upper bounds of O(n) for an exact counting and O(n4/5+e) for approximate counting.Finally, it is shown that the upper bounds are almost optimal. We prove that min([lg m], n − 1) is a lower bound on the worst-case complexity for any solo-terminating deterministic implementation of an m-valued bounded max register, which is exactly equal to the upper bound for m ≤ 2n−1. The same bound also holds m-valued counters. Furthermore, even in a solo-terminating randomized implementation of an n-valued max register with an oblivious adversary and global coins, there exist simple schedules containing n − 1 partial write operations and one read operation in which, with high probability, the worst-case step complexity of a read operation is Ω(log n log log n) if the write operations have polylogarithmic step complexity.

A new version of the nearest-neighbour approximating and eliminating search algorithm (AESA) with linear preprocessing time and memory requirements

In this paper we propose a new class of memory models, called transparent memory models, for implementing data structures so that they can be emulated in a distributed environment in a scalable, efficient and robust way. Transparent memory models aim at combining the advantages of the pointer model and the linear addressable memory model without inheriting their disadvantages. We demonstrate the effectiveness of our approach by looking at a specific memory model, called the hypertree memory model, and by implementing a search tree in it that matches, in an amortized sense, the performance of the best search trees in the pointer model yet can efficiently recover from arbitrary memory faults.

As the processing power of modern-day computers is increasing at an exponential rate, computers are able to process data at increasingly high speeds. Computers are able to surpass the limits of even the human mind by combining speed with ingenious algorithms. One example of this is the game of Chess where computers have proven to be a worthy foe to even the smartest of people. This fact is not so surprising to most people because they see the potential of artificial intelligence and machine learning algorithms. However, machines have been defeating humans for quite some time now, and not high-tech supercomputers but regular personal computers. How are computers performing better than humans in the most complex game on this planet while all our lives we have learned that computers are ‘dumb machines’ which do exactly what a programmer says. Well, a programmer cannot code all 10^120 possible games of chess so computers have to rely on two things: math and computation power. The computer uses brute force to analyse all the outcomes of the game by creating a search tree. It evaluates the search tree to find the best possible moves that would enhance the chances of winning through the minimax algorithm. However, the search tree constructed is huge, and to analyse every node in the minimax algorithm is not feasible. This is where alpha–beta pruning comes to picture. It is a search algorithm that decreases the number of nodes to be evaluated by the minimax algorithm in its search tree. As alpha–beta pruning became more popular, there were extensions made to it to increase the potency of it even further. The two mentioned in the thesis are the Transposition Tables and History Heuristic. This paper explains in detail about how they function and how effective they are at achieving the expected results.

Clause subsumption is of fundamental importance for reducing the search space in theorem proving systems. Since the subsumption problem is NP-complete, the design of efficient heuristics is of significant interest. The core of all subsumption algorithms is the search for suitable substitutions for the variables in a given clause, which in all previously known algorithms is implicitly embedded in the control structure of the algorithm. In this paper we adopt a more abstract view of subsumption and introduce the concept of a subsumption search tree to separate the search control from other computational tasks, such as computing substitutions and verifying their consistency. We study key algorithmic aspects of search trees and of heuristics for constructing them. For instance, the complexity of a search-tree based algorithm depends on the height of the search tree. We show that the problem of constructing minimal-height search trees is NP-complete. We also derive improved upper bounds on the height of search trees constructed according to an analysis based on variable dependencies, as proposed by Gottlob and Leitsch; and show that the bound is essentially tight in the worst case, by establishing suitable lower bounds for arbitrary search trees. In addition to these theoretical results, we propose further algorithmic improvements based on more sophisticated data structures for computing and representing substitutions. Finally, we have implemented several variants of our proposed algorithm and report on corresponding experiments.

When there are several different types of heat-storage ceramic bricks (checkers) that can be arranged in a regenerative combustion system, one must find an optimal arrangement (with the highest long-term Waste Heat Recovery Ratio) of these checkers, possibly of different types, in this regenerative combustion system. However, the number of possible arrangements of checkers in a heat regenerator could be huge. For example, when 5 different types of checkers are available for each of 14 positions in a heat regenerator, the total number of possible arrangements of checkers is 6,103,515,625. It is impractical to completely evaluate the efficiency of each of the 6,103,515,625 arrangements of checkers by 3D CFD simulations on Ansys Fluent. Here, we propose an optimization algorithm by tree search to tackle this optimization problem. This tree search method is motivated by the recent applications of Artificial Intelligence, based on combination of Deep Learning with Monte-Carlo Tree Search, to the incredibly complicated board game Go. Empirical evidence shows that this simple tree search algorithm leads to fast convergence of an optimization search and successfully suggests the optimal arrangement of checkers. This simple tree search method/algorithm may effectively enhance the thermal efficiency of a regenerative combustion system.

Many problems in real-world applications require searching graphs. Understanding the performance of search algorithms has been one of the eminent tasks of heuristic search research. Despite the importance of graph search algorithms, the research of analyzing their performance is limited, and most work on search algorithm analysis has been focused on tree search algorithms. One of the major obstacles to analyzing graph search is that no single graph is an appropriate representative of graph search problems. In this paper, we propose one possible approach to analyzing graph search: Analyzing the performance of graph search algorithms on a representative graph of a cluster of problems. We specifically consider job-sequencing problems in which a set of jobs must be sequenced on a machine such that a penalty function is minimized. We analyze the performance of A* graph search algorithm on an abstract model that closely represents job sequencing problems. It is an extension to a model widely used previously for analyzing tree search. One of the main results of our analysis is the existence of a gap of computational cost between two classes of job sequencing problems, one with exponential and the other with polynomial complexity. We provide experimental results showing that real job sequencing problems indeed have a huge difference on computational costs under different conditions.

We propose and investigate new algorithms permitting to find an identical object in the database using the number of operations not depending on the volume of the database. One algorithm requires memory size that depends linearly on the database volume in the average.

Variable neighborhood search for graphical model energy minimization

In this paper we advance the line of research launched by Knuth which was later improved by Chen for predicting the size of the search tree expanded by heuristic search algorithms such as IDA*. Chen's Stratified Sampling (SS) uses a partition of the nodes in the search tree called type system to guide its sampling. Recent work has shown that SS using type systems based on integer-valued heuristic functions can be quite effective. However, type systems based on real-valued heuristic functions are often too large to be practical. We use the k-means clustering algorithm for creating effective type systems for domains with real-valued heuristics. Orthogonal to the type systems, another contribution of this paper is the introduction of an algorithm called Active SS. SS allocates the same number of samples for each type. Active SS is the application of the idea of active sampling to search trees. Active SS allocates more samples to the types with higher uncertainty. Our empirical results show that (i) SS using clustering-based type systems tends to produce better predictions than competing schemes that do not use a type system, and that (ii) Active SS can produce better predictions than the regular version of SS.

In phylogenomic analysis the collection of trees with identical score (maximum likelihood or parsimony score) may hamper tree search algorithms. Such collections are coined phylogenetic terraces. For sparse supermatrices with a lot of missing data, the number of terraces and the number of trees on the terraces can be very large. If terraces are not taken into account, a lot of computation time might be unnecessarily spent to evaluate many trees that in fact have identical score. To save computation time during the tree search, it is worthwhile to quickly identify such cases. The score of a species tree is the sum of scores for all the so-called induced partition trees. Therefore, if the topological rearrangement applied to a species tree does not change the induced partition trees, the score of these partition trees is unchanged. Here, we provide the conditions under which the three most widely used topological rearrangements (nearest neighbor interchange, subtree pruning and regrafting, and tree bisection and reconnection) change the topologies of induced partition trees. During the tree search, these conditions allow us to quickly identify whether we can save computation time on the evaluation of newly encountered trees. We also introduce the concept of partial terraces and demonstrate that they occur more frequently than the original “full” terrace. Hence, partial terrace is the more important factor of timesaving compared to full terrace. Therefore, taking into account the above conditions and the partial terrace concept will help to speed up the tree search in phylogenomic inference.

Fuegito is an educational software package for learning about programming two player games. The package provides a simple, yet flexible and extensible framework which allows students to study the core search algorithms of computer game-playing, and extend them easily in projects. The current version of Fuegito supports alphabeta and Monte Carlo Tree Search as main game tree search methods, as well as a number of simple playing algorithms.

A wide variety of problems, ranging from planning and scheduling tasks in industry to making autonomous decisions in smart environments, can be modelled as constraint satisfaction problems (CSPs). While this general mathematical framework facilitates the reuse of algorithms for many different types of problems, the fact that CSPs are NP-complete gives them an exponential worst-case computational complexity. This complexity imposes severe restrictions on the size of the problem, as virtually all applications require problems to be solved in a timely fashion. However, despite being NP-complete problems, CSPs can often be solved efficiently in practice, making this worst-case complexity largely uninformative. The structure of a problem and of the search space have been used to identify classes of tractable problems and design more efficient solving techniques with better theoretical guarantees. Unfortunately, the worst-case bounds of these algorithms are hardly more informative when it comes to their practical efficiency, and exploiting this structure is challenging because efficient algorithms do not exist for many related tasks. This dissertation studies the structure of problems and their search space and how these influence the search times of algorithms. It provides additional insights into the computational complexity and practical efficiency of search algorithms, which can, for instance, be used to select the best algorithm for a given problem and aid in the design of more efficient solving techniques. This work also contributes towards improving the solving efficiency more directly by proposing improved ways to exploit the structure of a problem and the search space algorithmically.

Chapter 10 - State Space Pruning

Monte-Carlo Tree Search has delivered great results in two-player game-playing and its current success has turned it into a popular choice of study in different use cases. Recently, many works have applied MCTS and, especially, its neural variant, as an end-to-end approach to solve Combinatorial Optimization Problems. However, its efficiency for solving regular Combinatorial Problems has still to be studied. In this paper, we investigate the capability of the Monte-Carlo Tree Search algorithm to optimize permutation-based problems, making use of problem-specific knowledge. Particularly, we focus on the well-known Linear Ordering Problem (LOP), taking advantage of the easy computation of the expected and upper bound fitness of partial permutations. Moreover, we introduce a Multi-Objective Optimization approach to deal with the exploration-exploitation dilemma during the tree search. Conducted experiments show that MCTS obtains better results than classical constructive algorithms, though its performance is not obviously comparable to state-of-the-art results. Based on its ability for guiding structured searches, its scalability, convergence and search space coverage, MCTS could open new research trends in the optimization area.