TIGRIS: An Informed Sampling-based Algorithm for Informative Path Planning

End-to-end AutoML has attracted intensive interests from both academia and industry, which automatically searches for ML pipelines in a space induced by feature engineering, algorithm/model selection, and hyper-parameter tuning. Existing AutoML systems, however, suffer from scalability issues when applying to application domains with large, high-dimensional search spaces. We present VOLCANOML, a scalable and extensible framework that facilitates systematic exploration of large AutoML search spaces. VOLCANOML introduces and implements basic building blocks that decompose a large search space into smaller ones, and allows users to utilize these building blocks to compose an execution plan for the AutoML problem at hand. VOLCANOML further supports a Volcano-style execution model - akin to the one supported by modern database systems - to execute the plan constructed. Our evaluation demonstrates that, not only does VOLCANOML raise the level of expressiveness for search space decomposition in AutoML, it also leads to actual findings of decomposition strategies that are significantly more efficient than the ones employed by state-of-the-art AutoML systems such as auto-sklearn.

A heuristic approach to optimization of complex physicochemical processes in the form of a genetic algorithm for solving problems is presented. In comparison with other evolutionary methods, the genetic algorithm allows working with large search spaces and complex evaluation functions, which is especially important in the study of multifactor physicochemical systems. Due to the relatively high need for computing resources when working with large and complex search spaces, optimization of existing calculation organization schemes has a positive effect on the accuracy of the calculated results. The paper presents a modified genetic algorithm that minimizes the number of iterations to achieve a given accuracy when solving the problem of finding the optimal composition of the initial reaction mixture. For a complex physicochemical process, an optimization problem is formulated which consists in finding the composition of the initial reaction mixture that promotes maximization (or minimization) of a given target parameter. The optimality criterion is determined by the type of the problem being solved and, when organizing calculations, is focused on the maximum yield of the target product. The main steps of implementing the genetic algorithm include creating an initial set of solutions and subsequent iterative evaluation of their quality for subsequent combination and modification until optimal values are achieved using mechanisms similar to biological evolution. To improve the efficiency of the method and reduce the number of iterations, a modification of the genetic algorithm is proposed which boils down to a dynamic estimate of the “mutation” parameter, depending on the diversity of individuals in the formed population of solutions. In a series of computational experiments, an analysis was made of the influence of the genetic algorithm parameters on the accuracy and efficiency of solving the problem using the example of studying the kinetics of the Michaelis-Menten enzymatic reaction. The results of calculations to determine the optimal composition of the reaction mixture showed that the dynamic determination of the “mutation” parameter contributes to an increase in the accuracy of the problem solution and a multiple decrease in the relative error value reaching 0.77 % when performing 200 iterations and 0.21 % when performing 400 iterations. The presented modified approach to solving the optimization problem is not limited by the type and content of the studied physicochemical process. The calculations performed showed a high degree of influence of the “mutation” parameter on the accuracy and efficiency of the problem solution, and dynamic control of the value of this parameter allowed increasing the speed of the genetic algorithm and reduce the number of iterations to achieve an optimal solution of a given accuracy. This is especially relevant in the study of multifactorial systems when the influence of parameters is non-trivial.

Nearest neighbor search in high-dimensional space is an important operation in many applications, such as data mining and multimedia database. Evaluating similarities of a point to all other points in high-dimensional space need the high computational cost. For reducing the computational cost, index-structures are frequently used. Most of these index-structures are built by partitioning the data set based on a specific criterion. However, partitioning approaches potentially have a problem failing to find the nearest neighbor which is caused by disjoint partitions. In this paper, we propose an Error Minimizing Partitioning (E-MP) method with a novel tree structure, which minimizes the failure problem in finding the nearest neighbors. E-MP divides the data into subsets with considering the distribution of data set. For partitioning data set, the proposed method finds the first principal component of the data set using the principal component analysis (PCA). And then, the method finds the centroid of data set. Finally, it decides the partitioning hyper-plane that passes the centroid and is perpendicular to the principal component vector. We also make a comparative study of existing methods and the proposed method, to verify the usability of our method.

Advances and Prospect of Nearest Neighbor Search in High Dimensional Space(<Special Issue>Analysis of Very Large Scale Image Collections)

We present a parallel formulation for enumerative search in high dimensional spaces and apply it to planning paths for a 6-dof manipulator robot. Participating processors perform local A* search towards the goal configuration. To exploit all the processors at their maximum capacity at all times, a dynamic load-balancing scheme matches idle and busy processors for load transfer. For comparison purposes, we have also implemented an existing parallel static load-balancing formulation based on regular domain decomposition. Both methods achieved almost linear speed-up in our experiments. The two methods follow different search strategies in parallel and the implementation of the existing method (with tuned space decomposition) was more time efficient on average. However, the planning time of that method is highly dependent on the distribution of the search space among the processors and its tuned decomposition varies for different obstacle placements. Empirical selection of the space decomposition parameters for the existing method does not guarantee minimal planning time in all environments and leads to slower planning than our dynamic load-balancing method in some cases. The performance of the developed dynamic method is independent of the obstacle placements and the method can achieve consistent speed-up in all environments.

Exploiting lower bounds to accelerate approximate nearest neighbor search on high-dimensional data

Recent years see an explosive growth of digital images and we need efficient technologies to index and retrieve them. The content-based retrieval method is a hot topic in recent research, in which we index the features of image and then retrieve the images through nearest neighbor searching. The features of an image dwell in high dimensional space, which poses particular challenge for nearest neighbor search. The curse of dimensionality raises severe difficulty in traditional methods. In this paper, we proposed a novel method for nearest neighbor search in high dimensional space based on two-dimensional embedding. First, we filter out the non-nearest neighbors using low dimensional information via data embedding, and then efficient nearest neighbor search can be performed in a much smaller candidate set to achieve fast image retrieval. The experimental results on the dataset CIFAR validate the effectiveness and efficiency of our method.

Nearest neighbor search in high dimensional spaces is an interesting and important problem which is relevant for a wide variety of applications, including multimedia information retrieval, data mining, and pattern recognition. For such applications, the curse of high dimensionality tends to be a major obstacle in the development of efficient search methods. This paper addresses the problem of designing an efficient algorithm for high dimensional nearest neighbor search using a priority queue. The proposed algorithm is based on a simple linear search algorithm and eliminates unnecessary arithmetic operations from distance computations between multidimensional vectors. Moreover, we propose two techniques, a dimensional sorting method and a PCA-based method, to accelerate multidimensional search. Experimental results indicate that our scheme scales well even for a very large number of dimensions.

Nearest neighbor (NN) search in high dimensional space is an important problem in many applications. Ideally, a practical solution (i) should be implementable in a relational database, and (ii) its query cost should grow sub-linearly with the dataset size, regardless of the data and query distributions. Despite the bulk of NN literature, no solution fulfills both requirements, except locality sensitive hashing (LSH). The existing LSH implementations are either rigorous or adhoc. Rigorous-LSH ensures good quality of query results, but requires expensive space and query cost. Although adhoc-LSH is more efficient, it abandons quality control, i.e., the neighbor it outputs can be arbitrarily bad. As a result, currently no method is able to ensure both quality and efficiency simultaneously in practice.

In order to improve the relatively slow convergence of GA, in the presence of large search spaces, and reduce the high consuming time of evaluation functions in analog circuit design applications, this chapter will discuss the use of learning algorithms. These algorithms explore the successive generation of solutions, learn the tendency of the best optimization variables and will use this knowledge to predict future values. In other words, these techniques employ data mining theory, used to manage large databases and huge amount of internet information, to discover complex relationships among various factors and extract meaningful knowledge to improve the efficiency and quality of decision making. In this chapter a new hybrid optimization algorithm is presented together with a design methodology, which increases the efficiency on the analog circuit design cycle. This new algorithm combines an enhanced GA kernel with an automatic learning machine based on SVM model (GA-SVM) which efficiently guides the selection operator of the GA algorithm avoiding time-consuming SPICE evaluations of non-promising solutions. The SVM model is here defined as a classification model used to predict the feasibility region in the presence of large, non-linear and constraints search spaces that characterize analog design problems. The SVM modeling attempts to constraint the search space in order to accelerate the search towards the feasible region ensuring a proper operation of the circuit.KeywordsSupport Vector MachineDesign SpaceFeasibility RegionSupport Vector Machine ModelAnalog CircuitThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Search is considered to be an important functionality in a computational system. Search techniques are applied in file retrievals and indexing. Though there exists various search techniques, binary search is widely used in many applications due to its advantage over other search techniques namely linear and hash search. Binary search is easy to implement and is used to search for an element in a large search space. The worst case time complexity of binary search is O (log 2 n) where n is the number of elements (search space) in the array. However, in binary search, searching is performed on the entire search space. The complexity of binary search may be further reduced if the search space is reduced. This paper proposes an Ant Colony Optimization based Binary Search (ACOBS) algorithm to find an optimal search space for binary search. ACOBS algorithm categorizes the search space and the key element is searched only in a specific category where the key element can exist thereby reducing the search space. The time complexity of ACOBS algorithm is O (log 2 c) where c is the number of elements in the reduced search space and c < n. The proposal is best suited for real time applications where searching is performed on a large domain.KeywordsAnt colony optimizationBinary searchOptimal search space

Conventional evolutionary algorithms operate in a fixed search space with limiting parameter range, which is often predefined via a priori knowledge or trial and error in order to `guess' a suitable region comprising the global optimal solution. This requirement is hard, if not impossible, to fulfil in many real-world optimization problems since there is often no clue of where the desired solutions are located in these problems. Thus, this paper proposes an inductive---deductive learning approach for single- and multi-objective evolutionary optimization. The method is capable of directing evolution towards more promising search regions even if these regions are outside the initial predefined space. For problems where the global optimum is included in the initial search space, it is capable of shrinking the search space dynamically for better resolution in genetic representation to facilitate the evolutionary search towards more accurate optimal solutions. Validation results based on benchmark optimization problems show that the proposed inductive---deductive learning is capable of handling different fitness landscapes as well as distributing nondominated solutions uniformly along the final trade-offs in multi-objective optimization, even if there exist many local optima in a high-dimensional search space or the global optimum is outside the predefined search region.

Biasing, once regarded as "cheating" in the machine learning community, is now understood and accepted as a necessary part of learning. However, despite its wide acceptance and recognition, biasing has never been studied as a separate research issue, except by Hailu & Sommer (1999), who made an attempt to shed light on the relationship between the quality of the bias and learning trials. So far, the general view held in biasing a learning system is to look for a bias that maximally collapses the search space. It is well-known, however, that reckless reduction of the search space often leads to sub-optimal learning. Regardless of the final level of optimality, this paper challenges this broadily accepted biasing scheme from the point of view of accelerating the learning process itself. Is a large search space a definitive indication of slow learning? We give a non-affirmative answer to this dogma by presenting a typical robot learning scenario. Experiments clearly indicate that, in spite of its large search space, a bias that is derived from the unique characteristics of the task shows better learning behavior than a bias that reduces the search space aggressively.

Similarity search in high-dimensional vector spaces has been the subject of substantial research, motivated in part by the need to provide query support for images and other complex data types. The paper VLDB 1998 paper "Quantitative Analysis and Performance Study for Similarity-Search Methods in High-Dimensional Spaces" analyses why this search problem can be so tricky, and shows with intuitive yet formal proofs that nearest-neighbour search is fundamentally linear beyond a certain dimensionality. Consequently, the paper proposes a new, linear search structure (the VA-File) which focuses on accelerating the indispensable sequential scan with approximations and computational schemes to reduce both CPU and IO efforts. Experiments with both synthetic and image data showed -- surprisingly, at the time -- that such schemes outperform hierarchical methods in all cases where the dimensionality is greater than five. In this paper, we review that work and identify both what we got right in the paper and its impact, and also (with the benefit of hindsight) those elements of the work for which we were off the mark. The lessons learned are relevant not just to the narrow area of similarity search, but also more broadly across the fields of databases and computing.

Similarity search in high-dimensional metric spaces is routinely used in many applications including content-based image retrieval, bioinformatics, data mining, and recommender systems. Search can be accelerated by the use of an index. However, constructing a high-dimensional index can be quite expensive and may not pay off if the number of queries against the data is not large. In these circumstances, it is beneficial to construct an index adaptively , while responding to a query workload. Existing work on multidimensional adaptive indexing partitions space into orthotopes (i.e., hyperrectangular units). This approach, however, is highly ineffective in high-dimensional spaces. In this paper, we propose AV-tree: an alternative method for adaptive high-dimensional indexing that exploits previously computed distances, using query centers as vantage points. Our experimental study shows that AV-tree yields cumulative cost for the first several hundred or even thousand queries much lower than that of pre-built indices. After thousands of queries, the per-query performance of the AV-tree converges or even surpasses that of the state-of-the-art MVP-tree. Arguably, our approach is commendable in environments where the expected number of queries is not large while there is a need to start answering queries as soon as possible, such as applications where data are updated frequently and past data soon become obsolete.