A Constraint Optimization Method for Large-Scale Distributed View Selection

On-Line Analytical Processing (OLAP) tools are frequently used in business, science and health to extract useful knowledgefrom massive databases. An important and hard optimization problem in OLAP data warehouses is the view selection problem, consisting of selecting a set of aggregate views of the data for speeding up future query processing. A common variant of the view selection problem addressed in the literature minimizes the sum of maintenance cost and query time on the view set. Converting what is inherently an optimization problem with multiple conflicting objectives into one with a single objective ignores the need and value of a variety of solutions offering various levels of trade-off between the objectives. We apply two non-elitist multiobjective evolutionary algorithms (MOEAs) to view selection under a size constraint. Our emphasis is to determine the suitability of the combination of MOEAs with constraint handling to the view selection problem, compared to a widely used greedy algorithm. We observe that the evolutionary process mimics that of the greedy in terms of the convergence process in the population. The MOEAs are competitive with the greedy on a variety of problem instances, often finding solutions dominating it in a reasonable amount of time.

Developing and exploiting flexible techniques for optimizing the evaluation of queries over loosely structured data (e.g. tree or graph databases) is of crucial importance for modern database applications. In this context, we consider a new type of views which can be materialized as compressed bitmaps over tree data. We introduce the concept of view structural template to define classes of views. We then define and address a novel view selection problem (called view class selection (VCS) problem) where the goal is to select classes of bitmap views in order to optimize the overall evaluation cost of all tree pattern queries (TPQs) that can be issued against a database while satisfying a space constraint and ensuring that all the TPQs can be answered using exclusively the materialized views. We show that the VCS problem is NP-hard and we design two heuristic greedy algorithms which iteratively generate new batches of candidate view classes and make them available for selection. Each algorithm uses a different view class expansion technique to enable the systematic generation of candidate view classes from classes with smaller templates. We run extensive experiments to evaluate both the effectiveness of the algorithms and their efficiency on real, benchmark and synthetic datasets. Our algorithms are able to suggest high quality selections of view classes in a reasonable amount of time.

Materialized view selection under the maintenance time constraint

Nowadays, materialized views are increasingly being supported by a variety of commercial DBMS to speed up query response time. This technique is also very useful in dataware-housing for optimizing OLAP queries. However, most existing view selection methods are of static nature. Moreover, none of these methods have considered the problem of dematerializing the previously materialized views. This paper deals with the problem of dynamic view selection and the pending issue of removing materialized views in order to replace the less beneficial views with more beneficial views. More precisely, we have designed and implemented a view selection method, including a polynomial algorithm, to decide which views to materialize according to statistic metadata.

Efficient 3D space sampling to represent an underlying 3D object/scene is essential for 3D vision, robotics, and beyond. A standard approach is to explicitly sample a dense collection of views and formulate it as a view selection problem, or, more generally, a set cover problem. In this paper, we introduce a novel approach that avoids dense view sampling. The key idea is to learn a view prediction network and a trainable aggregation module that takes the predicted views as input and outputs an approximation of their generic scores (e.g., surface coverage, viewing angle from surface normals). This methodology allows us to turn the set cover problem (or multi-view representation optimization) into a continuous optimization problem. We then explain how to effectively solve the induced optimization problem using continuation, i.e., aggregating a hierarchy of smoothed scoring modules. Experimental results show that our approach arrives at similar or better solutions with about 10 x speed up in running time, comparing with the standard methods.

The data warehouse is a repository of information collected from multiple possibly heterogeneous autonomous distributed databases. The information stored at the data warehouse is in form of views referred to as materialized views. The selection of the materialized views is one of the most important decisions in designing a data warehouse. Materialized views are stored in the data warehouse for the purpose of efficiently implementing on-line analytical processing queries. The first issue for the user to consider is query response time. So in this paper, we develop algorithms to select a set of views to materialize in data warehouse in order to minimize the total view maintenance cost under the constraint of a given query response time. We call it query_cost view_ selection problem. First, cost graph and cost model of query_cost view_ selection problem are presented. Second, the methods for selecting materialized views by using random algorithms are presented. The genetic algorithm is applied to the materialized views selection problem. But with the development of genetic process, the legal solution produced become more and more difficult, so a lot of solutions are eliminated and producing time of the solutions is lengthened in genetic algorithm. Therefore, improved algorithm has been presented in this paper, which is the combination of simulated annealing algorithm and genetic algorithm for the purpose of solving the query cost view selection problem. Finally, in order to test the function and efficiency of our algorithms experiment simulation is adopted. The experiments show that the given methods can provide near-optimal solutions in limited time and works better in practical cases. Randomized algorithms will become invaluable tools for data warehouse evolution.

This paper presents a shuffled frog leaping algorithm (SFLA) based solution to solve the View Selection Problem (VSP) subject to dual constraints, which is often used to accelerate data warehouse queries. Since VSP is both discrete and constrained, a greedy-repaired strategy under dual constraints is proposed to handle unfeasible solutions. This proposed solution also profits from a mutation strategy in order to improve the quality of solutions, particularly to avoid being trapped in local optima. Experimental results show that under different constraints combinations, SFLA is able to find a near-optimal feasible solution, with maximum error less than 1%. Comparisons with GA and PSO show that SFLA has better solution quality and faster convergence rate, and also scales with the problem size.

We consider the problem of next-best view selection for volumetric reconstruction of an object by a mobile robot equipped with a camera. Based on a probabilistic volumetric map that is built in real time, the robot can quantify the expected information gain from a set of discrete candidate views. We propose and evaluate several formulations to quantify this information gain for the volumetric reconstruction task, including visibility likelihood and the likelihood of seeing new parts of the object. These metrics are combined with the cost of robot movement in utility functions. The next best view is selected by optimizing these functions, aiming to maximize the likelihood of discovering new parts of the object. We evaluate the functions with simulated and real world experiments within a modular software system that is adaptable to other robotic platforms and reconstruction problems. We release our implementation open source.

A Data warehouse is a structure that stores big amount of data. This data is exploited in the best possible ways in order to improve the efficiency of decision-making. The huge volume of data makes answering queries complex and time-consuming. Therefore, materialized views are used in order to reduce the query processing time. Since materializing all views is not possible, due to space and maintenance constraints, materialized view selection became one of the crucial decisions in designing a data warehouse for optimal efficiency. In this paper, we propose a discrete - quantum based - version of Differential Evolution DE algorithm named QDE algorithm to solve the materialized view selection (MVS) problem with space constraint. This algorithm is a merging of the original DE with Quantum Evolutionary (QEA) algorithm. The experimental results show the efficiency of the proposed algorithm compared to well known algorithms used to solve MVS problem with space constraint such as HRUA and GEA.

Most of the existing distributed feature selection schemes neglect how good the subsets are that are mapped to the computational nodes, which causes a waste of time and hardware resources. A distributed integrated feature selection scheme (DIFS) with Subset Quality Evaluation (SQE) is proposed. SQE studies the relevance between the quality of a subset and the number of selected features from this subset, which helps shorten the feature selection time efficiently. We have given the implementation of our scheme for the Column Subset Selection (CSS) problem. We integrate a CSS algorithm in DIFS and information entropy as the SQE metric. We prove that the speedup of DIFS can reach m^3 compared to the centralized algorithm in ideal situations where m is the number of computational nodes, and give a well bounded approximation guarantee of the solution for CSS problem. Extensive experiments on eight data sets are used to verify the performance of scheme. Experiments results demonstrate the effectiveness of SQE and the impressive speedup DIFS can achieve. Although there is a slight increase of the reconstruction error value in some situations. Additional experiments of classification tasks reveal that the performance of DIFS is better than existing state-of-the-art distributed algorithms.

The paper deals with the problem of the optimal selection of the real views used for the virtual view synthesis in the free navigation systems. The practical free navigation system requires the real-time view synthesis. Therefore, the virtual view cannot be synthesized using the information from all the real cameras, but only few, most proper ones. In the paper the method of fast selection of two proper real views is presented. The proposed approach allows to select the real views for different camera arrangements, thus can be applied to the practical free navigation systems.

Modern enterprise computing environments require integration architectures to meet both business and sustainability goals. MuleSoft's vision for API-led connectivity achieves these goals through a three-level integration architecture: System APIs, Process APIs, and Experience APIs aligned with the API-led connectivity architecture process model. The layering supports the principles of re-use, loose coupling, and modularization, as well as preventing duplicate processing in the enterprise integration portfolio. Event-driven architectural and connector-based microservices integration support asynchronous communication and independent scalability, and improve the resilience of the solution, compared to synchronous integration. In more complex scenarios, streaming data transformations, connection pooling, batched processing, and response caching can save CPU, memory, and network bandwidth, and coupling these with an event-based model (as opposed to periodic polling) avoids performing business logic for new events when there are none, thus saving computational resources. Cloud-native deployment models with container-based orchestration can achieve economies of scale through higher resource utilization, and, eventually, access to more renewable resources, as seen with hyperscale cloud providers. Through API reusability and consolidation of infrastructure, entire areas of duplicate functionality across enterprise portfolios can be avoided. Specification-first, performance testing and capacity management, monitoring, lifecycle management, and good practices ensure that APIs and architectures can perform well and with low levels of technical debt while supporting the sustainability goals of the organization.

A system's ability to precisely locate itself in a known physical environment is key to its capacity to interact with the environment in an intricate manner. The indoor localisation problem has been approached in a variety of ways, ranging from the identification of pre-defined features or topologies to the more general cloud-point matching. Cloud point matching can be achieved using a variety of algorithms, each with benefits and drawbacks. Recent improvements have focused on the application of genetic algorithms to solve the initial 'global' search for a solution, before refining this solution to a precise position through a non-genetic algorithm. This project aims to demonstrate the inefficiency of genetic algorithms applied to the global search problem for the issue of indoor localisation; this is thought to be caused by the solution space's low dimensionality, solution landscape topology and the inefficacy of crossover operators in the algorithm. Based on our assumptions of map topologies, we conclude that significant redundancies can be found in some purely genetic heuristics and suggest further development of landscape analysis to allow the use of algorithms appropriate to the scenario's complexity.

Video delivery from a server to a client across a network is an important component of many multimedia applications. While delivering a video stream across a resource constrained network, loss of frames may be unavoidable. Under such circumstances, it is desirable to find a server transmission schedule that can efficiently utilize the network resources while maximizing the perceived quality-of-service (QoS) at the client. To address this issue, we introduce the notion of selective frame discard at the server and formulate the optimal selective frame discard problem using a QoS based cost function. Given network bandwidth and client buffer constraints, we develop an O(N log N) algorithm to find the minimum number of frames that must be discarded in order to meet these constraints. The correctness of the algorithm is also formally established. Since the computational complexity of the optimal algorithm for solving the optimal selective frame discard problem is prohibitively high in general, we also develop several efficient heuristic algorithms for selective frame discard. These algorithms are evaluated using JPEG video traces.

The non-regular, incomplete and singular differential dynamical systems are discussed from a geometric point of view. The constitutive space of a dynamical system is considered as a subset of the tangent bundle to the manifold (the generalised hypersurface) being the ambient space of the system. The system is the essentially non-regular differential dynamical system if there is observed reduction of the configuration space to the 'proper' configuration space of the system. The dynamical system, which is not observed by the solutions in the entire configuration space, is called the incomplete system. The system such that the dynamic space (the solution space) is not the topologically closed subspace of the ambient space of the system is called the singular dynamical system. The points in the topological closure of the configuration space that are not contained in the solution space of the dynamical system are called the impasse points of the system. The points in the topological closure of the solution space that are not contained in the solution space of the system are called the singularity points of the dynamical system. The systematic classification of the impasse points in dynamical systems is included which extends the classification proposed for the systems defined by the differential algebraic equations. The well posed and the relatively well-posed dynamical systems are defined. These are the systems, which are considered most often in the modelling of real systems. The classification which includes the non-regular, the incomplete, the well posed, and the relatively well-posed dynamical systems is proposed. The trajectory available impasse points have been extended for the general differential dynamical systems. The trajectory available impasse points are important notion in the modelling and in the simulation of real systems which exhibit the discontinuous trajectory behaviour. At the time when the trajectory of the system approached the singularity point, one usually observes the discontinuity in the dynamic behaviour of the system or the 'switching' of the trajectories which is possibly continuous takes place in the system. The examples are enclosed which illustrate the considerations.