Optimizing bitmap index encoding for high performance queries

When using an out-of-core indexing method to answer a query, it is generally assumed that the I/O cost dominates the overall query response time. Because of this, most research on indexing methods concentrate on reducing the sizes of indices. For bitmap indices, compression has been used for this purpose. However, in most cases, operations on these compressed bitmaps, mostly bitwise logical operations such as AND, OR, and NOT, spend more time in CPU than in I/O. To speedup these operations, a number of specialized bitmap compression schemes have been developed; the best known of which is the byte-aligned bitmap code (BBC). They are usually faster in performing logical operations than the general purpose compression schemes, but, the time spent in CPU still dominates the total query response time. To reduce the query response time, we designed a CPU-friendly scheme named the word-aligned hybrid (WAH) code. In this paper, we prove that the sizes of WAH compressed bitmap indices are about two words per row for large range of attributes. This size is smaller than typical sizes of commonly used indices, such as a B-tree. Therefore, WAH compressed indices are not only appropriate for low cardinality attributes but also for high cardinality attributes.In the worst case, the time to operate on compressed bitmaps is proportional to the total size of the bitmaps involved. The total size of the bitmaps required to answer a query on one attribute is proportional to the number of hits. These indicate that WAH compressed bitmap indices are optimal. To verify their effectiveness, we generated bitmap indices for four different datasets and measured the response time of many range queries. Tests confirm that sizes of compressed bitmap indices are indeed smaller than B-tree indices, and query processing with WAH compressed indices is much faster than with BBC compressed indices, projection indices and B-tree indices. In addition, we also verified that the average query response time is proportional to the index size. This indicates that the compressed bitmap indices are efficient for very large datasets.

Spatial database is an essential component of Geographic Information System (GIS). With the development of modern remote sensors and data acquiring instruments, the amount of spatial data increases with geometric series. Retrieval required data in such massive database is a challenging issue to database engineers. Therefore, building efficient index is significant to spatial database. In this paper, bitmap index technology, which is rarely used in spatial database, is taken into consideration. In this paper, TM/ETM+ images covered main land of China are selected to establish a spatial database. In order to rapidly inquire and retrieval required data from the spatial database, an effective spatial index is very important. A bitmap index solution for original TM/ETM+ image is advanced in this paper. The bitmap index schema, which indexes field by building 0 and 1 binary bit vectors, is designed based on analyzing its principle and applicable conditions. In the image database, the strategy is programmed to implementation and applied for data retrieval. So that complex querying operations can be transformed to bitwise logical operations. A users' interface is developed based on building such bitmap index for original TM/ETM+ images database. Then the paper probes into the bitmap index update mechanism to address problems resulted from inserting and deleting images operation. In order to address the problem of high cardinality, an encoded bitmap index technology is proposed as well. At last, a simple comparison and efficiency analysis is carried out to illuminate its applicability.

On the Performance of Bitmap Indices for High Cardinality Attributes

Complete college finance and admin go under micro management

We present an efficient index structure to access spatial data. The proposed index, called SBS (Spatial Bit-Sliced), is based on the Bit-Sliced indexing method. The Bit-Sliced index has proven to be an efficient data structure for indexing one-dimensional data. The Bit-Sliced is an extension of bitmap indexing used to access data elements with high cardinality. Bitmap indexing improves I/O performance as well as saving storage by using single bits instead of multiple bytes of data to indicate a specific value of data. We present bulk loading, insertion, and deletion algorithms for SBS index structure. Using TIGER data files, we show the performance improvement gained by using the proposed SBS indexing in doing the bulk loading, insertion, and deletion operations over using R*-tree.

NoSQL and relational databases have been integrated in response to the increasing demand for scalable and efficient data management in hybrid cloud environments. The differences in data structures and query processing methods between these databases present both challenges and opportunities when designing an optimized hybrid system. This study explores the integration of NoSQL and relational databases to maximize data storage, retrieval, and processing efficiency across multiple cloud systems. NoSQL databases excel in handling unstructured and semi-structured data, offering flexibility and scalability, whereas relational databases provide robust consistency and structured query capabilities. Despite the wide availability of relational databases, certain applications require the dynamic adaptability of NoSQL systems, making integration a viable solution. The research evaluates key performance parameters, including query execution speed, scalability, data consistency, and resource utilization. Cloudsim is used for simulation, allowing for an in-depth comparison between standalone and hybrid database models. Experimental results indicate that the proposed hybrid model improves query performance by 30%, reduces latency by 20%, and enhances scalability by 40% compared to using relational or NoSQL databases alone. The novelty of this approach lies in its ability to overcome the limitations of each database type by their strengths in a dynamic cloud environment. The results highlight the effectiveness of hybrid database integration in optimizing cloud-based data management while ensuring seamless operation, adaptability, and resource efficiency.

The next generation of sensor node such as Lotus mote enables higher performance, low power consumption, higher storage/memory and higher speed processing capability than the older generation, such as TelosB or MicaZ motes, which facilitates multimedia data pre-processing and compression in wireless multimedia sensor network (WMSN). In WMSNs, multimedia compression is the main issue that performed on a multimedia sensor node because the complexity of multimedia coding can cause a major energy consumption problem. This paper proposes an optimal complexity H.264 encoding for video streaming over next generation of WMSN. It develops the mathematical model for achieving optimal complexity encoding and proposed the most reliable H.264 encoder configuration setting on H.264 coding. The proposed mechanism ensures an optimal encoding complexity, less power consumption and visual quality of the multimedia. The experimental results of this research give an evidence to verify that the proposed mechanism guarantees high QoS performance.

EDF (Earliest Deadline First) has been proved to be optimal scheduling algorithm for single processor real-time system. It also performs well for multiprocessor system. Limitation of EDF is that its performance decreases exponentially when system becomes slightly overloaded. ACO (Ant Colony Optimization) based scheduling algorithm performs well in both underloaded and overloaded conditions. But its limitation is that it takes more time for execution compared to EDF. In this paper, an adaptive algorithm for multiprocessor real-time system is proposed, which is combination of both of these algorithms. The proposed algorithm along with EDF and ACO based algorithm is simulated for real-time multiprocessor system and the results are obtained. The performance is measured in terms of Success Ratio (SR) and Effective CPU Utilization (ECU). Execution Time taken by each scheduling algorithm is also measured. From analysis and experiments, it reveals that the proposed algorithm is fast as well as efficient in both underloaded and overloaded conditions for real-time multiprocessor systems.

This paper deals with the solution of practical nonlinear optimization problems which arise especially in the fields of engineering design and production planning. In their mathematical description these problems frequently consist of highly nonlinear objective functions and constraints whose evaluation may lead to time-consuming simulation runs. In these cases assumptions about unimodality, convexity, and smoothness of well-known solution methods in nonlinear optimization are mostly invalid. An intuitive approach to overcoming these problems would be to apply a simultaneous combination of different optimization algorithms. In this situation the basis for a more reliable and even faster solution is the controlled information exchange on the continuing solution progress between the participating and parallel running methods. This idea has been implemented on a multiprocessor system with distributed memory for the solution of nonlinear optimization problems. Based on previous experiences with different algorithms, a coarse-grained parallelization approach under asynchronous control has been developed.

A Web-Based Perioperative Dashboard as a Platform for Anesthesia Informatics Innovation.

Many relational databases exhibit complex dependencies between data attributes, caused either by the nature of the underlying data or by explicitly denormalized schemas. In data warehouse scenarios, calculated key figures may be materialized or hierarchy levels may be held within a single dimension table. Such column correlations and the resulting data redundancy may result in additional storage requirements. They may also result in bad query performance if inappropriate independence assumptions are made during query compilation. In this paper, we tackle the specific problem of detecting functional dependencies between columns to improve the compression rate for column-based database systems, which both reduces main memory consumption and improves query performance. Although a huge variety of algorithms have been proposed for detecting column dependencies in databases, we maintain that increased data volumes and recent developments in hardware architectures demand novel algorithms with much lower runtime overhead and smaller memory footprint. Our novel approach is based on entropy estimations and exploits a combination of sampling and multiple heuristics to render it applicable for a wide range of use cases. We demonstrate the quality of our approach by means of an implementation within the SAP NetWeaver Business Warehouse Accelerator. Our experiments indicate that our approach scales well with the number of columns and produces reliable dependence structure information. This both reduces memory consumption and improves performance for nontrivial queries.

Dynamic Voltage Scaling (DVS) is one of the techniques used to obtain energy-saving in real-time DSP systems. In many DSP systems, some tasks contain conditional instructions that have different execution times for different inputs. Due to the uncertainties in execution time of these tasks, this paper models each varied execution time as a probabilistic random variable and solves the Voltage Assignment with Probability (VAP) Problem. VAP problem involves finding a voltage level to be used for each node of an date flow graph (DFG) in uniprocessor and multiprocessor DSP systems. This paper proposes two optimal algorithms, one for uniprocessor and one for multiprocessor DSP systems, to minimize the expected total energy consumption while satisfying the timing constraint with a guaranteed confidence probability. The experimental results show that our approach achieves significant energy saving than previous work. For example, our algorithm for multiprocessor achieves an average improvement of 56.1% on total energy-saving with 0.80 probability satisfying timing constraint.

This paper presents an optimal algorithm to solve the schedule compression problem, which is an open problem proposed by Trimberger [1] for time-multiplexed FPGA partitioning. Time-multiplexed FPGAs have the potential to dramatically improve logic density by time-sharing logic. Schedule compression is an important step in partitioning for time-multiplexed FPGAs [1,4,9,10] and can greatly influence the quality of the partitioning solution. We exactly solve the schedule compression problem by converting it to a constrained min-max path problem. We further extend our algorithm to minimize the communication cost during schedule compression. Experiments show that our optimal algorithm outperforms the existing heuristics and runs very efficiently.

In general, the hardware memory consistency model in a multiprocessor system is not identical to the memory model at the programming language level. Consequently, the programming language memory model must be mapped onto the hardware memory model. Memory fence instructions can be inserted by the compiler where needed to accomplish this mapping. We have developed and implemented several fence insertion and optimization algorithms in our Pensieve compiler project. We present the different fence insertion optimization techniques that were used in this system to guarantee sequential consistency at the language level, and compare them using performance data. Our techniques target two hardware relaxed memory consistency models provided by SMPs based on IBM Power 3 and Intel Pentium 4. Our fence insertion optimization shows up to 17.2% and 32.7% performance improvement on average, with the IBM PowerPC and Intel Pentium 4 (Xeon) multiprocessors respectively.

The rapid growth of smart grid systems has resulted in an exponential increase in data volume. This has led to significant challenges for efficient storage and transmission, thereby necessitating the development of advanced data compression techniques. Traditional techniques often struggle to maintain a balance between compression efficiency and data integrity, particularly when dealing with diverse and large datasets. To address this issue, this paper presents an adaptive data compression algorithm based on discrete wavelet transform (DWT) and Horse Herd Optimization (HHO). The proposed technique significantly improves storage by dynamically optimizing key performance metrics, such as signal-to-noise ratio (SNR), mean square error (MSE), and compression ratio (CR). By employing a robust optimization algorithm, it effectively addresses the computational challenges of processing large-scale and real-time data. This adaptability ensures the algorithm is well-suited for dynamic smart grid environments, providing a scalable and reliable solution to modern data management demands. This technique uses HHO to find the optimal thresholding for smart grid data compression. In general, DWT-based data compression is carried out using a universal threshold for ignoring particular wavelet coefficients. But, the performance of data compression varies for different threshold values. Hence, selecting an optimal threshold is a challenging task for data compression. Therefore, to solve this issue, an effective optimization algorithm is needed. In this work, a Multi-Objective Horse Herd Optimization (MO-HHO) algorithm is proposed to find the optimal threshold. The suggested MO-HHO algorithm accurately determines the global optimum threshold. Hence, it maintains a good compromise between SNR, MSE and CR. The effectiveness of the proposed algorithm is examined using three different data sets. Various datasets from the IEEE Power quality wave data, Household Electric Power Consumption data and Real-time dataset from an experimental set-up were used to test the proposed algorithm. The outcome demonstrates that the suggested MO-HHO algorithm performs better than the other conventional methods.