Multi Hashing Low Power Bloom Filter Architectures for Network Applications

A Bloom filter is a simple space-efficient randomized data structure for representing a set in order to support membership queries. According to its space-efficient character, Bloom filter can be used in the mobile ad hoc networks (MANET), whose nodes have limited storage space and computational ability. Load balancing can distribute the traffic equally in the network after detecting the flows in each node accurately. By using Bloom Filter, single-path load balancing mechanism is used to distribute the traffic evenly among the nodes in the network without importing the route coupling problem of multi-path load balancing. Simulations show that the network performance is enhanced by single- path load balancing mechanism, and using Bloom Filter can save lots of storage space without affecting the detection accuracy.

Bloom filters are widely used in many network applications but the high computation cost limits the system performance. In this paper, we introduce a new variation of Bloom filter named One-Hashing Bloom Filter (OHBF) to solve the problem. OHBF requires only one base hash function plus a few simple operations to implement a Bloom filter. While keeping nearly the same theoretical false positive ratio as an ideal Bloom filter, OHBF significantly reduces the hash computation overhead. We show that the false positive performance of a standard Bloom filter implementation strongly relies on the selection of hash functions, even if these hash functions are considered good. In contrast, OHBF presents consistently better performance with a proven mathematical foundation. OHBF is ideal for high throughput and low latency applications. As OHBF is a fundamental technique in Bloom filter theory, it can be applied to many other Bloom filter variations, such as Counting Bloom Filter and Space-Code Bloom Filter.

Bloom filter is a simple space-efficient randomized data structure for representing a set in order to support membership queries, which uses an m-bit array to represent a data set. Dynamic bloom filter (DBF) can support concisely representation and approximate membership queries of dynamic set instead of static set. It has been proved that DBF not only possess the advantage of standard bloom filter, but also has better features when dealing with dynamic set. But DBF also has a disadvantage: the addition operation which mapped element x into bloom filter s will become no sense, if some of the first s-1 bloom filters have already responded that element x is in set A with some false positive probability. We point out this shortcoming and improve the addition operation with a new algorithm. We call this improved dynamic bloom filter i-DBF. Finally, we prove that this i-DBF has better performance both in the storage space and in the false positive probability

Bloom filter is a compact and randomized data structure for representation of a set, in order to answer membership queries; Queries such as Is element x in set Y? There is no need to have set Y, for answering these kinds of questions; these questions can be answered by constructed Bloom filter from the set. Therefore, space saving is the main advantage of using Bloom filter. In this paper, first general structure of Bloom filter is explained and then by using the Bloom filter, a new method for error management and detection in computer networks is presented. In this approach, the sender makes a Bloom filter for the flow data that is to be sent and attaches this Bloom filter to data flow and sends them to receiver. The receiver by receiving the data flow and Bloom filter, makes the Bloom filter corresponding to data flow, then by comparing this Bloom filter with the received Bloom filter detects the possible errors. The results show that Bloom filter with the length of x input string is able to detects all errors except for errors with the length of exactly 2x, 4x and... etc.

Probabilistic techniques for verification of finite-state transition systems offer huge memory savings over deterministic techniques. The two leading probabilistic schemes are hash compaction and the bitstate method, which stores states in a Bloom filter. Bloom filters have been criticized for being slow, inaccurate, and memory-inefficient, but in this paper, we show how to obtain Bloom filters that are simultaneously fast, accurate, memory-efficient, scalable, and flexible. The idea is that we can introduce large dependences among the hash functions of a Bloom filter with almost no observable effect on accuracy, and because computation of independent hash functions was the dominant computational cost of accurate Bloom filters and model checkers based on them, our savings are tremendous. We present a mathematical analysis of Bloom filters in verification in unprecedented detail, which enables us to give a fresh comparison between hash compaction and Bloom filters. Finally, we validate our work and analyses with extensive testing using 3SPIN, a model checker we developed by extending SPIN.

Receiver-oriented design of Bloom filters for data-centric routing

A new analysis of the false positive rate of a Bloom filter

deepBF: Malicious URL detection using learned Bloom Filter and evolutionary deep learning

Bloom filters are a type of data structure that is used to test whether or not an element is a member of a set. They are known for being space-efficient and are commonly employed in various applications, such as network routers, web browsers, and databases. These filters work by allowing a fixed probability of incorrectly identifying an element as being a member of the set, known as the false positive rate (FPR). However, traditional bloom filters suffer from a high FPR and extensive memory usage, which can lead to incorrect query results and a slow performance. Thus, this study indicates that a content-based strategy could be a practical solution for these challenges. Specifically, our approach requires less bloom filter storage, consequently decreasing the probability of false positives. The effectiveness of several hash functions on our strategy’s performance was also evaluated. Experimental evaluations demonstrated that the proposed strategy could potentially decrease false positives by a substantial margin of up to 79.83%. The use of size-based content bits significantly contributes to the decrease in the number of false positives as well. However, as the volume of content bits rises, the impact on time is not considerably noticeable. Moreover, the evidence suggests that the application of a singular approach leads to a more than 50% decrease in false positives.

Bloom filters are widely used in networking and computing to accelerate membership checking. In many applications filters store sensitive data, so their privacy is of primary concern. At first glance, it seems that extracting the set of elements inserted from the filter would not be possible, because in Bloom filters elements are mapped to positions using hash functions. However, previous works have shown that for the Bloom filter, it may be possible to identify few of the elements inserted in the filter. In this work, we consider the case of counting Bloom filters (CBFs) and show that in some cases, the entire set of elements used to create the filter can be extracted from the filter. This poses serious privacy and security concerns when an attacker can get access to the filter contents. In this article, an algorithm to extract the elements inserted from the filter is presented and analyzed theoretically; then, the feasibility of the CBF inversion is shown by simulation. A case study is presented in detail to illustrate that in practical applications, these conditions can be met by using additional restrictions that are implicit in the nature of the application itself.

In this paper we propose a method for learning a category-sensitive hash function (i.e. a hash function that tends to map inputs from the same category to the same hash bucket) and a feature descriptor based on the Bloom filter. Category-sensitive hash functions are robust to intra-category variation. In this paper we use them to produce descriptors that are invariant to transformations caused by for example viewpoint changes, lighting variation and deformation. Since the descriptors are based on Bloom filters, they support a ”union” operation. So descriptors of matched features can be aggregated by taking their union. We thus end up with one descriptor per keypoint instead of one descriptor per feature (By keypoint we refer to a world-space reference point and by feature we refer to an image-space interest point. Features are typically observations of keypoints and matched features are observations of the same keypoint). In short, the proposed descriptor has data-defined invariance properties due to the category-sensitive hashing and is aggregatable due to its Bloom filter inheritance. This is useful whenever we require custom invariance properties (e.g. tracking of deformable objects) and/or when we make multiple observations of each keypoint (e.g. tracking, multi-view stereo or visual SLAM).

In this paper, we consider the problem of designing a data structure that can perform fast multiple-set membership testing in deterministic time. Our primary goal is to develop a hardware implementation of the data structure that uses only embedded memory blocks. Prior efforts to solve this problem involve hashing into multiple Bloom filters. Such approach needs a priori knowledge of the number of elements in each set in order to size the Bloom filter. We use a single-Bloom-filter-based approach and use multiple sets of hash functions to code for the set (group) id. Since a single Bloom filter is used, it does not need a priori knowledge of the distribution of the elements across the different sets. We show how to improve the performance of the data structure by using constant-weight error-correcting codes for coding the group id. Using error-correcting codes improves the performance of these data structures especially when there are a large number of sets. We also outline an efficient hardware-based approach to generate the large number of hash functions that we need for this data structure. The resulting data structure, COMB, is amenable to a variety of time-critical network applications.

Approximate Membership Query (AMQ) data structures, such as the Bloom filter, quotient filter, and cuckoo filter, have found numerous applications in databases, storage systems, networks, computational biology, and other domains. However, many applications must work around limitations in the capabilities or performance of current AMQs, making these applications more complex and less performant. For example, many current AMQs cannot delete or count the number of occurrences of each input item, take up large amounts of space, are slow, cannot be resized or merged, or have poor locality of reference and hence perform poorly when stored on SSD or disk. This paper proposes a new general-purpose AMQ, the counting quotient filter (CQF). The CQF supports approximate membership testing and counting the occurrences of items in a data set. This general-purpose AMQ is small and fast, has good locality of reference, scales out of RAM to SSD, and supports deletions, counting (even on skewed data sets), resizing, merging, and highly concurrent access. The paper reports on the structure's performance on both manufactured and application-generated data sets. In our experiments, the CQF performs in-memory inserts and queries up to an order-of magnitude faster than the original quotient filter, several times faster than a Bloom filter, and similarly to the cuckoo filter, even though none of these other data structures support counting. On SSD, the CQF outperforms all structures by a factor of at least 2 because the CQF has good data locality. The CQF achieves these performance gains by restructuring the metadata bits of the quotient filter to obtain fast lookups at high load factors (i.e., even when the data structure is almost full). As a result, the CQF offers good lookup performance even up to a load factor of 95%. Counting is essentially free in the CQF in the sense that the structure is comparable or more space efficient even than non-counting data structures (e.g., Bloom, quotient, and cuckoo filters). The paper also shows how to speed up CQF operations by using new x86 bit-manipulation instructions introduced in Intel's Haswell line of processors. The restructured metadata transforms many quotient filter metadata operations into rank-and-select bit-vector operations. Thus, our efficient implementations of rank and select may be useful for other rank-and-select-based data structures.

Hashing is an important function in many applications such as hash tables, caches and Bloom filters. In past, genetic programming was applied to evolve application-specific as well as general-purpose hash functions, where the main design target was the quality of hashing. As hash functions are frequently called in various time-critical applications, it is important to optimize their implementation with respect to the execution time. In this paper, linear genetic programming is combined with NSGA-II algorithm in order to obtain general-purpose, ultra-fast and high-quality hash functions. Evolved hash functions show highly competitive quality of hashing, but significantly reduced execution time in comparison with the state of the art hash functions available in literature.

Privacy-preserving record linkage (PPRL) facilitates the matching of records that correspond to the same real-world entities across different databases while preserving the privacy of the individuals in these databases. A Bloom filter (BF) is a space efficient probabilistic data structure that is becoming popular in PPRL as an efficient privacy technique to encode sensitive information in records while still enabling approximate similarity computations between attribute values. However, BF encoding is susceptible to privacy attacks which can re-identify the values that are being encoded. In this paper we propose two novel techniques that can be applied on BF encoding to improve privacy against attacks. Our techniques use neighbouring bits in a BF to generate new bit values. An empirical study on large real databases shows that our techniques provide high security against privacy attacks, and achieve better similarity computation accuracy and linkage quality compared to other privacy improvements that can be applied on BF encoding.