Fast Dynamic Multiple-Set Membership Testing Using Combinatorial Bloom Filters

Approximate Membership Query (AMQ) data structures, such as the Bloom filter, quotient filter, and cuckoo filter, have found numerous applications in databases, storage systems, networks, computational biology, and other domains. However, many applications must work around limitations in the capabilities or performance of current AMQs, making these applications more complex and less performant. For example, many current AMQs cannot delete or count the number of occurrences of each input item, take up large amounts of space, are slow, cannot be resized or merged, or have poor locality of reference and hence perform poorly when stored on SSD or disk. This paper proposes a new general-purpose AMQ, the counting quotient filter (CQF). The CQF supports approximate membership testing and counting the occurrences of items in a data set. This general-purpose AMQ is small and fast, has good locality of reference, scales out of RAM to SSD, and supports deletions, counting (even on skewed data sets), resizing, merging, and highly concurrent access. The paper reports on the structure's performance on both manufactured and application-generated data sets. In our experiments, the CQF performs in-memory inserts and queries up to an order-of magnitude faster than the original quotient filter, several times faster than a Bloom filter, and similarly to the cuckoo filter, even though none of these other data structures support counting. On SSD, the CQF outperforms all structures by a factor of at least 2 because the CQF has good data locality. The CQF achieves these performance gains by restructuring the metadata bits of the quotient filter to obtain fast lookups at high load factors (i.e., even when the data structure is almost full). As a result, the CQF offers good lookup performance even up to a load factor of 95%. Counting is essentially free in the CQF in the sense that the structure is comparable or more space efficient even than non-counting data structures (e.g., Bloom, quotient, and cuckoo filters). The paper also shows how to speed up CQF operations by using new x86 bit-manipulation instructions introduced in Intel's Haswell line of processors. The restructured metadata transforms many quotient filter metadata operations into rank-and-select bit-vector operations. Thus, our efficient implementations of rank and select may be useful for other rank-and-select-based data structures.

A Bloom filter is a simple space-efficient randomized data structure for representing a set in order to support membership queries. According to its space-efficient character, Bloom filter can be used in the mobile ad hoc networks (MANET), whose nodes have limited storage space and computational ability. Load balancing can distribute the traffic equally in the network after detecting the flows in each node accurately. By using Bloom Filter, single-path load balancing mechanism is used to distribute the traffic evenly among the nodes in the network without importing the route coupling problem of multi-path load balancing. Simulations show that the network performance is enhanced by single- path load balancing mechanism, and using Bloom Filter can save lots of storage space without affecting the detection accuracy.

Bloom filter is a simple space-efficient randomized data structure for representing a set in order to support membership queries, which uses an m-bit array to represent a data set. Dynamic bloom filter (DBF) can support concisely representation and approximate membership queries of dynamic set instead of static set. It has been proved that DBF not only possess the advantage of standard bloom filter, but also has better features when dealing with dynamic set. But DBF also has a disadvantage: the addition operation which mapped element x into bloom filter s will become no sense, if some of the first s-1 bloom filters have already responded that element x is in set A with some false positive probability. We point out this shortcoming and improve the addition operation with a new algorithm. We call this improved dynamic bloom filter i-DBF. Finally, we prove that this i-DBF has better performance both in the storage space and in the false positive probability

One of the most common threats to internet security is the Denial of Service attack. There are numerous methods and protocol changes in an effort to detect them. The most common DoS attack detection method relies on the number of the TCP control packet in the network flow. The Bloom filter represents a space efficient data structure that is commonly utilized to detect matching pairs. There are multiple algorithms for the DoS attack detection based on the Bloom filter. The SACK <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> algorithm uses the SYN/ACK - ACK matching pair detection with the Bloom filter data structure. The false positive error introduced by the Bloom filter influences on the matching pair detection in the algorithm. The improved SACK <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> algorithm significantly reduces the false positive error by replacing the Counting Bloom Filter (CBF) data structure with the Dual Counting Bloom Filter (DCBF) data structure. This improvement significantly improves the matching pair detection. It is expected that the false positive error should influence on the detection of the DoS attack. In this paper, the experimental study is performed to analyse this influence. This study confirms the correlation between the false positive error and the DoS attack detection.

Hierarchical packet classification using a Bloom filter and rule-priority tries

A Bloom filter is a simple space-efficient randomized data structure for representing a set in order to support network and database query systems. Although Bloom filters were invented in the 1970's and have been heavily used in database applications, they have only recently received widespread attention in the networking literature. Software applications frequently fail to identify so many signatures through comparisons at very high speeds. We introduce a low power bloom filter architectures which is space and power effective in hardware platforms. Instead of working on programming phase or technology, our work concentrated on look up techniques of Bloom filters. We have chosen better hash function to be utilized in hardware applications of Bloom filter by FPGA implementation. Power analysis shows that decrement in number of hash functions per stage results in power gain.

Bloom filter is a compact and randomized data structure for representation of a set, in order to answer membership queries; Queries such as Is element x in set Y? There is no need to have set Y, for answering these kinds of questions; these questions can be answered by constructed Bloom filter from the set. Therefore, space saving is the main advantage of using Bloom filter. In this paper, first general structure of Bloom filter is explained and then by using the Bloom filter, a new method for error management and detection in computer networks is presented. In this approach, the sender makes a Bloom filter for the flow data that is to be sent and attaches this Bloom filter to data flow and sends them to receiver. The receiver by receiving the data flow and Bloom filter, makes the Bloom filter corresponding to data flow, then by comparing this Bloom filter with the received Bloom filter detects the possible errors. The results show that Bloom filter with the length of x input string is able to detects all errors except for errors with the length of exactly 2x, 4x and... etc.

The Bloom filter is a data structure which the characteristics of contents is expressed by a bit pattern. The Bloom filter has become attractive as one of the methods of looking up data in the distributed system. Since two or more keywords can be used in the method based on the Bloom filter, it is more flexible than the distributed hash table (DHT). Especially, the key aspect of the structured Bloom filter is that the number of query forwarding is fixed. In the previous research, the ring of the Bloom filter, which is like Chord ring, was proposed. However, the node in the ring must maintain many filters corresponding to the number of nodes. In this paper, we propose the tree structured Bloom filter and evaluated the performance of the query forwarding. We also propose similarity-based tree management method which is effective to reduce the cost of reconstruction of the tree. To evaluate our method, the size of filter information and the number of average hops of query forwarding in this research are compared to the existing research. The tree reconstruction cost is also evaluated by simulation.

In many networking systems, Bloom filters are used for high-speed set membership tests. They permit a small fraction of false positive answers with very good space efficiency. However, they do not permit deletion of items from the set, and previous attempts to extend standard Bloom filters to support deletion all degrade either space or performance. We propose a new data structure called the cuckoo filter that can replace Bloom filters for approximate set membership tests. Cuckoo filters support adding and removing items dynamically while achieving even higher performance than Bloom filters. For applications that store many items and target moderately low false positive rates, cuckoo filters have lower space overhead than space-optimized Bloom filters. Our experimental results also show that cuckoo filters outperform previous data structures that extend Bloom filters to support deletions substantially in both time and space.

MANET, a self-organizing, infrastructure-less, wireless network is a fast-growing technology in day-to-day life. There is a rapid growth in the area of mobile computing due to the extent of economical and huge availability of wireless devices which leads to the extensive analysis of the mobile ad-hoc network. It consists of the collection of wireless dynamic nodes. Due to this dynamic nature, the routing of packets in the MANET is a complex one. The integration of distributed hash table (DHT) in MANET is performed to enhance the overlay of routing. The node status updating in the centralized hash table creates the storage overhead. The bloom filter is a data structure that is a space-effective randomized one but it allows the false-positive rates. However, this can be able to compensate for the issue of storage overhead in DHT (Distributed hash table). Hence, to overcome the storage overhead occurring in DHT, and reduce the false positives, the Bloom's filter is integrated with the DHT initially. Furthermore, the link stability is measured by the distance among mobile nodes. The optimal node selection should be done for the transmission of packets which is the lacking factor. If it fails to select the optimal path then the removal of malicious nodes may lead to the unwanted entry of nodes into the other clustering groups. Therefore, to solve this problem, the bloom's filter is modified for enhancing the link stability. The novelty of this proposed work is the integration of Bloom's filter with the Distributed Hash Table which provides good security on transmission data by removing false-positive errors and storage overhead.

As a key component in implementing Named Data Networking (NDN), Pending Interest Table (PIT) requires an efficient exact-matching algorithm for a scalable and fast PIT lookup. A Bloom filter (BF) is a memory-efficient data structure for performing exact matching operations. In this paper, three different BF-based PIT architectures are proposed: PIT using functional Bloom filters (FBF-PIT), PIT using counting Bloom filters with return values (rCBF-PIT), and a refined rCBF-PIT with signatures (R-rCBF-PIT). The proposed BF-based PITs incrementally allocate a new BF for storing multiple incoming faces of Interest packets with the same content name. For a Data packet lookup, the proposed PIT architectures simultaneously access every BF structure to find matching faces and delete the faces (i.e., matching Interest packet information). The functional Bloom filter (FBF) used in an FBF-PIT is a key-value data structure that stores values only without keys. However, because the number of non-reusable conflict cells in the FBF increases as the number of stored packets increases in the FBF-PIT, the indeterminable rate increases. To decrease the indeterminable rate, we propose the rCBF-PIT, which uses counting Bloom filters with return values (rCBFs), allowing reusable conflict cells. False positives for Interest packets lead to incorrect deletions that can cause false negatives for incoming Data packets. Because most of the false positives occur in the first BF structure, we finally propose the R-rCBF-PIT, in which the first rCBF is replaced with an rCBF with a signature field. The proposed PITs also provide an aging mechanism using a valid bit and a hit bit for entry expiration. Simulation results show that rCBF-PIT and R-rCBF-PIT both reduce the indeterminable rate by more than 81% compared with FBF-PIT. The results also show that R-rCBF-PIT resolves false negatives caused by incorrect deletions by including the signature fields in the first rCBF.

Robust image hashes are used to detect known illegal images, even after image processing. This is, for example, interesting for a forensic investigation, or for a company to protect their employees and customers by filtering content. The disadvantage of robust hashes is that they leak structural information of the pictures, which can lead to privacy issues. Our scientific contribution is to extend a robust image hash with privacy protection. We thus introduce and discuss such a privacy-preserving concept. The approach uses a probabilistic data structure - known as Bloom filter - to store robust image hashes. Bloom filter store elements by mapping hashes of each element to an internal data structure. We choose a cryptographic hash function to one-way encrypt and store elements. The privacy of the inserted elements is thus protected. We evaluate our implementation, and compare it to its underlying robust image hashing algorithm. Thereby, we show the cost with respect to error rates for introducing a privacy protection into robust hashing. Finally, we discuss our approach's results and usability, and suggest possible future improvements.

Bloom filters are probabilistic data structures that are popular in networking for set representation; however, they show an inherent inaccuracy due to false positives. One of the potential attacks on Bloom filters is to pollute them with elements that cause the filter to have a larger false positive probability than under normal operation; Pollution is simple when an attacker knows the details of the filter implementation. Recent research has shown that also black-box adversaries can pollute a counting Bloom filter (a common variant of the filter that also supports removals) with no knowledge of its implementation. As over time, many variants and improvements of Bloom filters have been proposed, it is of interest to study whether they can also be polluted and if so also the increase in their false positive probability. This paper first proposes and then evaluates pollution attacks for some of the most common variants including the Block Bloom filters (BBFs), the Variable Increment and Fingerprint Counting Bloom filters (VI-CBFs and FP-CBFs). The results show that with or without knowledge of the implementation, these variants of the Bloom filter are significantly more vulnerable to pollution attacks than the traditional Bloom filter. In particular, BBFs are extremely vulnerable, so providing an insight on their impact and use in practical systems when the number of memory accesses per lookup must be reduced.

A Bloom filter is a simple data structure that identifies the membership of an input against a given set. Various types of Bloom filters have been widely used in recent years. While standard Bloom filters only provide the membership identification of a given set by storing a value of 0 or 1 in a cell, we propose to provide the membership information of multiple sets through a single Bloom filter by storing different values in a cell. In other words, if two different sets are given, the proposed Bloom filter structure allocates 2 bits in one cell, and two different values indicating each set are specified in advance to program the sets. Hence the proposed Bloom filter structure accurately determines the membership of each set and the intersection of two sets by querying a single Bloom filter. In addition, we propose a circled Bloom filter structure to improve the accuracy of the membership identification. Experimental results show that the proposed Bloom filter structure provides the better accuracy with using the half of querying operations compared to two separate Bloom filter structure.

Content Centric Networking (CCN) is a novel network paradigm in which the communication process focusses on content rather than the host. In the current network architecture, routers forward the incoming packets based on the routing information that already had. Whereas, forwarding decisions in CCN are taken depending on networks situation. The contents in CCN are retrieved depending on their names which are URL like names that are composing of a number of string parts separated by’/’. These names are stored inside a specific data structure, called Forwarding Information Base (FIB), inside CCN routers and they are used to forward any incoming packet. Therefore, the main two challenges that face the design of FIB table are: search speed and storage utilization. Consequently, in this paper, we propose a new name matching technique (named BF-QF FIB) to design and implement a FIB table in CCN routers in order to decrease the storage utilization and to increase the lookup speed. This technique utilizes two kinds of query data structures: Bloom filter (BF) and Quotient filter (QF) as its main data structures. The utilization of these two data structures will ensure low memory usage and high lookup speed. The results of our evaluation show that BF-QF FIB can guarantee high search rate and offer perfect scalability to large FIB tables.