Software-based implementations of updateable data structures for high-speed URL matching

The designs and implementations of efficient aggregate data structures have been important issues in functional programming. It is not clear how to select a good representation for an aggregate when access patterns to the aggregate are highly variant, or even unpredictable. Previous approaches rely on compile--time analyses or programmer annotations. These methods can be unreliable because they try to predict a program's behavior before it is executed.We propose a probabilistic approach, which is based on Markov processes, for automatic selection of data representations. The selection is modeled as a random process moving in a graph with weighted edges. The proposed approach employs coin tossing at run--time to help choosing a suitable data representation. The transition probability function used by the coin tossing is constructed in a simple and common way from a measured cost function. We show that, under this setting, random selections of data representations can be quite effective. The probabilistic approach is used to implement bag aggregates, and the performance results are compared to those of deterministic selection strategies.

On the energy efficiency and performance of irregular application executions on multicore, NUMA and manycore platforms

In this paper, we present a new data structure called the packed compact trie (packed c-trie) which stores a set $S$ of $k$ strings of total length $n$ in $n \log\sigma + O(k \log n)$ bits of space and supports fast pattern matching queries and updates, where $\sigma$ is the size of an alphabet. Assume that $\alpha = \log_\sigma n$ letters are packed in a single machine word on the standard word RAM model, and let $f(k,n)$ denote the query and update times of the dynamic predecessor/successor data structure of our choice which stores $k$ integers from universe $[1,n]$ in $O(k \log n)$ bits of space. Then, given a string of length $m$, our packed c-tries support pattern matching queries and insert/delete operations in $O(\frac{m}{\alpha} f(k,n))$ worst-case time and in $O(\frac{m}{\alpha} + f(k,n))$ expected time. Our experiments show that our packed c-tries are faster than the standard compact tries (a.k.a. Patricia trees) on real data sets. As an application of our packed c-trie, we show that the sparse suffix tree for a string of length $n$ over prefix codes with $k$ sampled positions, such as evenly-spaced and word delimited sparse suffix trees, can be constructed online in $O((\frac{n}{\alpha} + k) f(k,n))$ worst-case time and $O(\frac{n}{\alpha} + k f(k,n))$ expected time with $n \log \sigma + O(k \log n)$ bits of space. When $k = O(\frac{n}{\alpha})$, by using the state-of-the-art dynamic predecessor/successor data structures, we obtain sub-linear time construction algorithms using only $O(\frac{n}{\alpha})$ bits of space in both cases. We also discuss an application of our packed c-tries to online LZD factorization.

© 2017 by John Wiley & Sons, Inc. All rights reserved. Concurrent data structures are the data sharing side of parallel programming. An implementation of a data structure is called lock-free, if it allows multiple processes/hreads to access the data structure concurrently and also guarantees that at least one operation among those finishes in a finite number of its own steps regardless of the state of the other operations. This chapter provides a sufficient background and intuition to help the interested reader to navigate in the complex research area of lock-free data structures. It offers the programmer familiarity to the subject that allows using truly concurrent methods. The chapter discusses the fundamental synchronization primitives on which efficient lock-free data structures rely. It discusses the problem of managing dynamically allocated memory in lock-free concurrent data structures and general concurrent environments. The idiosyncratic architectural features of graphics processors that is important to consider when designing efficient lock-free concurrent data structures for this emerging area.

Generalized association rule mining using an efficient data structure

Due to the rapid growth of Internet, there is an increasing need for efficiently classifying packets with many header fields in large rule sets. For example, in Software Defined Networking (SDN), the OpenFlow table lookup can require 15 packet header fields to be examined. In this paper, we present several decomposition-based packet classification implementations with efficient optimization techniques. In the searching phase, packet headers are split or combined. In the merging phase, the partial searching results from all the fields are merged to generate the final result. We prototype our implementations on state-of-the-art Field Programmable Gate Array (FPGA), multi-core General Purpose Processor (GPP), and Graphics Processing Unit (GPU). On FPGA, we propose two optimization techniques to divide generic ranges; modular processing elements are constructed and concatenated into a systolic array. On multi-core GPP, we parallelize both the searching and merging phases using parallel program threads. On the GPU-accelerated platform, we minimize branch divergence and reduce the data communication overhead. Experimental results show that 500 Million Packets Per Second (MPPS) throughput and 3 us latency can be achieved for 1.5 K rule sets on FPGA. We achieve 14.7 MPPS throughput and 30.5 MPPS throughput for 32 K rule sets on multi-core GPP and GPU-accelerated platforms, respectively. As a heterogeneous solution, our GPU-accelerated packet classifier shows 2X speedup compared to the implementation using multi-core GPP only. Compared with prior works, our designs can match long packet headers against very complex rule sets.

This paper studies a discrepancy-sensitive approach to dynamic fractional cascading. We provide an efficient data structure for dominated maxima searching in a dynamic set of points in the plane, which in turn leads to an efficient dynamic data structure that can answer queries for nearest neighbors using any Minkowski metric.

Paper 100020I, 11 S.

We have proposed and evaluated a scalable architecture for implementing multi-gigabit protocol engines. The architecture utilizes a combination of custom-made VLSI circuitry and a general-purpose processor, such as the Intel 960 or the IBM PowerPC 403. Time critical operations such as line coding/decoding, CRC generation/checking, context-independent header processing, and buffer management are implemented in the customized VLSI part. These designs are never-the-less scalable and can be cascaded to further increase throughput. Some of the packet level processing, such as context-dependent header processing, are performed by the general-purpose processor. As processing power increases, more and more functions can be included in the general purpose processor. The throughput of this architecture is shown to be adequate for the operations and bit rates currently specified by Fibre Channel. Future CMOS technology advances will have the potential to further improve the raw throughput.

Chapter 7 - Networks For The Small Business

As processor power density increases, thermal and power control becomes critical for application processing. In this paper, we consider network applications which feature ON/OFF execution pattern, that causes frequent temperature and power consumption changes in the processor. A novel power aware thermal management algorithm is designed to achieve power saving in multicore processors by employing a vacation scheme. We implement the scheme through the idle states (C-state) provided by the OS in the CPU and show their effectiveness both through analysis and experimental data. Then, we apply our scheme with the thermal constraint and propose a heterogeneous load distribution, which creates more opportunities for power saving. Besides maintaining processor temperature below the temperature constraint, our technique achieves higher sustainable load and better power saving with minimum latency increase compared to existing thermal management techniques. To the best of our knowledge, this is the first work to discuss and develop vacation algorithm considering power, temperature and latency for network application on a general purpose multicore processor.

Software defined radio (SDR) is characterized by its flexibility and convenience of modification, because significant amounts of signal processing are handed over to the general-purpose processor (GPP), rather than being done in special-purpose hardware. In the long term, software-defined radios work well in low data transmission rate systems such as 2G/3G cellular systems and other wireless radio systems. With rapid development of GPP power, more complex algorithms and higher data rate wireless communication systems can be implemented with GPP processor and general purpose RF front end. In this paper we propose a LTE physical layer parallel system architecture based on multicore GPP to match up the high capacity requests of LTE air interface. Firstly, we analyzed the complexity of each LTE physical layer module and made a statistics of their processing time. And then we used Linux CPU affinity to allocate each LTE module to particular CPU core carefully based on the statistics. CPU utilization rate shows that using multi-core makes the system to have enough capacity. Our system can only match up 2M bandwidth when the LTE physical layer procedures are running on single CPU core, while it can match up 5M bandwidth when it's running on multi-core.

<h3>Abstract</h3> Much of the DNA and RNA sequencing data available is in the form of high-throughput sequencing (HTS) reads and is currently unindexed by established sequence search databases. Recent succinct data structures for indexing both reference sequences and HTS data, along with associated metadata, have been based on either hashing or graph models, but many of these structures are static in nature, and thus, not well-suited as backends for dynamic databases. We propose a parallel construction method for and novel application of the <i>wavelet trie</i> as a dynamic data structure for compressing and indexing graph metadata. By developing an algorithm for merging wavelet tries, we are able to construct large tries in parallel by merging smaller tries constructed concurrently from batches of data. When compared against general compression algorithms and those developed specifically for graph colors (VARI and Rainbowfish), our method achieves compression ratios superior to gzip and VARI, converging to compression ratios of 6.5% to 2% on data sets constructed from over 600 virus genomes. While marginally worse than compression by bzip2 or Rainbowfish, this structure allows for both fast extension and query. We also found that additionally encoding graph topology metadata improved compression ratios, particularly on data sets consisting of several mutually-exclusive reference genomes. It was also observed that the compression ratio of wavelet tries grew sublinearly with the density of the annotation matrices. This work is a significant step towards implementing a dynamic data structure for indexing large annotated sequence data sets that supports fast query and update operations. At the time of writing, no established standard tool has filled this niche.

Neighborhood grid: A novel data structure for fluids animation with GPU computing

MapReduce is a widely used programming model for processing big data. Bloom filters are spatially efficient probabilistic data structures for fast queries that tell whether an element is a member of a set and allow false positive results. With reference to this, this article discusses the main characteristics, realisation techniques and scenarios for using Bloom filters when big data is analysed and processed. In addition to this, the study presents an implementation of a Bloom filter in a distributed MapReduce framework.