Pandora: An Efficient and Rapid Solution for Persistence-Based Tasks in High-Speed Data Streams

In this paper we study the problem of constructing histograms from high-speed time-changing data streams. Learning in this context requires the ability to process examples once at the rate they arrive, maintaining a histogram consistent with the most recent data, and forgetting out-date data whenever a change in the distribution is detected. To construct histogram from high-speed data streams we use the two layer structure used in the Partition Incremental Discretization (PiD) algorithm. Our contribution is a new method to detect whenever a change in the distribution generating examples occurs. The base idea consists of monitoring distributions from two different time windows: the reference time window, that reflects the distribution observed in the past; and the current time window reflecting the distribution observed in the most recent data. We compare both distributions and signal a change whenever they are greater than a threshold value, using three different methods: the Entropy Absolute Difference, the Kullback-Leibler divergence and the Cosine Distance. The experimental results suggest that Kullback-Leibler divergence exhibit high probability in change detection, faster detection rates, with few false positives alarms.

Histograms are a common technique for density estimation and they have been widely used as a tool in exploratory data analysis. Learning histograms from static and stationary data is a well known topic. Nevertheless, very few works discuss this problem when we have a continuous flow of data generated from dynamic environments. The scope of this paper is to detect changes from high-speed time-changing data streams. To address this problem, we construct histograms able to process examples once at the rate they arrive. The main goal of this work is continuously maintain a histogram consistent with the current status of the nature. We study strategies to detect changes in the distribution generating examples, and adapt the histogram to the most recent data by forgetting outdated data. We use the Partition Incremental Discretization algorithm that was designed to learn histograms from high-speed data streams. We present a method to detect whenever a change in the distribution generating examples occurs. The base idea consists of monitoring distributions from two different time windows: the reference window, reflecting the distribution observed in the past; and the current window which receives the most recent data. The current window is cumulative and can have a fixed or an adaptive step depending on the distance between distributions. We compared both distributions using Kullback-Leibler divergence, defining a threshold for change detection decision based on the asymmetry of this measure. We evaluated our algorithm with controlled artificial data sets and compare the proposed approach with nonparametric tests. We also present results with real word data sets from industrial and medical domains. Those results suggest that an adaptive window’s step exhibit high probability in change detection and faster detection rates, with few false positives alarms.

Discovering interesting patterns from high-speed data streams is a challenging problem in data mining. Recently, the support metric-based frequent pattern mining from data stream has achieved a great attention. However, the occurrence frequency of a pattern may not be an appropriate criterion for discovering meaningful patterns. Temporal regularity in occurrence behavior can be a key criterion for assessing the importance of patterns in several online applications such as market basket analysis, gene data analysis, network monitoring, and stock market. A pattern can be said regular if its occurrence behavior satisfies a user-given interval in the data steam. Mining regular patterns from static databases has recently been addressed. However, even though mining regular patterns from stream data is extremely required in online applications, no such algorithm has been proposed yet. Therefore, in this paper we develop a novel tree structure called Regular Pattern Stream tree (RPS-tree), and an efficient mining technique for discovering regular patterns over data stream. Using a sliding window method the RPS-tree captures the stream content, and with an efficient tree updating mechanism it constantly processes exact stream data when the stream flows. Extensive experimental analyses show that our RPS-tree is highly efficient in discovering regular patterns from a high-speed data stream.

Finding top- <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> frequent items has been a hot issue in databases. Finding top- <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> persistent items is a new issue, and has attracted increasing attention in recent years. In practice, users often want to know which items are significant, <i>i.e.</i> , not only frequent but also persistent. No prior art can address both of the above two issues at the same time. Also, for high-speed data streams, prior art cannot achieve high accuracy when the memory is tight. In this paper, we define a new issue, named finding significant items, and propose a novel algorithm namely LTC to address this issue. It includes two key techniques, Long-tail Restoring and CLOCK, as well as three optimizations. In addition, LTC is extended to support finding significant items with thresholds. We theoretically derive the correct rate and error bound, and conduct extensive experiments on three real datasets to test the performance of LTC. Our experimental results show that LTC can achieve <inline-formula><tex-math notation="LaTeX">$10^5$</tex-math></inline-formula> times higher accuracy in terms of average relative error than other related algorithms. Lastly, LTC is applied to a DDoS detection task and it shows that finding significant items is more powerful than finding frequent items.

A generic sketch for estimating super-spreaders and per-flow cardinality distribution in high-speed data streams

Outlier detection or anomaly detection is applied in numerous applications, such as fraud detection, network intrusion detection, manufacturing, and environmental monitoring. Due to the continuous and dynamic characteristics of streaming data, outlier detection over data streams becomes a very challenging task. When analyzing real-time data streams, it is typically impossible to store the entire set of data due to space limitations. Also due to the high data rates, it is necessary to produce the results in a limited amount of time. Parallel processing power of Graphics Processing Units (GPUs) can be used to accelerate the outlier detection process, and thus address the challenges of outlier detection over data streams. This paper proposes a GPU-accelerated outlier detection algorithm for continuous data streams using kernel density estimation approach. Experiments show that the proposed SOD_GPU algorithm is efficient for detecting outliers in high-dimensional, high-speed data streams, and produces results in a timely manner without compromising the outlier detection accuracy. The proposed method achieved up to 20X speedup compared to a respective multi-core CPU implementation, and the speedup increases with the number of data attributes and the input data rate.

A 28 nm CMOS Software-Defined Transceiver (SDTRX) enabling High-Speed Data (HSD) streaming, including Ultra HD TV, within home cable networks is presented. By making efficient use of available cable bandwidth, the SDTRX dynamically handles up to 1024QAM OFDM-modulated HSD streams. This paper addresses SDTRX system-level design methodology as the key driver in enabling performance optimization for achieving a wide frequency range of operation at the lowest power and area consumption. By employing an optimized architecture constructed on available state-of-the art 28 nm functional building blocks, the monolithic SDTRX consists of a mixer-based harmonic rejection RX with a DAC-based TX and a smart PLL system. It operates over a 0.4-to-1.7 GHz frequency range while consuming less than 475 mW in half-duplex mode. Moreover, by developing a simple TX-RX loopback circuit, the system is enabled to efficiently calibrate TX output power and to remove the need for a dedicated external pin. This low-cost SDTRX is embedded in various 28 nm CMOS multimedia SoCs and is, to the authors' knowledge, the first reported transceiver front end to enable true high-speed data streaming within home cable networks.

The methods of approximate query processing have been proposed for analytics over high-speed data streams, which compact continuous streams into a space-constrained sketch and provide reliable estimates for different queries. Count-Min (CM) is the state-of-the-art sketching structure supporting many queries with error-guaranteed estimates under limited space. However, we need to create a counter table beforehand in CM according to the size of data streams, while it is usually unpredictable for dynamic data streams. In this paper, we proposed an approach, called Dynamic Count-Min sketch (DCM), which is appropriate for dynamic data set and can provide accurate estimates for point query and self-join size query. Our approach constitutes incremental CM sketches and allocates space in a pay-as-you-go manner. Our mathematical analysis and substantial experiments both show that our approach is appropriate for data sets with dynamic or skewed inputs and can provide error-guaranteed estimates with less space compared to CM.

Motivated by scenarios in network anomaly detection, we consider the problem of detecting persistent items in a data stream, which are items that occur ‘regularly’ in the stream. In contrast with heavy hitters, persistent items do not necessarily contribute significantly to the volume of a stream, and may escape detection by traditional volume‐based anomaly detectors.\nWe first show that any online algorithm that tracks persistent items exactly must necessarily use a large workspace, and is infeasible to run on a traffic monitoring node. In light of this lower bound, we introduce an approximate formulation of the problem and present a small‐space algorithm to approximately track persistent items over a large data stream. We experimented with three different datasets to see how the accuracy and memory footprint of the algorithm varies with the skewness of the dataset. Our algorithms performed best for the two datasets out of three which had highest skewness of persistence and lowest mean persistence. To our knowledge, this is the first systematic study of the problem of detecting persistent items in a data stream, and our work can help detect anomalies that are temporal, rather than volume‐based.

Distributed systems have been widely used for applications that need real-time processing over high volume and high speed data streams. This work presents the architecture and the implementation of a correlation algorithm for streaming data on a Graphic Processing Unit (GPU). The proposed system accelerates the correlation calculation of the Hayashi-Yoshida algorithm up to 10x vs. a conventional distributed system, and these performance characteristics apply to a broad category of correlation estimators. Furthermore, our system offers real-time correlation computation over high-speed streaming data and demonstrates how a batch processing algorithm can be applied to real-time streaming data. The results show that GPUs are a highly promising platform for correlation estimators as they improve significantly the volume of streaming data that can be processed in real time vs. other approaches that use “unlimited” conventional computing resources.

Due to an ever-increasing demand for analyzing the large volumes of information issuing from high-speed data streams, multi-label stream classification is replacing the traditional offline multi-label classification system and has thus become a focal point in recent years. In this paper, we propose a new algorithm for multi-label stream classification that relies on an Online Dual Memory (ODM). It uses a short-term memory in order to adapt quickly to changes within the data stream, and employs a long-term memory for recurring concepts. Furthermore, via a modification of Learning Vector Quantization, ODM is not only capable of classifying efficiently but also of maintaining a low computational complexity. Experiments were conducted so as to establish a comprehensive overview of its performance, which was achieved by comparing four methods on two different kinds of datasets (nine stationary datasets and four datasets with concept drift). Our results confirm the high accuracy and low time complexity of ODM, as well as its ability to cope with concept drift.

Optical time-stretch imaging has shown potential in diverse fields for its capability of acquiring images at high speed and high resolution. However, its wide application is hindered by the stringent requirement on the instrumentation hardware caused by the high-speed serial data stream. Here we demonstrate temporally interleaved optical time-stretch imaging that lowers the requirement without sacrificing the frame rate or spatial resolution by interleaving the high-speed data stream into multiple channels in the time domain. Its performance is validated with both a United States Air Force (USAF)-1951 resolution chart and a single-crystal diamond film. We achieve a 101 Mfps 1D scanning rate and 3 µm spatial resolution with only a 2.5 GS/s sampling rate by using a two-channel-interleaved system.

Approaches that combine streaming algorithms and distributed computing have potential to deal with voluminous and high-speed data streams. Considering the data stream clustering task, also an important issue needs to be addressed, estimate the number of clusters dynamically, since it may vary due to concept drift. This work proposes three evolutionary-based algorithms to overcome these requirements. They are based in the discretized stream model, where the sequential batches of objects are distributed and processed in a parallel way using the MapReduce model. The proposed algorithms achieve superior experimental results, either in quality and processing time, overcoming the state-of-the-art.

How to extract meaningful information from big data has been a popular open problem. Decision tree, which has a high degree of knowledge interpretation, has been favored in many real world applications. However noisy values commonly exist in high-speed data streams, e.g. real-time online data feeds that are prone to interference. When processing big data, it is hard to implement pre-processing and sampling in full batches. To solve this tradeoff, this paper proposes a new incremental decision tree algorithm so called incrementally optimized very fast decision tree (iOVFDT). The experiment evaluates the proposed algorithm in comparison to existing methods under noisy data streams environment. Result shows iOVFDT has outperformance on the aspects of higher accuracy and smaller model size.

Many real life prediction problems involve predicting a structured output. Multi-target regression is an instance of structured output prediction whose task is to predict for multiple target variables. Structured output algorithms are usually computationally and memory demanding, hence are not suited for dealing with massive amounts of data. Most of these algorithms can be categorized as local or global methods. Local methods produce individual models for each output component and combine them to produce the structured prediction. Global methods adapt traditional learning algorithms to predict the output structure as a whole. We propose the first rule-based algorithm for solving multi-target regression problems from data streams. The algorithm builds on the adaptive model rules framework. In contrast to the majority of the structured output predictors, this particular algorithm does not fall into the local and global categories. Instead, each rule specializes on related subsets of the output attributes. To evaluate the performance of the proposed algorithm, two other rule-based algorithms were developed, one using the local strategy and the other using the global strategy. These methods were compared considering their prediction error, memory usage, computational time, and model complexity. Experimental results on synthetic and real data show that the local-strategy algorithm usually obtains the lowest error. However, the proposed and the global-strategy algorithms use much less memory and run significantly much faster at the cost of a slightly increase in the error, which make them very attractive when computation resources are an important factor. Also, the models produced by the latter approaches are much easier to understand since considerably less rules are produced.