Chunk Incremental LDA Computing on Data Streams

This paper presents a constructive method for deriving an updated discriminant eigenspace for classification when bursts of data that contains new classes is being added to an initial discriminant eigenspace in the form of random chunks. Basically, we propose an incremental linear discriminant analysis (ILDA) in its two forms: a sequential ILDA and a Chunk ILDA. In experiments, we have tested ILDA using datasets with a small number of classes and small-dimensional features, as well as datasets with a large number of classes and large-dimensional features. We have compared the proposed ILDA against the traditional batch LDA in terms of discriminability, execution time and memory usage with the increasing volume of data addition. The results show that the proposed ILDA can effectively evolve a discriminant eigenspace over a fast and large data stream, and extract features with superior discriminability in classification, when compared with other methods.

This paper proposes a new online feature extraction method called Incremental Recursive Fisher Linear Discriminant (IRFLD) whose batch learning algorithm called RFLD has been proposed by Xiang et al. In the conventional Linear Discriminant Analysis (LDA), the number of discriminant vectors is limited to the number of classes minus one due to the rank of the between-class covariance matrix. However, RFLD and the proposed IRFLD can break this limit; that is, an arbitrary number of discriminant vectors can be obtained. In the proposed IRFLD, the Pang et al.'s Incremental Linear Discriminant Analysis (ILDA) is extended such that effective discriminant vectors are recursively searched for the complementary space of a conventional discriminant subspace. In addition, to estimate a suitable number of effective discriminant vectors, the classification accuracy is evaluated with a cross-validation method in an online manner. For this purpose, validation data are obtained by performing the k-means clustering against incoming training data and previous validation data. The performance of IRFLD is evaluated for 16 benchmark data sets. The experimental results show that the final classification accuracies of IRFLD are always better than those of ILDA. We also reveal that this performance improvement is attained by adding discriminant vectors in a complementary LDA subspace.

In recent years, the progress in hardware technology has made it possible for organizations to store and record large streams of transactional data. This results in databases which grow without limit at a rapid rate. This data can often show important changes in trends over time. In such cases, it is useful to understand, visualize and diagnose the evolution of these trends. When the data streams are fast and continuous, it becomes important to analyze and predict the trends quickly in online fashion. In this paper, we discuss the concept of velocity density estimation, a technique used to understand, visualize and determine trends in the evolution of fast data streams. We show how to use velocity density estimation in order to create both temporal velocity profiles and spatial velocity profiles at periodic instants in time. These profiles are then used in order to predict three kinds of data evolution: dissolution, coagulation and shift. Methods are proposed to visualize the changing data trends in a single online scan of the data stream, and a computational requirement which is linear in the number of data points. In addition, batch processing techniques are proposed in order to identify combinations of dimensions which show the greatest amount of global evolution. The techniques discussed in this paper can be easily extended to spatio-temporal data, changes in data snapshots at fixed instances in time, or any other data which has a temporal component during its evolution.

Since the launch of the Chinese High-resolution Earth Observation System (CHEOS) program, China has strengthened its research and development in the field of satellite remote sensing. A large number of sensors has been or will be launched, providing very large data streams which all require processing of the engineering data, as provided by the instruments, to physical data which will be used for further processing and interpretation. To handle such large data streams we developed a one-click batch pre-processing toolkit for CHEOS remote sensing data as described in this paper. In this toolkit, IDL language and environment are used as the primary program combined with other programming languages developed in this research. In this paper, we first describe the Gaofen (GF) series data used in this research and then introduce the function design and realization of this one-click batch pre-processing toolkit. Some examples will be presented to illustrate the application of the toolkit to data from several CHEOS satellites.

In order to improve the monitoring ability of uncertain large data stream, an uncertain large data flow monitoring algorithm based on artificial intelligence is proposed. The collected uncertain big data flow is constructed by low dimensional feature set, and the rough set model of uncertain large data stream distribution is constructed. The fuzzy C-means clustering method is used to analyze the uncertain big data flow by fusion clustering and adaptive grid partition analysis. All the abnormal samples of large data stream are sampled and trained, and the feature quantities of association rules of uncertain large data stream are extracted. Combined with artificial intelligence method, the monitoring of uncertain large data stream is realized. The simulation results show that the method has high accuracy and good ability to resist abnormal traffic interference, and the traffic security monitoring ability of the network is improved.

In streaming applications, it is often required to detect situations of interest, by means of temporal pattern matching, with minimal latency. In the maritime domain, e.g., where it is crucial to prevent activities that are harmful to the environment, we need to report illegal fishing activities, based on streams of low-level vessel actions, as soon as possible. Streams often include events with delayed effects. In multi-agent voting protocols, e.g., a proposed motion may be seconded at the latest by some time in the future. In simulations of biological systems, a signal may lead to the deactivation of the functions of a gene after a time delay. We propose a formal computational framework that handles streams including events with delayed effects. We present the syntax, semantics and reasoning algorithms of our proposed framework, and demonstrate its correctness and complexity. Furthermore, we present a reproducible analysis on large synthetic and real data streams, from the fields of composite event recognition, multi-agent systems and biological feedback processes, and compare the efficiency of our approach with state-of-the-art systems that can perform stream reasoning in these domains. Our results demonstrate that our framework is capable of reasoning over very large streams, including events with delayed effects, while outperforming the state-of-the-art, often by orders of magnitude.

Face recognition efficiency is extremely challenged by image corruption, noise, shadowing and variant face expressions. In this paper, we propose a reliable incremental face recognition algorithm to address this problem. The algorithm is robust to face image misalignment, occlusion, corruption and different style variations. To apply this for large face data streams, the proposed algorithm uses incremental robust principal component analysis (PCA) to regain the intrinsic data of a bunch of images regarding one subject. A novel similarity metric is defined for face recognition and classification. Five different databases and a base of four different criteria are used in the experiments to illustrate the reliability of the proposed method. Experiments point that it outperforms other existing incremental PCA approaches namely incremental singular value decomposition, add block singular value decomposition and candid covariance-free incremental PCA in terms of recognition rate under occlusions, facial expressions and image perspectives.

Anytime algorithms have been proposed for many different applications e.g. in data mining. Their strengths are the ability to first provide a result after a very short initialization and second to improve their result with additional time. Therefore, anytime algorithms have so far been used when the available processing time varies, e.g. on varying data streams. In this paper we propose to employ anytime algorithms on constant data streams, i.e. for tasks with constant time allowance. We introduce two approaches that harness the strengths of anytime algorithms on constant data streams and thereby improve the over all quality of the result with respect to the corresponding budget algorithm. We derive formulas for the expected performance gain and demonstrate the effectiveness of our novel approaches using existing anytime algorithms on benchmark data sets.The goal that was set and reached in this paper is to improve the quality of the result over that of traditional budget approaches, which are used in an abundance of stream mining applications. Using anytime classification as an example application we show for SVM, Bayes and nearest neighbor classifiers that both our novel approaches improve the classification accuracy for slow and fast data streams. The results confirm our general theoretic models and show the effectiveness of our approaches. The simple yet effective idea can be employed for any anytime algorithm along with a quality measure and motivates further research in e.g. classification confidence measures or anytime algorithms.KeywordsData MiningData StreamQuality MeasureKnowledge DiscoveryConfidence MeasureThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Motivated by scenarios in network anomaly detection, we consider the problem of detecting persistent items in a data stream, which are items that occur ‘regularly’ in the stream. In contrast with heavy hitters, persistent items do not necessarily contribute significantly to the volume of a stream, and may escape detection by traditional volume‐based anomaly detectors.\nWe first show that any online algorithm that tracks persistent items exactly must necessarily use a large workspace, and is infeasible to run on a traffic monitoring node. In light of this lower bound, we introduce an approximate formulation of the problem and present a small‐space algorithm to approximately track persistent items over a large data stream. We experimented with three different datasets to see how the accuracy and memory footprint of the algorithm varies with the skewness of the dataset. Our algorithms performed best for the two datasets out of three which had highest skewness of persistence and lowest mean persistence. To our knowledge, this is the first systematic study of the problem of detecting persistent items in a data stream, and our work can help detect anomalies that are temporal, rather than volume‐based.

The dynamic data-driven applications system (DDDAS) paradigm uses the controlled measurement data to update models. When big data streams are available, signal processing can be used for situation awareness. However, many times, the physical world prevents sensor measurements availability which provides an opportunity to use heterogeneous sensor information; but methods are needed for data normalization, sampling alignment, and data mining. In this paper, we highlight a joint nonlinear manifold learning approach to that incorporates advances in machine learning, data fusion, and model-based simulation propagators. The context of the situation monitoring includes a multiple moving targets, a video sensor, and distributed signal sensors. The challenge is to determine the moving emitter amongst a cluttered scene during an experiment with large data streams. Resolving the signature triangulation of the emitter with that of the video sensor is both a nonlinear tracking problem as well as data learning issue. The paper presents the joint nonlinear manifold learning approach that is theoretically developed such sensor models could not only be the tracking scenario; but that of structure health monitoring as well as internet of things (IoT) examples. The theoretical analysis, heterogeneous data set, machine learning, and results are presented to showcase the importance of real-time DDDAS modeling updates of kinematic models.

Sliding windows are bounded sets which evolve together with an infinite data stream of records. Each new sliding window evicts records from the previous one while introducing newly arrived records as well. Aggregations on windows typically derive some metric such as an average or a sum of a value in each window. The main challenge of applying aggregations to sliding windows is that a naive execution can lead to a high degree of redundant computation due to a large number of common records across different windows. Special optimization techniques have been developed throughout the years to tackle redundancy and make sliding window aggregation feasible and more efficient in large data streams

An overview and comprehensive comparison of ensembles for concept drift

Since big data streams contain hidden meanings, there is a permanent motivation to store and process them. However, storing and processing this data requires special methods and tools. Today, the most effective approach for this situation is distributed computing mechanisms. However, this approach is economically expensive, since it requires a lot of computing resources. Therefore, users who do not have economic capabilities strive to solve problems with large data streams on a single server. In this situation, a sharp drop in efficiency in terms of time is observed. However, even for a single computing machine, the use of an internal distribution mechanism can lead to efficiency in terms of time. In this case, efficiency depends on several indicators, the most important of which is determining the number of effective distributions. However, determining the number of effective distributions is a complex process. To solve this problem, this research paper considers the use of artificial intelligence algorithms. First of all, the research methodology is developed and processes that are in it are explained. In the next step, Random Forest, XGBoost, Support Vector Regression, and Multiple Linear Regression algorithms are tested to determine the number of effective distributions. In order to improve the accuracy of the study, a multilayer neural network is improved, that is, a neural network ensemble method is developed that combines the above machine learning algorithms. At the end of the study, the research results are presented and explained in detail.

HeavySeparation: A Generic framework for stream processing faster and more accurate

The rapid innovations in technologies endorsed the emergence of sensory equipment's connection to the Internet for acquiring data from the environment. The increased number of devices generates the enormous amount of sensor data from diversified applications of Internet of things (IoT). The generation of data may be a fast or real-time data stream which depends on the nature of applications. Applying analytics and intelligent processing over the data streams discovers the useful information and predicts the insights. Decision-making is a prominent process which makes the IoT paradigm qualified. This chapter provides an overview of architecting IoT-based healthcare systems with different machine learning algorithms. This chapter elaborates the smart data characteristics and design considerations for efficient adoption of machine learning algorithms into IoT applications. In addition, various existing and hybrid classification algorithms are applied to sensory data for identifying falls from other daily activities.