A survey of techniques and open-source tools for processing streams of spatio-temporal events

Complex Event Processing (CEP) is a powerful technology used in complex and real-time environments. CEP is an Event Driven Architecture (EDA) style consists of processing different events within the distributed enterprise system attempting to discover interesting information from multiple streams of events in timely manner. In real world, the streams of events are uncertain, which means that is not guaranteed that an event has actually occurred, this uncertainty is due mainly to imprecise content from the event sources (Sensors, RFID,...). As a result, probabilistic CEP has become an important issue in complex environments that require a real-time reaction given streams of probabilistic events.

Soccer analytics is attracting increasing interest in academia and industry, thanks to the availability of sensing technologies that provide high-fidelity data streams for every match. Unfortunately, these detailed data are owned by specialized companies and hence are rarely publicly available for scientificresearch. To fill this gap, this paper describes the largest open collection of soccer-logs ever released, containing all the spatio-temporal events (passes, shots, fouls, etc.) that occured during each match for an entire season of seven prominent soccer competitions. Each match event contains information about its position, time, outcome, player and characteristics. The nature of team sports like soccer, halfway between the abstraction of a game and the reality of complex social systems, combined with the unique size and composition of this dataset, provide an ideal ground for tackling a wide range of data science problems, including the measurement and evaluation of performance, both at individual and at collective level, and the determinants of success and failure.Cite this article: Pappalardo, L., Cintia, P., Rossi, A. et al. A public data set of spatio-temporal match events in soccer competitions. Sci Data 6, 236 (2019) doi:10.1038/s41597-019-0247-7Link to paper online: https://www.nature.com/articles/s41597-019-0247-7.pdf

The goal of Real-time Linked Dataspaces is to support a real-time response from intelligent systems to situations of interests within a smart environment by providing data processing support services that follow the data management philosophy of dataspaces and meet the requirements of real-time data processing. This part of the book details support services to process streaming and event data which support loose semantic integration and administrative proximity. Support services include entity-centric queries, complex event processing, stream dissemination, and semantic matching for heterogeneous events. The goal of these services is to support a real-time linked dataspace to get setup and running with a low overhead for administrative and semantic integration costs (e.g. establishing data agreements, service selection, and service composition).

Cloud computing is the notion for delivering access to scalable on-demand computing resources and IT services. The resource management system in IaaS Clouds dynamically allocates the resources based on predefined customers' needs using Service Level Agreements (SLAs) between the Cloud provider and customers. One of the challenges of resource management is to continuously monitor resource utilization, manage, and adjust these resources in real-time fashion to meet the SLAs while not over provisioning resources. Though, there exist numerous Cloud monitoring solutions, they are often highly specialized and restrict the administrator in defining automation rules. Our work aims to develop a framework based on concepts from Complex Event Processing (CEP) and data stream processing where data from various primitive metrics streams are collected and treated as event streams (e.g., from CPU, memory, and disk sensors). By automatically detecting complex patterns and relationships among these primitive events, we can detect and understand more high-level situations from the information provided by sensor steams. In this paper, we describe our CEP-based resource monitoring framework and discuss a use case for implementing auto scaling facilities to the Proxmox platform.

With the arrival of cloud computing and the internet of things, the difficulty of extracting data from the cloud services is preventing some organizations from adopting cloud computing. At the same time, the number of web services increases dramatically, and the granularity of service is different, so how to choose a suitable granularity service also becomes a problem that needs to be resolved. So a storage strategy for cloud computing services is given and the method based on event stream is used to construct the hierarchal model of services which can provide different functional services or different granularity services to meet the business requirements. Firstly, the storage strategy for cloud computing services is that, when registering the services, the meta-data of services will be extracted and stored into database according to certain structure. Secondly, the Event Cloud can be constructed as the one-way event processing, which provide the raw data for more-way event processing. Then the more-way event processing can construct the hierarchical model of services with certain composition goal. The prototype system has been developed in order to fulfill the application, both human events and RFID events can trigger other events in the cloud events, which can create new business models, improve business processes, and reduce costs and risks.

Deep representation via convolutional neural network for classification of spatiotemporal event streams

Knowledge-infused and consistent Complex Event Processing over real-time and persistent streams

With the recent advances in IoT and other smart devices, an explosive amount of data, both spatially and temporally significant are generated. Discovering interesting or useful patterns from these spatiotemporal data is referred to as spatiotemporal data mining. These patterns could be unordered, totally ordered or partially ordered based on the temporal ordering. This work focusses on the totally ordered patterns or sequential patterns from spatiotemporal event database. Spatiotemporal event sequence miner finds sequence of events that overlaps spatially and temporally. Traditional approaches discover patterns that are frequent in the entire dataset. In this work a clustering-assisted approach to find regionally or locally frequent spatiotemporal pattern is proposed. The proposed Clustering assisted Regional Spatiotemporal Event Sequence (CReST) mining approach overcomes the bias caused by uneven distribution of spatiotemporal events while mining patterns. Chicago crime dataset is used for evaluating the proposed approach with traditional sequence mining algorithm.

How to effectively and efficiently deal with spatio-temporal event streams, where the events are generally sparse and non-uniform and have the μs temporal resolution, is of great value and has various real-life applications. Spiking neural network (SNN), as one of the brain-inspired event-triggered computing models, has the potential to extract effective spatio-temporal features from the event streams. However, when aggregating individual events into frames with a new higher temporal resolution, existing SNN models do not attach importance to that the serial frames have different signal-to-noise ratios since event streams are sparse and non-uniform. This situation interferes with the performance of existing SNNs. In this work, we propose a temporal-wise attention SNN (TA-SNN) model to learn frame-based representation for processing event streams. Concretely, we extend the attention concept to temporal-wise input to judge the significance of frames for the final decision at the training stage, and discard the irrelevant frames at the inference stage. We demonstrate that TA-SNN models improve the accuracy of event streams classification tasks. We also study the impact of multiple-scale temporal resolutions for frame-based representation. Our approach is tested on three different classification tasks: gesture recognition, image classification, and spoken digit recognition. We report the state-of-the-art results on these tasks, and get the essential improvement of accuracy (almost 19%) for gesture recognition with only 60 ms.

In such areas as transport, healthcare, business process management, financial services, complex event processing (CEP) systems are becoming in demand. Large volumes of streams of events are generated and collected by these systems. Real-time processing of event stream data is essential. Based on current and past events, a decision is made, and forecasting future events is an alternative goal. For the fastest reaction to non-standard behavior, it is necessary to process data streams in real time, it means, as quickly as possible. That's why it is important to find new technologies which can speed up data processing. In this article expensive queries in CEP is identified, effective optimizations to improve their performance significantly is proposed. It is developed and implemented algorithm, which should decrease processing time. The results are compared with state-of-the-art CEP system.

Sensors including RFID tags have been widely deployed for measuring environmental parameters such as temperature, humidity, oxygen concentration, monitoring the location and velocity of moving objects, tracking tagged objects, and many others. To support effective, efficient, and near real-time phenomena probing and objects monitoring, streaming sensor data have to be gracefully managed in an event processing manner. Different from the traditional events, sensor events come with temporal or spatio-temporal constraints and can be non-spontaneous. Meanwhile, like general event streams, sensor event streams can be generated with very high volumes and rates. Primitive sensor events need to be filtered, aggregated and correlated to generate more semantically rich complex events to facilitate the requirements of up-streaming applications. Motivated by such challenges, many new methods have been proposed in the past to support event processing in sensor event streams. In this chapter, we survey state-of-the-art research on event processing in sensor networks, and provide a broad overview of major topics in.

Chapter 4 - Frameworks for Big Data Integration, Warehousing, and Analytics

This paper proposes a system architecture for USN with a service robot to provide more active assisted living services for elderly persons by monitoring their mental and physical well-being with USN environments at home, hospital, or silver town. Sensors embedded in USN are used to detect preventive measures for chronic disease. Logged data are transferred to main controller of a service robot via wireless channel in which the analysis of data is performed. For the purpose of handling emergency situations, it needs real-time processing on gathering variety sensor data, routing algorithms for sensor nodes to a moving sink node and processing of logged data. This paper realized multi-hop sensor network to detect user movements with biometric data transmission and performed algorithms on Xenomai, a real-time embedded Linux. To leverage active sensing, a mobile robot is used of which task was implemented with a priority to process urgent data came from the sink-node. This software architecture is anticipated to integrate sensing, communication and computing with real-time manner. In order to verify the usefulness of a proposed system, the performance of data transferring and processing on a real-time OS with non real-time OS is also evaluated.

Complex Event Processing (CEP) is a branch of stream data mining, which is usually used to discover event patterns. Generally, a complex event processing engine uses nonprocedural declaration language and state machine to define event patterns. It generates synopses of universe data by sliding window model, therefore could identify event patterns which users care about from rapidly changing and potentially infinite event stream in a relatively short time. Among all the data streams, monitoring data flow is easy to describe by events, so CEP is suitable for supervision or decision control for business system. However, CEP does not support backtracking to eliminate invalid data, which limits CEP in supervision or decision control applications. The traditional complex event processing try to solve this problem by defining more complicated event patterns to avoid invalid results, the side effect of which would be heavier workload onto the engine. For this problem, we propose an optimized complex-event-processing model OCM. OCM will decompose complicated event patterns into small ones according to their relations, and distribute these patterns to more than one CEP engines to hierarchically filter invalid data. It can eliminate invalid data with better performance than the traditional approaches. We have also implemented the prototype and designed experiments to test the effectiveness of OCM. We have applied this model into Aviation Application in EU FP6 project Bridge.

\n In distributed complex event processing, event streams are processed\n over a chain of subsequent operators. For large-scale applications\n like a logistic chain these operators may be hosted by different\n entities and thus are spread over different security domains.\n Current approaches for complex event processing cannot preserve the\n privacy of an operators incoming event streams. An adversary may\n infer the original input stream from its legally received event\n streams.\n\n In this paper we present a fine-grained access management for\n complex event processing. We show how to enforce privacy of events\n throughout the chain of dependent operators by specifying\n appropriate access policies and proposing an algorithm for policy\n consolidation. Furthermore, we introduce the calculation of\n obfuscation achieved in a correlation step. This allows us to ignore\n access requirements once a sufficient obfuscation level has been\n achieved, the proposed algorithms is capable to reduce the required\n overhead in the enforcement of access policies. We prove correctness\n and evaluate the cost in establishing policy consolidation.\n