Modeling and Formal Analysis of Probabilistic Complex Event Processing (CEP) Applications

Complex event processing (CEP) over event streams has become increasingly important for real-time applications ranging from health care, supply chain management to business intelligence. These monitoring applications submit complex event queries to track sequences of events that match a given pattern. As these systems mature the need for increasingly complex nested sequence query support arises, while the state-of-art CEP systems mostly support the execution of only flat sequence queries. In this paper, we introduce our nested CEP query language NEEL for expressing nested queries composed of sequence, negation, AND and OR operators. Thereafter, we also define its formal semantics. Subtle issues with negation and predicates within the nested sequence context are discussed. An E-Analytics system for processing nested CEP queries expressed in the NEEL language has been developed. Lastly, we demonstrate the utility of this technology by describing a case study of applying this technology to a real-world application in health care.

In-network computing using programmable networking hardware is a strong trend in networking that promises to reduce latency and consumption of server resources through offloading to network elements (programmable switches and smart NICs). In particular, the data plane programming language P4 together with powerful P4 networking hardware has spawned projects offloading services into the network, e.g., consensus services or caching services. In this paper, we present a novel case for in-network computing, namely, Complex Event Processing (CEP). CEP processes streams of basic events, e.g., stemming from networked sensors, into meaningful complex events. Traditionally, CEP processing has been performed on servers or overlay networks. However, we argue in this paper that CEP is a good candidate for in-network computing along the communication path avoiding detouring streams to distant servers to minimize communication latency while also exploiting processing capabilities of novel networking hardware. We show that it is feasible to express CEP operations in P4 and also present a tool to compile CEP operations, formulated in our P4CEP rule specification language, to P4 code. Moreover, we identify challenges and problems that we have encountered to show future research directions for implementing full-fledged in-network CEP systems.

Knowledge-infused and consistent Complex Event Processing over real-time and persistent streams

\n In distributed complex event processing, event streams are processed\n over a chain of subsequent operators. For large-scale applications\n like a logistic chain these operators may be hosted by different\n entities and thus are spread over different security domains.\n Current approaches for complex event processing cannot preserve the\n privacy of an operators incoming event streams. An adversary may\n infer the original input stream from its legally received event\n streams.\n\n In this paper we present a fine-grained access management for\n complex event processing. We show how to enforce privacy of events\n throughout the chain of dependent operators by specifying\n appropriate access policies and proposing an algorithm for policy\n consolidation. Furthermore, we introduce the calculation of\n obfuscation achieved in a correlation step. This allows us to ignore\n access requirements once a sufficient obfuscation level has been\n achieved, the proposed algorithms is capable to reduce the required\n overhead in the enforcement of access policies. We prove correctness\n and evaluate the cost in establishing policy consolidation.\n

Complex event processing is a key technology for analyzing event patterns in real-time data streams. It has been widely applied in IoT monitoring, financial risk control, smart manufacturing and other time-sensitive fields. Query patterns serve as the core matching mechanism in complex event processing systems. These patterns are primarily defined through selection policies, predicate constraints and timestamps to establish complex event detection rules. The selection policy is particularly important as it determines the sequential relationships between events. This directly affects the system's event matching efficiency and accuracy. However, in practical applications, due to different business requirements for event sampling frequencies, event streams often contain numerous consecutive events of the same type representing identical states. While all these events participate in complex event matching, traditional processing methods generate large numbers of semantically redundant partial matches. Analysis shows that only a very small proportion of these intermediate results eventually form complete matches, creating serious computational redundancy issues. This redundancy not only significantly increases system resource consumption but also reduces overall processing efficiency due to ineffective computations. To address this critical problem, this paper innovatively proposes a cache-based optimized selection policy. The core concept of this policy is to effectively reduce redundant matching results by extracting representative events from continuous event sequences. Experimental results show that this policy can significantly decrease the redundant matching rate while alleviating system load, providing new insights for optimizing complex event processing systems.

Complex Event Processing (CEP) is a technology with support for matching patterns in a cloud or streams of events in order to support detection of specific combinations of event occurrences. A clever specification of event patterns may, for example, detect fraud attempts in a banking system, fire an alarm in response to hazardous situations in a control system or report suspicious customer behavior.Several CEP engines have support for graphically modelling applications as well as perform tests and provide execution traces to verify the application behavior. We argue that it is beneficial to complement testing with formal verification in order to detect errors in early stages of development.In this paper, we present the research prototype tool REX. REX is built as a loosely coupled front end to the timed-automata CASE tool UPPAAL. CEP applications and application specific properties can be specified in REX. To support formal verification, REX seamlessly transforms the CEP application together with the specified properties to the timed automata CASE tool UPPAAL where the properties are verified by the model-checker provided by UPPAAL.

Radio Frequency Identication (RFID) has been widely used in most applications. The RFID data generated are usually meaningless, real-time, and large-scale. Complex Event Processing (CEP) technology is used to extract meaningful information from the raw RFID data. This paper proposes a CEP model based on Open Service Gateway Initiative (OSGi) platform with the features of recongurable, customized and so on. Moreover, the open-source Esper technique for processing complex event and event stream is also introduced.

Sensors including RFID tags have been widely deployed for measuring environmental parameters such as temperature, humidity, oxygen concentration, monitoring the location and velocity of moving objects, tracking tagged objects, and many others. To support effective, efficient, and near real-time phenomena probing and objects monitoring, streaming sensor data have to be gracefully managed in an event processing manner. Different from the traditional events, sensor events come with temporal or spatio-temporal constraints and can be non-spontaneous. Meanwhile, like general event streams, sensor event streams can be generated with very high volumes and rates. Primitive sensor events need to be filtered, aggregated and correlated to generate more semantically rich complex events to facilitate the requirements of up-streaming applications. Motivated by such challenges, many new methods have been proposed in the past to support event processing in sensor event streams. In this chapter, we survey state-of-the-art research on event processing in sensor networks, and provide a broad overview of major topics in.

It is becoming increasingly crucial to integrate pattern matching functionality over live and archived event streams with hybrid event queries for various complex event processing(CEP) applications. As existing Stream Processing Engine(SPE) and DBMS alone can not accommodate hybrid event queries, we investigate system integration issues and scheduling algorithm optimizations of hybrid event processing based on discrete pattern in a single framework. First, hierarchical event stream storage is introduced to synchronize data access over live/archived event streams. Second, live and historical partial pattern matching caching mechanisms are proposed to provide effective partial pattern match search and update. Third, to reduce database access overhead, sub-window based event detect scheduling algorithms are proposed. Empirical performance study in a prototype hybrid event processing engine demonstrates effectiveness of our approaches.

Heterogeneity has to be taken into account when integrating a set of existing information sources into a distributed information system that are nowadays often based on Service- Oriented Architectures (SOA). This is also particularly applicable to distributed services such as event monitoring, which are useful in the context of Event Driven Architectures (EDA) and Complex Event Processing (CEP). Web services deal with this heterogeneity at a technical level, also providing little support for event processing. Our central thesis is that such a fully generic solution cannot provide complete support for event monitoring; instead, source specific semantics such as certain event types or support for certain event monitoring techniques have to be taken into account. Our core result is the design of a configurable event monitoring (Web) service that allows us to trade genericity for the exploitation of source specific characteristics. It thus delivers results for the areas of SOA, Web services, CEP and EDA.

SUMMARYRecently, event processing (EP) has gained considerable attention as an individual discipline in computer science. From a software engineering perspective, EP systems still lack the maturity of well‐established software architectures. For the development of industrial EP systems, generally accepted software architectures based on proven design patterns and principles are still missing. In this article, we introduce a catalog of design patterns that supports the development of event‐driven architectures (EDAs) and complex EP systems. The design principles originate from experiences reported in publications as well as from our own experiences in building EP systems with industrial and academic partners. We present several patterns on different layers of abstractions that define the overall structure as well as the building blocks for EP systems. Architectural patterns that determine the top‐level structure of an EDA can be distinguished from design patterns that specify the basic mechanisms of EP. The practical application of the catalog of patterns is described by the pattern‐based design of a sample EDA for a sensor‐based energy control system. Finally, we propose a coherent and general reference architecture for EP derived from the proposed patterns.Copyright © 2013 John Wiley & Sons, Ltd.

Goal of Complex Event Processing (CEP) is to identify meaningful events (such as opportunities or threats) and respond to them as quickly as possible. Traditional CEP frameworks do not have built-in security features. Any intruder may enter into the system and may send incorrect data to CEP engine. It causes to miss some patterns or it may lead to wrong prediction of patterns. Traditional CEP systems can be exploited through one or the other way. CEP system malfunctions when it is exploited. There is no built in procedure to check the health status of traditional CEP systems. Traditional CEP systems do not have procedures to take appropriate actions when system is affected or going to affect. When CEP system identifies some patterns, it will alert corresponding personnel about the occurrence of the event. Sometimes alert message may be undelivered or unnoticed by the corresponding people. Traditional CEP systems will not provide delivery assurance of alert message. These are few security challenges to existing CEP systems. In this paper Secure Complex Event Processing Framework is proposed to address above security challenges. Proposed framework is generic and it is pluggable into any existing CEP systems.

With the arrival of cloud computing and the internet of things, the difficulty of extracting data from the cloud services is preventing some organizations from adopting cloud computing. At the same time, the number of web services increases dramatically, and the granularity of service is different, so how to choose a suitable granularity service also becomes a problem that needs to be resolved. So a storage strategy for cloud computing services is given and the method based on event stream is used to construct the hierarchal model of services which can provide different functional services or different granularity services to meet the business requirements. Firstly, the storage strategy for cloud computing services is that, when registering the services, the meta-data of services will be extracted and stored into database according to certain structure. Secondly, the Event Cloud can be constructed as the one-way event processing, which provide the raw data for more-way event processing. Then the more-way event processing can construct the hierarchical model of services with certain composition goal. The prototype system has been developed in order to fulfill the application, both human events and RFID events can trigger other events in the cloud events, which can create new business models, improve business processes, and reduce costs and risks.

Complex Event Processing has been a growing field for the last ten years. It has seen the development of a number of methods and tools to aid in the processing of event streams and clouds though it has also been troubled by the lack of a cohesive definition. This paper aims to layout the technologies surrounding CEP and to distinguish it from the closely related field of Event Stream Processing. It also aims to explore the work done to apply Data Mining Techniques to both of these fields. An outline of stream processing technologies is laid out including the Data Stream Mining techniques that have been adapted for CEP.

With increasing demands for agility and reactivity, scientific circles and leading ICT companies have recognized the need for EDA(event-driven architecture) and complex event processing in factory automation, signal processing, data acquisition, manufacturing, andother applications of industrial electronics. We propose an EDA framework for defining and processing complex events based onontologies. The main novelty of our approach is introduction of semantic aspects into EDA. Our framework enables highly expressivesemantic definitions of complex events and a higher level of semantic integration of real-time events with the domain and eventontologies. The main component of the framework is the complex event service that acts as an event sink for member events and as anevent source for inferred complex events. Our approach is complementary to existing approaches and can enhance their support forcomplex events that require higher expressivity and semantic descriptions. Ill. 4, bibl. 22, tabl. 1 (in English; abstracts in English andLithuanian).http://dx.doi.org/10.5755/j01.eee.113.7.618