Modular Upper-Level Ontologies for Semantic Complex Event Processing

Complex event processing (CEP) systems have gained significant importance in various domains, such as finance, logistics, and security, where the real-time analysis of event streams is crucial. However, as the volume and complexity of event data continue to grow, optimizing the performance of CEP systems becomes a critical challenge. This paper investigates the impact of indexing strategies on the performance of databases handling complex event processing. We propose a novel indexing technique, called Hierarchical Temporal Indexing (HTI), specifically designed for the efficient processing of complex event queries. HTI leverages the temporal nature of event data and employs a multi-level indexing approach to optimize query execution. By combining temporal indexing with spatial- and attribute-based indexing, HTI aims to accelerate the retrieval and processing of relevant events, thereby improving overall query performance. In this study, we evaluate the effectiveness of HTI by implementing complex event queries on various CEP systems with different indexing strategies. We conduct a comprehensive performance analysis, measuring the query execution times and resource utilization (CPU, memory, etc.), and analyzing the execution plans and query optimization techniques employed by each system. Our experimental results demonstrate that the proposed HTI indexing strategy outperforms traditional indexing approaches, particularly for complex event queries involving temporal constraints and multi-dimensional event attributes. We provide insights into the strengths and weaknesses of each indexing strategy, identifying the factors that influence performance, such as data volume, query complexity, and event characteristics. Furthermore, we discuss the implications of our findings for the design and optimization of CEP systems, offering recommendations for indexing strategy selection based on the specific requirements and workload characteristics. Finally, we outline the potential limitations of our study and suggest future research directions in this domain.

With the broader dissemination of digital technologies, visionary concepts like the Internet of Things also affect an increasing number of use cases with interfaces to humans, e.g. manufacturing environments with technical operators monitoring the processes. This leads to additional challenges, as besides the technical issues also human aspects have to be considered for a successful implementation of strategic initiatives like Industrie 4.0. From a technical perspective, complex event processing has proven itself in practice to be capable of integrating and analyzing huge amounts of heterogeneous data and establishing a basic level of situation awareness by detecting situations of interests. Whereas this reactive nature of complex event processing systems may be sufficient for machine-to-machine use cases, the new characteristic of application fields with humans remaining in the control loop leads to an increasing action distance and delayed reactions. Taking human aspects into consideration leads to new requirements, with transparency and comprehensibility of the processing of events being the most important ones. Improving the comprehensibility of complex event processing and extending its capabilities towards an effective support of human operators allows tackling technical and non-technical challenges at the same time. The main contribution of this thesis answers the question of how to evolve state-of-the-art complex event processing from its reactive nature towards a transparent and holistic situation management system. The goal is to improve the interaction among systems and humans in use cases with interfaces between both worlds. Realizing a holistic situation management requires three missing capabilities to be introduced by the contributions of this thesis: First, based on the achieved transparency, the retrospective analysis of situations is enabled by collecting information related to a situation's occurrence and development. Therefore, CEP engine-specific situation descriptions are transformed into a common model, allowing the automatic decomposition of the underlying patterns to derive partial patterns describing the intermediate states of processing. Second, by introducing the psychological model of situation awareness into complex event processing, human aspects of information processing are taken into consideration and introduced into the complex event processing paradigm. Based on this model, an extended situation life-cycle and transition method are derived. The introduced concepts and methods allow the implementation of the controlling function of situation management and enable the effective acquisition and maintenance of situation awareness for human operators to purposefully direct their attention towards upcoming situations. Finally, completing the set of capabilities for situation management, an approach is presented to support the generation and integration of prediction models for predictive situation management. Therefore, methods are introduced to automatically label and extract relevant data for the generation of prediction models and to enable the embedding of the resulting models for an automatic evaluation and execution. The contributions are introduced, applied and evaluated along a scenario from the manufacturing domain.

As RFID is being applied in more and more business applications, events to be processed by RFID middleware is becoming more and more complex. It is needed that some original events are processed to create a complex event in order to get complex information. This paper introduced the design of an RFID middleware based on complex event process (CEP) concept, and presented checking method that is based on limited auto-machine and Hash. Moveover, a detail design of the event processing method of our RFID middleware is presented. Experiment study has also been performed with experimental results given.

In recent years there is a huge increase in real-time data, which cannot be processed efficiently. Complex event processing has become a very important method to get meaningful information. However, supporting complex event detection in multiple sources environments is a challenging problem. To allow for inferring high level information from vast amounts of continuous arriving data. In this paper, we present a complex event processing system based on a novel distributed computing platform Storm, which goes further than distributing queries and achieves better scalability by parallelizing event detection, and also higher efficiency through the use of some optimizations. The experimental shows that the event processing system is effective and better scalability.

Goal of Complex Event Processing (CEP) is to identify meaningful events (such as opportunities or threats) and respond to them as quickly as possible. Traditional CEP frameworks do not have built-in security features. Any intruder may enter into the system and may send incorrect data to CEP engine. It causes to miss some patterns or it may lead to wrong prediction of patterns. Traditional CEP systems can be exploited through one or the other way. CEP system malfunctions when it is exploited. There is no built in procedure to check the health status of traditional CEP systems. Traditional CEP systems do not have procedures to take appropriate actions when system is affected or going to affect. When CEP system identifies some patterns, it will alert corresponding personnel about the occurrence of the event. Sometimes alert message may be undelivered or unnoticed by the corresponding people. Traditional CEP systems will not provide delivery assurance of alert message. These are few security challenges to existing CEP systems. In this paper Secure Complex Event Processing Framework is proposed to address above security challenges. Proposed framework is generic and it is pluggable into any existing CEP systems.

In-network computing using programmable networking hardware is a strong trend in networking that promises to reduce latency and consumption of server resources through offloading to network elements (programmable switches and smart NICs). In particular, the data plane programming language P4 together with powerful P4 networking hardware has spawned projects offloading services into the network, e.g., consensus services or caching services. In this paper, we present a novel case for in-network computing, namely, Complex Event Processing (CEP). CEP processes streams of basic events, e.g., stemming from networked sensors, into meaningful complex events. Traditionally, CEP processing has been performed on servers or overlay networks. However, we argue in this paper that CEP is a good candidate for in-network computing along the communication path avoiding detouring streams to distant servers to minimize communication latency while also exploiting processing capabilities of novel networking hardware. We show that it is feasible to express CEP operations in P4 and also present a tool to compile CEP operations, formulated in our P4CEP rule specification language, to P4 code. Moreover, we identify challenges and problems that we have encountered to show future research directions for implementing full-fledged in-network CEP systems.

Complex event processing is a key technology for analyzing event patterns in real-time data streams. It has been widely applied in IoT monitoring, financial risk control, smart manufacturing and other time-sensitive fields. Query patterns serve as the core matching mechanism in complex event processing systems. These patterns are primarily defined through selection policies, predicate constraints and timestamps to establish complex event detection rules. The selection policy is particularly important as it determines the sequential relationships between events. This directly affects the system's event matching efficiency and accuracy. However, in practical applications, due to different business requirements for event sampling frequencies, event streams often contain numerous consecutive events of the same type representing identical states. While all these events participate in complex event matching, traditional processing methods generate large numbers of semantically redundant partial matches. Analysis shows that only a very small proportion of these intermediate results eventually form complete matches, creating serious computational redundancy issues. This redundancy not only significantly increases system resource consumption but also reduces overall processing efficiency due to ineffective computations. To address this critical problem, this paper innovatively proposes a cache-based optimized selection policy. The core concept of this policy is to effectively reduce redundant matching results by extracting representative events from continuous event sequences. Experimental results show that this policy can significantly decrease the redundant matching rate while alleviating system load, providing new insights for optimizing complex event processing systems.

Complex event processing (CEP) over event streams has become increasingly important for real-time applications ranging from health care, supply chain management to business intelligence. These monitoring applications submit complex event queries to track sequences of events that match a given pattern. As these systems mature the need for increasingly complex nested sequence query support arises, while the state-of-art CEP systems mostly support the execution of only flat sequence queries. In this paper, we introduce our nested CEP query language NEEL for expressing nested queries composed of sequence, negation, AND and OR operators. Thereafter, we also define its formal semantics. Subtle issues with negation and predicates within the nested sequence context are discussed. An E-Analytics system for processing nested CEP queries expressed in the NEEL language has been developed. Lastly, we demonstrate the utility of this technology by describing a case study of applying this technology to a real-world application in health care.

Complex Event Processing (CEP) systems extract interest situations from event flows based on event detection patterns. However, local event processing for distributed Internet of Things (IoT) has not been discussed yet. Besides, it is complex or impossible to discover such patterns in some applications of IoT. In this article, we design a complex event service to process event flows based on IoT resource models, which does not depend on existing patterns, and deals with both discrete events and continuous variables. To improve the CEP performance, local IoT resources are used for local event processing, and a lazy exchange method is designed to realize the collaborated event processing between network edges and a data center. Our evaluation shows that our solution is feasible and effective.

Enterprises have to be increasingly agile and responsive to address the challenges posed by the fast moving market. With the software architecture evolving into service-oriented architecture (SOA), and the adoption of radio frequency identification (RFID), event processing can fit well in enterprise information systems in terms of facilitation of event aggregation into high level actionable information, and event response to improve the responsiveness. To make it more applicable, the architecture of event processing in enterprise information systems is proposed; event meta model and context serve as the solid basis for event processing; the rules, operators and keys of complex event processing are defined. Especially, workflow model is firstly used to extract complex event pattern. We have implemented the event processing mechanism in enterprise information systems based on RFID, including the architecture, data structures, optimization strategies and algorithm. The performance evaluations show that the method is effective in terms of scalability and the capability of event processing. Complex event processing can improve operational performance and discover more actionable information, which is justified by application. Finally, lessons learned from the application are presented.

In this paper, we present a powerful logic-based approach for Complex Event Processing (CEP). The approach is founded on a logical transformation of event processing into logic programs. Many systems for event processing have ad-hoc semantics with unexpected behaviors. Further on, many formal approaches (with well defined semantics) cannot effectively be used in CEP, due to their inability to compute events in the data-driven fashion. Our approach enables both logic-based and data-driven complex event detection. Moreover, the backward chaining approach allows for very efficient reasoning of complex events and actions triggered by these events.

Event processing (EP) is a data processing technology that conducts online processing of event information. In this survey, we summarize the latest cutting-edge work done on EP from both industrial and academic research community viewpoints. We divide the entire field of EP into three subareas: EP system architectures, EP use cases, and EP open research topics. Then we deep dive into the details of each subsection. We investigate the system architecture characteristics of novel EP platforms, such as Apache Storm, Apache Spark, and Apache Flink. We found significant advancements made on novel application areas, such as the Internet of Things; streaming machine learning (ML); and processing of complex data types such as text, video data streams, and graphs. Furthermore, there has been significant body of contributions made on event ordering, system scalability, development of EP languages and exploration of use of heterogeneous devices for EP, which we investigate in the latter half of this article. Through our study, we found key areas that require significant attention from the EP community, such as Streaming ML, EP system benchmarking, and graph stream processing.

Complex event processing (CEP) is a technique for analyzing and correlating large amount of information about events that happen in a timely manner, and being in a position to derive conclusions or even respond to them as quickly as possible. Complex events are raised based on incoming sources productions and according to a set of user-defined rules. However, as the complexity of CEP systems grow, the process for manually defining rules becomes time and resource consuming or even impossible as dynamic changes occur in the domain environment. Moreover, it restricts the use of CEP to merely the detection of straightforward situations than in more advanced fields that require earliness and prediction. Therefore, we present a novel approach for completing the supervision of an unsupervised structure learning task. More precisely, we propose to incorporate an unsupervised technique that derives labels for unlabelled data, depended on their distance. From these results, we automatically generate CEP rules to feed the system. In order to evaluate our approach, we used a real world data-set with data labeled by experts. The evaluation indicates that our approach can effectively complete the missing labels and, in some cases, improve the accuracy of the underlying CEP structure learning system.

The execution of complex event processing (CEP) applications on a set of clustered homogenous computing nodes is latency-sensitive, especially when workload conditions widely change at runtime. To manage the varying workloads of nodes in a scalable and cost-effective manner, adjusting the application parallelism at runtime is critical. To tackle the scalability challenge, we have extended an existing parallelization model called PARS that only supports stateless CEP operators and runs operators in parallel regions without changing the number of computing nodes assigned to parallel regions. We have added new features to PARS in support of stateful operators by introducing local controllers and new initiator and terminator event types, making partitioning fully transparent to application developers. We have proved the correctness of this extended model, called PARS+ , with respect to its presented formal definition. We have then used PARS+ as the base parallelization model in formulation of an adaptive strategy called ACEP to auto-scale operators including the stateful operators. Scaling decisions are governed by a predictive performance model that uses a control-theoretic method for estimating the resource and latency costs of each operator at runtime. The loads of clustered compute nodes are monitored, and compute nodes in a parallel region are reconfigured at runtime to ensure a balanced load on all compute nodes, accruing minimum cost to parallelize a stateful operator. ACEP minimizes network delays because it does not force using the shared state or techniques that employ state migration. We have built an event generator to simulate event sources and experimentally evaluate the ACEP strategy in terms of response time and resource costs. Two variant implementations of ACEP have been compared with the elastic strategy presented by Xiao et al. (2018), and the findings demonstrated that our implementations had adapted themselves better in different resource and response-time-sensitive scenarios, with lower response time (6%) and lower resource cost (8%).

With increasing demands for agility and reactivity, scientific circles and leading ICT companies have recognized the need for EDA(event-driven architecture) and complex event processing in factory automation, signal processing, data acquisition, manufacturing, andother applications of industrial electronics. We propose an EDA framework for defining and processing complex events based onontologies. The main novelty of our approach is introduction of semantic aspects into EDA. Our framework enables highly expressivesemantic definitions of complex events and a higher level of semantic integration of real-time events with the domain and eventontologies. The main component of the framework is the complex event service that acts as an event sink for member events and as anevent source for inferred complex events. Our approach is complementary to existing approaches and can enhance their support forcomplex events that require higher expressivity and semantic descriptions. Ill. 4, bibl. 22, tabl. 1 (in English; abstracts in English andLithuanian).http://dx.doi.org/10.5755/j01.eee.113.7.618