A Markov logic framework for recognizing complex events from multimodal data

Event-driven Process Chain (EPC) is a known business process modeling language which is primarily renowned for efficient modeling of business requirements. It is a semi-formal modeling language which is commonly verified though various formal mechanism for simple atomic events. However, the complexity of business requirements cannot be modeled through EPC atomic events alone especially in case of real time systems. Therefore, the complex events patterns like Event Time, Event location, Event Cardinality, Event Sequence, Event Exclusion, Event Trend and Data dependency are introduced in EPC from the novel area of complex event processing. However, to the best of our knowledge, there is no mechanism available in EPC to model and verify such complex events. Therefore, in this paper, we investigate the verification possibilities of both (atomic and complex) events in the context of Timed-Automata (TA) and Time Petri Nets (TPN). Particularly, we try to map the semantics of simple and complex events with the corresponding concepts of TA and TPN. It is analyzed that the simple EPC events can be formalized through TA and TPN but limited support is available for complex events. The restaurant process case study has been used for further investigation of complex events. It has been concluded that TA formalism is more suitable than TPN for the verification of complex events in EPC.

Autonomous machines such as a planetary explorer must learn from the real universe, which can be seen as a single event of infinite complexity or as an infinite set of trivial events. Current inductive learning systems generalize from a finite set of events provided by a human teacher or a software environment. Even a constrained universe is difficult to represent as a finite set of events comprising various types of useful information. Cues used by humans to perform this representation task include temporal or spatial relationships among environmental data. We discuss a method for identifying meaningful complex events in the universe of an autonomous learning robot. When no temporal concept is known to link two descriptors, temporal proximity of events is used to pair simple descriptor events into complex events for inductive learning. When a temporal concept is known to link two descriptors, that concept is used to guide the pairing of descriptor events. We discuss the efficiency improvements which arise from using temporal concepts in this manner. The method is embedded in GPAL 1.3, a general purpose autonomous learner that uses a knowledge-based learning and control strategy which models the scientific method.

Event recognition becomes increasingly important and attracts growing interest from both academia and industry. It can provide awareness on possible threats and incidents such as acts of terrorism, illegal immigration, violence in crowded areas, and so on. It is essential to prepare and produce real-time and effective responses. However, it still faces great challenges when it comes to complex events that include more than one object/person. In this work, we target the problem of complex event recognition in visual surveillance by utilizing Unmanned Aerial Systems (UAS). Our approach is to detect and recognize relationship between events and object by determine set of predicates. We apply Markov Logic Network (MLN) to available online datasets to perform probabilistic inference for achieving this goal through grouping people together, who are doing same activities, and to recognize what these activities are. The performance of our approach is demonstrated on a number of existing videos obtained from UCLA that monitored wide spatial areas taking by flying UAS that contain complex relation between objects and activities.

Recognizing inference in texts (RITE) attracts growing attention of natural language processing (NLP) researchers in recent years. In this article, we propose a novel approach to recognize inference with probabilistic logical reasoning. Our approach is built on Markov logic networks (MLNs) framework, which is a probabilistic extension of first-order logic. We design specific semantic rules based on the surface, syntactic, and semantic representations of texts, and map these rules to logical representations. We also extract information from some knowledge bases as common sense logic rules. Then we utilize MLNs framework to make predictions with combining statistical and logical reasoning. Experiment results shows that our system can achieve better performance than state-of-the-art RITE systems.

In the context of computer vision, most of the traditional action recognition techniques assign a single label to a video after analyzing the whole video. We believe that under- standing of the visual world is not limited to recognizing a specific action class or individual object instances, but also extends to how those objects interact in the scene, which im- plies recognizing events happening in the scene. In this paper we present an approach for identifying complex events in videos, starting from detection of objects and simple events using a state-of-the-art object detector (YOLO). We provide a logic based representation of events by using a realization of the Event calculus that allows us to define complex events in terms of logical rules. Axioms of the calculus are encoded in a logic program under Answer Set semantics in order to reason and formulate queries over the extracted events. The applicability of the framework is demonstrated over the scenario of recognizing different kinds of kick events in soccer videos.

Composite event monitoring in XML repositories using generic rule framework for providing reactive e-services

The notion of "events" is extremely important in characterizing the contents of video. An event is typically triggered by some kind of change of state captured in the video, such as when an object starts moving. The ability to reason with events is a critical step toward video understanding. This article describes the findings of a recent workshop series that has produced an ontology framework for representing video events-called Video Event Representation Language (VERL) -and a companion annotation framework, called Video Event Markup Language (VEML). One of the key concepts in this work is the modeling of events as composable, whereby complex events are constructed from simpler events by operations such as sequencing, iteration, and alternation. The article presents an extensible event and object ontology expressed in VERL and discusses a detailed example of applying VERL and VEML to the description of a "tailgating" event in surveillance video.

While approaches based on bags of features excel at low-level action classification, they are ill-suited for recognizing complex events in video, where concept-based temporal representations currently dominate. This paper proposes a novel representation that captures the temporal dynamics of windowed mid-level concept detectors in order to improve complex event recognition. We first express each video as an ordered vector time series, where each time step consists of the vector formed from the concatenated confidences of the pre-trained concept detectors. We hypothesize that the dynamics of time series for different instances from the same event class, as captured by simple linear dynamical system (LDS) models, are likely to be similar even if the instances differ in terms of low-level visual features. We propose a two-part representation composed of fusing: (1) a singular value decomposition of block Hankel matrices (SSID-S) and (2) a harmonic signature (HS) computed from the corresponding eigen-dynamics matrix. The proposed method offers several benefits over alternate approaches: our approach is straightforward to implement, directly employs existing concept detectors and can be plugged into linear classification frameworks. Results on standard datasets such as NIST's TRECVID Multimedia Event Detection task demonstrate the improved accuracy of the proposed method.

Turbidity behavior in streams is a complex and dynamic function of both source material supply and event‐driven transport. While the primary controls on turbidity behavior across time and space are still not fully understood, recent increases in the availability of high temporal resolution, colocated stream turbidity, and discharge data provide an opportunity for more detailed analysis. Here we examine methods to quantitatively characterize event responses by modeling the shape of turbidity‐discharge hysteresis loops. A total of 1559 events from 20 gages in the Mid‐Atlantic region of the U.S. were modeled using both previously reported models and new models combining elements of existing models. The results suggest that a more general power law‐based model, utilizing both a discharge rate of change term and a “supply” term, allows characterization of a wide range of simple and complex events. Additionally, this study explores a decorrelation approach to address the strong correlation frequently observed between the power law model coefficient ( ) and exponent ( ), with the goal of exposing the underlying behavior of each parameter individually. An examination of seasonal parameter behavior suggests that this approach may facilitate greater physically based interpretation of the power law coefficient. The power law parameter decorrelation strategy and the loop models examined here provide a step toward the larger goal of understanding the physical controls on turbidity‐discharge hysteretic behavior.

Recent advancements in digital technologies have profoundly transformed healthcare delivery, sports analytics, and physical activity surveillance. The accelerated proliferation of wearable sensing devices and Internet of Things (IoT) infrastructures has facilitated unprecedented large-scale acquisition of multimodal physiological and kinematic data. However, accurate recognition of complex human activities remains a formidable challenge, primarily due to inadequate modeling of the spatiotemporal dependencies inherent in wearable-sensor signals. To address these fundamental limitations, this paper proposed a novel IoT-oriented activity recognition framework predicated on a Convolutional Recurrent Neural Network (CRNN) architecture, engineered to simultaneously model the spatial and temporal characteristics of multimodal wearable-sensor data streams. The proposed framework synergistically integrates Convolutional Neural Networks (CNNs) for hierarchical spatial feature extraction with Recurrent Neural Networks (RNNs) for temporal sequence modeling, thereby enabling more discriminative and robust activity classification. The methodological pipeline comprises several sequential stages: First, multidimensional wearable-sensor datasets are employed, encompassing physiological and inertial measurements including heart rate variability, triaxial accelerometer readings, gyroscopic angular velocity, magnetometer orientation data, and cutaneous temperature signals acquired from multiple anatomical locations. Second, raw sensor signals undergo preprocessing and temporal segmentation procedures to enhance data quality and optimize temporal representation. Third, spatiotemporal feature representations are learned autonomously within the hierarchical CRNN architecture. Finally, the proposed model is systematically evaluated through a comparative analysis with five representative baseline methods, encompassing both conventional machine learning algorithms and contemporary deep learning approaches. Experimental results demonstrate that the proposed CRNN framework achieves superior performance, achieving 98.2% classification accuracy, 97.2% sensitivity, 99.2% specificity, 97.4% recall, and 97.6% precision on the evaluated wearable-sensor datasets. Compared with existing methodologies, the proposed model consistently achieves higher recognition accuracy and greater generalization robustness, underscoring its efficacy for wearable-sensor-based activity recognition and its considerable potential for broader deployment in IoT-enabled monitoring paradigms and adaptable solutions in educational settings.

Complex event recognition is the problem of recognizing events in long and unconstrained videos. Examples of such events are birthday party, changing a tire, making a fire …etc. In this extremely challenging task, concepts have recently shown a promising direction, where core low-level events referred to as concepts are annotated and modelled using a portion of the training data, then each complex event is described using concept scores, which are features representing the occurrence confidence for the concepts in the event. However, because of the complex nature of the videos, both the concept models and the corresponding concept scores are significantly noisy. In order to address this problem, we discuss a low-rank formulation, which combines the precisely annotated videos used to train the concepts, with the rich concept scores. The approach presented in this chapter finds a new representation for each event, which is not only low-rank, but also constrained to adhere to the concept annotation, thus suppressing the noise, and maintaining a consistent occurrence of the concepts in each event. Extensive experiments on large scale real world dataset TRECVID Multimedia Event Detection 2011 and 2012 demonstrate that the approach consistently improves the discriminativity of the concept scores by a significant margin.KeywordsComplex Event RecognitionConcept ScoresOccurrence ConfidenceUnconstrained VideosLarge-scale Real-world DatasetsThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

An intelligent agent interacting with the real world will encounter individual people, courses, test results, drugs prescriptions, chairs, boxes, etc., and needs to reason about properties of these individuals and relations among them as well as cope with uncertainty. Uncertainty has been studied in probability theory and graphical models, and relations have been studied in logic, in particular in the predicate calculus and its extensions. This book examines the foundations of combining logic and probability into what are called relational probabilistic models. It introduces representations, inference, and learning techniques for probability, logic, and their combinations. The book focuses on two representations in detail: Markov logic networks, a relational extension of undirected graphical models and weighted first-order predicate calculus formula, and Problog, a probabilistic extension of logic programs that can also be viewed as a Turing-complete relational extension of Bayesian networks.

Is stillbirth associated with long-term atherosclerotic morbidity?

People often have knowledge about the chances of events but are unable to express their knowledge in the form of coherent probabilities. This study proposed to correct incoherent judgment via an optimization procedure that seeks the (coherent) probability distribution nearest to a judge's estimates of chance. This method was applied to the chances of simple and complex meteorological events, as estimated by college undergraduates. No judge responded coherently, but the optimization method found close (coherent) approximations to their estimates. Moreover, the approximations were reliably more accurate than the original estimates, as measured by the quadratic scoring rule. Methods for correcting incoherence facilitate the analysis of expected utility and allow human judgment to be more easily exploited in the construction of expert systems.

This paper investigates the distribution of reflexive and nonreflexive pronouns in the prepositional phrase, concluding that multiple semantic factors play a role in the appearance of one pronoun over the other. The distributional trends in English are explained by referencing the crucial role space plays in grammar, and the resulting implications for Binding Theory (Chomsky 1995) are discussed. The motivating forces for the corpus distribution are based on perceived directionality and location of the denoted event with respect to the body of the event's protagonist. The patterns found in the corpus data are attributed to a range of factors that play a role in the semantic specifications and associations of the pronouns themselves. First, it is argued that the high rate of reflexive pronouns in events that are metaphorically located in the body is due to the reflexive pronoun's close semantic association with the concept of self, a metaphorical body-internal entity. Second, it is argued that the reflexive pronoun is used to signal either an event which is performed on the body (in the referent's peri-personal space) or directed toward the body. Cases of these types are explained by a schematic, semantic parallelism between syntactically complex reflexive events and syntactically simple reflexive events. In both cases, the reflexive pronoun signals a contrastive element. In syntactically complex cases, the PP examples (e.g. John pushed the box toward himself), and syntactically simple cases, those with basic clause structure (e.g. John kicked himself), the reflexive is used to signal that the direction of the event is counter to the direction of expectation, thus explaining why certain reflexive events (e.g. bathe, or pull something toward you) do not have to, and most often, do not occur with the reflexive pronoun.