Complex Event Recognition Using Constrained Rank Optimization

While approaches based on bags of features excel at low-level action classification, they are ill-suited for recognizing complex events in video, where concept-based temporal representations currently dominate. This paper proposes a novel representation that captures the temporal dynamics of windowed mid-level concept detectors in order to improve complex event recognition. We first express each video as an ordered vector time series, where each time step consists of the vector formed from the concatenated confidences of the pre-trained concept detectors. We hypothesize that the dynamics of time series for different instances from the same event class, as captured by simple linear dynamical system (LDS) models, are likely to be similar even if the instances differ in terms of low-level visual features. We propose a two-part representation composed of fusing: (1) a singular value decomposition of block Hankel matrices (SSID-S) and (2) a harmonic signature (HS) computed from the corresponding eigen-dynamics matrix. The proposed method offers several benefits over alternate approaches: our approach is straightforward to implement, directly employs existing concept detectors and can be plugged into linear classification frameworks. Results on standard datasets such as NIST's TRECVID Multimedia Event Detection task demonstrate the improved accuracy of the proposed method.

In the context of computer vision, most of the traditional action recognition techniques assign a single label to a video after analyzing the whole video. We believe that under- standing of the visual world is not limited to recognizing a specific action class or individual object instances, but also extends to how those objects interact in the scene, which im- plies recognizing events happening in the scene. In this paper we present an approach for identifying complex events in videos, starting from detection of objects and simple events using a state-of-the-art object detector (YOLO). We provide a logic based representation of events by using a realization of the Event calculus that allows us to define complex events in terms of logical rules. Axioms of the calculus are encoded in a logic program under Answer Set semantics in order to reason and formulate queries over the extracted events. The applicability of the framework is demonstrated over the scenario of recognizing different kinds of kick events in soccer videos.

The growing popularity of online Earth visualization tools and geo-realistic games and the availability of high resolution terrain data have motivated a new class of queries to the interests of the GIS and spatial database community: spatial queries (e.g., kNN) over land surface. However, the fundamental challenges that restrict the applicability of these studies to real world applications are the prohibitive time complexity and storage overhead to precompute the shortest surface paths. In this paper, for the first time, we propose an approximate solution to address both challenges and allow browsing the shortest surface paths in O(log N + √N) time, where N is the size of the terrain. With this method, the time and space requirements for an exhaustive all-pair pre-computation have been reduced from O(N3) to O(N1.5) and O(N) respectively. The substantial savings in both time and storage are gained by taking advantage of the fact that the O(N2) surface paths only deviate from approximate straight lines at O(√N) points, termed rough vertices. As a result, we propose a linear time shortest surface path computation algorithm between two arbitrary vertices and a linear size storage structure, which captures all the shortest surface paths between any pair of vertices. We experimentally verified the applicability and scalability of the proposed methods with large real world and synthetic data sets and showed that accuracy higher than 97% can be obtained in most cases.

Emerging as an alternative to databases for continuous data processing, stream processing has evolved significantly since its inception in the early 2000s, leading to the emergence of numerous Stream Processing Engines (SPEs).

We present a system for Complex Event Recognition (CER) based on automata. While multiple such systems have been described in the literature, they typically suffer from a lack of clear and denotational semantics, a limitation which often leads to confusion with respect to their expressive power. In order to address this issue, our system is based on an automaton model which is a combination of symbolic and register automata. We extend previous work on these types of automata, in order to construct a formalism with clear semantics and a corresponding automaton model whose properties can be formally investigated. We call such automata Symbolic Register Transducers (SRT). The distinctive feature of SRT , compared to previous automaton models used in CER, is that they can encode patterns relating multiple input events from an event stream, without sacrificing rigor and clarity. We show how SRT can be used in CER in order to detect patterns upon streams of events, using our framework that provides declarative and compositional semantics. We also compare our SRT -based CER engine against other state-of-the-art CER systems and show that it is both more expressive and more efficient.

Privacy-preserving record linkage on large real world datasets

Utilizing reasonable components in his plays, Pinter’s dialect is not confusing or surprising, but genuine-because of its unadulterated English dialect. The Birthday Party is Pinter’s second full-length play and one of the most celebrated plays categorized under the title of the Theater of Absurd. This masterpiece follows the dilemma in the dialogue between the main characters. Meg, Petey, and Stanley demonstrate the preposterousness with the embodiment of human identity under practical components, such as goodness. The play’s central theme is loneliness and explains the story of the reborn man who realizes that his life cannot continue in the boarding house. The Birthday Party is outfitted with all the sensible components of an impeccable diversion; genuine characters with genuine sentiments, unpretentious dangers, ghastliness fulfilled with an inevitable setting that accentuates to blend of the real world and drivel. In The Birthday Party, the solidarity in the real world is clearly described, and the absurdity of social status by the real characters is uncovered. The threats of exploiting the hearts and minds of postwar individuals have shocked them both physically and mentally. Individuals did not want to be exposed to the outside world while they were afraid of strangers in society. To be amusing, the play coasts the title of Men’s Comedy of Menace while battling with the genuine risk of exploring the focal character.

The rise of the ``big data'' era has created a pressing demand for educating many data scientists and engineers quickly at low cost. It is essential they learn by working on assignments that involve real world data sets to develop the skills needed to be successful in the workplace. However, enabling instructors to flexibly deliver all kinds of data science assignments using real world data sets to large numbers of learners (both on-campus and off-campus) at low cost is a significant open challenge. To address this emerging challenge generally, we develop and deploy a novel Cloud-based Lab for Data Science (CLaDS) to enable many learners around the world to work on real-world data science problems without having to move or otherwise distribute prohibitively large data sets. Leveraging version control and continuous integration, CLaDS provides a general infrastructure to enable any instructor to conveniently deliver any hands-on data science assignment that uses large real world data sets to as many learners as our cloud-computing infrastructure allows at very low cost. In this paper, we present the design and implementation of CLaDS and discuss our experience with using CLaDS to deploy seven major text data assignments for students in both an on-campus course and an online course to work on for learning about text data retrieval and mining techniques; this shows that CLaDS is a very promising novel general infrastructure for efficiently delivering a wide range of hands-on data science assignments to a large number of learners at very low cost.

A2E2: Aerial-assisted energy-efficient edge sensing in intelligent public transportation systems

In this paper, we model the similar question retrieval task as a binary classification problem. We propose a novel approach of "1D-Siamese LSTM for cQA (1D-SLcQA)" to find the semantic similarity between a new question and existing question(s). In 1D-SLcQA, we use a combination of twin LSTM networks and a contrastive loss function to effectively memorize the long term dependencies i.e., capture semantic similarity even when the length of the answers/questions is very large (200 words). The similarity of the questions is modeled using a single network with (1D) (feature) convolution between feature vectors learned from twin LSTM layers. Experiments on large scale real world Yahoo Answers dataset show that 1D-SLcQA outperform the state of the art approach of Siamese cQA approach (SCQA).

Numerical data (e.g., DNA micro-array data, sensor data) pose a challenging problem to existing frequent pattern mining methods which hardly handle them. In this framework, gradual patterns have been recently proposed to extract covariations of attributes, such as: When X increases, Y decreases. There exist some algorithms for mining frequent gradual patterns, but they cannot scale to real-world databases. We present in this paper GLCM, the first algorithm for mining closed frequent gradual patterns, which proposes strong complexity guarantees: the mining time is linear with the number of closed frequent gradual item sets. Our experimental study shows that GLCM is two orders of magnitude faster than the state of the art, with a constant low memory usage. We also present PGLCM, a parallelization of GLCM capable of exploiting multicore processors, with good scale-up properties on complex datasets. These algorithms are the first algorithms capable of mining large real world datasets to discover gradual patterns.

Entity resolution is a challenging problem with unresolved and duplicated entities common in many large real world datasets. New methods are required for addressing this problem as the use of graphs to model data continues to proliferate. In this paper we propose a general framework for the fast resolution of duplicate vertices in graphs. Our framework utilises locality sensitive hashing for the quick identification of potential duplicates based on string similarity. However it is clear that in many tasks string similarity alone is not enough to determine duplication. This motivates the second aspect of our method which discovers the community structure in the graph using an ensemble of community detection algorithms. These communities are then used to augment the string similarity in the deduplication process. We evaluate our approach on a real world graph consisting of 620885 vertices and 1129986 edges and report a high accuracy score on a commercial real world graph.

Analysis and detection of complex events in videos require a semantic representation of the video content. Existing video semantic representation methods typically require users to pre-define an exhaustive concept lexicon and manually annotate the presence of the concepts in each video, which is infeasible for real-world video event detection problems. In this paper, we propose an automatic semantic concept discovery scheme by exploiting Internet images and their associated tags. Given a target event and its textual descriptions, we crawl a collection of images and their associated tags by performing text based image search using the noun and verb pairs extracted from the event textual descriptions. The system first identifies the candidate concepts for an event by measuring whether a tag is a meaningful word and visually detectable. Then a concept visual model is built for each candidate concept using a SVM classifier with probabilistic output. Finally, the concept models are applied to generate concept based video representations. We use the TRECVID Multimedia Event Detection (MED) 2013 as our video test set and crawl 400K Flickr images to automatically discover 2, 000 visual concepts. We show significant performance gains of the proposed concept discovery method over different video event detection tasks including supervised event modeling over concept space and semantic based zero-shot retrieval without training examples. Importantly, we show the proposed method of automatic concept discovery outperforms other well-known concept library construction approaches such as Classemes and ImageNet by a large margin (228%) in zero-shot event retrieval. Finally, subjective evaluation by humans also confirms clear superiority of the proposed method in discovering concepts for event representation.

Forensic analysis commonly involves searching an investigation target for personal identifiable information. An Inland Revenue Department (IRD) number is used for taxation purposes in New Zealand and can provide evidence of perpetrator identity, transaction information or electronic fraud. This research has designed and implemented a bulk_extractor feature scanner to detect and validate IRD numbers (features). The IRD scanner has been tested on a known data set to ensure tool functionality. A large real world data set was then used to determine scanner effectiveness in a realistic investigation scenario. Real world data set testing highlighted a high number of unrelated features detected by the scanner. To combat this, a novel post-processing technique was implemented to identify forensically interesting IRD numbers by performing feature context searching. The post-processing findings proved that feature context searching is an effective data reduction technique that identified a low number of directly relevant IRD numbers.

The notion of "events" is extremely important in characterizing the contents of video. An event is typically triggered by some kind of change of state captured in the video, such as when an object starts moving. The ability to reason with events is a critical step toward video understanding. This article describes the findings of a recent workshop series that has produced an ontology framework for representing video events-called Video Event Representation Language (VERL) -and a companion annotation framework, called Video Event Markup Language (VEML). One of the key concepts in this work is the modeling of events as composable, whereby complex events are constructed from simpler events by operations such as sequencing, iteration, and alternation. The article presents an extensible event and object ontology expressed in VERL and discusses a detailed example of applying VERL and VEML to the description of a "tailgating" event in surveillance video.