High Level Event Detection based on Spatial Occupancy and Interpersonal Relationships DOI: 10.14209/jcis.2015.8

With rapid growth of egocentric videos from wearable devices, the need for instant video event detection is emerging. Different from conventional video event detection, it requires more considerations on real-time event detection and immediate video recording due to the computational cost on wearable devices (e.g., Google Glass). Conventional work of video event detection analyzed video content in an offline process and it is time-consuming for visual analysis. Observing that wearable devices are usually along with sensors, we propose a novel approach for instant event detection in egocentric videos by leveraging sensor-based motion context. We compute statistics of sensor data as features. Next, we predict the user's current motion context by a hierarchical model, and then choose the corresponding ranking model to rate the importance score of the timestamp. With importance score provided in real-time, camera on the wearable device can dynamically record micro-videos without wasting power and storage. In addition, we collected a challenging daily-life dataset called EDS (Egocentric Daily-life Videos with Sensor Data), which contains both egocentric videos and sensor data recorded by Google Glass of different subjects. We evaluate the performance of our system on the EDS dataset, and the result shows that our method outperforms other baselines.

The paper presents a novel approach for event detection in sports videos by topic based graphical model learning. The characteristics features defining various sport events are extracted by contextual grouping of low-level video and audio features using topic modeling. Event detection is performed by learning the structure of context based distribution of characteristic features by CRF based graphical model. Experimental evaluation of the proposed concept is presented on recorded video of Handball and Soccer game.

This paper offers a natural extension of the newly introduced anti-face method to event detection, in both the gray-level and feature domains. For the gray-level domain, spatio-temporal templates are created by stacking the individual frames of the video sequence, and the detection is performed on these templates. In order to recognize the motion of features in a video sequence, the spatial locations of the features are modulated in time, thus creating a one-dimensional vector which represents the event in the detection process. The following applications are presented: 1) detection of an object under three-dimensional (3-D) rotations in a video sequence simulated from the COIL database; 2) visual recognition of spoken words; and 3) recognition of two-dimensional and 3-D sketched curves. The technique is capable of detecting 3-D curves in viewing directions which substantially differ from those in the training set. The resulting detection algorithm is very fast and can successfully detect events even in very low resolution. Also, it is capable of discriminating the desired event from arbitrary events, and not only from those in a negative training set. Possible applications of the techniques offered in this paper are in man-machine interaction, surveillance, and search and summarization in video databases.

Automatic event detection in cell videos is essential for monitoring cell populations in biomedicine. Deep learning methods have advantages over traditional approaches for cell event detection due to their ability to capture more discriminative features of cellular processes. Supervised deep learning methods, however, are inherently limited due to the scarcity of annotated data. Unsupervised deep learning methods have shown promise in general (non-cell) videos because they can learn the visual appearance and motion of regularly occurring events. Cell videos, however, can have rapid, irregular changes in cell appearance and motion, such as during cell division and death, which are often the events of most interest. We propose a novel unsupervised two-path input neural network architecture to capture these irregular events with three key elements: 1) a visual encoding path to capture regular spatiotemporal patterns of observed objects with convolutional long short-term memory units; 2) an event detection path to extract information related to irregular events with max-pooling layers; and 3) integration of the hidden states of the two paths to provide a comprehensive representation of the video that is used to simultaneously locate and classify cell events. We evaluated our network in detecting cell division in densely packed stem cells in phase-contrast microscopy videos. Our unsupervised method achieved higher or comparable accuracy to standard and state-of-the-art supervised methods.

This paper proposes human tracking and recognition method in a camera network. Human matching in a multi-camera surveillance system is a fundamental issue for increasing the accuracy of recognition in multiple views of cameras. In camera network, videos have different characteristics such as pose, scale and illumination. Therefore it is necessary to use a hybrid scheme of scale invariant feature transform to detection and recognition human's behaviors. The main focus of this paper is to analyze activities for tracking and recognition humans to extract trajectories. Extracting the trajectories help to detect abnormal behavior which may be occluded in single- camera surveillance. KEYWORDS: Camera network, Multi-camera surveillance, Human's behavior, Trajectories extraction. I. INTRODUCTION Tracking and behavior recognition are two fundamental tasks in video surveillance systems which are widely employed in commercial applications for purposes of statistics gathering and processing. The number of cameras and complexity of surveillance systems have been continuously increasing to have better coverage and accuracy. Multi-camera systems become increasingly attractive in machine vision. Applications include multi view object tracking, event detection, occlusion handling and etc. In this paper, we develop method for tracking and recognition by a traffic video surveillance system of two cameras with a partially overlapping field of view. This paper is organized as follows: an overview of the past works in section2. Our proposed architecture and algorithm is presented in section3. Results of subjective evaluations and objective performance measurements with respect to Ground-truth are presented in section4. Section5 contains the conclusion.

The security analysis of sensitive issues and medical diagnosis are immensely focus to determine exact location of event happening regions. In this paper we propose a model of clustering and pooling techniques to local features using Bag-of-Words (BoW) descriptor in SVM framework for event detection in video sequences. The proposed model extracts local features from six categories of Columbia Consumer Video (CCV) event detection benchmark. We developed the clusters of these features using KD-search tree and Lloyds algorithm. The clusters of features is pooled to vectors by using bag of words model. Introducing the inferring temporal instance labelling, the model performed fast for event detection. The significant performance of the research problem can thrilled out the social media by retrieving the best possible content. The proposed model can efficiently perform the experiments of event detection related to big data problem in visual media. Furthermore, the proposed approach in the model is invariant to rotation, translation and scale changes in the video sequence and robust to the illumination and viewpoints.

Event detection in crowded surveillance videos is a challenging yet important problem. In this paper, we present our eSur (Event detection system on SURveillance video) system, which is derived from TRECVid'12 surveillance tasks. Currently, eSur attempts to detect two categories of events: 1) pair-wise events (e.g., PeopleMeet, PeopleSplitUp and Embrace); 2) action-like events (e.g., ObjectPut, CellToEar, PersonRuns and Pointing). In eSur system, we first employ people detection and tracking algorithms to locate target persons in 3D space-time domain. Then the video sequences in which target persons occur are partitioned into several spatio-temporal cubes. Visual features (i.e. cubic feature and MoSIFT) are computed over these cubes. After that, a sequence learning method, (namely SVM with dynamic time alignment kernel), is employed to infer the existence of an event for the video sequence. According to the TRECVid SED formal evaluation, eSur has yielded fairly encouraging results on TRECVid'12 dataset.

Surveillance systems are mainly used to monitor activity and behaviour for the purpose of managing small human group. In particularly, the depicted approach focused on the small human groups stayed in the same place for a while and characterizing the behaviour of the group. The described approach has wide functions in several areas such as surveillance, group interface and behaviour classification. The video surveillance mainly deals with the recognition and classification of group behaviour with respect to several activities such as Normal speech, kicking, hugging and punching. The processing steps include frame generation, segmentation using Fuzzy C-Mean (FCM) clustering; features are extracted using Local Binary Pattern (LBP) and Hierarchical Centroid (HC). Convolution Neural Network (CNN) is used to classify the features. The proposed model is better than existing methods by achieving 80% accuracy.

No abstract available.

Objective To evaluate the effects of personalized psychological video intervention on patients of mental nursing in ICU. Methods A total of 124 conscious patients in ICU were divided into control and experiment group equally from December 2013 to October 2014. Common mental nursing was used in the control group while personalized psychological video intervention was used in the experiment group. The effects were evaluated and compared by SCL-90, PSQI and satisfaction questionnaire after 2 days admission and after 7 days nursing interventions. Results After intervention in the experimental group, the somatization, obsessive system, interpersonal relationship, depression, anxiety, hostile, terror, crankiness and psychosis of SCL-90 were (1.34±0.21), (1.62±0.36), (1.63±0.34), (1.56±0.22), (1.60±0.38), (1.43±0.51), (1.26±0.30), (1.50±0.48), (1.36±0.33) respectively (t=2.108, 2.934, 2.884, 2.335, 2.775, 2.696, 2.365, 2.005, 1.999; P<0.05). After intervention, the scores of each PSQI dimensions was decreased in two groups, and the scores of experimental group were lower than those in the control group (t=3.129, 2.635, 2.116, 2.568, 3.221, 3.227, 2.548, 2.494; P<0.05). Conclusions Personalized psychological video intervention can effectively reduce conscious patients negative emotions in ICU, and improve the sleep quality and the satisfaction. Key words: Intensive care unit; Psychological nursing; Negative emotions; Video

Complex events essentially include human, scenes, objects and actions that can be summarized by visual attributes, so leveraging relevant attributes properly could be helpful for event detection. Many works have exploited attributes at image level for various applications. However, attributes at image level are possibly insufficient for complex event detection in videos due to their limited capability in characterizing the dynamic properties of video data. Hence, we propose to leverage attributes at video level (named as video attributes in this work), i.e., the semantic labels of external videos are used as attributes. Compared to complex event videos, these external videos contain simple contents such as objects, scenes and actions which are the basic elements of complex events. Specifically, building upon a correlation vector which correlates the attributes and the complex event, we incorporate video attributes latently as extra informative cues into the event detector learnt from complex event videos. Extensive experiments on a real-world large-scale dataset validate the efficacy of the proposed approach.

Anomaly detection in video is a challenging computer vision problem. Due to the lack of anomalous events at training time, anomaly detection requires the design of learning methods without full supervision. In this paper, we approach anomalous event detection in video through self-supervised and multi-task learning at the object level. We first utilize a pre-trained detector to detect objects. Then, we train a 3D convolutional neural network to produce discriminative anomaly-specific information by jointly learning multiple proxy tasks: three self-supervised and one based on knowledge distillation. The self-supervised tasks are: (i) discrimination of forward/backward moving objects (arrow of time), (ii) discrimination of objects in consecutive/intermittent frames (motion irregularity) and (iii) reconstruction of object-specific appearance information. The knowledge distillation task takes into account both classification and detection information, generating large prediction discrepancies between teacher and student models when anomalies occur. To the best of our knowledge, we are the first to approach anomalous event detection in video as a multi-task learning problem, integrating multiple self-supervised and knowledge distillation proxy tasks in a single architecture. Our lightweight architecture outperforms the state-of-the-art methods on three benchmarks: Avenue, ShanghaiTech and UCSD Ped2. Additionally, we perform an ablation study demonstrating the importance of integrating self-supervised learning and normality-specific distillation in a multi-task learning setting.

Unsupervised anomaly detection defines an abnormal event as an event that does not conform to expected behavior. In the field of unsupervised anomaly detection, it is a pioneering work that leverages the difference between a future frame predicted by a generative adversarial network and its ground truth to detect an abnormal event. Based on the work, we improve the ability of video prediction framework to detect abnormal events by enhancing the difference between prediction results for normal and abnormal events. We incorporate super-resolution and self-attention mechanism to design a generative adversarial network. We propose an auto-encoder as a generator, which incorporates dense residual networks and self-attention. Moreover, we propose a new discriminator, which introduces self-attention on the basis of a relativistic discriminator. To predict a future frame with higher quality for normal events, we impose a constraint on the motion in video prediction by fusing optical flow and gradient difference between frames. We also introduce a perception constraint in video prediction to enrich the texture details of a frame. The AUC of our method on CUHK Avenue and Shanghai Tech datasets reaches 89.2% and 75.7% respectively, which is better than most existing methods. In addition, we propose a processing flow that can realize real-time anomaly detection in videos. The average running time of our video prediction framework is 37 frames per second. Among all real-time methods for abnormal event detection in videos, our method is competitive with the state-of-the-art methods.

Despite the popularity of Twitter for research, there are very few publicly available corpora, and those which are available are either too small or unsuitable for tasks such as event detection. This is partially due to a number of issues associated with the creation of Twitter corpora, including restrictions on the distribution of the tweets and the difficultly of creating relevance judgements at such a large scale. The difficulty of creating relevance judgements for the task of event detection is further hampered by ambiguity in the definition of event. In this paper, we propose a methodology for the creation of an event detection corpus. Specifically, we first create a new corpus that covers a period of 4 weeks and contains over 120 million tweets, which we make available for research. We then propose a definition of event which fits the characteristics of Twitter, and using this definition, we generate a set of relevance judgements aimed specifically at the task of event detection. To do so, we make use of existing state-of-the-art event detection approaches and Wikipedia to generate a set of candidate events with associated tweets. We then use crowdsourcing to gather relevance judgements, and discuss the quality of results, including how we ensured integrity and prevented spam. As a result of this process, along with our Twitter corpus, we release relevance judgements containing over 150,000 tweets, covering more than 500 events, which can be used for the evaluation of event detection approaches.

Analysis and detection of complex events in videos require a semantic representation of the video content. Existing video semantic representation methods typically require users to pre-define an exhaustive concept lexicon and manually annotate the presence of the concepts in each video, which is infeasible for real-world video event detection problems. In this paper, we propose an automatic semantic concept discovery scheme by exploiting Internet images and their associated tags. Given a target event and its textual descriptions, we crawl a collection of images and their associated tags by performing text based image search using the noun and verb pairs extracted from the event textual descriptions. The system first identifies the candidate concepts for an event by measuring whether a tag is a meaningful word and visually detectable. Then a concept visual model is built for each candidate concept using a SVM classifier with probabilistic output. Finally, the concept models are applied to generate concept based video representations. We use the TRECVID Multimedia Event Detection (MED) 2013 as our video test set and crawl 400K Flickr images to automatically discover 2, 000 visual concepts. We show significant performance gains of the proposed concept discovery method over different video event detection tasks including supervised event modeling over concept space and semantic based zero-shot retrieval without training examples. Importantly, we show the proposed method of automatic concept discovery outperforms other well-known concept library construction approaches such as Classemes and ImageNet by a large margin (228%) in zero-shot event retrieval. Finally, subjective evaluation by humans also confirms clear superiority of the proposed method in discovering concepts for event representation.