Joint representation learning of appearance and motion for abnormal event detection

In this paper, we propose a novel method for global abnormal events detection in crowded scenes. Each video is described as the set of overlapping space-time cubes. The histogram of optical flow orientation and motion magnitude are used as global feature descriptor to capture the motion magnitude and orientation of the normal and abnormal events. The motion-rich space-time cubes are selected to enhance the computational speed of the algorithm and omit background subtraction step, forms the proposed approach novel and unique. Simple and cost-effective one-class SVM classifier is used to distinguish between normal and abnormal events in a video. The one-class SVM classifier is trained with normal events from the training data. The proposed method is tested on benchmark UMN and PETS2009 dataset. The improved results are reported to evaluate the performance of the proposed method.

In order to recognize the abnormal event, such as emergency or panic, happened in public scenes timely, an algorithm based on features extraction and BP Adaboost to detect abnormal frame event from surveillance video of complex situation is proposed. The proposed method detects an abnormal event where people are running, and this panic situation is simulated by the frame in a video. Experiments show that the method can distinguish and detect the abnormal event effectively and efficiently, which has potentiality to be used in the real public monitoring.

Unsupervised anomaly detection defines an abnormal event as an event that does not conform to expected behavior. In the field of unsupervised anomaly detection, it is a pioneering work that leverages the difference between a future frame predicted by a generative adversarial network and its ground truth to detect an abnormal event. Based on the work, we improve the ability of video prediction framework to detect abnormal events by enhancing the difference between prediction results for normal and abnormal events. We incorporate super-resolution and self-attention mechanism to design a generative adversarial network. We propose an auto-encoder as a generator, which incorporates dense residual networks and self-attention. Moreover, we propose a new discriminator, which introduces self-attention on the basis of a relativistic discriminator. To predict a future frame with higher quality for normal events, we impose a constraint on the motion in video prediction by fusing optical flow and gradient difference between frames. We also introduce a perception constraint in video prediction to enrich the texture details of a frame. The AUC of our method on CUHK Avenue and Shanghai Tech datasets reaches 89.2% and 75.7% respectively, which is better than most existing methods. In addition, we propose a processing flow that can realize real-time anomaly detection in videos. The average running time of our video prediction framework is 37 frames per second. Among all real-time methods for abnormal event detection in videos, our method is competitive with the state-of-the-art methods.

In this paper, a novel method termed Multi-Instance Dictionary Learning (MIDL) is presented for detecting abnormal events in crowded video scenes. With respect to multi-instance learning, each event (video clip) in videos is modeled as a bag containing several sub-events (local observations); while each sub-event is regarded as an instance. The MIDL jointly learns a dictionary for sparse representations of sub-events (instances) and multi-instance classifiers for classifying events into normal or abnormal. We further adopt three different multi-instance models, yielding the Max-Pooling-based MIDL (MP-MIDL), Instance-based MIDL (Inst-MIDL) and Bag-based MIDL (Bag-MIDL), for detecting both global and local abnormalities. The MP-MIDL classifies observed events by using bag features extracted via max-pooling over sparse representations. The Inst-MIDL and Bag-MIDL classify observed events by the predicted values of corresponding instances. The proposed MIDL is evaluated and compared with the state-of-the-art methods for abnormal event detection on the UMN (for global abnormalities) and the UCSD (for local abnormalities) datasets and results show that the proposed MP-MIDL and Bag-MIDL achieve either comparable or improved detection performances. The proposed MIDL method is also compared with other multi-instance learning methods on the task and superior results are obtained by the MP-MIDL scheme.

Abnormal event detection, also known as anomaly detection, is one challenging task in security video surveillance. It is important to develop effective and robust movement representation models for global and local abnormal event detection to fight against factors such as occlusion and illumination change. In this paper, a new algorithm is proposed. It can locate the abnormal events on one frame, and detect the global abnormal frame. The proposed algorithm employs a sparse measurement matrix designed to represent the movement feature based on optical flow efficiently. Then, the abnormal detection mission is constructed as a one-class classification task via merely learning from the training normal samples. Experiments demonstrate that our algorithm performs well on the benchmark abnormal detection datasets against state-of-the-art methods.

With the booming development of tourism, travel security problems are becoming more and more prominent. Congestion, stampedes, fights and other tourism emergency events occurred frequently, which should be a wake-up call for tourism security. Therefore, it is of great research value and application prospect to real-time monitor tourists and detect abnormal events in tourism surveillance video by using computer vision and video intelligent processing technology, which can realize the timely forecast and early warning of tourism emergencies. At present, although most of the video-based abnormal event detection methods work well in simple scenes, there are often problems such as low detection rate and high false positive rate in complex motion scenarios, and the detection of abnormal events can’t be processed in real time. To tackle these issues, we propose an abnormal event detection model in tourism video based on salient spatio-temporal features and sparse combination learning, which has good robustness and timeliness in complex motion scenarios and can be adapted to real-time anomaly detection in practical applications. Specifically, spatio-temporal gradient model is combined with foreground detection to extract 3D gradient features on the foreground target of video sequence as the salient spatio-temporal features, which can eliminate the interference of the background. Sparse combination learning algorithm is used to establish the abnormal event detection model, which can realize the real-time detection of abnormal events. In addition, we construct a new ScenicSpot dataset with 18 video clips (5964 frames) containing both normal and abnormal events. The experimental results on ScenicSpot dataset and two standard benchmark datasets show that our method can realize the automatic detection and recognition of tourists’ abnormal behavior, and has better performance compared with the classical methods.

Detecting abnormal events in video is commonly framed as a one-class classification task, where training videos contain only normal events, while test videos encompass both normal and abnormal events. In this scenario, anomaly detection is an open-set problem. However, some studies assimilate anomaly detection to action recognition. This is a closed-set scenario that fails to test the capability of systems at detecting new anomaly types. To this end, we propose UBnormal, a new supervised open-set benchmark composed of multiple virtual scenes for video anomaly detection. Unlike existing data sets, we introduce abnormal events annotated at the pixel level at training time, for the first time enabling the use of fully-supervised learning methods for abnormal event detection. To preserve the typical open-set formulation, we make sure to include dis-joint sets of anomaly types in our training and test collections of videos. To our knowledge, UBnormal is the first video anomaly detection benchmark to allow a fair head-to-head comparison between one-class open-set models and supervised closed-set models, as shown in our experiments. Moreover, we provide empirical evidence showing that UB-normal can enhance the performance of a state-of-the-art anomaly detection framework on two prominent data sets, Avenue and ShanghaiTech. Our benchmark is freely available at https://github.com/lilygeorgescu/UBnormal.

Intelligent Video Investigation is on nice interest in trade applications because of increasing demand to scale back the force of analyzing the large-scale video information. Sleuthing the abnormal events from crowded video scenes offer varied difficulties. Initially, an oversized variety of moving persons will simply distract the native anomaly detector. Secondly it’s tough to model the abnormal events in real time. Thirdly, the inaccessibility of ample samples of coaching information for abnormal events ends up in problem in sturdy detection of abnormal events. Our planned system provides a peculiar approach to find anomaly in crowded video scenes. We are initially divide the video frame into patches and apply the Difference-of-Gaussian (DoG) filter to extract edges. Then we work out Multiscale Histogram of Optical Flow (MHOF) and Edge directed bar chart (EOH) for every patch. Then exploitation of Normalized Cuts (NCuts) and Gaussian Expectation-Maximization (GEM) techniques, and to cluster the similar patches into cluster and assign the motion context. Finally exploitation of k-Nearest neighbor (k-NN) search, and establish the abnormal activity at intervals in the crowded scenes. Our spatio-temporal anomaly search system helps to boost the accuracy and computation time for detection of irregular patterns. This technique is helpful for investigation, trade specific and market applications like public transportation, enforcement, etc.KeywordsMultiscale Histogram of Optical FlowGaussian Expectation MaximizationSpatio-temporal anomaly search

Event detection plays an essential role in video content analysis. On the other hand, according to our analysis, the coding structures in new video coding standard High Efficient Video Coding (HEVC) have a high correlation with video contents. Hence there is large potential to identify events by reusing coding structures in HEVC, which can save a huge amount of computational resources. In this paper, we proposed a new compressed-domain feature for abnormal event detection, namely Motion Intensity Count (MIC), which makes use of motion vectors, coding unit and prediction unit modes in HEVC with little computational cost. MIC can well predict the normal paths of moving objects, which enables us to identify motions in unexpected locations where abnormal events are likely to happen. Our experiments show that MIC can correctly detect abnormal events at about 1250 fps.

To categorize and synthesize medication safety event detection methods in the critically ill in order to provide clinicians and administrators with approaches to event detection that are intended to expand and complement traditional voluntary reporting systems. A literature search of OvidMEDLINE was performed to identify articles related to medication safety involving critically ill patients in the intensive care unit setting. The inclusion of articles was restricted to comparative studies. The bibliographies of all retrieved articles were reviewed to obtain additional articles of relevance. The various event detection methods were compared by: evidence supporting their use; number, type and severity of events detected; phase of the medication use process in which events were detected; and ease and cost of implementation. Major limitations of each method were also collated. There are a number of methods that can be used to identify medication safety events in the critically ill. These can broadly be categorized as: 1) voluntary reporting, 2) record review, 3) rules/triggers and 4) direct observation and 5) interviews/surveys. Relatively few studies have directly compared these assessment methods in the ICU setting, although the limitations of the traditional voluntary reporting system as the sole method of event detection are well established. Although not truly dichotomous, these methods can be broken down into more proactive and reactive approaches. Rules/triggers and direct observation of the medication use process in the ICU are examples of proactive approaches to event detection, while the traditional unsolicited voluntary reporting is typically reactive. However, each of the event detection methods has advantages and disadvantages, so the methods should not be considered mutually exclusive with respect to obtaining information about medication safety. Given the limitations of traditional voluntary reporting systems, a multimodal approach used to identify medication safety events is most likely to capture the largest number and type of events. We would advise not trying to implement additional approaches beyond voluntary reporting systems all at once. This would be difficult and costly. Rather, we suggest a systematic implementation of additional event detection approaches that takes into account hospital-specific considerations.

The growing interest in deep learning approaches to video surveillance raises concerns about the accuracy and efficiency of neural networks. However, fast and reliable detection of abnormal events is still a challenging work. Here, we introduce a two-stream approach that offers an autoencoder-based structure for fast and efficient detection to facilitate anomaly detection from surveillance video without labeled abnormal events. Furthermore, we present post hoc interpretability of feature map visualization to show the process of feature learning, revealing uncertain and ambiguous decision boundaries in the video sequence. Experimental results on Avenue, UCSD Ped2, and Subway datasets show that our method can detect abnormal events well and explain the internal logic of the model at the object level.

BackgroundPreventable adverse drug reactions (PADRs) in inpatients are associated with harm, including increased length of stay and potential loss of life, and result in elevated costs of care. We conducted an overview of reviews (i.e., a systematic review of systematic reviews) to determine the incidence of PADRs experienced by inpatients. Secondary review objectives were related to assessment of the effects of patient age, setting, and clinical specialty on PADR incidence.MethodsThe protocol was registered in PROSPERO (CRD42016043220). We performed a search of Medline, Embase, and the Cochrane Library, limiting languages of publication to English and French. We included published systematic reviews that reported quantitative data on the incidence of PADRs in patients receiving acute or ambulatory care in a hospital setting. The full texts of all primary studies for which PADR data were reported in the included reviews were obtained and data relevant to review objectives were extracted. Quality of the included reviews was assessed using the AMSTAR-2 tool. Both narrative summaries of findings and meta-analyses of primary study data were undertaken.ResultsThirteen systematic reviews encompassing 37 unique primary studies were included. Across primary studies, the PADR incidence was highly varied, ranging from 0.006 to 13.3 PADRs per 100 patients, with a pooled incidence estimate of 0.59 PADRs per 100 patients. Substantial heterogeneity was present across both reviews and primary studies with respect to review/study objectives, patient age, hospital setting, medical discipline, definitions and assessment tools used, event detection methods, endpoints of interest, and units of measure. Thirteen primary studies used prospective event detection methods and had a pooled PADR incidence of 3.13 (2.87–3.38) PADRs per 100 patients; however, extreme statistical heterogeneity (I2 = 97%) indicated this finding should be considered with caution. Subgroup meta-analyses demonstrated that PADR incidence varied significantly with event detection method (prospective > retrospective > voluntary reporting methods), hospital setting (ICU > wards), and medical discipline (medical > surgical). High statistical heterogeneity (I2 > 80%) was present across all analyses, indicating results should be interpreted with caution. Effects of patient age could not be assessed due to poor reporting of age groups used in primary studies.DiscussionThe method of event detection appeared to significantly influence PADR incidence, with prospective methods having the highest reported PADR rate. This finding is in agreement with the background literature. High methodological and statistical heterogeneity across primary studies evaluating adverse drug events reduces the validity of the overall PADR incidence derived from the meta-analyses of the pooled data. Data pooled from studies using only prospective methods of event detection should provide an overall estimate closest to the true PADR incidence; however, our estimate should be considered with caution due to the statistical heterogeneity found in this group of studies. Future studies should employ prospective methods of detection. This review demonstrates that the true overall incidence of PADRs is likely much greater than the overall pooled incidence estimate of 0.59 PADRs per 100 patients obtained when event detection method was not taken into consideration.

These days consciousness about public safety increases and CCTV cameras are installed at almost all public places. But generally automatic smart surveillance systems are not available. In this manuscript, emphasis is given to detect and classify abnormal events in surveillance video especially in crowd environments. Abnormal event detection is a challenging task because the definition of abnormality is subjective. A normal event in one situation can be considered an abnormal event in another case. In the surveillance video with a dense crowd, automatic anomaly detection becomes very difficult because of clutter and severe occlusion. This manuscript represents CNN (Convolutional Neural Network) and CNN-LSTM (Convolutional Neural Network-Long Short-Term Memory) based approaches for detection and classification of abnormal events. The CNN architecture is developed from scratch and can be used for spatial domains. LSTM architecture is developed for the temporal domain. Feature sequences are generated using CNN model and given as input to LSTM model. Experiments are carried out using five different publicly available benchmark datasets. The performance is measured by accuracy and area under the ROC (receiver operating characteristic) curve (AUC). CNN-LSTM approach works better than only CNN.

Motivation: Staining the mRNA of a gene via in situ hybridization (ISH) during the development of a Drosophila melanogaster embryo delivers the detailed spatio-temporal patterns of the gene expression. Many related biological problems such as the detection of co-expressed genes, co-regulated genes and transcription factor binding motifs rely heavily on the analysis of these image patterns. To provide the text-based pattern searching for facilitating related biological studies, the images in the Berkeley Drosophila Genome Project (BDGP) study are annotated with developmental stage term and anatomical ontology terms manually by domain experts. Due to the rapid increase in the number of such images and the inevitable bias annotations by human curators, it is necessary to develop an automatic method to recognize the developmental stage and annotate anatomical terms.Results: In this article, we propose a novel computational model for jointly stage classification and anatomical terms annotation of Drosophila gene expression patterns. We propose a novel Tri-Relational Graph (TG) model that comprises the data graph, anatomical term graph, developmental stage term graph, and connect them by two additional graphs induced from stage or annotation label assignments. Upon the TG model, we introduce a Preferential Random Walk (PRW) method to jointly recognize developmental stage and annotate anatomical terms by utilizing the interrelations between two tasks. The experimental results on two refined BDGP datasets demonstrate that our joint learning method can achieve superior prediction results on both tasks than the state-of-the-art methods.Availability:http://ranger.uta.edu/%7eheng/Drosophila/Contact:heng@uta.edu

Existing methods have addressed the issue of detecting abnormal events at a smart home for medical care or security monitoring services extensively in the past decades. However, most of approaches use wearable sensors that require users to be equipped with the sensor devices at every moment. If the monitored users stop or pause the sensors, any abnormal events are not able to be detected. The use of non-wearable and non-intrusive sensors (e.g., IP cameras) is necessary for providing better user experiences and achieving sustainable and reliable detection model. However, it is still very challenging to analyze such non-wearable sensor data with a high accuracy. In this work, we propose an event detection model using a Bayesian Network. We first obtain the features by analyzing the daily videos and audios captured from different angles by multiple IP cameras at a smart home. These features are then used to construct a Bayesian network. We propose a probabilistic graph model where the dependence relations are defined in the graph as opposed to the naive Bayesian network. The experiments are presented to demonstrate the performance and utility of our model.