A multilayer backpropagation saliency detection algorithm and its applications

During past few years’ significant usage of video monitoring has been observed in several places. Video monitoring has always helped in improvement of security specific aspects. These systems provide major helping hand in monitoring through multiple object detection methods. However, there are certain challenges related to weather conditions which needs to be addressed. In view of this in this paper, we study multiple object detection in surveillance videos through hybrid deep gaussian mixture ensembles. This method integrates multiple deep gaussian mixture components to form ensembles which helps to develop multiple object detection solutions in surveillance videos. The computational system consists of ensemble of three major deep gaussian mixture components. The ensemble pipeline is formed with augmenting, bagging and stacking methods. The ensemble consists of hybrid object detection paradigm with steps background analysis, video pre-processing, data integration, object detection, foreground detection and post-processing. The computational pipeline is successfully experimented with ViSOR and CDnet 2014 benchmarked datasets. All results are validated with accuracy, precision, sensitivity or recall, specificity, F1-Score and RMSE metrics. Several comparative studies are performed with state-of-the-art methods as well as baselines. The experimental results demonstrate superiority of this method in comparison with other methods. Ablation studies have also been performed here with superior results. This system presents robustness in multiple object detection for real life challenges such as sudden illumination variation, shadow presence, long term occlusion and formation of dynamic backgrounds. It provides cost-effective, more profitable, efficient and sustainable real time multiple object detection solutions

Salient object detection in video using deep non-local neural networks

Moving objects detection in video is essential for many computer vision applications and it is considered as a challenging research issue due to dynamic changes in object size, shape, complex background and illumination changes. In this research article a novel method to detect moving objects in video is proposed grounded on Locality Preserving Projections (LPP) and also thorough analysis of variations in neighbor mode and weight mode for constructing adjacency graph is given. LPP is an unsupervised subspace learning approach used for dimensionality reduction, which preserves the neighborhood structure of dataset while detection, which is vital in further steps accurate tracking and recognition of objects. The proposed method is tested on standard datasets with complex environments and experimental results obtained with variations in neighborhood modes and weight modes are encouraging.

In this paper, we propose a novel framework for detecting multiple objects in 2D and 3D images. Since a joint multi-object model is difficult to obtain in most practical situations, we focus here on detecting the objects sequentially, one-by-one. The interdependence of object poses and strong prior information embedded in our domain of medical images results in better performance than detecting the objects individually. Our approach is based on Sequential Estimation techniques, frequently applied to visual tracking. Unlike in tracking, where the sequential order is naturally determined by the time sequence, the order of detection of multiple objects must be selected, leading to a Hierarchical Detection Network (HDN). We present an algorithm that optimally selects the order based on probability of states (object poses) within the ground truth region. The posterior distribution of the object pose is approximated at each step by sequential Monte Carlo. The samples are propagated within the sequence across multiple objects and hierarchical levels. We show on 2D ultrasound images of left atrium, that the automatically selected sequential order yields low mean detection error. We also quantitatively evaluate the hierarchical detection of fetal faces and three fetal brain structures in 3D ultrasound images.

Detecting multiple salient objects in complex scenes is a challenging task. In this paper, we present a novel method to detect salient objects in images. The proposed method is based on the general ‘center-surround’ visual attention mechanism and the spatial frequency response of the human visual system (HVS). The saliency computation is performed in a statistical way. This method is modeled following three biologically inspired principles and compute saliency by two ‘scatter matrices’ which are used to measure the variability within and between two classes, i.e., the center and surrounding regions, respectively. In order to detect multiple salient objects of different sizes in a scene, the saliency of a pixel is estimated via its saliency support region which is defined as the most salient region centered at the pixel. Compliance with human perceptual characteristics enables the proposed method to detect salient objects in complex scenes and predict human fixations. Experimental results on three eye tracking datasets verify the effectiveness of the method and show that the proposed method outperforms the state-of-the-art methods on the visual saliency detection task.

Video motion target tracking has been widely used in the field of sports. Content-based video search has a broad application prospect and attracts more and more researchers’ attention. In view of the shortcomings of moving object detection in motion video at present, a human motion recognition method based on correlation vector machine is proposed. Combined with the improved Gaussian mixture model of moving object detection and tracking method in motion video, moving object detection and tracking are carried out. The design of the system is realized. Firstly, it collects sports videos, considers all kinds of sports objects in sports videos, uses improved background update difference method to detect sports objects, then uses correlation vector machine to recognize human motion, and finally uses AdaBoost classifier to use it in multiple sports videos and sports objects. Carry out follow-up experiments. Results in this way, we can track the sports target in sports video accurately and quickly, and the real-time performance of sports target tracking is better than other tracking methods, which provides a new research tool for tracking sports target in sports video.

Saliency and co-saliency detection aim to distinguish conspicuous foreground objects from single and multiple images, thus are essential in many multimedia and vision applications. To achieve balanced efficiency and accuracy, most recent successful saliency detectors are based on superpixels. However, saliency detection with single-scale superpixel segmentation may fail in capturing intrinsic salient objects of complex natural scenes with small-scale high-contrast backgrounds. To tackle this problem, we present a simple strategy using multiscale superpixels to jointly detect salient object via low-rank optimisation. Specifically, we first build a multiscale superpixel pyramid and derive the corresponding saliency map by multimodal saliency features and priors at each single scale. Then, we use joint low-rank analysis of multiscale saliency maps to obtain a more reliable and adaptively-fused saliency map, which properly takes all scales saliency into account. We further propose a GMM generative co-saliency prior to enable the above approach to detect co-salient objects from multiple images. Extensive experiments on benchmark datasets validate the effectiveness and superiority of the proposed saliency and co-saliency detector over state-of-the-arts.

Object detection in videos is widely used in entertainment, robotics, surveillance etc. A major challenge in effective object detection is observed when there is occlusion, bad illumination or cluttered background. Besides, cameras do not provide any mechanism for detecting moving objects after capture. Researchers have proposed different methods for object detection in video frames, ranging from traditional to deep learning approaches. However, using the right method in the right situation for efficient and accurate detection is a concern. Even though deep learning has shown high accuracy in detection, the training and testing time and cost is a concern. In our research, we explore the best motion models for tracking objects in a video sequence. The characteristics of each model was analyzed and the results were compared using different types of videos. Our observations determine that depending on the quality of input video, traditional approaches show high accuracy in detection, comparable to state-of-the art methods.

Salient object detection in video has attracted enormous research efforts for its wide applicability. But there are still some issues in restraining the disturbance of background, which make it difficult to detect salient object in complex scenarios. Inspired by the hypothesis of center prior in image domain, we novelly introduced the concept of dynamic attention center in video. The distance between specific regions and this center is used as a weighting term to restrain the influence of background disturbance and obtain more accurate spatial and temporal saliency maps. Besides, we develop a dynamic fusion method to combine the temporal and spatial saliency map, leading to higher spatiotemporal consistency. The experiments on Freiburg-Berkeley Motion Segmentation Dataset show that our method outperforms several state-of-art methods on subjective visual perception and objective measurements.

Video saliency detection aims to pop out the most salient regions in every frame of a video. Up to now, many efforts have been made from various aspects for video saliency detection. Unfortunately, the existing video saliency models are very likely to fail in challenging videos with complicated motions and complex scenes. Therefore, in this paper, we propose a novel framework to improve the saliency detection results generated by existing video saliency models. The proposed framework consists of three key steps including localized estimation, spatiotemporal refinement, and saliency update. Specifically, the initial saliency map of each frame in a video is first generated by using an existing saliency model. Then, by considering the temporal consistency and strong correlation among adjacent frames, the localized estimation models, which are generated by training the random forest regressor within a local temporal window, are employed to generate the temporary saliency map. Finally, by taking the appearance and motion information of salient objects into consideration, the spatiotemporal refinement step is deployed to further improve the temporary saliency map and generate the final saliency map. Furthermore, such an improved saliency map is then utilized to update the initial saliency map and provide reliable cues for saliency detection in the next frame. The experimental results on four challenging datasets demonstrate that the proposed framework is able to consistently and significantly improve the saliency detection performance of various video saliency models, thereby achieving the state-of-the-art performance.

The goal of this special issue was to bring together experts working on applying models, principles, and knowledge of human audio-visual perception and cognition to optimize video processing algorithms and applications. This is an area where our deeper understanding of visual perception to video processing will lead to new breakthroughs in visual information processing. Attentional focus is an important aspect of perception that helps understand user interest and intent. Determining points of saliency is computationally complex. Compressed-domain methods thus become valuable tools for developing computationally efficient and practical solutions. In BCompressed-Domain Correlates of Human Fixations in Dynamic Scenes^ (10. 1007/s11042-015-2802-3) authors present a method for detecting points of fixation in H.264/ AVC video using motion vectors, block coding modes and coded residuals parsed from a H. 164/AVC bitstream. Egocentric videos are captured usingwearable cameras and used to detect a wearer’s point of view. Amount of video captured using wearable cameras is increasing and saliency detection in such videos enables applications such as efficient video summarization. In BGeometrical Cues in Visual Saliency Models for Active Object Recognition in Egocentric Videos^ (10.1007/ s11042-015-2803-2) authors use geometrical cues to improve saliency detection in videos. Action recognition from videos is a challenging task with many applications including surveillance and social behavior understanding. Inspired by models of neural response to visual input, the paper entitled BDeep Learning Human Actions from Video via Sparse Filtering and Locally Competitive Algorithms^ (10.1007/s11042-015-2808-x) presents a method that combines sparse filtering with locally competitive algorithms for applications in action recognition. Multimed Tools Appl (2015) 74:10053–10056 DOI 10.1007/s11042-015-2957-y

Multiple Object detection and tracking is a predominant area in computer vision and has numerous applications, including healthcare monitoring, autonomous driving, and anomaly detection. The proposed study is focused on multiple object detection and tracking under complex environments using advanced deep learning frameworks. Multiple object detection from videos under complex environments presents several challenges due to the dynamic and unpredictable nature of real-world conditions. The study summarizes the state-of-the-art methods, including important algorithms, datasets, and assessment measures, making it a fundamental resource for scholars and practitioners. It also highlights the shortcomings and difficulties in current solutions, offering information about areas that need more research. In conclusion, this study highlights new developments and approaches for the future, promoting the creation of innovative models and frameworks that can tackle outstanding problems in object tracking and detection.

The tremendous volume of video data generated by industrial surveillance networks presents a number of difficulties when examining such videos for a variety of purposes, including video summarization (VS), analysis, indexing and retrieval. The task of creating video summaries is extremely difficult because of the huge amount of data, redundancy, interleaved views and light variations. Multiple object detection and identification in video is difficult for machines to recognize and classify. To address all such issues, multiple low-feature and clustering-based machine learning strategies that fail to completely exploit VS are recommended. In this work, we achieved VS by embedding deep neural network-based soft computing methods. Firstly, the objects in extracted frames are detected using YOLOv5, and then the frames without objects (useless frames) are removed. Video summary generation occurs with the help of frames containing Objects. To check the quality of the proposed work Summary length, precision, recall, PR curve, and mean average precision (mAP) are used and system resource utilization during the model training are also tracked. As a result, the proposed work was able to identify the most effective video summarization framework with best summary length under varying conditions.

The past decade has witnessed the popularity of video conferencing, such as FaceTime and Skype. In video conferencing, almost every frame has a human face. Hence, it is necessary to predict attention on face videos by saliency detection, as saliency can be used as a guidance of regionof- interest (ROI) for the content-based applications. To this end, this paper proposes a novel approach for saliency detection in single-face videos. From the data-driven perspective, we first establish an eye tracking database which contains fixations of 70 single-face videos viewed by 40 subjects. Through analysis on our database, we investigate that most attention is attracted by face in videos, and that attention distribution within a face varies with regard to face size and mouth movement. Inspired by the previous work which applies Gaussian mixture model (GMM) for face saliency detection in still images, we propose to model visual attention on face region for videos by dynamic GMM (DGMM), the variation of which relies on face size, mouth movement and facial landmarks. Then, we develop a long shortterm memory (LSTM) neural network in estimating DGMM for saliency detection of single-face videos, so called LSTM-DGMM. Finally, the experimental results show that our approach outperforms other state-of-the-art approaches in saliency detection of single-face videos.

Saliency detection for images and videos becomes increasingly popular due to its wide applicability. Enormous research efforts have been focused on saliency detection, but it still has some issues in maintaining spatiotemporal consistency of videos and uniformly highlighting entire objects. To address these issues, this paper proposes a superpixel-level spatiotemporal saliency model for saliency detection in videos. To detect salient object, we extract multiple spatiotemporal features combined with intra-consistency motion information preliminarily. Meanwhile, considering inter-consistency of foreground in videos, a set of foreground locations are obtained from previous frames. Then, we introduce foreground-background and local foreground contrast saliency cues of those features using the location prior information of foreground. These two improved contrast saliency cues uniformly highlight the entire object and suppress the background effectively. Finally, we use an interactively dynamic fusion method to integrate the output spatial and temporal saliency maps. The proposed approach is validated on challenging sets of video sequences. Subjective observations and objective evaluations demonstrate that the proposed model achieves a better performance on saliency detection compared with the state-of-the-art spatiotemporal saliency methods.