An End to End Encoder-Decoder Network with Multi-scale Feature Pulling for Detecting Local Changes From Video Scene

Moving Object Detection (MOD) is one of the prominent application in the field of computer vision. This paper contributes towards designing a fast and accurate MOD technique for real time applications by addressing the issues like dynamic background, illumination changes, occlusion, camera jitter, moving shadows, etc. The brilliance of the proposed method is confined in the effective utilization of QR decomposition and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$l_{2,1}$</tex> -norm minimization in the well-known Robust Principal Component Analysis (RPCA) framework. The <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$l_{2,1}$</tex> norm is robust to outliers and exploits the low rankness of the background component. While the use of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$l_{1}$</tex> -norm enhances the sparsity in the foreground part. Moreover, Total Variation(TV) regularization technique helps in retaining the details about the edges. Experimental result shows the supremacy of the proposed method w.r.t. the state of the art methods in terms of computational complexity and F-measure score.

One of the fundamental operations in computer vision applications is change detection, in which moving foreground objects are segmented from a static background. A common approach for change detection is the comparison of an image frame with the stored background model using a matching algorithm, a process known as background subtraction. However, such techniques fail in environments with dynamic backgrounds, illumination changes, or shadow and camera jitters. This study focuses on effectively detecting scene changes in complex environments. To this end, we proposed a new colour descriptor named local colour difference pattern (LCDP) that is insusceptible to shadow and is able to capture both colour and texture features at a pixel location. Furthermore, a scene change detection framework was proposed to handle dynamic scenes based on sample consensus that integrates LCDP and a novel spatial model fusion mechanism. Experiments using the CDnet benchmark dataset demonstrated the effectiveness of the proposed approach to change detection in complex environments.

Background subtraction models based on mixture of Gaussians have been extensively used for detecting objects inmotion in a wide variety of computer vision applications. However, background subtraction modeling is still an openproblem particularly in video scenes with drastic illumination changes and dynamic backgrounds (complexbackgrounds). The purpose of the present work is focused on increasing the robustness of background subtractionmodels to complex environments. For this, we proposed the following enhancements: a) redefine the modeldistribution parameters involved in the detection of moving objects (distribution weight, mean and variance), b)improve pixel classification (background/foreground) and variable update mechanism by a new time-space dependentlearning-rate parameter, and c) replace the pixel-based modeling currently used in the literature by a new space-timeregion-based model that eliminates the noise effect caused by drastic changes in illumination. Our proposed schemecan be implemented on any state of the art background subtraction scheme based on mixture of Gaussians toimprove its resilient to complex backgrounds. Experimental results show excellent noise removal and object motiondetection properties under complex environments.

A detection algorithm, based on the combined local-global (CLG) optical-flow model and Gaussian pyramid for a moving target appearing against a dynamic background, can compensate for the inadaptability of the classic Horn–Schunck algorithm to illumination changes and reduce the number of needed calculations. Incorporating the hypothesis of gradient conservation into the traditional CLG optical-flow model and combining structure and texture decomposition enable this algorithm to minimize the impact of illumination changes on optical-flow estimates. Further, calculating optical-flow with the Gaussian pyramid by layers and computing optical-flow at other points using an optical-flow iterative with higher gray-level points together reduce the number of calculations required to improve detection efficiency. Finally, this proposed method achieves the detection of a moving target against a dynamic background, according to the background motion vector determined by the displacement and magnitude of the optical-flow. Simulation results indicate that this algorithm, in comparison to the traditional Horn-Schunck optical-flow algorithm, accurately detects a moving target undergoing illumination changes against a dynamic background and simultaneously demonstrates a significant reduction in the number of computations needed to improve detection efficiency.

Background modeling methods in video analysis: A review and comparative evaluation

Moving object detection in the presence of complex dynamic backgrounds such as swaying of trees, spouting of water from fountain, ripples in water, flag fluttering in the wind, camera jitters, noise, etc., is known to be very difficult and challenging task. In addition to this, illumination variation, camouflage and real-time constraint aggravate the problem further. Background subtraction (BS) is a widely used algorithm for moving object detection in the presence of static cameras. Its performance purely depends on the choice of features used for background modeling. In this paper, we have proposed a novel multi-feature and multi-modal based background subtraction using Local Binary Pattern (LBP) and Histogram of oriented Gradients (HOG) for complex dynamic scene. Each pixel is modeled as a set of multi-feature calculated from its neighborhood and multi-modal BS is performed using Gaussian mixture model (GMM). To show its efficacy, the proposed algorithm is compared with some of the state-of-the-art BS techniques. In order to evaluate the algorithm in uncontrolled environments, a collection of publicly available database has been used. Quantitative and qualitative results justify our algorithm for efficient moving object detection in the presence of swaying of trees, camouflage and ripples in the water surface.

Moving object detection is critical for automated video analysis in many vision-related tasks, such as surveillance tracking, video compression coding, etc. Robust Principal Component Analysis (RPCA), as one of the most popular moving object modelling methods, aims to separate the temporally-varying (i.e., moving) foreground objects from the static background in video, assuming the background frames to be low-rank while the foreground to be spatially sparse. Classic RPCA imposes sparsity of the foreground component using <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _1$</tex-math></inline-formula> -norm, and minimizes the modeling error via <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _2$</tex-math></inline-formula> -norm. We show that such assumptions can be too restrictive in practice, which limits the effectiveness of the classic RPCA, especially when processing videos with dynamic background, camera jitter, camouflaged moving object, etc. In this paper, we propose a novel RPCA-based model, called Hyper RPCA, to detect moving objects on the fly. Different from classic RPCA, the proposed Hyper RPCA jointly applies the maximum correntropy criterion (MCC) for the modeling error, and Laplacian scale mixture (LSM) model for foreground objects. Extensive experiments have been conducted, and the results demonstrate that the proposed Hyper RPCA has competitive performance for foreground detection to the state-of-the-art algorithms on several well-known benchmark datasets.

Background subtraction is a widely used technique in motion detection. There are many challenges for motion detection in oceanic video like camera jitter, dynamic background and low visibility. If the background is dynamic or if the background changes overtime, a background update should be done in real time to precisely detect any kind of moving objects. In order to achieve an accurate underwater detection of motion in case of static camera with dynamic background in marine video, a novel detection scheme called sliding windowed fuzzy correlation analysis is proposed. The background modelling is based on sliding window technique, and the detection scheme is based on fuzzy correlation analysis. The window size is fixed to 12 in the algorithm to obtain better results and to reduce the latency in execution. The dataset considered here is ‘dataset on underwater change detection’ (Kaghyan and Sarukhanyan, Int J Inf Model Anal 1:146–156, 2012) that consists of five marine videos along with its ground truth. We qualitatively and quantitatively prove that the proposed method attains better motion detection as compared to other existing methods. The computational complexity involved is Intel Core i5 processor with MATLAB® software for simulation.KeywordsBackground subtractionCamera jitterDynamic backgroundSliding windowUnderwater motion detection

The analysis of moving objects in video sequences has been a paramount issue in applications related to intelligent surveillance systems, robotics, and medicine. Although several works aimed to analyze objects in video sequences have been reported, many of them need manual parameter adjustments and they are not tolerant to illumination changes and dynamic backgrounds. Therefore, a novel scheme termed Dynamic Retinotopic SOM based on an adaptive artificial neural network, to detect moving objects is proposed in this work. The neural network is a model based on the mechanisms of the visual cortex that we called Retinotopic SOM (RESOM) and it is also proposed in this paper. Furthermore, RESOM is a real-time neural network that can adapt its learning parameters based on the scene behavior and it mimics perception abilities. A quantitative comparison with other segmentation methods reported in the literature using real video scenes showed that the proposed DR-SOM segmentation method automatically adjusts its parameters and outperforms the reported methods in condition of dynamic backgrounds, and gradual and sudden illumination changes.

Moving object detection is an important and basic step in any video surveillance application. Detected object has used an input to higher level tasks such as tracking, event detection, and behavioral analysis. There are several challenges in moving object detection such as illumination changes, occlusion, shadow, bootstrap, dynamic background etc. Detection of moving object having similar color as background structure is called as camouflage, which is another big challenge. The method of detecting camouflage moving object is known as Decamouflaging. In recent years, the researchers are attracted towards the camouflage moving object detection due to its application in the military. In this review paper, different techniques using various features for camouflaged moving obj ect detection were studied.

Understanding the actions of humans from videos is a demanding task in Computer Vision. Identifying the actions being accomplished by the human in the video sequence automatically and tagging their actions is the prime functionality of intelligent video systems. In general the major issues of Human Action Recognition are dynamic backgrounds, camouflage, occlusion handling, similar action recognition, incremental learning etc. Among these issues the major hurdle is to differentiate the similar actions and incremental learning. To resolve the above aforementioned issues, an adaptive framework for silhouette based Human Action Recognition is proposed. Here the silhouettes are detected by combining Adaptive Guassian Mixture Model and Contour Saliency. Similar Action Handling is done by integrating Bipartite matching with the class certainty measure. The new actions are identified by invoking adaptation method thus making the system adaptable to dynamic conditions. The action labels are identified using Support Vector Machine — Nearest Neighbor classifier. Experimental results show the higher recognition rate achieved for various similar actions with the benchmarking datasets such as Weizmann, UCF-Sports Action dataset.

According to the problem of camera jittering under natural environments when detecting moving objects, a background adaptive scheme is proposed in the paper. First, the Harris operator is used to detect corners in the region-of-interest for background and foreground image respectively. A correlation and relaxation method is also applied to a small region to obtain several stable matched points. Then, the camera jitter parameter is estimated with offsets of these matched points and used to recover background image to match against the current image. At last, background difference algorithm based on the multi-resolution pyramid is adopted to detect moving object. It can remove the environment dynamic background noises and some small offset estimation errors caused by image blurring. The proposed algorithm is verified with camera jittering sequence of the public test image and compared with several state-of-the-art algorithms qualitatively and quantitatively. Experimental results demonstrate that the proposed algorithm can solve the problem of camera jittering in natural environment effectively. The detected effect evaluation parameter is better than the current algorithms.

Face Recognition is one of the challenging tasks in computer vision. Recognizing various features correctly, which differs from one face to another face, is very difficult. This task becomes more difficult with the change of illumination, pose, expression, beard, age, and the quality of the picture. Before the rise of deep learning in computer vision, hand engineering feature extractor was used to recognize faces. However, they suffered mostly on a variety of illumination, pose, expression, and age. After deep learning evolution, these problems can be overcome easily by feeding a massive amount of data. Facenet is one of the deep learning architectures to recognize faces. It produces a point in a 128-dimensional vector space where the distance between two faces of the same person is small, and the distance between two faces of different persons is large. If we have a large amount of face data to compare it will be computationally expensive, and thus real-time face recognition becomes slow. Support Vector Machine or Logistic Regression can solve this problem. However, SVM or Logistic Regression cannot be able to recognize faces that not in the database as 'Unknown'. For solving these problems, we proposed a method called 'Mean Embeddings'. By using 'Mean Embeddings,' we achieved 6.5 times faster and 1.6% better performance on average with the ability to recognize unknown faces.

Robust visual tracking is the important stage in the computer vision applications such as robotics, man-free control systems, and the visual surveillance. Accurate motion states estimation and the target representation in visual tracking system are based on the appearances of the target. The factor affects the learning of target representation is the accumulated error due to the pose, illumination changes, and the uneven background. The presence of dynamic background and the shadowing effects causes the visual drift and destructive information. Besides, the misclassification of target region induces the false detection of moving objects. The K-means and Fuzzy-C-means clustering algorithms are available to segment the foreground/background and suppress the shadow region on the basis of the non-changing background of the surveillance area. This paper proposes the novel background normalization technique with textural pattern analysis to suppress the shadow region. The Neighborhood Chain Prediction (NCP) algorithm is used to cluster the uneven background and the Differential Boundary Pattern (DBP) extracts the texture of the video frame to suppress the shadow pixels present in the frame. The lower intensity estimation and the prediction of the area around the lower intensity in proposed work enhance the pixels for shadow removal. The shadow-free frame split up into several grids and the histograms of features are extracted from the grid formatted frame. Finally, the Machine Level Classification (MLC) finds the matching grid corresponds to the tracking region and provides the binary labeling to separate the background and foreground. The proposed DBP-based visual tracking system is high robustness over the sudden illumination changes and the dynamic background due to the texture pattern analysis. The comparison of proposed NCP-DBP combination with the existing segmentation techniques regarding the accuracy, precision, recall, F-measure, success and error rate assured the effectiveness in visual tracking applications.

Dynamic backgrounds and sudden illumination changes are two of the major problems associated with background subtraction techniques. In this paper, we present a novel approach to background subtraction that addresses both of these challenges. In particular, we present an improved codebook background modelling and subtraction technique. We utilise image segmentation on the background image and model the background with a codebook for each pixel along with a pseudo background layer. We perceive background motion as an occlusion of one background layer by a nearby background layer. In other words, sliding of one background layer over a neighbouring layer causes background motion and will hence result in false segmentation. We present our approach of codeword spreading across layer boundaries to handle background motion. Furthermore, we present a two-step update of the background codebook to handle both sudden and gradual illumination changes.