Moving Object Detection in Complex Scene Using Spatiotemporal Structured-Sparse RPCA.

Moving object detection is critical for automated video analysis in many vision-related tasks, such as surveillance tracking, video compression coding, etc. Robust Principal Component Analysis (RPCA), as one of the most popular moving object modelling methods, aims to separate the temporally-varying (i.e., moving) foreground objects from the static background in video, assuming the background frames to be low-rank while the foreground to be spatially sparse. Classic RPCA imposes sparsity of the foreground component using <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _1$</tex-math></inline-formula> -norm, and minimizes the modeling error via <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _2$</tex-math></inline-formula> -norm. We show that such assumptions can be too restrictive in practice, which limits the effectiveness of the classic RPCA, especially when processing videos with dynamic background, camera jitter, camouflaged moving object, etc. In this paper, we propose a novel RPCA-based model, called Hyper RPCA, to detect moving objects on the fly. Different from classic RPCA, the proposed Hyper RPCA jointly applies the maximum correntropy criterion (MCC) for the modeling error, and Laplacian scale mixture (LSM) model for foreground objects. Extensive experiments have been conducted, and the results demonstrate that the proposed Hyper RPCA has competitive performance for foreground detection to the state-of-the-art algorithms on several well-known benchmark datasets.

Background/foreground separation is one of the most fundamental tasks in computer vision, especially for video data. Robust PCA (RPCA) and its tensor extension, namely, Robust Tensor PCA (RTPCA), provide an effective framework for background/foreground separation by decomposing the data into low-rank and sparse components, which contain the background and the foreground (moving objects), respectively. However, in real-world applications, the video data is contaminated with noise. For example, in metal additive manufacturing (AM), the processed X-ray video to study melt pool dynamics is very noisy. RPCA and RTPCA are not able to separate the background, foreground, and noise simultaneously. As a result, the noise will contaminate the background or the foreground or both. There is a need to remove the noise from the background and foreground. To achieve the three components decomposition, a smooth sparse Robust Tensor Decomposition (SS-RTD) model is proposed to decompose the data into static background, smooth foreground, and noise, respectively. Specifically, the static background is modeled by the low-rank tucker decomposition, the smooth foreground (moving objects) is modeled by the spatio-temporal continuity, which is enforced by the total variation regularization, and the noise is modeled by the sparsity, which is enforced by the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\ell _{1}$ </tex-math></inline-formula> norm. An efficient algorithm based on alternating direction method of multipliers (ADMM) is implemented to solve the proposed model. Extensive experiments on both simulated and real data demonstrate that the proposed method significantly outperforms the state-of-the-art approaches for background/foreground separation in noisy cases. Note to Practitioners—This work is motivated by melt pool detection in metal additive manufacturing where the processed X-ray video from the monitoring system is very noisy. The objective is to recover the background with porosity defects and the foreground with melt pool in the presence of noise. Existing methods fail to separate the noise from the background and foreground since RPCA and RTPCA have only two components, which cannot explain the three components in the data. This paper puts forward a smooth sparse Robust Tensor Decomposition by decomposing the tensor data into low-rank, smooth, and sparse components, respectively. It is a highly effective method for background/foreground separation in noisy case. In the case studies on simulated video and X-ray data, the proposed method can handle non-additive noise, and even the case of high noise-ratio. In the proposed algorithm, there is only one tuning parameter <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\lambda $ </tex-math></inline-formula> . Based on the case studies, our method achieves satisfying performance by taking any <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\lambda \in [{0.2,1}]$ </tex-math></inline-formula> with anisotropic total variation regularization. With this observation, practitioners can apply the proposed method without extensive parameter tuning work. Furthermore, the proposed method is also applicable to other popular industrial applications. Practitioners can use the proposed SS-RTD for degradation processes monitoring, where the degradation image contains the static background, anomaly, and random disturbance, respectively.

Moving Object Detection (MOD) is one of the prominent application in the field of computer vision. This paper contributes towards designing a fast and accurate MOD technique for real time applications by addressing the issues like dynamic background, illumination changes, occlusion, camera jitter, moving shadows, etc. The brilliance of the proposed method is confined in the effective utilization of QR decomposition and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$l_{2,1}$</tex> -norm minimization in the well-known Robust Principal Component Analysis (RPCA) framework. The <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$l_{2,1}$</tex> norm is robust to outliers and exploits the low rankness of the background component. While the use of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$l_{1}$</tex> -norm enhances the sparsity in the foreground part. Moreover, Total Variation(TV) regularization technique helps in retaining the details about the edges. Experimental result shows the supremacy of the proposed method w.r.t. the state of the art methods in terms of computational complexity and F-measure score.

Digital subtraction angiography (DSA) is one of imaging methods using X-ray image for clear visualization of the vessel information during intervention with a catheter. In order to obtain a fine DSA image, patients have to hold their breathing. However, steady breath hold is a burden for the patients and is sometimes difficult for elder patients. We propose a blood vessel enhancement method with consecutive digital angiographic images acquired under the natural breathing. Robust principal component analysis (RPCA) is used to enhance blood vessel information from consecutive angiographic images acquired under the natural breathing. RPCA can separate the consecutive images into a low-rank component and a sparse component. The information of contrast media is included in the sparse component. We implemented it on GPU and applied the proposed method to 13 sets of angiographic images and confirmed that it enables to generate satisfactory enhanced angiographic images.

Intrinsic image decomposition is an important topic in computer vision and computer graphics applications. However, this is a challenging problem by adopting the information of a single image. Therefore, additional priors or supplementary information such as multiply images or user interactions are necessary to address this problem. In this paper, we propose a novel scheme to use multiple images for intrinsic image decomposition, based on a similar strategy in robust principal component analysis (RPCA). RPCA utilizes the fact that the reflectance layer is common in multiple lights of a scene, and attempts to decompose the data matrix constructed from input images into a low-rank matrix and a sparse matrix. This is possible if the sparse matrix is sufficiently sparse, which is often not the case in computer vision applications. Moreover, the weighting parameter between the low-rank and sparse matrices greatly affects the accuracy of the results, and tuning this parameter can be tricky. This paper proposes a rank-constrained PCA algorithm (RCPCA) for solving background recovering problems. Fixing the rank of the low-rank matrix to be 1 allows RCPCA to better recover the low-rank matrix from the data matrix. Comprehensive tests show that RCPCA produces more stable and accurate results than RP-CA.

RPCA (Robust Principal Component Analysis) recovers sparse and low rank components from the original observation data, RPCA commonly uses ADM (Alternate Direction Method) for solving, the efficiency of algorithm depends on the nuclear norm optimization solution, that is SVD. And the application of RPCA in computer vision, image and video and so on, large amounts of data, such as images and video, make it difficult for large-scale data SVD. In this paper, we used the random matrix algorithm to improve the SVD, respectively the Count Sketch algorithm, the Prototype Randomized k-SVD algorithm and the Faster Randomized k-SVD algorithm, The main idea is to reduce the size of the original large-scale data matrix and sample randomly. Using the random projection algorithm to obtain an approximation of the original matrix, and operate QR decomposition of the approximate matrix, get the unitary matrix corresponding to the approximate matrix, and do the corresponding SVD, finally we can get the results which is similar to the original matrix calculation. Obtaining approximation of the original data matrix. But the cost time and space have been greatly optimized. Simulation experiments based on single image and video foreground detection show that the proposed methods can greatly improve the efficiency of RPCA iterative optimization.

We study the problem of low-rank and sparse decomposition from possibly noisy observations, known as Robust PCA, where the sparse component can be seen as outliers. We first propose a modified objective function where the nuclear norm captures the low-rank term, $\ell _0$ -“norm” addresses the sparse outlier term, and an $\ell _1$ -norm to deal with the additive noise term. The associated algorithm, termed sparsity regularized principal component pursuit (SRPCP), is shown to converge. Under certain model and algorithm parameter settings, it is shown that SRPCP can recover the low-rank component and sparse component exactly in the noiseless case. In the noisy case, we first prove that the widely used principal component pursuit (PCP) method, which was designed for the noiseless case, is actually stable to dense noise. Then, we show that SRPCP has smaller estimation error bound, and can identify outlier entries without any false alarm. Another important byproduct of our analysis is the result that PCP with missing entries is also stable to dense noise. We further propose another objective function which replaces the above nuclear norm by the log-determinant. The proposed algorithm, termed iterative reweighted sparsity regularized principal component pursuit, is also shown to converge. In each iteration, it solves a weighted nuclear norm regularized robust matrix completion problem. We propose an alternating direction method of multipliers algorithm to solve this nonconvex subproblem, which also converges. Empirical studies demonstrate the efficacy of the proposed $\ell _{0}$ - $\ell _{1}$ regularization framework to deal with the outliers as well as its advantage over the existing state-of-the-art methods.

Video background recovery is a very important task in computer vision applications. Recent research offers robust principal component analysis (RPCA) as a promising approach for solving video background recovery. RPCA works by decomposing a data matrix into a low-rank matrix and a sparse matrix. Our previous work shows that when the desired rank of the low-rank matrix is known, fixing the rank in the algorithm called FrALM (fixed-rank ALM) yields more robust and accurate results than existing RPCA algorithms. However, application of RPCA to video background recovery requires that each frame in the video is encoded as a column in the data matrix. This is impractical in real applications because the videos can be easily larger than the amount of memory in a computer. This paper presents an algorithm called iFrALM (incremental fixed-rank ALM) that computes fixed-rank RPCA incrementally by splitting the video frames into an initial batch and an incremental batch. Comprehensive tests show that iFrALM uses less memory and time compared to FrALM. Moreover, the initial batch size and batch quality can be carefully selected to ensure that iFrALM reduces memory and time complexity without sacrificing accuracy.KeywordsBackground recoveryIncremental SVDFixed-rank robust PCA

Effective vehicle detection technique for traffic surveillance systems

Moving object detection in a video plays an important role in many vision applications. Recently, moving object detection using appearance modeling based on a convolutional neural network has been actively developed. However, the CNN- based methods usually require the user’s supervision of the first frame so that it becomes highly dependent on the training dataset. In contrast, the method of finding a foreground, which models a background occupying a large proportion in an image, can detect a moving object efficiently in an unsupervised manner. However, existing methods based on background modeling in a pan-tilt-zoom (PTZ) camera suffer many false positives or loss of moving objects due to the estimation error of camera motion. To overcome the aforementioned limitations, we propose a moving object detection method for a PTZ camera through two background models. In an unsupervised way, our method builds the two background models that have different roles: 1) a coarse background model for detecting large changes, and 2) a fine background model for detecting small changes. In more detail, the coarse background model builds a block-based Gaussian model, and the fine model builds a sample consensus model. Both models are adaptively updated according to the estimated camera motion in the video recorded by a PTZ camera. Then, each foreground result from two background models is incorporated to fill the moving object region. Through experiments, the proposed method achieves better performance than the state-of-the-art methods and operates in real-time without parallel processing. In addition, we showed the effectiveness of the proposed model through improved results of moving object detection through combination with the latest supervised method.

Most schemes exhibit low robustness due to LSB’s (Least Significant Bit) and MSB’s (Most Significant Bit) based information hiding in the cover image. However, most of these IW schemes have low imperceptibility as the cover image distortion reveals to the attacker due to information hiding in MSB’s. In this paper, a hybrid image watermarking scheme is proposed based on integrating Robust Principal Component Analysis (R-PCA), Discrete Tchebichef Transform (DTT), and Singular Value Decomposition (SVD). A grayscale watermark image is twisted/scrambled using a 2D Discrete Hyper-chaotic Encryption System (2D-DHCES) to boost up the robustness/heftiness and security. The original cover image is crumbled into sparse components using R-PCA and using DTT the substantial component is additionally decomposed and the watermark will be embedded in the cover image using SVD processing. In DTT, scarcer coefficients hold the utmost energy, also provide an optimum sparse depiction of the substantial image edges and features that supports proficient retrieval of the watermark image even after unadorned image distortion based channel attacks. The imperceptibility and robustness of the proposed method are corroborated against a variety of signal processing channel attacks (salt and pepper noise, multi-directional shearing, cropping, and frequency filtering, etc.). The visual and quantifiable outcomes reveal that the proposed image watermarking scheme is much effective and delivers high forbearance against several image processing and geometric attacks.

Community detection in network-type data provides a powerful tool in analyzing and understanding real-world systems. In fact, community detection approaches aim to reduce the network&#x2019;s dimensionality and partition it into a set of disjoint clusters or communities. However, real networks are usually corrupted with noise or outliers which affect the detected community structure quality. In this paper, a new robust community detection algorithm that is capable of recovering a clean or a smoothed version of the corrupted graph and detecting the correct community structure is introduced. The proposed approach combines robust principal component analysis (RPCA) and symmetric nonnegative matrix factorization (SymNMF) in a single optimization problem. The proposed problem is solved under the framework of alternating direction methods of multipliers (ADMM). In particular, the corrupted adjacency matrix is decomposed into a low-rank and sparse components using RPCA and the community structure is detected by applying SymNMF to the extracted low-rank component. Extensive experiments that have been conducted on real and simulated binary and weighted networks show that the proposed approach significantly outperforms existing algorithms in detecting the correct community structure even in grossly corrupted networks.

Moving objects have been detected using various object detection techniques. Two categories for moving object detection techniques are frame differencing based and background subtraction based. These techniques are limited by camera scene complexity, light conditions, video type etc. Frame differencing based techniques process videos faster compared to background subtraction based techniques. Frame differencing based techniques detects only the boundary of the moving object and may fail for slow moving objects. These techniques for moving object detection can be improved by using sound data as most video recording cameras are equipped with a microphone. Sounds from human footsteps can be recorded with video and used with frame differencing techniques to improve moving object detection results. Camera microphones also record background noise with other background sound. This noisy data has been filtered out using the Fourier transform. When peak locations for each footstep sound are determined, and a Full Width at Half Maxima is computed for each peak, the number of frames within this width are counted, these frames are verify the presence of a moving object.

Moving object detection in the presence of complex dynamic backgrounds such as swaying of trees, spouting of water from fountain, ripples in water, flag fluttering in the wind, camera jitters, noise, etc., is known to be very difficult and challenging task. In addition to this, illumination variation, camouflage and real-time constraint aggravate the problem further. Background subtraction (BS) is a widely used algorithm for moving object detection in the presence of static cameras. Its performance purely depends on the choice of features used for background modeling. In this paper, we have proposed a novel multi-feature and multi-modal based background subtraction using Local Binary Pattern (LBP) and Histogram of oriented Gradients (HOG) for complex dynamic scene. Each pixel is modeled as a set of multi-feature calculated from its neighborhood and multi-modal BS is performed using Gaussian mixture model (GMM). To show its efficacy, the proposed algorithm is compared with some of the state-of-the-art BS techniques. In order to evaluate the algorithm in uncontrolled environments, a collection of publicly available database has been used. Quantitative and qualitative results justify our algorithm for efficient moving object detection in the presence of swaying of trees, camouflage and ripples in the water surface.

Moving object detection in urban scenes is important for the guidance of autonomous vehicles, robot navigation, and monitoring. In this paper moving objects are automatically detected using three sequential frames and tracked over a longer period. To this extend we modify the plane+parallax, fundamental matrix, and trifocal tensor algorithms to operate on three sequential frames automatically, and test their ability to detect moving objects in challenging urban scenes. Frame-to-frame correspondences are established with the use of SIFT keys. The keys that are consistently matched over three frames are used by the algorithms to distinguish between static objects and moving objects. The tracking of keys for the detected moving objects increases their reliability over time, which is quantified by our results. To evaluate the three different algorithms, we manually segment the moving objects in real world data and report the fraction of true positives versus false positives. Results show that the plane+parallax method performs very well on our datasets and we prove that our modification to this method outperforms the original method. The proposed combination of the advanced plane+parallax method with the trifocal tensor method improves on the moving object detection and their tracking for one of the four video sequences.