Video background subtraction using semi-supervised robust matrix completion

The singular value thresholding (SVT) algorithm plays an important role in the well-known matrix reconstruction problem, and it has many applications in computer vision and recommendation systems. In this paper, an SVT with diagonal-update (D-SVT) algorithm was put forward, which allows the algorithm to make use of simple arithmetic operation and keep the computational cost of each iteration low. The low-rank matrix would be reconstructed well. The convergence of the new algorithm was discussed in detail. Finally, the numerical experiments show the effectiveness of the new algorithm for low-rank matrix completion.

We provide parallel algorithms for large-scale matrix completion on problems with millions of rows, millions of columns, and billions of revealed entries. We focus on in-memory algorithms that run either in a shared-memory environment on a powerful compute node or in a shared-nothing environment on a small cluster of commodity nodes; even very large problems can be handled effectively in these settings. Our ASGD, DSGD-MR, DSGD++, and CSGD algorithms are novel variants of the popular stochastic gradient descent (SGD) algorithm, with the latter three algorithms based on a new "stratified SGD" approach. All of the algorithms are cache-friendly and exploit thread-level parallelism, in-memory processing, and asynchronous communication. We investigate the performance of both new and existing algorithms via a theoretical complexity analysis and a set of large-scale experiments. The results show that CSGD is more scalable, and up to 60 % faster, than the best-performing alternative method in the shared-memory setting. DSGD++ is superior in terms of overall runtime, memory consumption, and scalability in the shared-nothing setting. For example, DSGD++ can solve a difficult matrix completion problem on a high-variance matrix with 10M rows, 1M columns, and 10B revealed entries in around 40 min on 16 compute nodes. In general, algorithms based on SGD appear to perform better than algorithms based on alternating minimizations, such as the PALS and DALS alternating least-squares algorithms.

The matrix completion problem consists of finding or approximating a low-rank matrix based on a few samples of this matrix. We propose a new algorithm for matrix completion that minimizes the least-square distance on the sampling set over the Riemannian manifold of fixed-rank matrices. The algorithm is an adaptation of classical nonlinear conjugate gradients, developed within the framework of retraction-based optimization on manifolds. We describe all the necessary objects from differential geometry necessary to perform optimization over this low-rank matrix manifold, seen as a submanifold embedded in the space of matrices. In particular, we describe how metric projection can be used as retraction and how vector transport lets us obtain the conjugate search directions. Finally, we prove convergence of a regularized version of our algorithm under the assumption that the restricted isometry property holds for incoherent matrices throughout the iterations. The numerical experiments indicate that our approach scales very well for large-scale problems and compares favorably with the state-of-the-art, while outperforming most existing solvers.

With the rapid development of nowadays technologies, TV could keep its position as one of the most important entertainment and sometimes educative utilities in our daily life. However, keeping this position required a lot of major changes to take place in order for the TV to follow up with the digital revolution, such as, digital broadcasting, High Definition TV, TV on demand. TV-REPLAY, WebTV, etc. This evolution accompanied with many other factors such as the vast spread of communication means and the low prices of storing media have all resulted in many other indispensable technologies for video content storing, structuring, searching and retrieval. Video content can be of various types: a sequence of frames, a sequence of shots, a sequence of scenes, or a sequence of programs which is what the TV stream is usually composed of. Video content structuring would be of a great benefit to help indexing searching and retrieving information from the content efficiently. For example, structuring a soccer game into Play/Break phases facilitates later the detection of goals or summarizing the soccer video. Another example is to structure a news program into stories where each story is composed of an anchorperson segment followed by a report, which facilitates later the search of a specific story or an intelligent navigation inside the news program. However, all the existing analysis methods are dedicated for one type of video content. Such methods generate very poor results if it is applied on a TV stream that is composed of several video programs. So, it is important to detect a priori the boundaries of each program and then identify the type of each program in order to run the dedicated analysis method based on the type. For a TV viewer, a TV stream is a sequence of programs (P) and breaks (B). Programs may be separated by breaks and may include also breaks. For analysis purpose, the stream can be considered as a sequence of audio and video frames with no markers of the start and end points of the included programs or breaks. Most of TV channels that produce TV streams provide a program guide about the broadcasted programs. However, such guides usually lack precision, especially with the existence of live programs which makes the prediction of their start and end very hard. Moreover, program guides do not include any information about the breaks (i.e. commercials). Hence, one of the important steps to structure TV video content is to segment it into different programs and then choose the appropriate method to segment each program separately based on its type. The TV stream structuring consists in detecting the start and end of all the programs and breaks in the stream and later trying to annotate automatically each program by some metadata that summarizes its content or identifies its type. This step can be performed by analyzing the metadata provided with the stream (EPG or EIT), or analyzing the audio-visual stream itself. In this article, we define what we call TvToC (TV stream table of content) that adds a new level in the hierarchical video decomposition (traditional video ToC). Then, we provide a comparative study of all the methods and techniques in the domain of TV stream segmentation. Besides, a comparison of the different approaches is done to highlight the advantages and the weaknesses of each of them.

The paper proposes a novel matrix completion algorithm for data collection in wireless sensor networks through incorporating a randomised version of Kaczmarz method. By splitting the matrix completion problem into two convex sub-problems and solving the optimal probability computing problem in the randomised Kaczmarz method approximately with the D-optimal design solution, we reduce the reconstruction error and accelerate the convergence speed of the matrix completion computation. The synthetic data experiments show that the proposed algorithm presents more accurate reconstruction accuracy and faster reconstruction speed than the state-of-the-art matrix completion algorithms. Furthermore, we verify the practicality of the proposed matrix completion algorithm in real data collection scenario of wireless sensor networks through the experiments based on the real sensed dataset.

We discuss a problem of automatic defect detection in welded joints of stainless steel pipes in the production process. Possible defects that occur during tungsten inert gas welding are shown. The substantiation of the choice of the method for solving the problem based on modeling and background subtraction is given. An algorithm for defect detection in welded joints on frames of video sequences is proposed, taking into account the features of a specific area. The background models are built using the methods of averaging and a mixture of Gaussians. Experimental studies of the algorithm are carried out using examples of processing frames of video sequences received from a static camera. The obtained results confirm that the background modeling method based on frame averaging is suitable for the automatic detection of welding defects since the defects are different and have characteristic features. The proposed algorithm makes it possible to detect and highlight the defective area in a welded joint on frames of video sequences. The experimental results show that the algorithm satisfies the requirements for continuous rapid detection of surface defects.

Matrix completion is the task of predicting missing entries of matrix from a subset of known entries. Notions of structured matrix completion include any setting in which whether an entry is observed does not occur uniformly at random. In recent work, a modification to the standard nuclear norm minimization for matrix completion has been made to take into account sparsity-based structure in the missing entries, which is motivated e.g. in recommender systems. In this work, we propose adjusting an Iteratively Reweighted Least Squares (IRLS) algorithm for low-rank matrix completion to take into account sparsity-based structure. We also outline an iterative gradient-projection-based implementation of the algorithm that can handle large-scale matrices. Lastly, we present preliminary numerical experiments showcasing the performance of the proposed method compared to the standard IRLS algorithm in structured settings.

Positive semidefinite matrix completion (PSDMC) aims to recover positive semidefinite and low-rank matrices from a subset of entries of a matrix. It is widely applicable in many fields, such as statistic analysis and system control. This task can be conducted by solving the nuclear norm regularized linear least squares model with positive semidefinite constraints. We apply the widely used alternating direction method of multipliers to solve the model and get a novel algorithm. The applicability and efficiency of the new algorithm are demonstrated in numerical experiments. Recovery results show that our algorithm is helpful.

In this work, a novel technique for real-time clutter rejection in ultrasound Color Flow Imaging (CFI) is proposed. Suppressing undesired clutter signal is important because clutter prohibits an unambiguous view of the vascular network. Although conventional eigen-based filters are potentially efficient in suppressing clutter signal, their performance is highly dependent on proper selection of a clutter to blood boundary which is done manually. Herein, we resolve this limitation by formulating the clutter suppression problem as a foreground-background separation problem to extract the moving blood component. To that end, we adapt the fast Robust Matrix Completion (fRMC) algorithm, and utilize the in-face extended Frank-Wolfe method to minimize the rank of the matrix of ultrasound frames. Our method automates the clutter suppression process, which is critical for clinical use. We name the method RAPID (Robust mAtrix decomPosition for suppressIng clutter in ultrasounD) since the automation step can substantially streamline clutter suppression. The technique is validated with simulation, flow phantom and two sets of in-vivo data. RAPID code as well as most of the data in this paper can be downloaded from RAPID.sonography.ai.

We propose a joint registration and segmentation method for cardiac perfusion images using robust PCA (RPCA) to decompose the time series into a low rank and sparse component. Registration maximizes the smoothness of the intensity signal in the low rank component. Segmentation minimizes the sparse component's pixel intensity difference with other pixels having the same label. B-splines are used to combine the registration and segmentation costs. Tests on real patient datasets show the improved registration and segmentation accuracy over conventional methods that perform them separately.

The variable interference filter imaging spectrometer (VIFIS), is a modified video‐based remote sensing instrument. The creation of a hyperspectral cube for a flight line from the VIFIS instrument relies on the successful alignment and mosaic of sequential video frames. A number of techniques have been tested to assess which is the most reliable at finding accurately matched positions between sequential frames. Results show correlation techniques to be more successful than ordinal measures and invariant moments. Once a list of matching points had been created the transform between frames can be calculated and a mosaic generated. The application of an affine transform between every frame led to severe distortions in the imagery and rendered the final product useless. Experiments have shown that an affine transform was not required in each case and should only be applied when it provides an improvement over the mapping provided by a lower level transform.

Orthogonal waveforms are often desirable to multiple-input multiple-output (MIMO) radar systems. Unfortunately, the orthogonality may not be always guaranteed in practice. In this letter, we consider the direction-of-arrival (DOA) estimation problem in colocated MIMO radar with imperfect waveforms, and a new methodology is presented. The noiseless cross-covariance matrix is obtained by utilizing the spatial cross-correlation technique. DOAs are obtained via reduced-dimension multiple signal classification (RD-MUSIC). In contrast to the state-of-the-art matrix completion (MC) algorithm, the proposed RD-MUSIC method is computationally more efficient and may have an estimation performance more accurate than the existing MC approach. The numerical results show the improvement in the proposed scheme.

Due to the heavy workload of RSS collection, the instability of WLAN signal strength and the disappearance of signals caused by complex indoor environments, the construction of radio map for wireless local area network (WLAN) fingerprint-based indoor positioning system is time-consuming and laborious. In order to rapidly deploy indoor WLAN positioning system, the bidirectional encoder representation from transformers (BERT) model is used to fill the missing signal in radio map and quickly build radio map. The radio map is imported into the BERT model in the form of natural language text, and the missing signal is filled by the BERT model. Since the number of input data in BERT model cannot exceed 512 words, the structure of BERT model is not suitable for WLAN signals with large data volume. Therefore, we redefine the model structure based on the original BERT model and fill in the missing signals in the radio map in parallel. In addition, the loss function is redefined. Except that each segment has a loss function, the weighted average value of all segment loss functions is defined as the total loss function. The experimental results show that the BERT model is better than the traditional linear interpolation method, compressed sensing algorithm and matrix completion algorithm in filling the missing signals in the fingerprint database, and the probability of error within 2 m reaches about 94%.

A novel matrix completion algorithm which iteratively minimizes the fitting error and the matrix rank is presented. Unlike conventional matrix completion algorithms, which usually require some relaxation technique to cope with the low rank constraints, the proposed algorithm does not require any such techniques, thus making the selection of the parameter q of the matrix q-norm (0 < q ≤1) or the regularization parameter unnecessary. Simulation results of the random generated data and Jester joke data set verify our algorithm's effectiveness and superiority over the reported algorithms in literature.

Trust networks, where people leave trust and distrust feedback, are becoming increasingly common. These networks may be regarded as signed graphs, where a positive edge weight captures the degree of trust while a negative edge weight captures the degree of distrust. Analysis of such signed networks has become an increasingly important research topic. One important analysis task is that of sign inference, i.e., infer unknown (or future) trust or distrust relationships given a partially observed signed network. Most state-of-the-art approaches consider the notion of structural balance in signed networks, building inference algorithms based on information about links, triads, and cycles in the network. In this paper, we first show that the notion of weak structural balance in signed networks naturally leads to a global low-rank model for the network. Under such a model, the sign inference problem can be formulated as a low-rank matrix completion problem. We show that we can perfectly recover missing relationships, under certain conditions, using state-of-the-art matrix completion algorithms. We also propose the use of a low-rank matrix factorization approach with generalized loss functions as a practical method for sign inference - this approach yields high accuracy while being scalable to large signed networks, for instance, we show that this analysis can be performed on a synthetic graph with 1.1 million nodes and 120 million edges in 10 minutes. We further show that the low-rank model can be used for other analysis tasks on signed networks, such as user segmentation through signed graph clustering, with theoretical guarantees. Experiments on synthetic as well as real data show that our low rank model substantially improves accuracy of sign inference as well as clustering. As an example, on the largest real dataset available to us (Epinions data with 130K nodes and 840K edges), our matrix factorization approach yields 94.6% accuracy on the sign inference task as compared to 90.8% accuracy using a state-of-the-art cycle-based method - moreover, our method runs in 40 seconds as compared to 10,000 seconds for the cycle-based method.