Adapted Windows Detection of Moving Objects in Video Scenes

As more and more visual information is available on video, information indexing and retrieval of digital video data is becoming important. A digital video database embedded with visual information processing using image analysis and image understanding techniques such as automated target detection, classification, and identification can provide query results of higher quality. We address in this paper a robust digital video database system within which a target detection module is implemented and applied onto the keyframe images extracted by our digital library system. The tasks and application scenarios under consideration involve indexing video with information about detection and verification of artificial objects that exist in video scenes. Based on the scenario that the video sequences are acquired by an onboard camera mounted on Predator unmanned aircraft, we demonstrate how an incoming video stream is structured into different levels -- video program level, scene level, shot level, and object level, based on the analysis of video contents using global imagery information. We then consider that the keyframe representation is most appropriate for video processing and it holds the property that can be used as the input for our detection module. As a result, video processing becomes feasible in terms of decreased computational resources spent and increased confidence in the (detection) decisions reached. The architecture we proposed can respond to the query of whether artificial structures and suspected combat vehicles are detected. The architecture for ground detection takes advantage of the image understanding paradigm and it involves different methods to locate and identify the artificial object rather than nature background such as tree, grass, and cloud. Edge detection, morphological transformation, line and parallel line detection using Hough transform applied on key frame images at video shot level are introduced in our detection module. This function can also help rapidly filter incoming video and extract only those video sequences of potential interest under real time combating environment. Experimental results on video sequences acquired by Predator prove the feasibility of our approach.

A novel approach for the detection and classification of human movements in videos scenes is presented in this paper. It consists in detecting, segmenting and tracking foreground objects in video scenes to further classify their movements as conventional or non-conventional. From each tracked object in the scene, features such as position, speed, changes in direction and temporal consistency of the bounding box dimension are extracted. These features make up feature vectors that are stored together with labels that categorize the movement and which are assigned by human supervisors. At the classification step, an instance-based learning algorithm is used to classify the object movement as conventional or non-conventional. For this aim, feature vectors computed from objects in motion are matched against reference feature vectors previously labeled. Experimental results on video clips from two different databases (Parking Lot and CAVIAR) have shown that the proposed approach is able to detect non-conventional human movements in video scenes with accuracies between 77% and 82%.

Surveillance systems in video sequences have been widely used to monitor scenes in various environments, such as airports, banks, schools, industries, bus and train stations, highways and stores. Due to the large amount of information obtained via surveillance cameras, the use of visual inspection by camera operators becomes a task subject to fatigue and failure, in addition to consuming a lot of time. One challenge is the development of intelligent surveillance systems capable of analyzing long video sequences captured by a network of cameras in order to identify a certain behavior. In this work, we propose and analyze the use of several classification techniques, based on the CENTRIST (Transformation Census Histogram) operator, in the context of identifying violent events in video scenes. Additionally, we evaluated other traditional descriptors, such as HoG (Oriented Gradient Histogram), HOF (Optical Flow Histogram) and descriptors extracted from pre-trained deep machine learning models. In order to allow the evaluation only in regions of interest present in the video frames, we investigated techniques for removing the background from the scene. A sliding window-based approach was used to assess smaller regions of the scene in combination with a voting criterion. The sliding window is then applied along with block filtering using the optical flow of the scene. To demonstrate the effectiveness of our method for discriminating violence in crowd scenes, we compared the results to other approaches available in the literature in two public databases (Violence in Crowds and Hockey Fights). The combination of CENTRIST and HoG was demonstrated in comparison to the use of these operators individually. The combination of both operators obtained approximately 88% against 81% using only HoG and 86% using CENTRIST. From the refinement of the proposed method, we identified that evaluating blocks of the frame with the sliding window-based approach made the method more effective. Techniques for generating a codebook with sparse coding, distance measurement with a Gaussian mixture model and distance measurement between clusters were evaluated and discussed. Also we dynamically calculate the threshold for class voting, which obtained superior results in some cases. Finally, strategies for restricting the actors present in the scenes using optical flow were analyzed. By using the Otsu's method to calculate the threshold from the optical flow at the scene, the effectiveness surpasses our most competitive results: 91.46% accuracy for the Violence in Crowds dataset and 92.79% for the Hockey Fights dataset.

Background estimation is one of the most challenging phases in extracting foreground objects from video sequences. In this paper we present a background modeling approach that uses the similarity of frames to extract background areas from the video sequence. We use a window over the frames history and compute the similarity between the selected frames of this window as a similarity window. The properties of similarity window depend on the characteristics of the scene and can be adjusted parametrically. Our primary results show that if proper parameters are chosen, this method can give a good approximation of the background model.

Background estimation can be regarded as a problem to construct the background from a series of video frames including moving objects in the scene. Scene background estimation is the essential prerequisite, or at least can be helpful for many applications such as video surveillance, video segmentation, and privacy protection for videos. To perform this task, in this paper we propose a robust framework for scene background modelling based on temporal median filter with Gaussian filtering. Specifically, each background pixel is firstly modeled with a probability density function (PDF) learned over a series of video sequence. Then, pixels in video sequence with low probabilities are filtered, taken as foreground moving objects or noises. Finally, a temporal median filter is employed on video sequence, with pixels left. The performance of our framework is evaluated visually and numerically using different metrics on the scene background modeling contest 2016(SBMnet 2016). Results on the SBMnet dataset indicate that the proposed scheme can improve performance, in comparison with the classic temporal median filter to some extend, for background motion and clutter.

In this paper, an algorithm is introduced combining a template matching technique with background subtraction to detect moving targets on a road captured on video using a static camera. The designed algorithm based on combining the two techniques is to reduce the difficulties in background estimation and improve the processing time, etc. The proposed algorithm was tested in two different video sequences. The first video sequence was filmed with stabilized camera, while the second video was acquired with unstable camera. The results in both cases indicated that the background subtraction algorithm is successful in detecting moving objects and the template matching technique is suitable for object replacement from background frame to acquire the standard background frame.

Given a video sequence decompressed from a compressed video sequence associated with motion vector information for each image frame, this short note presents an efficient predictive watershed-based video segmentation algorithm using motion vectors. The proposed algorithm has much computation-saving merit since the next frame can inherit the segmented results of the current frame according to the motion vector information of the next frame. Experimental results demonstrate that with little perceptual segmentation degradation, the proposed algorithm has better execution-time performance when compared to the previous Chien et al.'s video segmentation algorithm whose input video sequence is assumed to have no motion vector information. Further, the alleviation of the over-segmentation problem existed in both algorithms is investigated.

In this paper, we propose the prediction search algorithm for block matching using the temporal/spatial correlation of the video sequence and the renter-biased property of motion vectors The proposed algorithm determines the location of a better starting point for the search of an exact motion vector using the point of the smallest SAD(Sum of Absolute Difference) value by the predicted motion vector from the same block of the previous frame and the predictor candidate pint in each search region and the predicted motion vector from the neighbour blocks of the current frame. And the searching process after moving the starting point is processed a adaptive search pattern according to the magnitude of motion vector Simulation results show that PSNR(Peak-to-Signal Noise Ratio) values are improved up to the 0.75dB as depend on the video sequences and improved about 0.05∼0.34dB on an average except the FS (Full Search) algorithm.

This paper presents a robust foreground detection method capable of adapting to different motion speeds in scenes. A key contribution of this paper is the background estimation using a proposed novel algorithm, neighbor-based intensity correction (NIC), that identifies and modifies the motion pixels from the difference of the background and the current frame. Concretely, the first frame is considered as an initial background that is updated with the pixel intensity from each new frame based on the examination of neighborhood pixels. These pixels are formed into windows generated from the background and the current frame to identify whether a pixel belongs to the background or the current frame. The intensity modification procedure is based on the comparison of the standard deviation values calculated from two pixel windows. The robustness of the current background is further measured using pixel steadiness as an additional condition for the updating process. Finally, the foreground is detected by the background subtraction scheme with an optimal threshold calculated by the Otsu method. This method is benchmarked on several well-known data sets in the object detection and tracking domain, such as CAVIAR 2004, AVSS 2007, PETS 2009, PETS 2014, and CDNET 2014. We also compare the accuracy of the proposed method with other state-of-the-art methods via standard quantitative metrics under different parameter configurations. In the experiments, NIC approach outperforms several advanced methods on depressing the detected foreground confusions due to light artifact, illumination change, and camera jitter in dynamic scenes.

Automated traffic surveillance systems are widely used in intelligent transportation systems (ITS). However, the accuracy of video-based Vehicle Detection is heavily affected by complex environmental factors such as shadows, rain, illumination and glare. This paper introduces an approach to motion vehicle detection in complex condition over highway surveillance video. The framework is composed of two parts: background estimation and multi-feature extraction. A fast constrained Delaunay triangulation (CDT) algorithm based on constrained-edge priority is presented instead of complicated segmentation algorithms. We present a background block update modeling theory based on triangulation to estimate background self-adaptively. Consequently, we can get the difference between the current frame and the background model. After extracting features in triangular candidates, multi-feature eigenvector is created for each vehicle with Principal Component Analysis (PCA). We design a classifier to classify triangular candidate as a part of a real vehicle or not by Support Vector Machine (SVM). And then, a parallelogram is used to represent the vehicle's shape robustly. Finally, experiments using real video sequence are performed to verify the method proposed for complex environmental factors.

Background initialization is the task of computing a background model by processing a set of preliminary frames in a video scene. The initial background estimation serves as bootstrap model for video segmentation of foreground objects, although the background estimation could be refined and updated in steady state operation of video processing systems. In this paper we approach the background modeling problem with a weightless neural network called WiSARD $$^{rp}$$ . The proposed approach is straightforward, since the computation is pixel–based and it exploits a dedicated neural network to model the pixel background by using the same training rule.

In recent years, global motion estimation (GME) has become an important tool in the fields of video coding, video compression and computer vision. Estimating the correct global motion is often more difficult when the video scene contains large foreground objects. Previous approaches to addressing this problem have required the application of algorithm parameters that are sequence dependent or give inconsistent results for different video sequences. In this paper, we propose a fully automatic approach that can successfully estimate global motion in the presence of large foreground objects. The proposed algorithm determines an initial estimate of the foreground pixels and then reduces the effect of the remaining foreground by using a modified Lorentzian estimator. Experimental results show the proposed method produces superior and more consistent performance than some recent approaches for a wide range of sequences.

Real time object tracking is the process of locating moving objects over time using the camera in video sequences in real time. The objective of object tracking is to associate target objects in consecutive video frames. Object tracking requires location and shape or features of objects in the video frames. Every tracking algorithm needs to detect moving object. So object detection is the preceding step of object tracking in computer vision applications. After that, detected object can be extracted by the feature of moving object to track that moving object into video scene. It is challenging task in image processing to track the objects into consecutive frames. Various challenges can arise due to complex object motion, irregular shape of object, occlusion of object to object and object to scene and real time processing requirements. Object tracking has a variety of uses, some of which are: surveillance and security, traffic monitoring, video communication, robot vision and animation.

A method is presented for extracting object information from an image sequence taken by a static monocular camera. The method was developed towards a low computational complexity in order to be used in real-time surveillance applications. Our approach makes use of both intensity and edge information of each frame and works efficiently in an indoor environment. It consists of two major parts: background processing and foreground extraction. The background estimation and updating makes the object detection robust to environment changes like illumination changes and camera jitter. The fusion of intensity and edge information allows a more precise estimation of the position of the different foreground objects in a video sequence. The result obtained are quite reliable, under a variety of environmental conditions.

This paper describes a video surveillance system capable of recognizing human postures from video sequences. The system comprises of two key modules: human detection and posture classification. In the module of human detection, human blobs are extracted by the technique of background subtraction. An adaptive background model is employed to characterize the dynamics and complexity of outdoor scenes based on the mixture of Gaussians. In order to formulate the variations of human postures, pseudo 2D hidden Markov models (P2DHMM) is employed for representing and recognizing human postures based on its '2-D elastic matching' property. It is trained to differentiate human postures and tolerate the variations of the same human posture using embedded Viterbi and segmental K means algorithms. In the classification of human postures, observation sequence is extracted from current image frame. The probabilities of observation sequence corresponding to each P2DHMM model are computed by doubly embedded Viterbi optimization, and human blob is classified as the human posture with the highest likelihood.