<title>Ground-target detection system for digital video database</title>

In this paper we propose temporal and optimized video scene segmentation techniques. Temporal video scene segmentation methods aims to partition the video to elementary image sequences termed scenes. The main purpose of video scene segmentation is to extract objects from series of consecutive video frames. One important goal in video analysis is to group the shots, such that all shots in a single scene are related to the same subject. In this paper we make an analysis on four methods of video scene segmentation. The methods proposed for segmentation is Backward shot coherence(BSC), Best first model merging(BFMM) and Markov chain Monte Carlo (MCMC) and Hierarchical chain partitioning(HCP). Video scene segmentation is performed by considering both content coherence and temporally contextual dissimilarity. This task is formulated as a ratio function of content coherence within a scene and contextual dissimilarity of adjacent scenes. Finally we propose an optimized video scene segmentation which is a novel approach that is used to segment the video scene and thereby optimize them to produce efficiency. An effective and efficient hierarchical chain partitioning(HCP) approach is used to find the optimal scene segmentation. HCP gets competitive performance at low tolerance and better performance at high tolerance The HCP method approaches 100 % performance at tolerance = 1. The output segments can be used for further analysis and processing of the videos that is for indexing, storage, searching and retrieval of particular portion of video. It can be used in several applications in the fields of Image processing, video content analysis, Computer vision, Movies, Medical, Sports, and News. We have tested the proposed methods in Feature Films, News videos, TV Shows and high degree of accuracy has been obtained.

The perception of Human Visual System (HVS) for the video scene is selective. Different regions in the video scene have distinctive levels of visual importance. In this study, we present a novel H.264 visual perceptual video coding (VPVC) method in a uniform analyzing and encoding framework, which can allocate bit and computation resources effectively. The framework presented in this work consists of a visual perception model, a H.264 perceptual encoder and a corresponding sharing channel of information. The visual perception model, composed of motion perception, texture perception and spatial position perception sub-models, can compute the visual perception map (VPM) by fusing these spatiotemporal visual features. Visual perception results of HVS for various regions can be simulated well by VPM. The side encoding information of H.264 encoder, including motion vectors (MVs) and sum of absolute differences (SADs), is applied as input features for motion perception sub-model. A novel VPVC method is proposed based on the VPM and the global motion type of video scene. Using an adaptive frequency coefficient suppression technique and a novel encoding strategy, the optimal bit resource allocation is achieved by classifying video scene based on the VPM. In order to allocate computation resource effectively in VPVC method, the relation between optimal encoding mode and image features at video scene level is experimentally analyzed. As a result, a fast and effective H.264 mode analysis algorithm is deduced. When compared with the conventional H.264 coding method, our results on four video sequences show that the proposed method can obtain a high PSNR gain up to about 2.0 dB for visual important regions and decrease about 38% of total encoding time on average.

Surveillance systems in video sequences have been widely used to monitor scenes in various environments, such as airports, banks, schools, industries, bus and train stations, highways and stores. Due to the large amount of information obtained via surveillance cameras, the use of visual inspection by camera operators becomes a task subject to fatigue and failure, in addition to consuming a lot of time. One challenge is the development of intelligent surveillance systems capable of analyzing long video sequences captured by a network of cameras in order to identify a certain behavior. In this work, we propose and analyze the use of several classification techniques, based on the CENTRIST (Transformation Census Histogram) operator, in the context of identifying violent events in video scenes. Additionally, we evaluated other traditional descriptors, such as HoG (Oriented Gradient Histogram), HOF (Optical Flow Histogram) and descriptors extracted from pre-trained deep machine learning models. In order to allow the evaluation only in regions of interest present in the video frames, we investigated techniques for removing the background from the scene. A sliding window-based approach was used to assess smaller regions of the scene in combination with a voting criterion. The sliding window is then applied along with block filtering using the optical flow of the scene. To demonstrate the effectiveness of our method for discriminating violence in crowd scenes, we compared the results to other approaches available in the literature in two public databases (Violence in Crowds and Hockey Fights). The combination of CENTRIST and HoG was demonstrated in comparison to the use of these operators individually. The combination of both operators obtained approximately 88% against 81% using only HoG and 86% using CENTRIST. From the refinement of the proposed method, we identified that evaluating blocks of the frame with the sliding window-based approach made the method more effective. Techniques for generating a codebook with sparse coding, distance measurement with a Gaussian mixture model and distance measurement between clusters were evaluated and discussed. Also we dynamically calculate the threshold for class voting, which obtained superior results in some cases. Finally, strategies for restricting the actors present in the scenes using optical flow were analyzed. By using the Otsu's method to calculate the threshold from the optical flow at the scene, the effectiveness surpasses our most competitive results: 91.46% accuracy for the Violence in Crowds dataset and 92.79% for the Hockey Fights dataset.

This paper presents a fast method for detecting textured objects moving in a video sequence. It is based on a known background estimation and a fixed camera position. The algorithm is able to detect the presence of moving objects and locates them on-line. It is a frame-by-frame method. First, a difference image is computed from the background and the current frame. This yields three classes of pixels, those for which something changed with respect to the background, those for which nothing changed, and finally the pixels for which no decision can be made. Then an a contrario model allows an automatic clustering, by using adapted rectangular windows, of the pixels for which changes have been detected. If necessary, these regions are corrected in order to better fit the moving objects' boundaries. Experimental results show that the algorithm is very robust with respect to noise and to the quality of the background estimation. The choice of the model parameters is quite natural and user friendly. The algorithm has been successfully tested on video sequences coming from different databases, including indoor and outdoor sequences.

The rapid development of multimedia applications over the past decade requires efficient methods for video browsing. In this paper, we present an algorithm for video summarization with shot comparison. We analyze video content in the shot level, and we calculate the shot distance using the advanced Hausdorff distance. The advanced Hausdorff distance combines the Hausdorff distance and Boolean model, and it could compare two shots from the global view. When the shot similarity matrix is obtained, we group these video shots into several clusters using the affinity propagation cluster method to remove redundant video content. Performance evaluation on ten video sequences are given to illustrate the proposed algorithm.

In recent years, global motion estimation (GME) has become an important tool in the fields of video coding, video compression and computer vision. Estimating the correct global motion is often more difficult when the video scene contains large foreground objects. Previous approaches to addressing this problem have required the application of algorithm parameters that are sequence dependent or give inconsistent results for different video sequences. In this paper, we propose a fully automatic approach that can successfully estimate global motion in the presence of large foreground objects. The proposed algorithm determines an initial estimate of the foreground pixels and then reduces the effect of the remaining foreground by using a modified Lorentzian estimator. Experimental results show the proposed method produces superior and more consistent performance than some recent approaches for a wide range of sequences.

Real time object tracking is the process of locating moving objects over time using the camera in video sequences in real time. The objective of object tracking is to associate target objects in consecutive video frames. Object tracking requires location and shape or features of objects in the video frames. Every tracking algorithm needs to detect moving object. So object detection is the preceding step of object tracking in computer vision applications. After that, detected object can be extracted by the feature of moving object to track that moving object into video scene. It is challenging task in image processing to track the objects into consecutive frames. Various challenges can arise due to complex object motion, irregular shape of object, occlusion of object to object and object to scene and real time processing requirements. Object tracking has a variety of uses, some of which are: surveillance and security, traffic monitoring, video communication, robot vision and animation.

Humans assimilate information from the traffic environment mainly through visual perception. Obviously, the dominant information required to conduct a vehicle can be acquired with visual sensors. However, in contrast to most other sensor principles, video signals contain relevant information in a highly indirect manner and hence visual sensing requires sophisticated machine vision and image understanding techniques. This paper provides an overview on the state of research in the field of machine vision for intelligent vehicles. The functional spectrum addressed covers the range from advanced driver assistance systems to autonomous driving. The organization of the article adopts the typical order in image processing pipelines that successively condense the rich information and vast amount of data in video sequences. Data-intensive low-level “early vision” techniques first extract features that are later grouped and further processed to obtain information of direct relevance for vehicle guidance. Recognition and classification schemes allow to identify specific objects in a traffic scene. Recently, semantic labeling techniques using convolutional neural networks have achieved impressive results in this field. High-level decisions of intelligent vehicles are often influenced by map data. The emerging role of machine vision in the mapping and localization process is illustrated at the example of autonomous driving. Scene representation methods are discussed that organize the information from all sensors and data sources and thus build the interface between perception and planning. Recently, vision benchmarks have been tailored to various tasks in traffic scene perception that provide a metric for the rich diversity of machine vision methods. Finally, the paper addresses computing architectures suited to real-time implementation. Throughout the paper, numerous specific examples and real world experiments with prototype vehicles are presented.

In this paper we present video event representation and recognition approaches that are based on Generalized Stochastic Petri Nets (GSPN). Along with the typical modeling capabilities of GSPN for video recognition, we propose to integrate the Petri net marking analysis for better scene understanding. This work focuses on behavior modeling and uses the results of an external module for object detection, tracking and classification. The proposed approach is evaluated using the developed surveillance system which can recognize events from videos and give a textual expression for the detected behavior. The experimental results illustrate the ability of the system to create complex spatiotemporal relations and to recognize the behavior of one or multiple objects in various video scenes.

We propose in this paper a model-based approach to clustering video scenes based on scenelets. We define a video scenelet as a short consecutive sample of frames of a video sequence. The approach makes use of an unsupervised method to represent scenelets of a video with a concise Gaussian mixture model and cluster them into different video scenes according to their visual similarities. In particular the expectation-maximization algorithm is employed to estimate the unknown model parameters, and Bayesian information criterion is used to determine the optimal number and model of scene clusters in a principled manner. This approach is fundamentally different from many existing video clustering methods, as it does not require explicit knowledge of shot boundaries. Instead, the shot boundaries can also be obtained as a by-product of the scene clustering process. The proposed methods have been tested with various types of sports videos and promising results are reported in this paper.

Video surveillance systems are growing in size and complexity. Such systems typically consist of integrated modules of different vendors to cope with the increasing demands on network and storage capacity, intelligent video analytics, picture quality, and enhanced visual interfaces. Within a surveillance system, relevant information (like technical details on the video sequences, or analysis results of the monitored environment) is described using metadata standards. However, different modules typically use different standards, resulting in metadata interoperability problems. In this paper, we introduce the application of Semantic Web Technologies to overcome such problems. We present a semantic, layered metadata model and integrate it within a video surveillance system. Besides dealing with the metadata interoperability problem, the advantages of using Semantic Web Technologies and the inherent rule support are shown. A practical use case scenario is presented to illustrate the benefits of our novel approach.

Chapter 11 - Social network inference in videos

We develop a system which generates summaries from seniors' indoor-activity videos captured by a social robot to help remote family members know their seniors' daily activities at home. Unlike the traditional video summarization datasets, indoor videos captured from a moving robot poses additional challenges, namely, (i) the video sequences are very long (ii) a significant number of videoframes contain no-subject or with subjects at ill-posed locations and scales (iii) most of the well-posed frames contain highly redundant information. To address this problem, we propose to exploit pose estimation for detecting people in frames. This guides the robot to follow the user and capture effective videos. We use person identification to distinguish a target senior from other people. We also make use of action recognition to analyze seniors' major activities at different moments, and develop a video summarization method to select diverse and representative keyframes as summaries.

Caption graphically superimposed in video frames can provide important indexing information. The automatic detection and recognition of video captions can be of great help in querying topics of interest in digital news library. To detect the caption from video sequence, we present algorithms based on fuzzy clustering neural networks. Since neural networks have the capabilities of learning and self-organizing and parallel computing mechanism, with the great increasing of digital images and video databases, neural networks based techniques become more efficient and popular tools for multimedia processing. Experimental results show that our caption detection scheme is effective and robust.

This paper describes a video surveillance system capable of recognizing human postures from video sequences. The system comprises of two key modules: human detection and posture classification. In the module of human detection, human blobs are extracted by the technique of background subtraction. An adaptive background model is employed to characterize the dynamics and complexity of outdoor scenes based on the mixture of Gaussians. In order to formulate the variations of human postures, pseudo 2D hidden Markov models (P2DHMM) is employed for representing and recognizing human postures based on its '2-D elastic matching' property. It is trained to differentiate human postures and tolerate the variations of the same human posture using embedded Viterbi and segmental K means algorithms. In the classification of human postures, observation sequence is extracted from current image frame. The probabilities of observation sequence corresponding to each P2DHMM model are computed by doubly embedded Viterbi optimization, and human blob is classified as the human posture with the highest likelihood.