Certain investigations on video scene segementaion techniques

Video scene segmentation is very important research in the field of computer vision, because it helps in efficient storage, indexing and retrieval of videos. Achieving this kind of scene segmentation cannot be done by just calculating the similarity of low-level features presented in the video; high-level features should also be considered to achieve a better performance. Even though much research has been conducted on video scene segmentation, most of these studies failed to semantically segment a video into scenes. Thus, in this study, we propose a Deep-learning Semantic-based Scene-segmentation model (called DeepSSS) that considers image captioning to segment a video into scenes semantically. First, the DeepSSS performs shot boundary detection by comparing colour histograms and then employs maximum-entropy-applied keyframe extraction. Second, for semantic analysis, using image captioning that benefits from deep learning generates a semantic text description of the keyframes. Finally, by comparing and analysing the generated texts, it assembles the keyframes into a scene grouped under a semantic narrative. That said, DeepSSS considers both low- and high-level features of videos to achieve a more meaningful scene segmentation. By applying DeepSSS to data sets from MS COCO for caption generation and evaluating its semantic scene-segmentation task results with the data sets from TRECVid 2016, we demonstrate quantitatively that DeepSSS outperforms other existing scene-segmentation methods using shot boundary detection and keyframes. What’s more, the experiments were done by comparing scenes segmented by humans and scene segmented by the DeepSSS. The results verified that the DeepSSS’ segmentation resembled that of humans. This is a new kind of result that was enabled by semantic analysis, which was impossible by just using low-level features of videos.

Currently, video scene segmentation is an important part of realizing content-based video retrieval (CBVR). Aiming at the problem that low efficiency of video scene segmentation in CBVR, this paper proposed a multi-modal video scene segmentation optimization algorithm based on feature extraction of convolutional neural network (CNN). According to the large amount of information contained in the multi-modal data of video, the VGG19 network has been improved in a targeted manner and the underlying features and semantic features of various modes are extracted from each video shots. By forming these features into vectors and using the method such as triplet loss learning and shot similarity calculation, scene segmentation task is converted to a binary classification problem for shot boundary. Then the scoring mechanism is established to optimize the results, finally the scene segmentation task is completed. Experimental results show that the algorithm can be effective in video scene segmentation, and the overall recall and precision can reach 85.77% and 87.01%, respectively. Compared with the shot similarity graph method, two indicators have increased by 10% and 9% respectively. Compared with the DeepSSS method that also uses the deep learning network model, the comprehensive metric F-messure has increased by 8%.

In order to quickly and effectively segment the video scene, a multi-modality video scene segmentation algorithm with shot force competition is proposed in this paper. This method is take full account of temporal associated co-occurrence of multimodal media data, to calculate the similarity between video shot by merging the video low-level features, then go to the video scene segmentation based on the judgment method of shot competition. The authors experiments show that the video scene can be efficiently separated by the method proposed in the paper.

Video scene segmentation is a process for dividing video into semantically meaningful blocks. This can help e.g. search engines to divide video into better manageable parts and enable more relevant search in video. Unfortunately, scene segmentation is based on the semantic and therefore it is a difficult task for computers. This work is preliminary study involved into supervised video scene segmentation, which is driven by the way how human segments scenes in a movie. Since these video segments represent semantic parts in video, it can be used for better video annotation and also for searching in videos. As a training set, only high quality movies were used and from these movies 100 training samples have been extracted and used for evaluation. Resulting model is a method based on general color layout, Tamura similarity measure and k-nearest neighbors achieving 97.00% accuracy.

In this paper, we have developed a novel framework to enable more effective investigation of large-scale news video database via knowledge visualization. To relieve users from the burdensome exploration of well-known and uninteresting knowledge of news reports, a novel interestingness measurement for video news reports is presented to enable users to find news stories of interest at first glance and capture the relevant knowledge in large-scale video news databases efficiently. Our framework takes advantage of both automatic semantic video analysis and human intelligence by integrating with visualization techniques on semantic video retrieval systems. Our techniques on intelligent news video analysis and knowledge discovery have the capacity to enable more effective visualization and exploration of large-scale news video collections. In addition, news video visualization and exploration can provide valuable feedback to improve our techniques for intelligent news video analysis and knowledge discovery.

In this paper, we present a schema for content-based analysis of broadcast news video. First, we separate commercials from news using audiovisual features. Then, we automatically organize news programs into a content hierarchy at various levels of abstraction via effective integration of video, audio, and text data available from the news programs. Based on these news video structure and content analysis technologies, a TV news video Library is generated, from which users can retrieve definite news story according to their demands.

In this paper, we propose an optimized video scene segmentation approach with considering both content coherence and temporally contextual dissimilarity. First, a chain structure is constructed by connecting temporally adjacent shots to represent a video. Then the chain is partitioned such that the content within a chain segment is coherent enough and the contextual similarity of temporally adjacent chain segments is small enough. This task is formulated as a ratio function of content coherence and contextual similarity. Finally, we present an effective and efficient hierarchical chain partitioning approach to find the optimal scene segmentation. Experimental results on a set of home videos and feature movies demonstrate the superiority of the proposed approach over several existing key approaches.

As more and more visual information is available on video, information indexing and retrieval of digital video data is becoming important. A digital video database embedded with visual information processing using image analysis and image understanding techniques such as automated target detection, classification, and identification can provide query results of higher quality. We address in this paper a robust digital video database system within which a target detection module is implemented and applied onto the keyframe images extracted by our digital library system. The tasks and application scenarios under consideration involve indexing video with information about detection and verification of artificial objects that exist in video scenes. Based on the scenario that the video sequences are acquired by an onboard camera mounted on Predator unmanned aircraft, we demonstrate how an incoming video stream is structured into different levels -- video program level, scene level, shot level, and object level, based on the analysis of video contents using global imagery information. We then consider that the keyframe representation is most appropriate for video processing and it holds the property that can be used as the input for our detection module. As a result, video processing becomes feasible in terms of decreased computational resources spent and increased confidence in the (detection) decisions reached. The architecture we proposed can respond to the query of whether artificial structures and suspected combat vehicles are detected. The architecture for ground detection takes advantage of the image understanding paradigm and it involves different methods to locate and identify the artificial object rather than nature background such as tree, grass, and cloud. Edge detection, morphological transformation, line and parallel line detection using Hough transform applied on key frame images at video shot level are introduced in our detection module. This function can also help rapidly filter incoming video and extract only those video sequences of potential interest under real time combating environment. Experimental results on video sequences acquired by Predator prove the feasibility of our approach.

Most questions in the Fish4Knowledge project are related to the ability to count fish, fish species or events using video analysis Chap. 2. Automatic video analysis however brings uncertainty due to False Positive/False Negative classifications, which makes determining the counts based on automatic video analysis difficult. Automatic video analysis software is often able to express a measure of uncertainty for a single detection/recognition using a similarity score, indicating how certain the software is about a single decision. Logistic Regression allows us to combine these similarity scores to compute an estimated count, outperforming the estimates of the underlying machine learning methods used for the original video analysis. We show this works both for the two-class and multi-class problem. Furthermore, we identify potential pitfalls where ensuring a correct sampling procedure is essential. The error in the estimated species counts using Logistic Regression was on average around 15 on a set of 11,585 fish images, while the machine methods had an average error of 800. The key to understanding this huge improvement in accuracy is that we use Logistic Regression over the individual classifications to estimate the size of the whole population, rather than the species classification of any individual observation.

The preconditioned Crank--Nicolson (pCN) method is a Markov chain Monte Carlo (MCMC) scheme, specifically designed to perform Bayesian inferences in function spaces. Unlike many standard MCMC algorithms, the pCN method can preserve the sampling efficiency under the mesh refinement, a property referred to as being dimension independent. In this work we consider an adaptive strategy to further improve the efficiency of pCN. In particular we develop a hybrid adaptive MCMC method: the algorithm performs an adaptive Metropolis scheme in a chosen finite dimensional subspace and a standard pCN algorithm in the complement space of the chosen subspace. We show that the proposed algorithm satisfies certain important ergodicity conditions. Finally with numerical examples we demonstrate that the proposed method has competitive performance with existing adaptive algorithms.

With the continuous development of video detection technology, the video analysis technology based on campus security has become an important part of the construction of safe campus. As the college students still are a group that has poor ability of security protection, campus security issue is closely related to the stability of society and family happiness, and has become a topic of concern to the whole society. The intelligent vision-based campus public safety monitoring system is an important means to achieve security monitoring, it can automatically analyze the video image sequence, and detect, track and identify objects in the monitoring scene without human intervention, and make high-level understanding and analysis of behaviors on this basis. Most of the existing visual monitoring systems can collect and store video data, and the real-time event detection task can automatically be generated through background analysis. Intelligent visual monitoring system should not only be used for accident investigation, but also be used to prevent potential disasters and accidents. The system is consisted of system management platform, event mining and analysis, monitoring and extraction of moving targets, forecasting and tracking targets. The paper makes an in-depth study on the application of intelligent visual detection technology on campus. Based on the intelligent visual video analysis, hidden Markov model is adopted in the paper for video event detection and analysis, motion features and shape features are taken as the observation data, and segmentation method is adopted to analyze the influence of video viewing height and angle on the detection result.

Nowadays the vast amount of video data is generated, which needs to be processed fast for getting correct insights into it. Manpower is not enough to process all of it in a short span of time due to their limited capacity. Video analytics facilitates useful tools to extract more valuable video data. This paper lays stress on different videos analysis activities. Here we explore various techniques which are used in video analysis along with presenting a comparison with image analysis techniques. Further, the application of video analytics along with the challenges being faced while analyzing the real time videos is discussed.

Chapter 11 - Social network inference in videos

Video-surveillance systems are one of the main source of information during investigations. However, the adopted closed-circuit devices are often affected by poor quality mainly because of economical and practical problems. In many cases a low quality image can give useful information during the first phase of the investigation. As a consequence, the images and sequences coming from video-surveillance systems need to be digitalized in order to be processed to enhance and extract features useful for crime analysis. Visual information is the most appealing and intuitive mode of conveying information. Further, the amount of information that video carries is significantly greater than that carried by any other media. From surveillance for security, to capturing and analyzing medical procedures, to forensic applications. The greatest users of video information constitute the surveillance cameras used for security. The content in such applications constitutes hours of video usually taken from a single camera that is stationary. For such applications, it is usually necessary to trace certain anomalous events or individuals .In particular, airport surveillance cameras, surveillance camera in traffic, shopping malls are very good examples. The purpose of these cameras is to detect suspicious activity, like a perpetrator leaving a piece of luggage that might be harmful, or picking up something that might not belong to him or her, Firing from a vehicle, suspicious activity in shopping malls etc. In this work, the task is the automatic segmentation of continuous video into activity-based segments. It is different from the conventional shot-detection and scene change problems of video processing. The information being searched for is not the boundary between two different video contents in a single video, but a significant change in the activity within a single scene in the video. Thus, a temporal segmentation of the video into different events is desired. The approach used is to combine some of the conventional shot-detection methods with some new techniques for video-content analysis . KeywordsVideo-surveillance systems, crime analysis, automatic segmentation, activity-based segments, video-content analysis. International Journal of Machine Intelligence ISSN: 0975-2927 & E-ISSN: 0975-9166, Volume 4, Issue 1, 2012 Citation: Angadi S.A., Vilas Naik and Ashwin Kumar (2012) Automatic Activity Segmentation from Surveillance Video Using Conventional Shot Boundary Detection Methods. International Journal of Machine Intelligence, ISSN: 0975-2927 & E-ISSN: 0975-9166, Volume 4, Issue 1, pp.-404. Copyright: Copyright©2012 Angadi S.A., Vilas Naik and Ashwin Kumar. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

The voluminous data analysis is an obstacle for video indexing and retrieval, a novel method based on video frame difference is proposed to make the fast indexing: firstly frame clustering with FSVM is used to extract the important scene in video; secondly the scenes are labelled with characteristic features; finally, the associated rule data-mining is used to fabricate the last video analysis. The experimental results suggest that the proposed method is an effective approach for video analysis.