Video shot segmentation and classification

In this paper, a novel camera motion descriptor is proposed for video shot classification. In the proposed method, raw motion information of consecutive video frames are extracted by computing the motion vector of each macroblock to form motion vector fields (MVFs). Next, a motion consistency analysis is applied on MVFs to eliminate the inconsistent motion vectors. Then, MVFs are divided into nine (3 × 3) local regions and the singular value decomposition (SVD) technique is applied on the motion vectors extracted from each local region in the temporal direction. Consistent motion vectors of a number of MVFs are compactly represented at a time to characterize temporal camera motion. Accordingly, each local region of the whole video shot is represented using a sequence of compactly represented vectors. Finally, the sequence of vectors is converted into a histogram to describe the camera motions of each local region. Combination of all the local histograms is considered as the camera motion descriptor of a video shot. The shot descriptors are used in a classifier to classify video shots. In this work, we use support vector machine (SVM) for performing classification tasks. The experimental results show that the proposed camera motion descriptor has strong discriminative capability to classify different camera motion patterns in professionally captured video shots effectively. We also show that our proposed approach outperforms two state-of-the-art video shot classification methods.

A new method for detecting shot boundaries in video sequences by fusing features obtained by singular value decomposition (SVD) and mutual information (MI) is proposed. The first method relies on performing singular value decomposition on a matrix created from 3D color histograms of single frames. The method can detect cuts and gradual transitions, such as dissolves, fades and wipes. The second method relies on evaluating mutual information between two consecutive frames. It can detect abrupt cuts, fade-ins and fade-outs with very high accuracy. A combination of features derived from these methods and subsequent processing through a clustering procedure results in very efficient detection of abrupt cuts and gradual transitions, as demonstrated by experiments on the TRECVID2004 video test set containing different types of shots with significant object and camera motion inside the shots.

The properties of singular value decomposition (SVD) are used to implement an SVD spatial domain pseudoinverse restoration filter. This type of filter is attractive for poor imaging conditions (low spatial resolution, high image noise) and is thus appealing for nuclear medicine images. The method might offer some advantages over more traditional frequency domain filter techniques since the restoration is performed on a local rather than global basis. High-contrast thyroid phantom images collected at different count densities and low-contrast liver phantom images were processed with the SVD filter. Restored images yielded improved spatial resolution, lesion contrast, and signal-to-noise ratio. The SVD pseudoinverse restoration filter implemented as an interactive process permits the operator to terminate filtering at a stage where the visually "best" image is obtained compared to the original data. Processed images suggest that the technique may have potential for improving lesion detection in nuclear medicine.

The security of information exchange is very important on the network. Authentication and information hiding have also become important issues. Information hiding techniques are acquiring an increasing importance due to the widespread diffusion of multimedia contents. The aim of this chapter is to focus on the Singular Value Decomposition (SVD) transform, with the aim of providing an exhaustive overview on those steganography, image cryptography and watermarking techniques leveraging on the important properties of such a transform. Despite the attention it has received in the last years, SVD in image processing and security is still in its infancy. Many SVD characteristics are still unutilized in image processing. In this chapter the author tries to highlight the basic properties of SVD and some of their applications in the field of security to encourage researchers to discover more about SVD properties which are not yet utilized.

ABSTRACTThe proposed content-based video retrieval method in this study uses feature extraction, shot boundary detection, shot noise filter, shot cluster, the key feature set of shot extraction and shot classification. Shot boundary detection is conducted using a weight selection genetic algorithm. In order to verify the video retrieval method in this paper, three video databases were used in the experiment. These video databases are used to verify the shot boundary detection, the shot classification and video retrieval of this study. A series of comparisons and analyses were then conducted using the above video databases. The experiment results showed that the shot boundary detection method exhibits better performance than the other compared methods. Moreover, the video retrieval experiment results showed that the proposed method is able to precisely catch a shot in the search video, and do so with reduced video retrieval time.

In this paper, two new properties of singular value decomposition (SVD) on images are proved. The first property demonstrates the quantitative relationship between singular values and power spectrum. The second one proves that under the condition of losing equal power spectrum, the square-error of the reconstructed image is much smaller when we reduce all singular values proportionally instead of neglect the smaller ones. Based on the two properties, a new data-hiding scheme is proposed. It performs well as for robustness, for it satisfies power-spectrum condition (PSC), and PSC-compliant watermarks are proven to be most robust. Besides, the proposed scheme has a good performance as for capacity and adaptability.

Block-wise reversible data hiding in encrypted domain using SVD

In this article, we have improved the quantitative inversion performance of the cross-correlated contrast source inversion (CC-CSI) method by incorporating the subspace optimization strategy. The proposed method is called the cross-correlated subspace optimization method (CC-SOM). Meanwhile, multifrequency data are used to improve the inversion performance of high-contrast scatterers, where the L-curve method is introduced to select the regularization parameters of each frequency point without relying on experience. Finally, a fast algorithm is implemented by using the property of singular value decomposition (SVD) to simplify the large-scale matrix, and the fast Fourier transform (FFT) to accelerate the calculation. Synthetic and experimental inversion results demonstrate that both CC-SOM and CC-CSI show better robustness than SOM. In comparison to CC-CSI, CC-SOM is superior in terms of inversion accuracy when the regularization parameters have been appropriately selected. However, these advantages come at the cost of higher computational complexity.

Author profiling is a challenging task that consists of identifying a person’s relevant attributes based on the content he/she generates. In this article, we validate a classification method based on the reconstructive classification to identify two demographic attributes, age and gender, for users of social networks based on the text content they publish. For the problem, we consider balanced and unbalanced data, where a gender or age group has a larger presence than the others. The proposed method is based on using the reconstructive property of singular value decomposition (SVD), and its suitability to work with sparse data, to find a matrix with the main latent components to represent the information of specific gender or age classes. Afterward, we use such a matrix to project and reconstruct new users’ information to identify their demographic variables. We test our method with a set of datasets in several languages collected from Twitter and Pinterest users, and we use different evaluation metrics to compare their performance with the ones of several popular classifiers based on traditional machine learning, and on deep learning, and with some relevant works in the literature. The results show that the proposed classifier performs generally well in identifying the age and gender of users in social networks.

To achieve more efficient video indexing and access, we introduce a video database management framework and strategies for video content structure and events mining. The video shot segmentation and representative frame selection strategy are first utilized to parse the continuous video stream into physical units. Video shot grouping, group merging, and scene clustering schemes are then proposed to organize the video shots into a hierarchical structure using clustered scenes, scenes, groups, and shots, in increasing granularity from top to bottom. Then, audio and video processing techniques are integrated to mine event information, such as dialog, presentation and clinical operation, from the detected scenes. Finally, the acquired video content structure and events are integrated to construct a scalable video skimming tool which can be used to visualize the video content hierarchy and event information for efficient access. Experimental results are also presented to evaluate the performance of the proposed framework and algorithms.

Optimized deep learning enabled lecture audio video summarization

This work investigates the effectiveness of a state-of-the-art concept detection framework for the automatic classification of multimedia content, namely images and videos, embedded in publicly available Web resources containing recipes for the synthesis of Home Made Explosives (HMEs), to a set of predefined semantic concepts relevant to the HME domain. The concept detection framework employs advanced methods for video (shot) segmentation, visual feature extraction (using SIFT, SURF, and their variations), and classification based on machine learning techniques (logistic regression). The evaluation experiments are performed using an annotated collection of multimedia HME content discovered on the Web, and a set of concepts, which emerged both from an empirical study, and were also provided by domain experts and interested stakeholders, including Law Enforcement Agencies personnel. The experiments demonstrate the satisfactory performance of our framework, which in turn indicates the significant potential of the adopted approaches on the HME domain.

Short videos on the Internet are growing exponentially, and the number of videos uploaded every day is huge; people also involve a lot of video data in real life. People can retrieve and view all kinds of videos, but it also brings a lot of problems. On the one hand, the accumulation of a large number of videos makes people unable to find the videos they want quickly, and the repeated scenes in the videos will also waste people's time and energy; on the other hand, a large amount of video data also brings enormous pressure to storage. Aiming at the problems of inaccurate selection of key frames and how to select video frame features in existing video summarization models, this paper proposes a multi-feature-based video summarization generation model (DME-VSNet), which extracts multiple features of video frames. Including importance score, image memory strength and image entropy. Aiming at the problem of inaccurate video shot segmentation, this model proposes a video shot segmentation algorithm based on TransNet network, which divides the original video into several short shots through shot boundaries; the model inputs the above three features into the proposed The video frame score is obtained in the MLP architecture, and the key frame is selected by the score to generate a video summary. The effectiveness of the video shot segmentation method based on TransNet network and the overall model based on convolutional neural network is verified by comparative experiments. The experimental results show that the evaluation results of the video summaries generated by the three features are better.

The segmentation and classification of news video into single-story semantic units is a challenging problem. This research proposes a two-level, multi-modal framework to tackle this problem. The video is analyzed at the shot and story unit (or scene) levels using a variety of features and techniques. At the shot level, we employ a Decision Tree to classify the shot into one of 13 predefined categories. At the scene level, we perform the HMM (Hidden Markov Models) analysis to eliminate shot classification errors and to locate story boundaries. We test the performance of our system using two days of news video obtained from the MediaCorp of Singapore. Our initial results indicate that we could achieve a high accuracy of over 95 % for shot classification. The use of HMM analysis helps to improve the accuracy of the shot classification and achieve over 89% accuracy on story segmentation.KeywordsNews Video classificationmulti-modal approachlearning-based approachStory Segmentation

The extraction of playing shots is helpful for video analysis. In this paper, we propose an algorithm: first, the shots are segmented by detecting the shot boundaries (both abrupt cuts and gradual transitions), and wipes are discarded directly; then from the rest shots, playing shots are extracted by shot classification. Experimental results confirm the efficiency of our proposed algorithm.