Minimum dependency key frames selection via quadratic mutual information

Based on compressed video segment detection, the decompression step can be omitted, features can be directly extracted from the original video data stream, and the detection rate can be accelerated. This paper analyzes the characteristics of video data, video segmentation and key frame selection. Based on the analysis of the key technologies of video retrieval technology, the compressed video stream extraction method based on video segmentation and key frame is introduced. A method of extracting key frames from MPEG compressed video based on DC coefficients and motion vectors is proposed. Experiments show that the proposed method can reduce the computational burden, and can better represent the video content.

Key frame extraction is an important manner of video summarization. It can be used to interpret video content quickly. Existing approaches first partition the entire video into video clips by shot boundary detection, and then, extract key frames by frame clustering. However, in most team-sport videos, a video clip usually includes many events, and it is difficult to extract the key frames related to all of these events accurately, because different events of a game shot can have features of similar appearance. As is well known, most events in team-sport videos are attack and defense conversions, which are related to global translation. Therefore, by using fine-grained partition based on the global motion, a shot could be further partitioned into more video clips, from which more key frames could be extracted and they are related to the events. In this study, global horizontal motion is introduced to further partition video clips into fine-grained video clips. Furthermore, global motion statistics are utilized to extract candidate key frames. Finally, the representative key frames are extracted based on the spatial–temporal consistence and hierarchical clustering, and the redundant frames are removed. A dataset called SportKF is built, which includes 25 videos of 197,878 frames in 112 min and 764 key frames from four types of sports (basketball, football, American football and field hockey). The experimental results demonstrate that the proposed scheme achieves state-of-the-art performance by introducing global motion statistics.

Due to video data exponential growth supported by quick advances in multimedia technology, it became difficult for the user to retrieve information from a large videos collection. Key frame extraction consists in providing an abstract of the entire video, containing the most informative frames. In this paper, we present an efficient key frame extraction method. This method is based on local interest points description and repeatability graph clustering via approaching modularity value. To minimize the data to be treated, the process will be applied only on a set candidate frames selected with a windowing rule. Indeed, the first step consists in extracting interest points on a set of candidate frames. After that, we compute repeatability values between each two frames from the candidate set. These values are represented by a repeatability direct graph. The selection of key frames is performed using graph clustering by approaching modularity principle. The experiments performed showed that the proposed method succeeds in extracting key frames that preserve the relevant video content without redundancy.

Surface hardness monitoring of laser shock Peening: Acoustic emission and key frame selection

In an interactive multimedia information system, users should have the flexibility to browse and choose various scenarios they want to see. This means that two-way communications should be captured by the conceptual model. Digital video has gained increasing popularity in many multimedia applications. Instead of sequential access to the video contents, the structuring and modeling of video data so that users can quickly and easily browse and retrieve interesting materials has become an important issue in designing multimedia information systems. An abstract semantic model called the augmented transition network (ATN), which can model video data and user interactions, is proposed in this paper. An ATN and its subnetworks can model video data based on different granularities, such as scenes, shots and key frames. Multimedia input strings are used as inputs for ATNs. The details of how to use multimedia input strings to model video data are also discussed. Key frame selection is based on the temporal and spatial relations of semantic objects in each shot. These relations are captured from our proposed unsupervised video segmentation method, which considers the problem of partitioning each frame as a joint estimation of the partition and class parameter variables. Unlike existing semantic models, which only model multimedia presentation, multimedia database searching or browsing, ATNs together with multimedia input strings can model these three in one framework.

In this paper we carry out a survey on key frame extraction methods for Video Summary. We also discuss the summary evaluation criteria and compare the approaches based on the method, data set and the results. Video Summary is a process of presenting an abstract of entire video within a short period of time. It aims to provide a compact video representation, while preserving the essential activities of the original video. It is an essential task in video analysis and indexing applications. Most of the video summaries are based on selection of key frames within the shots of a video. Many of them use motion features and few use visual features for extracting the key frames. The video summary quality assessment methods are based more on subjective and less on objective measures. Tong wei Ren et al has provided a framework to assess the quality of the video against a given reference summary using both subjective and objective measures. Ciocca et al used the objective measures for evaluation of summary and most of them evaluate by taking the subjective opinion of experts. A framework for automatic evalution is needed based on both subjective and objective measures without the reference summary.

The continuous advancement of computer vision and deep learning technology provides powerful technical support for supervising the occurrence of falling behaviors. Considering that some frames in the video contribute little to the recognition of falling behaviors, in order to eliminate the video frames irrelevant to falling behaviors, this paper proposes an algorithm to extract key frames in a video by combining LUV local maximum and Mask-RCNN. Firstly, candidate key frames are extracted based on the local maximum of LUV. Video frames containing motion changes can be obtained. Afterwards, in order to further eliminate the video frames that are less relevant to the fall action, Mask-RCNN is used for human body detection. According to the aspect ratio of the bounding box and the motion speed of the human body, the video frames that are more relevant to the fall action are selected as key frames. Experiments and results analysis are carried out on the UR fall detection dataset, Multiple cameras fall dataset, Le2i Fall detection dataset and falling video dataset of real scenes. The effectiveness and accuracy of the proposed method are verified.

Shot reconstruction degree: a novel criterion for key frame selection

In this paper, we present a fast method of key frame selection and an accurate feature point matching method for 3D reconstruction through the control of quadrotor. The quadrotor is controlled to fly at a fixed altitude and the gimbal camera has always been a downward direction. Therefore, the additional information of the quadrotor could be easily added to the 3D reconstruction process. As a result, we find a fast method to select key frames and an optimized matching method through the geometric constraints of the flight path and direction. The method is mainly to improve the efficiency of key frame selection and to get more accurate matching points. The advantage of our approach is to combine the two processes between video capture and video processing by adding flight control to the 3D scene reconstruction method. Therefore, it can use less time to complete the entire reconstruction task. The proposed approach is tested by quadrotor platform. Experiment results show that our method can greatly reduce the key frame selection time and get more matching points at the same accuracy.

An efficient video summarization for surveillance system using normalized k-means and quick sort method

In this paper, a generalized statistical tool is introduced to estimate key frames in a video sequence. The tool works based on the inter-relationship between different features of image frames in a video. The image feature vectors are plotted in feature space as points and a randomness measure is determined from the distribution of these points. The randomness measure of the feature vectors is defined with respect to simulated random point patterns and expressed as a probability value of a frame being a key frame. Since, depending on the video content more than one inter-relationship of features can be used to determine a single key frame, different probability values are derived to support a frame as a key frame. To integrate these probability values a combiner model is designed to uniquely decide the status of a key frame. The combiner model is based on the Dempster-Shafer theory of evidence. To demonstrate the idea, randomness measures, and consequently the probabilities of a frame to be a key frame, are obtained separately from spatial domain and frequency domain features. The combined probability value enhances the confidence in selecting a frame as a key frame. The result is tested on a number of standard video sequences and it outperforms the related approach

The main objective of this study framework is to help the user to management the interesting video images from the huge storage device. The study can be divided into three main steps is listing as follows: shot boundary detection, key frame selection and evaluation method are designed by analyzing the video content. In this paper we proposed a novel algorithm for shot boundary detection. First, transform the RGB color model into LUV color model according our experimental results. The useful features of each frame are extracted from sequence images by the automatically computation of their color moment-preserving and matching 64-predefined bit planes in order to detect the shot boundary. The excellent shot boundary detection will result in the good key frames selection. Second, the key frame selection is proceed by applying the geometric invariant moments which successful be used in target tracing, pattern recognition, and facial recognition. In our experiments we used five test sequences that belong to five different program categories: movies, advertising, sport game, news, and commercial documentary. The experimental results showed that our method is feasible, effective, and is better than the former approaches.

Mining human movement evolution for complex action recognition

We present a framework for efficient reconstruction of dense scene structure from video. Sequential structure-from-motion recovers camera information from video, providing only sparse 3D points. We build a dense 3D point cloud by performing full-frame tracking and depth estimation across sequences. First, we present a novel algorithm for sequential frame selection to extract a set of key frames with sufficient parallax for accurate depth reconstruction. Second, we introduce a technique for efficient reconstruction using dense tracking with geometrically correct optimisation of depth and orientation. Key frame selection is also performed in optimisation to provide accurate depth reconstruction for different scene elements. We test our work on benchmark footage and scenes containing local non-rigid motion, foreground clutter and occlusions to show comparable performance to state of the art techniques. We also show a substantial increase in speed on real world footage compared to existing methods, when they succeed, and successful reconstructions when they fail.

In 2D-to-3D video conversion, key-frame selection is essential and it affects the quality and workload of the conversion. In this paper, a key-frame selection method based on edge classification is proposed. In the proposed method, the candidate key-frames are selected out based on the occlusion area and feature point correspondence. The optimal key-frames are selected out from the candidates according to the edge point classification so that the depth of the key-frames can be estimated accurately and automatically referring to the depth estimation based on focus cue. Experiments show that the proposed method can bring 2D-to-3D conversion better objective and subject quality and it is promising in practical applications.