Shot reconstruction degree: a novel criterion for key frame selection

Surface hardness monitoring of laser shock Peening: Acoustic emission and key frame selection

Key frame selection, which aims to choose a small number of frames that well represent the entire video contents, is one of the important research issues in video content analysis. Many existing methods developed so far use some prior information about the contents of video frames based on high-level semantic features, and thus they are applicable only in particular problems. Without any prior information, approaches based on low-level index features such as clusters, correlation, and consecutive frame difference are useful. Key frames found by these approaches are expected to be as much different from each other as possible. Therefore, accurately evaluating the difference among candidate key frames is a key challenge. However, a standard similarity metric such as the squared error and the cross correlation tends to provide redundant key frames. To cope with this problem, we propose to use quadratic mutual information (QMI) as an alternative similarity measure, which allows us to capture higher-order correlation for more than two variables simultaneously. Through experiments, we demonstrate the usefulness of the proposed QMI-based key frame selection method that uses an accurate and computationally efficient least-squares QMI estimator.

In this paper, we present a fast method of key frame selection and an accurate feature point matching method for 3D reconstruction through the control of quadrotor. The quadrotor is controlled to fly at a fixed altitude and the gimbal camera has always been a downward direction. Therefore, the additional information of the quadrotor could be easily added to the 3D reconstruction process. As a result, we find a fast method to select key frames and an optimized matching method through the geometric constraints of the flight path and direction. The method is mainly to improve the efficiency of key frame selection and to get more accurate matching points. The advantage of our approach is to combine the two processes between video capture and video processing by adding flight control to the 3D scene reconstruction method. Therefore, it can use less time to complete the entire reconstruction task. The proposed approach is tested by quadrotor platform. Experiment results show that our method can greatly reduce the key frame selection time and get more matching points at the same accuracy.

Due to video data exponential growth supported by quick advances in multimedia technology, it became difficult for the user to retrieve information from a large videos collection. Key frame extraction consists in providing an abstract of the entire video, containing the most informative frames. In this paper, we present an efficient key frame extraction method. This method is based on local interest points description and repeatability graph clustering via approaching modularity value. To minimize the data to be treated, the process will be applied only on a set candidate frames selected with a windowing rule. Indeed, the first step consists in extracting interest points on a set of candidate frames. After that, we compute repeatability values between each two frames from the candidate set. These values are represented by a repeatability direct graph. The selection of key frames is performed using graph clustering by approaching modularity principle. The experiments performed showed that the proposed method succeeds in extracting key frames that preserve the relevant video content without redundancy.

Wireless capsule endoscopy (WCE) is a non-invasive imaging technique to examine the entire small intestine. This paper presents a novel hierarchical key frame extraction algorithm based on the saliency map to automatically select a small number of key informative frames in the WCE video to assist the diagnostic process. We start from calculating the saliency map and extracting the inside colour, texture and shape features. We then compute the image information entropies with adapted weights on three features; calculate the differences of entropies in the successive frames and extract the first level key frames to divide the video into multi-segments. Finally, the mean shift clustering method is applied on each segment to obtain the final key frames. The proposed technique achieves the fidelity of 95.07% and the compression ratio of 87.9% on average, validating that the proposed scheme is highly promising for the key frame extraction in the WCE video.

Based on compressed video segment detection, the decompression step can be omitted, features can be directly extracted from the original video data stream, and the detection rate can be accelerated. This paper analyzes the characteristics of video data, video segmentation and key frame selection. Based on the analysis of the key technologies of video retrieval technology, the compressed video stream extraction method based on video segmentation and key frame is introduced. A method of extracting key frames from MPEG compressed video based on DC coefficients and motion vectors is proposed. Experiments show that the proposed method can reduce the computational burden, and can better represent the video content.

In this paper we carry out a survey on key frame extraction methods for Video Summary. We also discuss the summary evaluation criteria and compare the approaches based on the method, data set and the results. Video Summary is a process of presenting an abstract of entire video within a short period of time. It aims to provide a compact video representation, while preserving the essential activities of the original video. It is an essential task in video analysis and indexing applications. Most of the video summaries are based on selection of key frames within the shots of a video. Many of them use motion features and few use visual features for extracting the key frames. The video summary quality assessment methods are based more on subjective and less on objective measures. Tong wei Ren et al has provided a framework to assess the quality of the video against a given reference summary using both subjective and objective measures. Ciocca et al used the objective measures for evaluation of summary and most of them evaluate by taking the subjective opinion of experts. A framework for automatic evalution is needed based on both subjective and objective measures without the reference summary.

With the improvement of people's security awareness, numerous monitoring equipment has been put into use, resulting in the explosive growth of surveillance video data. Key frame extraction technology is a paramount technology for improving video storage efficiency and enhancing the accuracy of video retrieval. It can extract key frame sets that can express video content from massive videos. However, the existing key frame extraction algorithms of surveillance video still have deficiencies, such as the destruction of image information integrity and the inability to extract key frames accurately. To this end, this paper proposes a key frame extraction algorithm of surveillance video based on quaternion Fourier saliency detection. Firstly, the algorithm used colors, and intensity features to perform quaternion Fourier transform on surveillance video sequences. Next, the phase spectrum of the quaternion Fourier transformed image was obtained, and he image visual saliency map was obtained according to the quaternion Fourier phase spectrum. Then, the image visual saliency map of two adjacent frames is used to characterize the change of target motion state. Finally, the frames that can accurately express the motion state of the target are selected as key frames. The experimental results show that the method proposed in this paper can accurately capture the changes of the local motion state of the target while maintaining the integrity of the image information.

Key frame extraction is one of the critical techniques in computer vision fields such as video search, video identification and video forgeries detection. The extracted key frames should be sufficient key frames that preserve main actions in a video with compact representation. The objective of this work is to improve our previous action key frames extraction algorithm (AKF) by adapting a threshold which selects action key frames as final key frames. The threshold adaptation was achieved by using the mean value, the standard deviation, and the L1-norm instead of the comparison of user summaries evaluation method to obtain a fully automatic video summarisation algorithm, and by eliminating the conditions in selecting the final key frames to reduce the complexity of the algorithm. We have validated our proposed Improved AKF on complex colour video shots instead of the simple grey level video shots. The Improved AKF algorithm was able to extract a compact number of action key frames by preventing redundant key frames, reduce processing complexity, and preserve sufficient information about the main actions in a video shot. We then evaluated the Improved AKF algorithm with the-state-of-the-art algorithms in terms of compression ratio using Paul videos and Shih-Tang dataset. The evaluation results showed that the Improved AKF algorithm achieved better compression ratio and retained sufficient information in the extracted action key frames under different testing video shots. Therefore, the improved AKF algorithm is a suitable technique for applications in computer vision fields such as passive object-based video authentication systems

Aiming at the problem of video key frame extraction, a density peak clustering algorithm is proposed, which uses the HSV histogram to transform high-dimensional abstract video image data into quantifiable low-dimensional data, and reduces the computational complexity while capturing image features. On this basis, the density peak clustering algorithm is used to cluster these low-dimensional data and find the cluster centers. Combining the clustering results, the final key frames are obtained. A large number of key frame extraction experiments for different types of videos show that the algorithm can extract different number of key frames by combining video content, overcome the shortcoming of traditional key frame extraction algorithm which can only extract a fixed number of key frames, and the extracted key frames can represent the main content of video accurately.

In this work, a novel method for color video compression using key-frame based color transfer has been proposed. In this scheme, compression is achieved by discarding the color information of all but few selected frames. These selected frames are either the key frames (frame selected by a key frame selection algorithm) or the Intra coded (I) frames. The partially colored video is compressed using a standard encoder thereby achieving higher compression. In the proposed decoder, a standard decoder first generates the partially colored video sequence from the compressed input. A color transfer algorithm is then used for generating the fully colored video sequence. The complexity of the proposed decoder is close to a standard decoder, allowing its use in wide variety of applications like video broadcasting, video streaming, hand-held devices etc.

We present a framework for efficient reconstruction of dense scene structure from video. Sequential structure-from-motion recovers camera information from video, providing only sparse 3D points. We build a dense 3D point cloud by performing full-frame tracking and depth estimation across sequences. First, we present a novel algorithm for sequential frame selection to extract a set of key frames with sufficient parallax for accurate depth reconstruction. Second, we introduce a technique for efficient reconstruction using dense tracking with geometrically correct optimisation of depth and orientation. Key frame selection is also performed in optimisation to provide accurate depth reconstruction for different scene elements. We test our work on benchmark footage and scenes containing local non-rigid motion, foreground clutter and occlusions to show comparable performance to state of the art techniques. We also show a substantial increase in speed on real world footage compared to existing methods, when they succeed, and successful reconstructions when they fail.

Key frame extraction is an important manner of video summarization. It can be used to interpret video content quickly. Existing approaches first partition the entire video into video clips by shot boundary detection, and then, extract key frames by frame clustering. However, in most team-sport videos, a video clip usually includes many events, and it is difficult to extract the key frames related to all of these events accurately, because different events of a game shot can have features of similar appearance. As is well known, most events in team-sport videos are attack and defense conversions, which are related to global translation. Therefore, by using fine-grained partition based on the global motion, a shot could be further partitioned into more video clips, from which more key frames could be extracted and they are related to the events. In this study, global horizontal motion is introduced to further partition video clips into fine-grained video clips. Furthermore, global motion statistics are utilized to extract candidate key frames. Finally, the representative key frames are extracted based on the spatial–temporal consistence and hierarchical clustering, and the redundant frames are removed. A dataset called SportKF is built, which includes 25 videos of 197,878 frames in 112 min and 764 key frames from four types of sports (basketball, football, American football and field hockey). The experimental results demonstrate that the proposed scheme achieves state-of-the-art performance by introducing global motion statistics.

In this paper, we present a new solution for extracting key frames from motion capture data using an optimization algorithm to obtain compact and sparse key frame data that can represent the original dense human body motion capture animation. The use of the genetic algorithm helps determine the optimal solution with global exploration capability while the use of a probabilistic simplex method helps expedite the speed of convergence. By finding the chromosome that maximizes the fitness function, the algorithm provides the optimal number of key frames as well as the low reconstruction error with an ordinary interpolation technique. The reconstruction error is computed between the original motion and the reconstruction one by the weighted differences of joint positions and velocities. The resulting set of key frames is obtained by iterative application of the algorithm with initial populations generated randomly and intelligently. We also present experiments which demonstrate that the method can effectively extract key frames with a high compression ratio and reconstruct all other non key frames with high quality.

We present a key frames selection algorithm based on three iso-content principles (iso-content distance, iso-content error and iso-content distortion), so that the selected key frames are equidistant in video content according to the used principle. Two automatic approaches for defining the most appropriate number of key frames are proposed by exploiting supervised and unsupervised content criteria. Experimental results and the comparisons with existing methods from literature on large dataset of real-life video sequences illustrate the high performance of the proposed schemata.