Semantic home video categorization

Recognizing Human action in video is an very active research topic. There are a growing variety of human action datasets with different video length, different practitioners. Make human action recognition becomes a very difficult topic. A majority researchers solve the problem by extracting key frames from the videos. Most paper use feature Clustering methods to extract key frames in videos. On one hand, the large variety of visual content in videos make handcraft feature isn't effective enough, since there are no fixed descriptors can describe all video cases. On the other hand, traditional clustering algorithms are easily influenced by the choice of initial clustering centers. An Unsupervised feature learning and clustering method for key frame extraction is proposed in this paper, which can be used for human action recognition. Stacked auto-encoder(SAE) is trained using videos from 10 different human actions, SAE is used as a feature extractor to learn features representing human actions. Affinity Propagation Clustering algorithm is used to select key frames from video sequences. We use a variety of videos to do the experiments. Experiments demonstrate that our method can be effectively summarizing video shots considering different human actions.

For the existing traditional method to extract key frames from the video sequence with gradual change of content, there are problems of having too much redundancy or too much loss in the extraction results and other issues, a new key frames extracting method for adaptively extracting the video key frames has proposed in the paper. The experimental results have shown that the proposed method can effectively extract video key frames, especially for the video with gradual change of content, which can express more succinctly and comprehensively. The experiments on a large number of videos containing various scenes have shown that the key frames extracted by the proposed algorithm can more fully express the main content of video. When dealing with content gradient video, the effect is better. The content adaptively determines the number of key frames based on video.

Aiming at the problem of video key frame extraction, a density peak clustering algorithm is proposed, which uses the HSV histogram to transform high-dimensional abstract video image data into quantifiable low-dimensional data, and reduces the computational complexity while capturing image features. On this basis, the density peak clustering algorithm is used to cluster these low-dimensional data and find the cluster centers. Combining the clustering results, the final key frames are obtained. A large number of key frame extraction experiments for different types of videos show that the algorithm can extract different number of key frames by combining video content, overcome the shortcoming of traditional key frame extraction algorithm which can only extract a fixed number of key frames, and the extracted key frames can represent the main content of video accurately.

Wireless capsule endoscopy (WCE) is a non-invasive imaging technique to examine the entire small intestine. This paper presents a novel hierarchical key frame extraction algorithm based on the saliency map to automatically select a small number of key informative frames in the WCE video to assist the diagnostic process. We start from calculating the saliency map and extracting the inside colour, texture and shape features. We then compute the image information entropies with adapted weights on three features; calculate the differences of entropies in the successive frames and extract the first level key frames to divide the video into multi-segments. Finally, the mean shift clustering method is applied on each segment to obtain the final key frames. The proposed technique achieves the fidelity of 95.07% and the compression ratio of 87.9% on average, validating that the proposed scheme is highly promising for the key frame extraction in the WCE video.

The purpose of video key frame extraction is to use as few video frames as possible to represent as much video content as possible, reduce redundant video frames, and reduce the amount of computation, so as to facilitate quick browsing, content summarization, indexing, and retrieval of videos. In this paper, a method of dance motion recognition and video key frame extraction based on multifeature fusion is designed to learn the complicated and changeable dancer motion recognition. Firstly, multiple features are fused, and then the similarity is measured. Then, the video sequences are clustered by the clustering algorithm according to the scene. Finally, the key frames are extracted according to the minimum amount of motion. Through the quantitative analysis and research of the simulation results of different models, it can be seen that the model proposed in this paper can show high performance and stability. The breakthrough of video clip retrieval technology is bound to effectively promote the inheritance and development of dance, which is of great theoretical significance and practical value.

Key frame extraction is an important manner of video summarization. It can be used to interpret video content quickly. Existing approaches first partition the entire video into video clips by shot boundary detection, and then, extract key frames by frame clustering. However, in most team-sport videos, a video clip usually includes many events, and it is difficult to extract the key frames related to all of these events accurately, because different events of a game shot can have features of similar appearance. As is well known, most events in team-sport videos are attack and defense conversions, which are related to global translation. Therefore, by using fine-grained partition based on the global motion, a shot could be further partitioned into more video clips, from which more key frames could be extracted and they are related to the events. In this study, global horizontal motion is introduced to further partition video clips into fine-grained video clips. Furthermore, global motion statistics are utilized to extract candidate key frames. Finally, the representative key frames are extracted based on the spatial–temporal consistence and hierarchical clustering, and the redundant frames are removed. A dataset called SportKF is built, which includes 25 videos of 197,878 frames in 112 min and 764 key frames from four types of sports (basketball, football, American football and field hockey). The experimental results demonstrate that the proposed scheme achieves state-of-the-art performance by introducing global motion statistics.

In this work, the idea of key frames extraction from single shots in video sequences is presented. The method is implemented by an efficient two-step algorithm, which is classified neither to clustering nor to temporal variations based techniques. In the first step, an MST (minimal spanning tree) graph is constructed, where each node is associated to a single frame of the shot. In the second step, extracts key frames based on the principle of their maximum spread, are extracted. The number of the selected key frames is controlled by an adaptively defined threshold, while the validity of the results is evaluated by the fidelity measure.

In this paper, a generalized statistical tool is introduced to estimate key frames in a video sequence. The tool works based on the inter-relationship between different features of image frames in a video. The image feature vectors are plotted in feature space as points and a randomness measure is determined from the distribution of these points. The randomness measure of the feature vectors is defined with respect to simulated random point patterns and expressed as a probability value of a frame being a key frame. Since, depending on the video content more than one inter-relationship of features can be used to determine a single key frame, different probability values are derived to support a frame as a key frame. To integrate these probability values a combiner model is designed to uniquely decide the status of a key frame. The combiner model is based on the Dempster-Shafer theory of evidence. To demonstrate the idea, randomness measures, and consequently the probabilities of a frame to be a key frame, are obtained separately from spatial domain and frequency domain features. The combined probability value enhances the confidence in selecting a frame as a key frame. The result is tested on a number of standard video sequences and it outperforms the related approach

This paper is based on the Chinese sign language video library, and discusses the algorithm design of video classification based on handshape recognition of key frames in video. Video classification in sign language video library is an important part of sign language arrangement and is also the premise of video feature retrieval. At present, sign language video’s handshape classification work is done manually. The accuracy and correctness of the results are quite erroneous and erroneous. In this paper, from the angle of computer image analysis, the definition and extraction of key frames are carried out, and then the region of interest is identified. Finally, an improved SURF algorithm is used to match the area of interest and the existing hand image, and the classification of the video is completed. The entire process is based on the actual development environment, and it can be used for reference based on the classification of video image features.

For the question of how to classification of cultural relic videos, this paper put forward the analysis method based on semantic and it's two-steps in classification of video: Firstly, based on algorithm that has been used, separating the cultural relic shots and extracting the key frames; Then extracting the features of the key frames. Next training classification of the key frames' features using SVM. The system composed by the two steps' algorithms implementing classification of cultural relic video. Experiments show this method can better classify based on the characteristics of cultural relic videos then other methods.

Key frame extraction is one of the critical techniques in computer vision fields such as video search, video identification and video forgeries detection. The extracted key frames should be sufficient key frames that preserve main actions in a video with compact representation. The objective of this work is to improve our previous action key frames extraction algorithm (AKF) by adapting a threshold which selects action key frames as final key frames. The threshold adaptation was achieved by using the mean value, the standard deviation, and the L1-norm instead of the comparison of user summaries evaluation method to obtain a fully automatic video summarisation algorithm, and by eliminating the conditions in selecting the final key frames to reduce the complexity of the algorithm. We have validated our proposed Improved AKF on complex colour video shots instead of the simple grey level video shots. The Improved AKF algorithm was able to extract a compact number of action key frames by preventing redundant key frames, reduce processing complexity, and preserve sufficient information about the main actions in a video shot. We then evaluated the Improved AKF algorithm with the-state-of-the-art algorithms in terms of compression ratio using Paul videos and Shih-Tang dataset. The evaluation results showed that the Improved AKF algorithm achieved better compression ratio and retained sufficient information in the extracted action key frames under different testing video shots. Therefore, the improved AKF algorithm is a suitable technique for applications in computer vision fields such as passive object-based video authentication systems

The process of key frames extraction is facing human subjectivity. The objectives of key frames extraction are mainly two objectives. The first objective is minimizing the number of extracted key frames. The second objective is maximizing the visual representation of a video shot that the key frames should have. There is a tradeoff between the two objectives. Also the needs of individuals are varied. Some individuals favor the compression ratio and others favor the representation level. Many works depend on a subjective ground truth of the key frames for assessing the performance of key frames extraction methods. In this paper we propose a key frame extraction system that takes a confidence level as input from the user to satisfy the different needs. We evaluate our system using fair measures which are the compression ratio and the fidelity that are directly related to the confidence level.

Key Frame Extraction Using Unsupervised Clustering Based on a Statistical Model

With the improvement of people's security awareness, numerous monitoring equipment has been put into use, resulting in the explosive growth of surveillance video data. Key frame extraction technology is a paramount technology for improving video storage efficiency and enhancing the accuracy of video retrieval. It can extract key frame sets that can express video content from massive videos. However, the existing key frame extraction algorithms of surveillance video still have deficiencies, such as the destruction of image information integrity and the inability to extract key frames accurately. To this end, this paper proposes a key frame extraction algorithm of surveillance video based on quaternion Fourier saliency detection. Firstly, the algorithm used colors, and intensity features to perform quaternion Fourier transform on surveillance video sequences. Next, the phase spectrum of the quaternion Fourier transformed image was obtained, and he image visual saliency map was obtained according to the quaternion Fourier phase spectrum. Then, the image visual saliency map of two adjacent frames is used to characterize the change of target motion state. Finally, the frames that can accurately express the motion state of the target are selected as key frames. The experimental results show that the method proposed in this paper can accurately capture the changes of the local motion state of the target while maintaining the integrity of the image information.

In this paper, we present a new solution for extracting key frames from motion capture data using an optimization algorithm to obtain compact and sparse key frame data that can represent the original dense human body motion capture animation. The use of the genetic algorithm helps determine the optimal solution with global exploration capability while the use of a probabilistic simplex method helps expedite the speed of convergence. By finding the chromosome that maximizes the fitness function, the algorithm provides the optimal number of key frames as well as the low reconstruction error with an ordinary interpolation technique. The reconstruction error is computed between the original motion and the reconstruction one by the weighted differences of joint positions and velocities. The resulting set of key frames is obtained by iterative application of the algorithm with initial populations generated randomly and intelligently. We also present experiments which demonstrate that the method can effectively extract key frames with a high compression ratio and reconstruct all other non key frames with high quality.