A Video Segmentation and Key Frame Extraction Method for Compressed Video Stream

Based on compressed video segment detection, the decompression step can be omitted, features can be directly extracted from the original video data stream, and the detection rate can be accelerated. This paper analyzes the characteristics of video data, video segmentation and key frame selection. Based on the analysis of the key technologies of video retrieval technology, the compressed video stream extraction method based on video segmentation and key frame is introduced. A method of extracting key frames from MPEG compressed video based on DC coefficients and motion vectors is proposed. Experiments show that the proposed method can reduce the computational burden, and can better represent the video content.

Keyframe extraction is a very important part of the video retrieval steps. However, current keyframe extraction techniques have problems such as underrepresentation, large redundancy, poor adaptability and large computation. So, we propose an accelerated alternative to the basic task with comparable performance, and by rephrasing the task as fragment generation and prompting. We use the key frame extraction method combined with the SAM-based feature extraction model of lightweight-SAM, without additional training, to obtain an interactive video key frame extraction method for focused preference objects. In this model, we used the Yolov8-seg model to replace the VIT part in Segment-Anything Model (SAM), while adding the tracking module XMem in the keyframe extraction step, which can effectively reduce the computational amount while also ensuring the accuracy of the keyframes. Using our approach, we achieve comparable performance to the SAM approach with a 50x higher runtime speed. Recall at 50% precision (R50) is high and has significantly fewer parameters (only 70M). The ratio of key frame extraction can reach 90.3% under the premise of an average fidelity of 69.4%. We compare the performance of these methods on UCF-101, COCO, BSDS500 and other datasets, and this interactive design performs impressively in video object tracking and segmentation, and performs well in keyframe extraction for selected objects.

We present a method for motion-based video segmentation and segment classification as a step towards video summarization. The sequential segmentation of the video is performed by detecting changes in the dominant image motion, assumed to be related to camera motion and represented by a 2D affine model. The detection is achieved by analysing the temporal variations of some coefficients of the 2D affine model (robustly) estimated. The obtained video segments supply reasonable temporal units to be further classified. For the second stage, we adopt a statistical representation of the residual motion content of the video scene, relying on the distribution of temporal co-occurrences of local motion-related measurements. Pre-identified classes of dynamic events are learned off-line from a training set of video samples of the genre of interest. Each video segment is then classified according to a Maximum Likelihood criterion. Finally, excerpts of the relevant classes can be selected for video summarization. Experiments regarding the two steps of the method are presented on different video genres leading to very encouraging results while only low-level motion information is considered.

The purpose of video key frame extraction is to use as few video frames as possible to represent as much video content as possible, reduce redundant video frames, and reduce the amount of computation, so as to facilitate quick browsing, content summarization, indexing, and retrieval of videos. In this paper, a method of dance motion recognition and video key frame extraction based on multifeature fusion is designed to learn the complicated and changeable dancer motion recognition. Firstly, multiple features are fused, and then the similarity is measured. Then, the video sequences are clustered by the clustering algorithm according to the scene. Finally, the key frames are extracted according to the minimum amount of motion. Through the quantitative analysis and research of the simulation results of different models, it can be seen that the model proposed in this paper can show high performance and stability. The breakthrough of video clip retrieval technology is bound to effectively promote the inheritance and development of dance, which is of great theoretical significance and practical value.

In the current era, most of the digital information is in the form of multimedia with a giant share of videos. Videos do have audio and visual content where the visual content has number of frames put in a sequence. Most of the consecutive frames do have very little discriminative contents. In video summarization process, several frames containing similar information do need to get processed. This leads to redundant slow processing speed and complexity, time consumption. Video summarization using key frames can ease the speed up of video processing. In this paper, novel key frames extraction method is proposed with Kekere's Proportionate Error (KPE) codebook generation techniques of vector quantization with ten different codebook sizes and two color spaces (RGB and KLUV). Experimentation done with help of the test bed of videos has shown that higher codebook sizes of KPE have given better completeness in key frame extraction for video summarization. The LUV color space with Euclidean Distance with 512 codebook size gives best performance. In square chord Distance, Mean Square Error and Euclidean Distance LUV color space gives better completeness than RGB color space for proposed KPE based video Key frame Extraction

Recognizing Human action in video is an very active research topic. There are a growing variety of human action datasets with different video length, different practitioners. Make human action recognition becomes a very difficult topic. A majority researchers solve the problem by extracting key frames from the videos. Most paper use feature Clustering methods to extract key frames in videos. On one hand, the large variety of visual content in videos make handcraft feature isn't effective enough, since there are no fixed descriptors can describe all video cases. On the other hand, traditional clustering algorithms are easily influenced by the choice of initial clustering centers. An Unsupervised feature learning and clustering method for key frame extraction is proposed in this paper, which can be used for human action recognition. Stacked auto-encoder(SAE) is trained using videos from 10 different human actions, SAE is used as a feature extractor to learn features representing human actions. Affinity Propagation Clustering algorithm is used to select key frames from video sequences. We use a variety of videos to do the experiments. Experiments demonstrate that our method can be effectively summarizing video shots considering different human actions.

The key frame extraction algorithm based on the indigenous disturbance variation difference video

The video segmentation is the basis of video analysis and digging, the results directly affect the accuracy of follow-up processing. For the video segmentation, the predecessors have done a IoT of research. We have achieved very good results in regard to the mutant lens, but there is no good way for the gradient. Because football is a popular sports, it has a very practical significance for the process. But since the football video screen changes rapidly, which increases the difficulty of segmentation. This paper first introduces five commonly used methods of shot segmentation, and then introduces three methods for soccer video segmentation, and finally gives a combination of methods used in soccer video segmentation. Hope that this article allows you to more quickly understand the methods of the shot segmentation about soccer videoes.

Key frame extraction is an important component of video analysis, and has gradually become a research hotspot in the computer vision community in recent years. Early key frame extraction algorithms were mostly based on fixed time intervals, and their accuracy could not meet practical application requirements. Thanks to the rapid development of machine learning technology, key frame extraction algorithms based on image quality, motion analysis, and deep learning are gradually becoming mature. Although the above methods significantly improve the accuracy of key frame extraction, few work pay attention to the extraction effect in different scenarios. In this article, selecting the key frame extraction algorithm based on optical flow method, we compared the key frame extraction algorithm based on optical flow method in detail for different video scenarios. Extensive experimental results demonstrate the robuteness of key frame extraction algorithm based on optical flow.

Triple attention network for video segmentation

There has been remarkable progress in the field of Semantic segmentation in recent years. Yet, it remains a challenging problem to apply segmentation to the video-based applications. Videos usually involve significantly larger volume of data compared to images. Particularly, a video contains around 30 frames per second. Segmentation of the similar frames unnecessarily adds to the time required for segmentation of complete video. In this paper, we propose a contour detection-based approach for detection of salient frames for faster semantic segmentation of videos. We propose to detect the salient frames of the video and pass only the salient frames through the segmentation block. Then, the segmented labels of the salient frames are mapped to the non-salient frames. The salient frame is defined by the variation in the pixel values of the background subtracted frames. The background subtraction is done using MOG2 background subtractor algorithm for background subtraction in various lighting conditions. We demonstrate the results using the Pytorch model for semantic segmentation of images. We propose to concatenate the semantic segmentation model to our proposed framework. We evaluate our result by comparing the time taken and the mean Intersection over Union (mIoU) for segmentation of the video with and without passing the video input through our proposed framework. We evaluate the results of Saliency Detection Block using Retention and Condensation ratio as the quality metrics.

The amount of user created contents has been increasing rapidly and is associated with a serious copyright problem. Automatic logo detection and recognition in videos is a natural and efficient way of overcoming the copyright problem. However, logos have varying characteristics, which make logo detection and recognition very difficult. Moreover, logo transitions between two different logos exist in one video comprising several video contents. This disrupts the automatic logo detection and recognition. Therefore, in order to improve logo detection, it is necessary to take into account the logo transitions explicitly. This paper proposes an accurate logo transition detection method for recognizing logos in digital video contents. The proposed method accurately segments a video according to logo and efficiently recognizes various types of logos. The experimental results demonstrate the effectiveness of the proposed method for logo detection and video segmentation according to logo.

In this paper, we propose a general Eulerian framework for region-based active contours named DREAM2S. We introduce a general criterion including both region-based and boundary-based terms where the information on a region is named “descriptor”. The originality of this work is twofold. Firstly we propose to use shape optimization principles to compute the evolution equation of the active contour that will make it evolve as fast as possible towards a minimum of the criterion. Secondly, we take into account the variation of the descriptors during the propagation of the curve. Indeed, a descriptor is generally globally attached to the region and thus “region-dependent”. This case arises for example if the mean or the variance of a region are chosen as descriptors. We show that the dependence of the descriptors with the region induces additional terms in the evolution equation of the active contour that have never been previously computed. DREAM2S gives an easy way to take such a dependence into account and to compute the resulting additional terms. Experimental results point out the importance of the additional terms to reach a true minimum of the criterion and so to obtain accurate results. The covariance matrix determinant appears to be a very relevant tool for homogeneous color regions segmentation. As an example, it has been successfully applied to face detection in real video sequences.KeywordsEvolution EquationVideo SequenceAdditional TermActive ContourHomogeneous RegionThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This paper presents a new method for unsupervised video segmentation based on mean shift clustering in spatio-temporal domain. The main novelties of the proposed approach are dynamic temporal adaptation of clusters due to which the segmentation evolves quickly and smoothly over time. The proposed method consists of a short initialization phase and an update phase. The proposed method significantly reduce the computation load for the mean shift clustering. In the update phase only the positions of relatively small number of cluster centers are updated and new frames are segmented based on the segmentation of previous frames. The method segments video in real-time and tracks video objects effectively.

For the traditional method to extract the surveillance video key frame, there are problems of redundant information, substandard representative content and other issues. A key frame extraction method based on motion target detection and image similarity is proposed in this paper. This method first uses the ViBe algorithm fusing the inter-frame difference method to divide the original video into several segments containing the moving object. Then, the global similarity of the video frame is obtained by using the peak signal to noise ratio, the local similarity is obtained through the SURF feature point, and the comprehensive similarity of the video image is obtained by weighted fusion of them. Finally, the key frames are extracted from the critical video sequence by adaptive selection threshold. The experimental results show that the method can effectively extract the video key frame, reduce the redundant information of the video data, and express the main content of the video concisely. Moreover, the complexity of the algorithm is not high, so it is suitable for the key frame extraction of the surveillance video.