Sign Language Video Classification Based on Image Recognition of Specified Key Frames

Recognizing Human action in video is an very active research topic. There are a growing variety of human action datasets with different video length, different practitioners. Make human action recognition becomes a very difficult topic. A majority researchers solve the problem by extracting key frames from the videos. Most paper use feature Clustering methods to extract key frames in videos. On one hand, the large variety of visual content in videos make handcraft feature isn't effective enough, since there are no fixed descriptors can describe all video cases. On the other hand, traditional clustering algorithms are easily influenced by the choice of initial clustering centers. An Unsupervised feature learning and clustering method for key frame extraction is proposed in this paper, which can be used for human action recognition. Stacked auto-encoder(SAE) is trained using videos from 10 different human actions, SAE is used as a feature extractor to learn features representing human actions. Affinity Propagation Clustering algorithm is used to select key frames from video sequences. We use a variety of videos to do the experiments. Experiments demonstrate that our method can be effectively summarizing video shots considering different human actions.

For the traditional method to extract the surveillance video key frame, there are problems of redundant information, substandard representative content and other issues. A key frame extraction method based on motion target detection and image similarity is proposed in this paper. This method first uses the ViBe algorithm fusing the inter-frame difference method to divide the original video into several segments containing the moving object. Then, the global similarity of the video frame is obtained by using the peak signal to noise ratio, the local similarity is obtained through the SURF feature point, and the comprehensive similarity of the video image is obtained by weighted fusion of them. Finally, the key frames are extracted from the critical video sequence by adaptive selection threshold. The experimental results show that the method can effectively extract the video key frame, reduce the redundant information of the video data, and express the main content of the video concisely. Moreover, the complexity of the algorithm is not high, so it is suitable for the key frame extraction of the surveillance video.

Aiming at the problem of video key frame extraction, a density peak clustering algorithm is proposed, which uses the HSV histogram to transform high-dimensional abstract video image data into quantifiable low-dimensional data, and reduces the computational complexity while capturing image features. On this basis, the density peak clustering algorithm is used to cluster these low-dimensional data and find the cluster centers. Combining the clustering results, the final key frames are obtained. A large number of key frame extraction experiments for different types of videos show that the algorithm can extract different number of key frames by combining video content, overcome the shortcoming of traditional key frame extraction algorithm which can only extract a fixed number of key frames, and the extracted key frames can represent the main content of video accurately.

Wireless capsule endoscopy (WCE) is a non-invasive imaging technique to examine the entire small intestine. This paper presents a novel hierarchical key frame extraction algorithm based on the saliency map to automatically select a small number of key informative frames in the WCE video to assist the diagnostic process. We start from calculating the saliency map and extracting the inside colour, texture and shape features. We then compute the image information entropies with adapted weights on three features; calculate the differences of entropies in the successive frames and extract the first level key frames to divide the video into multi-segments. Finally, the mean shift clustering method is applied on each segment to obtain the final key frames. The proposed technique achieves the fidelity of 95.07% and the compression ratio of 87.9% on average, validating that the proposed scheme is highly promising for the key frame extraction in the WCE video.

This paper presents U-LanD, a framework for automatic detection of landmarks on key frames of the video by leveraging the uncertainty of landmark prediction. We tackle a specifically challenging problem, where training labels are noisy and highly sparse. U-LanD builds upon a pivotal observation: a deep Bayesian landmark detector solely trained on key video frames, has significantly lower predictive uncertainty on those frames vs. other frames in videos. We use this observation as an unsupervised signal to automatically recognize key frames on which we detect landmarks. As a test-bed for our framework, we use ultrasound imaging videos of the heart, where sparse and noisy clinical labels are only available for a single frame in each video. Using data from 4,493 patients, we demonstrate that U-LanD can exceedingly outperform the state-of-the-art non-Bayesian counterpart by a noticeable absolute margin of 42% in R2 score, with almost no overhead imposed on the model size.

As the fast development of network and multimedia technology, there are mass of videos on the Internet. The problem of how to browse the large volume of video data is urgent to be solved. Key frame extraction is a very important technology in online video retrieval and video abstraction. Efficient key frame extraction can boost the development of video browsing technology. In this paper, we first review some commonly-used key frame extraction technology, and then propose a improved key frame extraction algorithm, which measures the similarity of two adjacent frames' contents in terms of the information of frame difference and extracts key frames after optimizing the frame difference. The experiment results show that the proposed algorithm can extract key information from online video efficiently, and has a good description in the aspect of motion of the camera lens.

Nowadays, a strong need exists for the efficient organization of an increasing amount of home video content. To create an efficient system for the management of home video content, it is required to categorize home video content in a semantic way. So far, a significant amount of research has already been dedicated to semantic video categorization. However, conventional categorization approaches often rely on unnecessary concepts and complicated algorithms that are not suited in the context of home video categorization. To overcome the aforementioned problem, this paper proposes a novel home video categorization method that adopts semantic home photo categorization. To use home photo categorization in the context of home video, we segment video content into shots and extract key frames that represent each shot. To extract the semantics from key frames, we divide each key frame into ten local regions and extract lowlevel features. Based on the low level features extracted for each local region, we can predict the semantics of a particular key frame. To verify the usefulness of the proposed home video categorization method, experiments were performed with home video sequences, labeled by concepts part of the MPEG-7 VCE2 dataset. To verify the usefulness of the proposed home video categorization method, experiments were performed with 70 home video sequences. For the home video sequences used, the proposed system produced a recall of 77% and an accuracy of 78%.

The purpose of video key frame extraction is to use as few video frames as possible to represent as much video content as possible, reduce redundant video frames, and reduce the amount of computation, so as to facilitate quick browsing, content summarization, indexing, and retrieval of videos. In this paper, a method of dance motion recognition and video key frame extraction based on multifeature fusion is designed to learn the complicated and changeable dancer motion recognition. Firstly, multiple features are fused, and then the similarity is measured. Then, the video sequences are clustered by the clustering algorithm according to the scene. Finally, the key frames are extracted according to the minimum amount of motion. Through the quantitative analysis and research of the simulation results of different models, it can be seen that the model proposed in this paper can show high performance and stability. The breakthrough of video clip retrieval technology is bound to effectively promote the inheritance and development of dance, which is of great theoretical significance and practical value.

Video sharing sites provide an opportunity for the collection and use of sign language presentations about a wide range of topics. Currently, locating sign language videos (SL videos) in such sharing sites relies on the existence and accuracy of tags, titles or other metadata indicating the content is in sign language. In this paper, we describe the design and evaluation of a classifier for distinguishing between sign language videos and other videos. A test collection of SL videos and videos likely to be incorrectly recognized as SL videos (likely false positives) was created for evaluating alternative classifiers. Five video features thought to be potentially valuable for this task were developed based on common video analysis techniques. A comparison of the relative value of the five video features shows that a measure of the symmetry of movement relative to the face is the best feature for distinguishing sign language videos. Overall, an SVM classifier provided with all five features achieves 82% precision and 90% recall when tested on the challenging test collection. The performance would be considerably higher when applied to the more varied collections of large video sharing sites.

Key Frame Extraction Techniques: A Survey

In this paper, a new approach for key frame extraction for sports video summarization is presented. Our approach employs dominant value of shot volume and energy. For the dominant value of each shot audio energy corresponding video key frame is extracted. This key frame of each shot serves as accurate localization of the events on the field. This is applied to cricket, soccer and lawn tennis videos and the experimental results show our proposed method is promising. Abstract In this paper, a new video summarization approach for lawn tennis video is presented. The proposed method uses frame color histogram to classify video into play field color shots (PFCS) and non play field color shots (NPFCS). Play field color shots are the segments of interest and used to recognize the tournament class. A dominant colored frame from every PFCS is extracted as a salient frame. Our approach also employs dominant value of PFCS volume and energy. For the each PFCS dominant audio energy value is computed and corresponding key frame is extracted. On-screen text presented in play field color shots provides important semantic information so the last frame of the shot is extracted as key caption. Three key frames based on audio, visual and text information conveyed in every PFCS are extracted and summary of lawn tennis video is created. These key frames of every PFCS serve as accurate localization of the events on the lawn tennis field. Experiments are performed on lawn tennis videos to confirm the efficiency of the proposed method.

For the existing traditional method to extract key frames from the video sequence with gradual change of content, there are problems of having too much redundancy or too much loss in the extraction results and other issues, a new key frames extracting method for adaptively extracting the video key frames has proposed in the paper. The experimental results have shown that the proposed method can effectively extract video key frames, especially for the video with gradual change of content, which can express more succinctly and comprehensively. The experiments on a large number of videos containing various scenes have shown that the key frames extracted by the proposed algorithm can more fully express the main content of video. When dealing with content gradient video, the effect is better. The content adaptively determines the number of key frames based on video.

In this paper, we demonstrate a system for the key frame summarization, extraction, and browsing of the echocardiogram videos. Echocardiograms are video sequences produced by the ultrasound scanning of the heart. Echocardiography is the most common type of medical imaging modality in the field of cardiology. Our approach combines the domain-specific knowledge and the automatic analysis of the spatio-temporal structure of the echocardiogram videos. The videos are temporally sampled using the embedded electrocardiogram graph (ECG). The consecutive sampled frames are compared based on the shape of the region of interest to detect the boundaries between the different segments. The content of each segment of the video is summarized into two forms: the static and the dynamic summaries. Finally the summary is displayed in the user interface in an intuitive form for the purpose of browsing.

With rapid increasing research in emotion detection, facial expression recognition becomes more popular as a vital emotion measurement instrument. Happy is one of the most common facial expressions that indicates a positive human emotional status. Although many facial expression recognition methods have been well established for recognizing multiple emotions, studies on happy detection are very limited, especially for processing the videos. In this paper, we propose a new single emotion recognition method to recognize happy emotion from key frames of facial expression videos. In our method, we extract key frames that demonstrate the highest intensity level of expressions among all frames in two steps. In the first step, an Inception ResNet framework is utilized to segment an facial expression process in a video into three parts: onset, apex and offset. In the second phase, we choose first three frames in the apex segment as key frames. A fine-tuned convolutional neural network (CNN) then classifies the key frames into happy and non-happy classes. Our experimental results demonstrate that the proposed approach achieves higher accuracy than four counterpart methods in recognizing happy emotions, with the accuracy of 98.4% and 94.2% on two benchmark facial expression datasets, i.e., the extended Cohn-Kanade (CK+) and MMI.

The process of key frames extraction is facing human subjectivity. The objectives of key frames extraction are mainly two objectives. The first objective is minimizing the number of extracted key frames. The second objective is maximizing the visual representation of a video shot that the key frames should have. There is a tradeoff between the two objectives. Also the needs of individuals are varied. Some individuals favor the compression ratio and others favor the representation level. Many works depend on a subjective ground truth of the key frames for assessing the performance of key frames extraction methods. In this paper we propose a key frame extraction system that takes a confidence level as input from the user to satisfy the different needs. We evaluate our system using fair measures which are the compression ratio and the fidelity that are directly related to the confidence level.