Towards genre-specific frameworks for video summarisation: A survey

An efficient video summarization for surveillance system using normalized k-means and quick sort method

In this study, the authors present a new system for sign language hand gesture recognition. Using video input, the system can recognise any spelled word or alphabetic sequence signed in American Sign Language. The three main steps in the recognition process include detection of the region of interest (the hands), detection of key frames and recognition of gestures from these key frames. The proposed segmentation algorithm distinguishes regions of interest from both uniform and non-uniform backgrounds with an efficiency of 95%. The proposed key frame detection algorithm achieves an efficiency of 96.50%. A rotation-invariant algorithm for feature extraction is additionally proposed, which provides an overall gesture recognition efficiency of 84.2%.

Key Frame Extraction Techniques: A Survey

Video summarization focuses on creating concise versions of lengthy videos by preserving essential content. This study reveals a multimodal machine learning strategy that integrates visual and auditory information by using pretrained Gated Recurrent Neural Network architectures referred to as GARNN, combining Gated Recurrent Units (GRUs) and AlexNet, to extract audio, image, and visual features. The keyframe detection is improved by removing redundant frames and applying motion-compensated feature reduction, followed by optional PCA-based dimensionality reduction. An adversarial encoder-based Long Short-Term Memory (AE-LSTM) classifier is employed for temporal modeling by achieving high accuracy in summarization. The results got evaluated by using sensitivity, F-scores, and positive predictive values, and the method attained an average F-score of 0.985. A gated AlexNet is introduced in a multimodal GARNN-AE-LSTM framework, where motion-compensated PCA-based reduction eliminates redundancy, GRUs record temporal progression, and gating fine-tunes spatial feature selection, all of which contribute to a more accurate and efficient video summarization system. The improved F1 score demonstrates the model's effectiveness in generating accuracy in video summaries by creating a meaningful video. This approach highlights the potential of multimodal feature extraction and advanced deep learning techniques for robust video analysis and compression.

An Improved Audio Based Key Frame Detection Model for Movie Summarization

In this paper, a new approach for key frame extraction for sports video summarization is presented. Our approach employs dominant value of shot volume and energy. For the dominant value of each shot audio energy corresponding video key frame is extracted. This key frame of each shot serves as accurate localization of the events on the field. This is applied to cricket, soccer and lawn tennis videos and the experimental results show our proposed method is promising. Abstract In this paper, a new video summarization approach for lawn tennis video is presented. The proposed method uses frame color histogram to classify video into play field color shots (PFCS) and non play field color shots (NPFCS). Play field color shots are the segments of interest and used to recognize the tournament class. A dominant colored frame from every PFCS is extracted as a salient frame. Our approach also employs dominant value of PFCS volume and energy. For the each PFCS dominant audio energy value is computed and corresponding key frame is extracted. On-screen text presented in play field color shots provides important semantic information so the last frame of the shot is extracted as key caption. Three key frames based on audio, visual and text information conveyed in every PFCS are extracted and summary of lawn tennis video is created. These key frames of every PFCS serve as accurate localization of the events on the lawn tennis field. Experiments are performed on lawn tennis videos to confirm the efficiency of the proposed method.

Surveillance video cameras capture large amounts of continuous video streams every day. To analyze or investigate any significant events, it is a laborious and boring job to identify these events from the huge video data if it is done manually. Existing approaches sometimes neglect key frames with significant visual contents and/or select some unimportant frames with low/no activity. To solve this problem, in this paper, a video summarization technique is proposed by combining three multimodal human visual sensitive features, such as foreground objects, motion information, and visual saliency. In a video stream, foreground objects are one of the most important pieces of a video as they contain more detailed information and play a major role in important events. Moreover, motion is another stimulus of a video that significantly attracts human visual attention. To obtain this, motion information is calculated in the spatial domain as well as the frequency domain. Spatial motion information can select object motion accurately; however, it is sensitive to illumination changes. On the other hand, frequency motion information is robust to illumination change, although it is easily affected by noise. Therefore, motion information in both the spatial and the frequency domains is employed. Furthermore, the visual attention cue is a sensitive feature to measure the indication of a user's attraction label for determining key frames. As these features individually cannot perform very well, they are combined to obtain better results. For this purpose, an adaptive linear weighted fusion scheme is proposed to combine the features to rank video frames for summarization. Experimental results reveal that the proposed method outperforms the state-of-the-art methods.

Sparse dictionary selection (SDS) has demonstrated to be an effective solution for keyframe based video summarization (VS), which generally assumes a linear relation among similar video frames. However, such a linear assumption is not always true for videos. In this paper, the nonlinearity among frames is taken into consideration and a nonlinear SDS model is formulated for VS, in which the nonlinearity is transformed to linearity by projecting a video to a high dimensional feature space induced by a kernel function. Moreover, a kernel simultaneous orthogonal matching pursuit (KSOMP) is proposed to solve the problem. In order to achieve an intuitive and flexible configuration of the VS process, an adaptive criterion is devised to produce video summaries with different lengths for different video content. Experimental results on benchmark video datasets demonstrate that the proposed algorithm outperforms several state-of-the-art VS algorithms.

Anaplastic lymphoma kinase (ALK) positive (+) lung cancers are predictive for response to crizotinib and alectinib. There are many cases of lung cancer in which surgery cannot be performed, and such cases require diagnosis by cytological specimen or biopsy. Estimating ALK (+) lung cancer from cytomorphology would allow molecular testing to proceed without the waste of a small amount of specimen. The purpose of this study was to assess whether qualitative and quantitative cytomorphological features are sufficient for distinguishing primary ALK (+) from ALK (-) lung cancer. We examined eight qualitative cytomorphological parameters and three quantitative nuclear morphometric parameters in 17 cases of primary ALK (+) lung cancer, diagnosed by fluorescence insitu hybridisation (FISH) using histological specimens, and in 41 cases of ALK (-) lung cancer. Quantitative nuclear morphometric parameters were analysed by a computer-assisted image analysis system. In ALK (+) lung cancer, three qualitative parameters (signet ring cells, nuclear grooves and single type nucleoli) and two quantitative parameters (large nuclear area and irregular nuclear shape) were observed in significantly higher proportions. However, in ALK (-) lung cancer, one qualitative parameter (unclear and multiple type nucleoli) was seen significantly more often. These results show that the cytomorphological features of signet ring cells, nuclear grooves and nucleoli shape can help to triage a small amount of cytological and biopsy specimens for appropriate molecular testing of primary ALK (+) lung cancer.

Short videos on the Internet are growing exponentially, and the number of videos uploaded every day is huge; people also involve a lot of video data in real life. People can retrieve and view all kinds of videos, but it also brings a lot of problems. On the one hand, the accumulation of a large number of videos makes people unable to find the videos they want quickly, and the repeated scenes in the videos will also waste people's time and energy; on the other hand, a large amount of video data also brings enormous pressure to storage. Aiming at the problems of inaccurate selection of key frames and how to select video frame features in existing video summarization models, this paper proposes a multi-feature-based video summarization generation model (DME-VSNet), which extracts multiple features of video frames. Including importance score, image memory strength and image entropy. Aiming at the problem of inaccurate video shot segmentation, this model proposes a video shot segmentation algorithm based on TransNet network, which divides the original video into several short shots through shot boundaries; the model inputs the above three features into the proposed The video frame score is obtained in the MLP architecture, and the key frame is selected by the score to generate a video summary. The effectiveness of the video shot segmentation method based on TransNet network and the overall model based on convolutional neural network is verified by comparative experiments. The experimental results show that the evaluation results of the video summaries generated by the three features are better.

In the current era, most of the digital information is in the form of multimedia with a giant share of videos. Videos do have audio and visual content where the visual content has number of frames put in a sequence. Most of the consecutive frames do have very little discriminative contents. In video summarization process, several frames containing similar information do need to get processed. This leads to redundant slow processing speed and complexity, time consumption. Video summarization using key frames can ease the speed up of video processing. In this paper, novel key frames extraction method is proposed with Kekere's Proportionate Error (KPE) codebook generation techniques of vector quantization with ten different codebook sizes and two color spaces (RGB and KLUV). Experimentation done with help of the test bed of videos has shown that higher codebook sizes of KPE have given better completeness in key frame extraction for video summarization. The LUV color space with Euclidean Distance with 512 codebook size gives best performance. In square chord Distance, Mean Square Error and Euclidean Distance LUV color space gives better completeness than RGB color space for proposed KPE based video Key frame Extraction

Video summarization is an economical way of representing video contents and is useful for effective and quick browsing of the relevant activity present in the video. Most current video summarization approaches do not provide the correct insight of the events occurring in the video, such as moving the objects in a nonchronological order and tampering with the background and size of the objects. In this paper, we present an approach to create a video summarization that is a precise representation of the video content. First, our approach finds out the salient activities that are taking place in the video using an optimization framework for static and dynamic scenes. Second, the frames with the salient activities are stitched using alpha matting to form a single frame. Third, the summarized frame over multiple video shots obtained by our approach gives superior retrieval performance with image queries while reducing retrieval latency and memory requirement. This paper proposes a single frame-based indexing of the video database instead of multi-frame indexing as it is an interesting and potentially promising approach. Finally, several experiments are carried out to evaluate the proposed approach. The experiments demonstrate that the video summarization produced by our approach gives a realistic view of the events in the video, while the reduction in computational complexity and memory requirement is high.

A Study on Multi-view Video Summarization Techniques

In recent years, as a result of climate change as well as rainfall reduction in arid and semi‐arid regions, modelling qualitative and quantitative parameters belonging to aquifers has become crucially important. In Iran, as aquifers are treated as the most commonly used drinking water resources, modelling their qualitative and quantitative parameters is enormously important. In this paper, for the first time, values of salinity, total dissolved solids (TDS), groundwater level (GWL) and electrical conductivity (EC) of the Arak Plain, located in Markazi Province, Iran, are simulated by means of four modern artificial intelligence models including extreme learning machine (ELM), wavelet extreme learning machine (WELM), online sequential extreme learning machine (OSELM) and wavelet online sequential extreme learning machine (WOSELM) as well as the MODFLOW software for a 15‐year period monthly. To develop the hybrid artificial intelligence models, the wavelet is employed. First, the effective lags in estimating the qualitative and quantitative parameters of the groundwater are identified using the autocorrelation function (ACF) and the partial autocorrelation function (PACF) analysis. After that, four different models are developed by the selected input combinations and also the ACF and the PACF in the form of different lags for each of ELM, WAELM, OSELM and WOSELM methods. Then, the superior models in simulating the groundwater qualitative and qualitative parameters are detected by conducting a sensitivity analysis. To forecast the electrical conductivity (EC) by the best WOSELM model, the values of the Nash–Sutcliffe efficiency coefficient (NSC), Mean Absolute Error (MAE) and the scatter index (SI) are obtained to be 0.991, 18.005 and 4.28E‐03, respectively. In addition, the most effective lags in estimating these parameters are introduced. Subsequently, the results found by the MODFLOW model are compared with those of the artificial intelligence models and it is concluded that the latter are more accurate. For instance, the scatter index and Nash–Sutcliffe efficiency coefficient values calculated by WOSELM for TDS, respectively, are 5.34E‐03 and 0.991. Finally, an uncertainty analysis is conducted to evaluate the performance of different numerical models. For example, MODFLOW has an underestimated performance in simulating the salinity parameter.

To evaluate the clinical value of contrast-enhanced ultrasound (CEUS) with time-intensity curve (TIC) in distinguishing different types of hypovascular solid pancreatic lesions. A total of 89 patients with 90 pancreatic lesions (all confirmed by surgery or biopsy pathology) that manifested hypoenhancement on contrast-enhanced ultrasound (CEUS) were included in this study. Six peak enhancement patterns were proposed for differentiating hypovascular pancreatic lesions. CEUS qualitative and TIC-based quantitative parameters were analyzed, and each lesion was scored based on the statistically significant qualitative parameters to evaluate the diagnostic ability of CEUS for hypovascular solid pancreatic lesions. Qualitative parameters such as peak enhancement pattern II/III/IV, penetrating vessels, centripetal enhancement, and early washout were reliable indicators of malignant lesions, and lesions scored based on these qualitative parameters, with a score ≥ 2, were highly suspected to be malignant lesions. Pattern I had an accuracy of 83.33% for predicting mass-forming pancreatitis (MFP), pattern V had an accuracy of 96.67% for predicting solid pseudopapillary tumors of the pancreas (SPTP), and pattern VI had an accuracy of 81.11% for predicting neuroendocrine tumors/carcinomas (NETs/NECs). For quantitative analysis, nodule/pancreatic parenchyma echo intensity reduction ratio was significantly greater in malignant lesions. CEUS qualitative and TIC-based quantitative parameters have clinical value in distinguishing malignant from benign hypovascular pancreatic lesions. • Contrast-enhanced ultrasound helps clinicians assess patients with pancreatic lesions. • Six peak enhancement patterns are proposed for differentiating pancreatic hypovascular lesions. • Qualitative parameters such as peak enhancement pattern II/III/IV, penetrating vessels, centripetal enhancement, early washout, and quantitative parameter nodule/pancreatic parenchyma echo intensity reduction ratio were important characteristics to discriminate malignant from hypovascular benign lesions.