Reusing a compound-based infrastructure for searching video stories

With the exponential growth of video data, video summarization techniques are urgently needed for reducing people’s efforts in the videos' content exploration by generating succinct but informative summaries from original lengthy videos. Though supervised video summarization approaches have demonstrated the state-of-the-art performance, unsupervised methods are still highly demanded due to resourcefully expensive human annotations and the subjectiveness of video summarization tasks. In this paper, a novel unsupervised-based Deep Self-attention Recurrent summarization network with Reinforcement Learning (DSR-RL) for video summarization is proposed. The model can learn the input video sequence and suggest the key-shot summary without additional human annotations by integrating self-attention, BRNN, and reinforcement learning mechanisms. The DSR-RL improves not only importance score through the attention map vector of self-attention network but also the diversity of summaries via the reward function of reinforcement learning. Our method outperforms the state-of-the-art unsupervised video summarization methods on both SumMe and TVSum datasets. The source code is available at https://github.com/phaphuang/DSR-RL.

Content Based Video Retrieval (CBVR) is so popular these days, because of the increased utilization of video based analytical systems. Video based analytics is quite effective than image analysis, as a series of actions are captured by the video. This ends up with better decision making ability. The CBVR systems play an important role in boosting the human–computer interaction. This paper presents a multimodal CBVR that takes both the visual and audio information into account for retrieving relevant videos to the user. Two modules are employed by this work to deal with video and audio data. The video data is processed to detect the significant frame from shots and is achieved by Lion Optimization Algorithm (LOA). The features are extracted from the visual data and with respect to the audio data, MHEC and LPCC features are extracted. The extracted features are clustered by Kernelized Fuzzy C Mean (KFCM) algorithm. Finally, the feature database is formed and is utilized in the query matching process during the testing phase. The performance of the proposed work is tested in terms of precision, recall, F-measure and time consumption rates. The proposed CBVR system proves better performance than the existing approaches and is evident through attained results.

An efficient video summarization for surveillance system using normalized k-means and quick sort method

Evolutions in image-data processing is rapidly changing from text-based retrieval systems to video retrieval which is created on content based systems due to its eminent demand in digital market and that can be possible only due to available multimedia data types and available bandwidth. In proposed method of Content Based Video Retrieval (CBVR) the Ordered-Dither Block Truncation Coding (ODBTC) technique is employed which generates appropriate image contents. Combinations of Void-and-cluster half-toning and Block Truncation Coding (BTC) offers low complexity in algorithm and provides better video image quality. Dither array Look-Up-Table (LUT) is a distinctive feature of ODBTC which reduces the difficulties by providing look up values of segmented blocks. ODBTC encoded streams are used for generation of two distinct features including of color features namely Color Co-occurrence Feature (CCF) and Bit Pattern Features (BPF). After quantizing and bit-mapping from ODBTC encoder, BPF is obtained by LUT. In the presented system, CBVR is achieved by Block Truncation (BT) of expected video information to be retrieved. Proposed system provides good remedy for CBVR for large digital video-data processing in the fields of Image and Video Processing.

As technology progresses, a gigantic amount of video data is generated day-by-day. Processing of such a huge video requires time, increased storage, and a computational power. Sometimes it is convenient for the user to watch a summary or highlight rather than watching a complete video, which is a time-consuming task. So, a fully automated solution is required to extract important segments from video. Researchers have proposed multiple approaches / techniques for summarizing the videos which resolve the problem of long videos and summarize them according to the video type. This survey and comparative evaluation of video summarizing techniques based on several domains are presented in this study. Primarily, these methods are classified into different categories based on their methods or techniques used. Furthermore, an overview of some of the latest literature is presented with the dataset and the evaluation approaches used. The review is also made related to the domain direction, and is concluded by presenting the benefits and difficulties associated with the current video summarization techniques.

Surveillance video cameras capture large amounts of continuous video streams every day. To analyze or investigate any significant events, it is a laborious and boring job to identify these events from the huge video data if it is done manually. Existing approaches sometimes neglect key frames with significant visual contents and/or select some unimportant frames with low/no activity. To solve this problem, in this paper, a video summarization technique is proposed by combining three multimodal human visual sensitive features, such as foreground objects, motion information, and visual saliency. In a video stream, foreground objects are one of the most important pieces of a video as they contain more detailed information and play a major role in important events. Moreover, motion is another stimulus of a video that significantly attracts human visual attention. To obtain this, motion information is calculated in the spatial domain as well as the frequency domain. Spatial motion information can select object motion accurately; however, it is sensitive to illumination changes. On the other hand, frequency motion information is robust to illumination change, although it is easily affected by noise. Therefore, motion information in both the spatial and the frequency domains is employed. Furthermore, the visual attention cue is a sensitive feature to measure the indication of a user's attraction label for determining key frames. As these features individually cannot perform very well, they are combined to obtain better results. For this purpose, an adaptive linear weighted fusion scheme is proposed to combine the features to rank video frames for summarization. Experimental results reveal that the proposed method outperforms the state-of-the-art methods.

Content-based video retrieval is a proper solution to handle the video data. But because of their huge volumes and high dimensionality, finding a proper way to organize them for efficient search and retrieval becomes a challenging and important problem. In this paper, we present a framework for content-based video retrieval using the Shot Cluster Tree, while the latter organizes the content of shots in the tree structure. The framework can supply users with two types of queries: query-by-content and query-by-example. We also put up a novel and efficient clustering method for generating the Shot Cluster Tree structure, where shots which are visual similar and time adjacent are grouped into shot groups using the method of Sliding Shot Window firstly and then shot groups are clustered into shot clusters with the simple agglomerative hierarchical clustering method. In addition to constructing the structure of content, the Shot Cluster Tree also provides better ways to video summary and video annotation, which facilitate the video retrieval. An experimental system has been built up. Experiments verify the effectiveness of the proposed approach.

Human action recognition is an important area of research in the field of computer vision due to its extensive applications like security surveillance; content based video retrieval and annotation, human computer interaction, human fall detection, video summarization, robotics, etc.The surveillance system deals with the monitoring and analysing the human behaviour and activities.The main aim of the smart surveillance system is to recognize anomalous behaviour in given scene and provide real time intimation to relevant person.We have designed and tested Smart Surveillance System for College Corridor Scene (3S2CS).The system recognises the anomalous behaviour and an intimation is provided in the form of Firebase Cloud Messaging (FCM) alert on the android mobile phone to the authorised user.This paper mainly discuss the methodologies used for the human action recognition.The basic step is to provide video as an input.These videos are further divided into number of frames.The videos are used for training and for each video, Scale Invariant Feature Transform (SIFT) is applied for extracting features and developing feature vectors.The actions are classified using K Nearest Neighbour (KNN) and Support Vector Machine (SVM) classifier.Two standard offline datasets considered for testing are Weizmann and UTD-MHAD.For real time scenario we have created dataset in college campus called as College Corridor dataset.It contains student activities like falling, fighting, walking, running, sitting and other general actions.If falling or fighting action is detected, the notification is sent to the authorized user who has installed ActionDetector android application and registered a device for the same.Action recognition accuracy is 92.91% using SVM and 90.83% using KNN.

The video retrieval system refers to the task of retrieving the most relevant video collection, given a user query. By applying some feature extraction models the contents of the video can be extracted. With the exponential increase in video data in online and offline databases as well as a huge implementation of multiple applications in health, military, social media, and art, the Content-Based Video Retrieval (CBVR) system has emerged. The CBVR system takes the inner contents of the video frame and analyses features of each frame, through which similar videos are retrieved from the database. However, searching and retrieving the same clips from huge video collection is a hard job because of the presence of complex properties of visual data. Video clips have many frames and every frame has multiple properties that have many visual properties like color, shape, and texture. In this research, an efficient content-based video retrieval system using the AlexNet model of Convolutional Neural Network (CNN) on the keyframes system has been proposed. Firstly, select the keyframes from the video. Secondly, the color histogram is then calculated. Then the features of the color histogram are compared and analyzed for CBVR. The proposed system is based on the AlexNet model of CNN and color histogram, and extracted features from the frames are together to store in the feature vector. From MATLAB simulation results, the proposed method has been evaluated on benchmark dataset UCF101 which has 13320 videos from 101 action categories. The experiments of our system give a better performance as compared to the other state-of-the-art techniques. In contrast to the existing work, the proposed video retrieval system has shown a dramatic and outstanding performance by using accuracy and loss as performance evaluation parameters.

A review on video summarization techniques

A privacy-preserving content-based image retrieval method based on deep learning in cloud computing

With the development of multimedia data types and available bandwidth there is huge demand of video retrieval systems as users shift from text based retrieval systems to content based retrieval systems (CBVRS). Selection of extracted features play an important role in content based video retrieval, regardless of video attributes under consideration. Good features selection also allows the time, space and cost saving in the retrieval process. This leads to the interest of selecting only those features that can be extracted from video data and used for indexing and retrieval along with similarity measurement methods. There are certain issues like video store, search, index and retrieve as a trend in area of content based video retrieval systems. In this paper, we have proposed a method. In this paper fractional discrete cosine transform coefficients are used to extract key frames which eventually save space and time with better completeness in the retrieval process.

Key Frame Extraction Techniques: A Survey

Due to the enormous increase in image database sizes, the need for an image search and indexing tool is crucial. Content-based image retrieval systems (CBIR) have become very popular for browsing, searching and retrieving images in different fields including web based searching, industry inspection, satellite images, medical diagnosis images, etc. The challenge, however, is in designing a system that returns a set of relevant images i.e. if the query image represents a horse then the first images returned from a large image dataset must return horse images as first responses. In this paper, we have combined YACBIR [7], a CBIR that relies on color, texture and points of interest and Multiple Support Vector Machines Ensemble to reduce the existing gap between high-level semantic and low-level descriptors and enhance the performance of retrieval by minimize the empirical classification error and maximize the geometric margin classifiers. The experimental results show that the method proposed reaches high recall and precision.

With the advance of multimediatechnology and communications, image and video data over the Internet contribute to the sea of media. Methods on how to search and findout user desired media contents becomesimportant. In this paper, we proposed an efficient content-based video retrieval (CBVR) method that integrated color, texture and SIFT-BOW (Bag of Word) image features for robust and higher retrieval performance. Experimental results showed that our proposed CBVR method can well manipulate both global and local features for retrieval and can outperforms the previous methods by 12.5% and 19.7 %in precision performance that adopts single and multi-feature types.