Video retrieval for shot cluster and classification based on key feature set

Video Shot Boundary Detection (SBD) is the fundamental process towards video summarization and retrieval. A fast and efficient SBD algorithm is necessary for real-time video processing applications. Extensive work has focused on accurate shot boundary detection at the expense of demanding computational costs. In this paper, we propose a fast SBD approach that reduces the computation pixel-wise and frame-wise while still giving satisfactory accuracy. The proposed approach substantially speeds up the computation through reducing both detection region and scope. Color histogram and mutual information are used together to measure the difference between frames. Corner distribution of frames is utilized to exclude most of false boundaries. We conduct extensive experiments to evaluate the proposed approach, and the results show that our approach can not only speed up SBD, but also detect shot boundaries with high accuracy in both Cut (CUT) and Gradual Transition (GT) boundaries.

As digital sports video data become more and more pervasive, automatic highlight detection is particularly demanding. We present an automatic and effective framework for goal event detection based on the cinematic features. The proposed framework includes 4 parts: shot boundary detection, shot classification, slow-motion detection, and goal event detection. The shot boundary detector employs the multi-filter structure. Then, the goal detector uses the slow-motion replay to locate the goal shot and the new cinematic template to model and identify the goal event. By the experiments over diverse soccer game videos, the proposed framework is effective and efficient to find the goal shot.

This paper proposes a method for content-based sports video retrieval using camera work information. Since particular camera work for a typical scene exists in sports videos, camera work transition becomes an effective cue for retrieving a sports scene based on its content. The proposed method extracts a series of camera parameters from both a user-specified scene of a retrieval key and a video stream of a retrieval target, and detects scenes having content similar to that of the key from the target by applying continuous DP matching. The method was evaluated using a video stream of a baseball game. Recall-Precision curves make its effectiveness clear.

Video is a rich information source containing both audio and visual information along with motion information embedded in it. Applications such as e-learning, live TV, video on demand, traffic monitoring, etc. need an efficient video retrieval strategy. Content-based video retrieval and superpixel segmentation are two diverse application areas of computer vision. In this work, we are presenting an algorithm for content-based video retrieval with help of Integration of Curvelet transform and Simple Linear Iterative Clustering (ICTSLIC) algorithm. Proposed algorithm consists of two steps: off line processing and online processing. In offline processing, keyframes of the database videos are extracted by employing features: Pearson Correlation Coefficient (PCC) and color moments (CM) and on the extracted keyframes superpixel generation algorithm ICTSLIC is applied. The superpixels generated by applying ICTSLIC on keyframes are used to represent database videos. On other side, in online processing, ICTSLIC superpixel segmentation is applied on query frame and the superpixels generated by segmentation are used to represent query frame. Then videos similar to query frame are retrieved through matching done by calculation of Euclidean distance between superpixels of query frame and database keyframes. Results of the proposed method are irrespective of query frame features such as camera motion, object’s pose, orientation and motion due to the incorporation of ICTSLIC superpixels as base feature for matching and retrieval purpose. The proposed method is tested on the dataset comprising of different categories of video clips such as animations, serials, personal interviews, news, movies and songs which is publicly available. For evaluation, the proposed method randomly picks frames from database videos, instead of selecting keyframes as query frames. Experiments were conducted on the developed dataset and the performance is assessed with different parameters Precision, Recall, Jaccard Index, Accuracy and Specificity. The experimental results shown that the proposed method is performing better than the other state-of-art methods.

In this paper, we propose a novel motion-based video retrieval approach to find desired videos from video databases through trajectory matching. The main component of our approach is to extract representative motion features from the video, which could be broken down to the following three steps. First, we extract the motion vectors from each frame of videos and utilize Harris corner points to compensate the effect of the camera motion. Second, we find interesting motion flows from frames using sliding window mechanism and a clustering algorithm. Third, we merge the generated motion flows and select representative ones to capture the motion features of videos. Furthermore, we design a symbolic based trajectory matching method for effective video retrieval. The experimental results show that our algorithm is capable to effectively extract motion flows with high accuracy and outperforms existing approaches for video retrieval.

In this paper, we provided adaptive key feature sets to describe the content of every shot in video. Return the most similar results to build a multi-video retrieval function. By using K-mean cluster and the gradient histogram of local binary pattern, we respectively transferred video into color and texture features. According to calculated the features distance between continuous frames to detect the shot boundaries of the video. Thus, different shot can be detected. After videos were cut into representative shots, filter and cluster the shots by video analysis function, so that each shot can be more representative. Finally, we could find out the similar results by calculating the distance between query video and video database. In the experimental results, we detected the correct shot boundary. In video retrieval, it can be divided into two ways - the image query videos and the video query videos. The query results showed the similar shot of the target clips.

We propose a technique for video shot segmentation and classification based on singular value decomposition (SVD). For the input video sequence, we create a feature-frame matrix A, and perform the SVD on it. From this SVD, we are able to not only derive the refined feature space to better segment the video sequence along the time axis, but also define metrics to enable classifications of the detected video shots. Using these SVD properties, we achieve the two goals of accurate video shot segmentation, and visual content-based shot classification at the same time.

We propose a method of shot boundary detection based on the co-occurrence of global motion in video stream. In addition to the conventional features based on appearance and local motion, we apply ST (Space-Time) patch analysis for detecting global motion in video stream. And then we perform shot boundary detection by constructing AdaBoost classifiers which represent the co-occurrence of global motion and the conventional features. Experimental results show that our method had 3.8% higher F-measure value than that of the conventional method for gradual shot boundary detection.

Pyramid regional graph representation learning for content-based video retrieval

Video shot boundary detection (SBD) is the first step of video analysis, summarization, indexing, and retrieval. In SBD process, videos are segmented into basic units called shots. In this paper, a new SBD method is proposed using color, edge, texture, and motion strength as vector of features (feature vector). Features are extracted by projecting the frames on selected basis vectors of Walsh-Hadamard transform (WHT) kernel and WHT matrix. After extracting the features, based on the significance of the features, weights are calculated. The weighted features are combined to form a single continuity signal, used as input for Procedure Based shot transition Identification process (PBI). Using the procedure, shot transitions are classified into abrupt and gradual transitions. Experimental results are examined using large-scale test sets provided by the TRECVID 2007, which has evaluated hard cut and gradual transition detection. To evaluate the robustness of the proposed method, the system evaluation is performed. The proposed method yields F1-Score of 97.4% for cut, 78% for gradual, and 96.1% for overall transitions. We have also evaluated the proposed feature vector with support vector machine classifier. The results show that WHT-based features can perform well than the other existing methods. In addition to this, few more video sequences are taken from the Openvideo project and the performance of the proposed method is compared with the recent existing SBD method.

At present, content-based video retrieval (CBVR) is the most mainstream video retrieval method, using the video features of its own to perform automatic identification and retrieval. This method involves a key technology, i.e. shot segmentation. In this paper, the method of automatic video shot boundary detection with K-means clustering and improved adaptive dual threshold comparison is proposed. First, extract the visual features of every frame and divide them into two categories using K-means clustering algorithm, namely, one with significant change and one with no significant change. Then, as to the classification results, utilize the improved adaptive dual threshold comparison method to determine the abrupt as well as gradual shot boundaries.Finally, achieve automatic video shot boundary detection system.

Shot boundary detection (SBD) is the first step in video management and retrieval system. Existing SBD techniques, however, are mostly based on original frame histograms, which process too much data and are sensitive to flashes and small motions. Furthermore, they mainly perform boundary detection between each two frames, which are unreliable when there appear noises. To address these problems, we investigate a wavelet based shot boundary detection technique. It employs wavelet transform to get thumbnails of the original frames. Furthermore, it employs between cluster distances instead of adjacent distances to detect boundary and is robust to noises. Extensive experiments have been carried out on the platform of TRECVID benchmark. The experimental results show that the proposed technique outmatches some others and achieves satisfactory results.

Video scene segmentation is very important research in the field of computer vision, because it helps in efficient storage, indexing and retrieval of videos. Achieving this kind of scene segmentation cannot be done by just calculating the similarity of low-level features presented in the video; high-level features should also be considered to achieve a better performance. Even though much research has been conducted on video scene segmentation, most of these studies failed to semantically segment a video into scenes. Thus, in this study, we propose a Deep-learning Semantic-based Scene-segmentation model (called DeepSSS) that considers image captioning to segment a video into scenes semantically. First, the DeepSSS performs shot boundary detection by comparing colour histograms and then employs maximum-entropy-applied keyframe extraction. Second, for semantic analysis, using image captioning that benefits from deep learning generates a semantic text description of the keyframes. Finally, by comparing and analysing the generated texts, it assembles the keyframes into a scene grouped under a semantic narrative. That said, DeepSSS considers both low- and high-level features of videos to achieve a more meaningful scene segmentation. By applying DeepSSS to data sets from MS COCO for caption generation and evaluating its semantic scene-segmentation task results with the data sets from TRECVid 2016, we demonstrate quantitatively that DeepSSS outperforms other existing scene-segmentation methods using shot boundary detection and keyframes. What’s more, the experiments were done by comparing scenes segmented by humans and scene segmented by the DeepSSS. The results verified that the DeepSSS’ segmentation resembled that of humans. This is a new kind of result that was enabled by semantic analysis, which was impossible by just using low-level features of videos.

Many recent cases of illegal activities have been reported regarding the video and all the unlawful activities designed to destroy the originality of the video data. A slight modification can alter the meaning and intent of the video. Nowadays, many devices have gained the power of tapping or capturing digital video, and all these videos can be approved over the internet and many other unsafe channels. Therefore, the detection o of video authentication proving is a widespread and critical problem in various areas such as criminal, forensic investigations, and court cases. The preceding study proposed numerous techniques to ensure the authentication of video data and structure. This study emphasized retrieving a video structure to detect an alteration made in the video using shot boundary detection to prove authentication. The detection of shot boundary is a complicated task and is usually utilized for structured video because these techniques can identify modifications made with a fundamental shot change frequency. This study focuses on retrieving a video structure to detect an alteration made in the video by detecting shot boundary to prove authentication. The detection of shot boundary is a complicated task and is usually utilized for structured video because these techniques can identify modifications made with a regular shot change frequency. Furthermore, this study aims to develop a new model for video authentication detection using the shot boundary detection method, which is considered the practical tool to track down the originality based on their fundamental structure.

Shot boundary detection is the fundamental part in many real applications as video retrieval and so on. This paper tackles the problem of video segment obtaining in complex movie videos. Firstly, intermediate descriptor is proposed to depict the variation of both abrupt and gradual change in shot boundaries, which is formed by distance vector on Local Binary Pattern (LBP), GIST (GIST) or their fusion. Instead of just using the adjacent frames distance, intermediate descriptor keeps the distances between current frame and consecutive frames. It comprehensively characterizes local temporal structure, which is especially important for gradual change. For the excellent ability for feature fusion in random forests, it is adopted here to verify the fusion effect of intermediate descriptor on LBP and GIST. The whole experiments are designed on the subset of TRECVid 2013 INS (INstance Search) task to verify the effectiveness of proposed intermediate descriptor and the fusion ability for random forest. Compared with static and adaptive thresholds approaches, the best performance can be achieved by post-fusion of intermediate descriptor on LBP and GIST.