Multiplayer semantic modeling of video and its semantic extraction frame

Content-based video retrieval is a proper solution to handle the video data. But because of their huge volumes and high dimensionality, finding a proper way to organize them for efficient search and retrieval becomes a challenging and important problem. In this paper, we present a framework for content-based video retrieval using the Shot Cluster Tree, while the latter organizes the content of shots in the tree structure. The framework can supply users with two types of queries: query-by-content and query-by-example. We also put up a novel and efficient clustering method for generating the Shot Cluster Tree structure, where shots which are visual similar and time adjacent are grouped into shot groups using the method of Sliding Shot Window firstly and then shot groups are clustered into shot clusters with the simple agglomerative hierarchical clustering method. In addition to constructing the structure of content, the Shot Cluster Tree also provides better ways to video summary and video annotation, which facilitate the video retrieval. An experimental system has been built up. Experiments verify the effectiveness of the proposed approach.

This paper proposes a method for content-based sports video retrieval using camera work information. Since particular camera work for a typical scene exists in sports videos, camera work transition becomes an effective cue for retrieving a sports scene based on its content. The proposed method extracts a series of camera parameters from both a user-specified scene of a retrieval key and a video stream of a retrieval target, and detects scenes having content similar to that of the key from the target by applying continuous DP matching. The method was evaluated using a video stream of a baseball game. Recall-Precision curves make its effectiveness clear.

Content-based image retrieval (CBIR) systems use multiple image features to represent an image. These systems suffer from curse of dimensionality since a high-dimensional feature vectors are formed to represent images in the target dataset. Reduction in length of feature vector can speed-up the retrieval process, but it reduces the retrieval accuracy. To overcome the aforementioned problem, an efficient layer-wise feature incremental approach for the CBIR system has been proposed. The proposed approach uses three primitive image features namely color, texture, and shape. The retrieval process is accomplished in three layers, at first a layer-complete dataset is searched but only single-feature space is used. Top 10% of images most similar to query image are retained in the second layer. The second layer uses two features for similarity computation, and only 50% of the most similar images are passed to the third layer. Finally, the third layer uses all three features to compute the similarity. Our aim is to reduce the search space at subsequent layers and use multiple features for a reduced dataset at the final layer. The proposed approach is evaluated on four publicly available image datasets. The retrieval results on the basis of precision, recall, and f-score show the performance improvement in comparison to state-of-the-art CBIR systems.

Scene change detection is an essential step to automatic and content-based video indexing, retrieval, and browsing. In this paper, a robust scene change detection method is presented, which analyzes both audio and visual information sources and accounts for their inter-relations and coincidence to semantically identify video scenes. Audio analysis focuses on the segmentation of audio source into four types of semantic data such as silence, speech, music, and environmental sound. Speech data are further decomposed into different elements according to different speakers. Meanwhile, visual analysis partitions video source into shots. Results from single source segmentation are in some cases suboptimal. By combining visual and audio features, the scene extraction accuracy is enhanced, and more semantic segmentations are developed. Experimental results are proven to be appropriate for content-based video indexing and retrieval.

With the development of multimedia data types and available bandwidth there is huge demand of video retrieval systems as users shift from text based retrieval systems to content based retrieval systems (CBVRS). Selection of extracted features play an important role in content based video retrieval, regardless of video attributes under consideration. Good features selection also allows the time, space and cost saving in the retrieval process. This leads to the interest of selecting only those features that can be extracted from video data and used for indexing and retrieval along with similarity measurement methods. There are certain issues like video store, search, index and retrieve as a trend in area of content based video retrieval systems. In this paper, we have proposed a method. In this paper fractional discrete cosine transform coefficients are used to extract key frames which eventually save space and time with better completeness in the retrieval process.

Video is a rich information source containing both audio and visual information along with motion information embedded in it. Applications such as e-learning, live TV, video on demand, traffic monitoring, etc. need an efficient video retrieval strategy. Content-based video retrieval and superpixel segmentation are two diverse application areas of computer vision. In this work, we are presenting an algorithm for content-based video retrieval with help of Integration of Curvelet transform and Simple Linear Iterative Clustering (ICTSLIC) algorithm. Proposed algorithm consists of two steps: off line processing and online processing. In offline processing, keyframes of the database videos are extracted by employing features: Pearson Correlation Coefficient (PCC) and color moments (CM) and on the extracted keyframes superpixel generation algorithm ICTSLIC is applied. The superpixels generated by applying ICTSLIC on keyframes are used to represent database videos. On other side, in online processing, ICTSLIC superpixel segmentation is applied on query frame and the superpixels generated by segmentation are used to represent query frame. Then videos similar to query frame are retrieved through matching done by calculation of Euclidean distance between superpixels of query frame and database keyframes. Results of the proposed method are irrespective of query frame features such as camera motion, object’s pose, orientation and motion due to the incorporation of ICTSLIC superpixels as base feature for matching and retrieval purpose. The proposed method is tested on the dataset comprising of different categories of video clips such as animations, serials, personal interviews, news, movies and songs which is publicly available. For evaluation, the proposed method randomly picks frames from database videos, instead of selecting keyframes as query frames. Experiments were conducted on the developed dataset and the performance is assessed with different parameters Precision, Recall, Jaccard Index, Accuracy and Specificity. The experimental results shown that the proposed method is performing better than the other state-of-art methods.

Content-based video retrieval is very interesting point where it can be used in our daily life. Video retrieval is regarded as one of the most important in multimedia research. The development of multimedia data types there is demand of video retrieval system. Video retrieval can be used for video search and browsing which are useful in web applications. Selection of extracted features play an important role in content based video retrieval. There are two types of feature extraction, low level feature extraction and high level feature extraction. Low level feature extraction based on color, shape, texture, spatial relationship. The main goal of this paper is that, user can give the two different types of input in the form of image query and the text query. First one is that give the input in the form of image query and retrieved image which is similar to the query image by using the CBIR algorithm. CBIR is still developing science. Retrieval of images based on visual features such as color, texture and shape. In this paper gives a detail description of a system developed for retrieving images similar to a query image from a different large set of image. Second one is that give the input in the form of text query and retrieved image by using the ABIR technique. ABIR is powerful algorithm for the information retrieval it can utilize their powerful natural language. Annotation is never complete.

Content-based search and retrieval of video data has become a challenging and important issue. Different methods[1,2,3] have been proposed to develop video retrievals systems to achieve better performance in terms of accuracy. The proposed technique uses wavelettransforms that areCosine Wavelet Transform, Walsh, Kekre, Slant and Hartley Wavelet Transform. The benefit of energy compaction of fewer transformedcoefficients is taken to reduce the feature vector size by taking fractional coefficients[5]of transformed frames of video. Smaller feature vector size results in less time for comparison of feature vectors resulting in faster retrieval of images. The feature vectors are extracted and coefficients sets are considered as feature vectors (100%, 6.25%, 3.125%, 1.5625%, 0.7813%, 0.39%, 0.195%, 0.097%, 0.048%, 0.024%, 0.012%, 0.006% and 0.003% of complete transformed coefficients).The database consists of 500 videos spread across 10 categories.Performance using wavelets is compared with respect to their transforms. In all considered wavelet transform, experimental result show the performance improvement in fractional coefficient of wavelet transform in content based video retrieval as compared to 100% coefficient considered.

With the vast growth and progress of multimedia on the Internet, particularly videos, which caused a growing demand for video retrieval, organizing, and automate systems, as many users mandate content-based retrieval systems over text-based retrieval systems. The process of extracting and selecting features, plays a very important role in content-based video retrieval regardless the significance of video characteristics. The reduction of time and space costs of the retrieval process is dependent on effective features selection. In this paper, we will discuss content-based video indexing and retrieval, in the application of video search engines CBVSE, which is the problem of searching for digital videos over the Internet. Addressing diverse approaches regarding video indexing and retrieval with brief outline and classification. Furthermore, we will propose a framework for a CBVSE that will offer a less complex solution for content-based video retrieval for video search engines over the web using reduced features vector with better accuracy to decrease the complexity trade-off. These features include temporal boundaries patterns combination and key-object concept features, as well as classifying the index list with a multi key-object concepts arrangement built on the statistical redundancy of the detected key-object concepts for each video shot in the sequence.

In content-based image retrieval (CBIR), one of the most challenging and ambiguous tasks are to correctly understand the human query intention and measure its semantic relevance with images in the database. Due to the impressive capability of visual saliency in predicting human visual attention that is closely related to the query intention, this paper attempts to explicitly discover the essential effect of visual saliency in CBIR via qualitative and quantitative experiments. Toward this end, we first generate the fixation density maps of images from a widely used CBIR dataset by using an eye-tracking apparatus. These ground-truth saliency maps are then used to measure the influence of visual saliency to the task of CBIR by exploring several probable ways of incorporating such saliency cues into the retrieval process. We find that visual saliency is indeed beneficial to the CBIR task, and the best saliency involving scheme is possibly different for different image retrieval models. Inspired by the findings, this paper presents two-stream attentive CNNs with saliency embedded inside for CBIR. The proposed network has two streams that simultaneously handle two tasks. The main stream focuses on extracting discriminative visual features that are tightly related to semantic attributes. Meanwhile, the auxiliary stream aims to facilitate the main stream by redirecting the feature extraction to the salient image content that human may pay attention to. By fusing these two streams into the Main and Auxiliary CNNs (MAC), image similarity can be computed as the human being does by reserving conspicuous content and suppressing irrelevant regions. Extensive experiments show that the proposed model achieves impressive performance in image retrieval on four public datasets.

The work presented here introduces a real-time automatic scene classifier within content-based video retrieval. In our envisioned approach end users like documentalists, not image processing experts, build classifiers interactively, by simply indicating positive examples of a scene. Classification consists of a two-stage procedure. First, small image fragments called patches are classified. Second, frequency vectors of these patch classifications are fed into a second classifier for global scene classification (e.g., city, portraits, or countryside). The first stage classifiers can be seen as a set of highly specialized, learned feature detectors, as an alternative to letting an image processing expert determine features a priori. The end user or domain expert thus builds a visual alphabet that can be used to describe the image in features that are relevant for the task at hand.We present results for experiments on a variety of patch and image classes. The scene classifier approach has been successfully applied to other domains of video content analysis, such as content-based video retrieval in television archives, automated sewer inspection, and porn filtering.

At present, content-based video retrieval (CBVR) is the most mainstream video retrieval method, using the video features of its own to perform automatic identification and retrieval. This method involves a key technology, i.e. shot segmentation. In this paper, the method of automatic video shot boundary detection with K-means clustering and improved adaptive dual threshold comparison is proposed. First, extract the visual features of every frame and divide them into two categories using K-means clustering algorithm, namely, one with significant change and one with no significant change. Then, as to the classification results, utilize the improved adaptive dual threshold comparison method to determine the abrupt as well as gradual shot boundaries.Finally, achieve automatic video shot boundary detection system.

In content based image retrieval (CBIR) process, every image has been represented in a compact set of local visual features i.e. color, texture, and/or shape of images. This set of local visual features is known as feature vector. In the CBIR process, feature vectors of images have been used to represent or to identify similar images in adequate way. As a result, feature vector construction has always been considered as an important issue since it must reflect proper image semantics using minimal amount of data. The proposed CBIR scheme is based on the combination of color and texture features. In this work initially, we have converted the given RGB image into HSV color image. Subsequently, we have considered H (hue), S (saturation), and V (intensity) components for extraction of visual image features. The texture features have been extracted from the V component of the image using 2D dual-tree complex wavelet transform (2D DT-CWT) where it analyzes the textural patterns in six different directions i.e. ±15\(^{\circ }\), ±45\(^{\circ }\), and ±75\(^{\circ }\). At the same time, we have computed the probability histograms of H and S components of the image respectively and subsequently those are divided into non-uniform bins based on cumulative probability for extraction of color based features. So, in this work both the color and texture features have been extracted simultaneously. Finally, the obtained features have been concatenated to attain the final feature vector and same is considered in image retrieval process. We have tested the novelty and performance of the proposed work in two Corel, two objects, and, a texture image datasets. The experimental results reveal the acceptable retrieval performances for different types of datasets.

The elementary technology for video retrieval technique in computing is very sensitive to the user's query not to the contents of video. Every user has perspective to the video in the world of videos. So to identify this is difficult using text based video retrieval as this retrieval is based on text as processing input for video retrieval. The better approach for this video retrieval is content based video retrieval which is gaining acceptance in world of videos as a technique. In this paper, a novel approach for increasing the performance of retrieval by using intermediate levels with Block Truncation Coding (BTC) is presented. For this, the experimental data set is built up of 10 different categories in which 50 videos are present per category. In this paper, the approach of generation of feature by using 20 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">th</sup> frequency frames of videos from data set considered. Each of the extracted frame is intermediated by intermediate blocks (which is considered as 4 and 9) and then BTC applied on each block to get feature vector suing on 7 different color spaces as KLUV YCbCr, YCgCb, YIQ, YUV RGB, and XYZ. The feature vectors generated are processed and best match results are achieved. The similarity criteria used is absolute difference (AD) for calculating similarity or difference among videos and query. The performance of this technique is confirmed with help of cross over points of precision and recall values for each color space. The higher performance is achieved is in YIQ color space using intermediate block level 16.

Content-based image retrieval (CBIR) is a technique for images retrieval based on their visual features, i.e. induced by their pixels. The images are, classically, described by the image feature vectors. Those vectors reflect the texture, color or a combination of them. The accuracy of the CBIR system is highly influenced by the (i) definition of the image feature vector describing the image, (ii) indexing and (iii) retrieval process. In this paper, we propose a new CBIR system entitled ISE (Image Search Engine). Our ISE system defines the optimum combination of color and texture features as an image feature vector, including the Particle Swarm Optimization (PSO) algorithm and employing an Interactive Genetic Approach (GA) for the indexing process. The performance analysis shows that our suggested PCM (Proposed Combination Method) upgrades the average precision metric from 66.6% to 89.30% for the “Food” category color histogram, from 77.7% to 100% concerning CCVs (Color Coherence Vectors) for the “Flower” category and from 58% to 87.65% regarding the DCD (Dominant Color Descriptor) for the “Building” category using the Corel dataset. Besides, our ISE system showcases an average precision of 98.23%, which is significantly higher than other CBIR systems presented in related works.