Visual Alphabets: Video Classification by End Users

Various relevance feedback techniques have been applied in content-based image retrieval. However, many are either heuristics-based, or computationally too expensive to be implemented in real-time, or limited to deal with only positive examples. We propose a fast and optimal linear relevance feedback scheme that takes both positive and negative examples from the user. This scheme can be regarded as a generalization of discriminant analysis on one hang, and on the other hand, it is also a generalization of an existing optimal scheme that takes only positive examples. We first define biased classification problem for the case where the data samples are labeled as positive or negative as to whether belonging to the target class (the biased class) or not; then biased discriminant analysis (BDA) is proposed as an optimal linear solution for dimensionality reduction. We also propose biased whitening transformation on the data when Euclidean distance is applied afterwards. Toy problems are designed to show the theoretical advantages of the proposed scheme over traditional discriminant analysis. It is implemented in real-time image retrieval for large databases and experimental results are presented to show the improvement achieved by the new scheme.© (2000) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

With the emergence and explosion of huge image databases there is an increasing necessity for effective methods to assess visual information on the level of objects and scene types. A wide variety of Content – Based Image Retrieval (CBIR) systems already exists. As a key issue in CBIR, similarity measure quantifies the resemblance in contents between a pair of images. Depending on the type of features, the formulation of the similarity measure varies greatly. The primary goal of our study is to reduce the computation time and user interaction. The secondary goal is to reduce the semantic gap between high level concepts and low level features. A third goal is to evaluate system performance with regard to speed and accuracy. In the proposed study transform color after statistical transform, such as the Adaptive Color Karhunen Loeve Transform (ACKLT) is used as a color descriptor. The results are showing the advantage of the new algorithm for ACKLT in comparison with the YCrCb color model. Based on the experimental results, we concluded that correct selection of descriptors invariant to light intensity and light color changes affects object and scene category recognition.Keywordscontent-based image retrievalAdaptive Color Karhunen Loeve Transform

Research has been devoted in the past few years to relevance feedback as an effective solution to improve performance of content-based image retrieval (CBIR). In this paper, we propose a new feedback approach with progressive learning capability combined with a novel method for the feature subspace extraction. The proposed approach is based on a Bayesian classifier and treats positive and negative feedback examples with different strategies. Positive examples are used to estimate a Gaussian distribution that represents the desired images for a given query; while the negative examples are used to modify the ranking of the retrieved candidates. In addition, feature subspace is extracted and updated during the feedback process using a principal component analysis (PCA) technique and based on user's feedback. That is, in addition to reducing the dimensionality of feature spaces, a proper subspace for each type of features is obtained in the feedback process to further improve the retrieval accuracy. Experiments demonstrate that the proposed method increases the retrieval speed, reduces the required memory and improves the retrieval accuracy significantly.

Content Based Image Retrieval (CBIR) deals withthe automatic extraction of images from a database based ona query. For efficient retrieval the digital image CBIR requiressupport of scene classification algorithms. The Cognitive psychology suggests that the basic level classification is efficient withthe global features. However, a detailed classification requires acombination of the global and the local features. In this paper, we propose a decision fusion of the classification results based onlocal and global features. The proposed algorithm is a multi stageapproach, in the stage-1 the algorithm separates the completedatabase into natural and artificial images using spectral features. In the stage-2, the texture and color features are used to furtherclassify the image database into subcategories. The results of theproposed decision fusion algorithm give a 5% better classificationaccuracy than the single best classifier.

Content-based video retrieval is a proper solution to handle the video data. But because of their huge volumes and high dimensionality, finding a proper way to organize them for efficient search and retrieval becomes a challenging and important problem. In this paper, we present a framework for content-based video retrieval using the Shot Cluster Tree, while the latter organizes the content of shots in the tree structure. The framework can supply users with two types of queries: query-by-content and query-by-example. We also put up a novel and efficient clustering method for generating the Shot Cluster Tree structure, where shots which are visual similar and time adjacent are grouped into shot groups using the method of Sliding Shot Window firstly and then shot groups are clustered into shot clusters with the simple agglomerative hierarchical clustering method. In addition to constructing the structure of content, the Shot Cluster Tree also provides better ways to video summary and video annotation, which facilitate the video retrieval. An experimental system has been built up. Experiments verify the effectiveness of the proposed approach.

Indexing chromatic and achromatic patterns for content-based colour image retrieval

Merits and defects of note-based video retrieval and content-based video retrieval are investigated in this paper, and complementarity of two aspects each other are also discussed. A new theory of semantic processing in content-based video retrieval is explored. An object-oriented representation modeling of the video semantics, based on the multi-layers, is presented. And on this basis, a semantic extraction frame based on the multi-semantics is constructed.

Content Based Image Retrieval (CBIR) systems that search similar images in a large database are attracting more and more research interests recently, and have been applied to medical image characterization for expert's experience sharing. One challenging task in CBIR is how to extract features for effective image representation. Therein sparse coding technique has been proven to be an effective way to learn inherent structure features for image analysis. However, it is necessary to first vectorize the 2- or 3-dimensional spatial structure for analysis with sparse coding, and then destroy the spatial relation of nearby voxels. In this study, we propose a multilinear sparse coding method to learn features from multi-dimensional medical images. We regard high dimensional local structures as tensors and propose a K-CP (CANDECOMP/PARAFAC) algorithm to learn a tensor dictionary in an iterative way. With the learned tensor dictionary, sparse coefficients of tensor local structures are calculated by multilinear orthogonal matching pursuit (MOMP) algorithm, which is an extended multilinear version of the conventional linear OMP. The proposed multilinear sparse coding method is prospected to be more efficient and effective for inherent feature extraction compared with conventional linear methods. The proposed method is applied to a CBIR system for retrieval of focal liver lesions (FLLs) using a medical database consisting of contrast-enhanced multi-phase computer-tomography (CT) images. Experiments show that the constructed CBIR with multilinear sparse coding method can achieve promising retrieval performance.

The explosive growth in the volume of images available on the Web and in enterprise databases continues unabated. However, images on the Web are of many variations in type and format. Selecting relevant images based on image content is a difficult problem. Managing a large amount of images using their semantic meaning with content-based retrieval remains a challenge for both database and image processing researchers. This paper contributes by proposing an architecture for Web-based image mining and management systems with content-based image retrieval (CBIR), ImgWeb. ImgWeb is a Web-based system because it not only enables image mining and management for images in the database but also images from the Internet (Web images). Image agents play a key role in retrieving and mining Web images that are relevant to textual queries. ImgWeb is an object-oriented image database system, and it represents images as content-objects for processing. Thus, it supports CBIR and a wide range of graphical and textual querying methods. The proposed architecture permits the development of new image mining and management systems by utilizing existing components. Examples of image database applications include graphic design, recognition and entertainment.

Similarity-based retrieval of images is an important task in image databases. Most of the user's queries are on retrieving those database images that are spatially similar to a query image. In defence strategies, one wants to know a number of armoured vehicles, such as battle tanks, portable missile launching vehicles, etc. moving towards it, so that one can decide counter strategy. Content-based spatial similarity retrieval of images can be used to locate spatial relationship of various objects in a specific area from the aerial photographs and to retrieve images similar to the query image from image database. A content-based image retrieval system that efficiently and effectively retrieves information from a defence image database along with the architecture for retrieving images by spatial similarity is presented. A robust algorithm SIMdef for retrieval by spatial similarity is proposed that utilises both directional and topological relations for computing similarity between images, retrieves similar images and recognises images even after they undergo modelling transformations (translation, scale and rotation). A case study for some of the common objects, used in defence applications using SIMdef algorithm, has been done.

Image classification is very helpful for organizing large image databases and content based image retrieval (CBIR). However, it is very complex and challenging because of lacking effective methods. In this paper, we present a tree representation of images based on rectangular-shape partition. Then an adaptive processing algorithm is adopted to perform the classification task. Experimental results on seven categories of scenery images show that the structural representations are better than the traditional methods and our previous work based on quadtree representation of fixed partition.

In this paper, a prototype system is described for the management and content-based retrieval of defect images in huge image databases. This is a real problem in surface inspection applications, since modern inspection systems may produce up to thousands of defect images in a day. The retrieval is based on shape and internal structure characteristics of defects, so neither manual labeling nor annotation needs to be made. We are using a noncommercial, generic content-based image retrieval (CBIR) system called PicSOM that is modified to fit the special requirements of our application. The results of experiments show that the prototype system works fast with good retrieval results.

With the rapid development of 3S technologies, content-based retrieval from remote sensing images provides an important means of information acquisition and sharing and has become a key technology in digital earth construction. This paper puts the emphasis on using texture features for retrieval of aerial image database. M-band wavelet theory is introduced and M-band wavelet histogram technology is applied to texture extraction. Some experimental results are given to evaluate the retrieval performance of our method and to compare with that of method based on traditional two-band wavelet transform, followed by conclusions

This paper proposes a formal representation of the operations required to perform content-based image retrieval (CBIR) in large relational databases, using similarity queries. In this paper, we consider similarity as a numerical value obtained comparing a pair of images, which is calculated by a distance (dissimilarity) function. Distance functions usually rely on a set of features extracted from each image through a set of image processing algorithms called feature extractors. Before extracting features, other image processing algorithms are usually employed to pre-process each image, preparing it for the extractors. Usually there are several criteria that can be considered when measuring how much two images are similar. Therefore, to compare images in current CBIR environments one must define (1) the criteria, (2) the image pre-processing needed before the extractors can be executed, (3) which are those extractors, (4) which features must be considered, (5) and which distance function must be used. All of these definitions must have been set before a comparison can be performed. The complexity of defining how to compare images has lead to the development of systems aiming CBIR that allow relatively few options to configure the image comparison operations. Moreover, no formal representation of the entire CBIR process exists. In this paper we present such a formal environment, where all above-mentioned definitions are represented, entailing the development of flexible and highly-configurable CBIR systems. We also report a system developed using this formalism that enables the content-based retrieval of medical images from a hospital database, thus showing results of applying the presented formalism in a real environment.

Abstract-With the explosion in demand for visual information retrieval in soccer videos, many Content-Based Video Retrieval (CBVR) models were born. However these current CBVR models still remain some shortcomings such as inflexible retrieval frameworks because they are majorly based on a specific training data set and a specific language. In this paper we propose a framework which contains flexible and easy-to-use queries based on visual and audio data indexing in soccer videos. Our framework understands a soccer event which is constructed by the following visual components: shot with shot types, interest objects, and audio components. In which, an event can be retrieved from coarse to fine by finding relevant shot-sequence, after that some specific visual and audio objects are related to events. Our experiment on AFF-Suzuki Cup Tournament 2009 video data set showed better retrieval results with events such as goal, foul, shoot, corner kick, off side, red (yellow) card regarding precision and recall than the traditional systems only based on visual or audio queries.