Role of kmax mapping image annotation method combined with R-tree index in civil engineering supervision

Joint multi-view representation and image annotation via optimal predictive subspace learning

Automatic image annotation is crucial for keyword-based image retrieval because it can be used to improve the textual description of images efficiently. For this purpose, many methods have been developed. Due to the restrictions of computational complexity and small training set, the image annotation methods are usually based on the probability of individual word, instead of the joint probability of a set of words. Therefore the correlation between words is omitted. In this paper, we propose a method to approximate the joint probability of words in a progressive way. Given an image, the word with highest probability is first annotated. Then, the successive words are annotated by incorporating the information of previously annotated words. It can be seen as a greedy algorithm to calculate the joint probability of multiple words. The experiments show that the proposed progressive annotation method can effectively improve the annotation performance.

In order to investigate the performance of visual feature extraction method for automatic image annotation, three visual feature extraction methods, namely discrete cosine transform, Gabor transform and discrete wavelet transform, are studied in this paper. These three methods are used to extract low-level visual feature vectors from images in a given database separately, then these feature vectors are mapped to high-level semantic words to annotate images with labels in a given semantic label set. As it is more efficient to depict the visual features of an image by the feature distribution than to resort to image segmentation technology for semantic image blocks, this paper is going to find out which of the three feature extraction methods performs better in image annotation based on the distribution of feature vectors from the image. The performance of three different kinds of feature extraction method is fully analyzed, and it is found that discrete cosine transform method is more suitable for Gaussian mixture model in automatic image annotation.

Automatic Image Annotation is important research topic in machine vision as it enables one to retrieve images from large databases by using textual queries. In recent years many machine learning techniques have been proposed to build detectors of concepts present on the images. In this paper we present a novel approach for image auto-annotation based on transfer of annotations from most similar images to the query image. We model image features by Multivariate Gaussian Distribution and measure distance between images by using Jensen-Shannon divergence. In spite of its simplicity, the proposed solution outperforms the state-of-the-art methods for image annotation and thus can be used as a baseline for developing other more elaborate methods.KeywordsQuery ImageSimilar ImageImage AnnotationSemantic LabelAutomatic Image AnnotationThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

With the advance of internet, multi-modal data can be easily collected from many social websites such as Wikipedia, Flickr, YouTube, etc. Images shared on the web are usually associated with social tags or other textual information. Although existing multi-modal methods can make use of associated text to improve image annotation, the disadvantages of them are that associated text is also required for a new image to be predicted. In this paper, we propose the cross-modal multi-task learning (CMMTL) framework for image annotation. Labeled and unlabeled multi-modal data are both levaraged for training in CMMTL, and it finally obtains visual classifiers which can predict concepts for a single image without any associated information. CMMTL integrates graph learning, multi-task learning and cross-modal learning into a joint framework, where a shared subspace is learned to preserve both cross-modal correlation and concept correlation. The optimal solution of the proposed framework can be obtained by solving a generalized eigenvalue problem. We conduct comprehensive experiments on two real world image datasets: MIR Flickr and NUS-WIDE, to evaluate the performance of the proposed framework. Experimental results demonstrate that CMMTL obtains a significant improvement over several representative methods for cross-modal image annotation.

This paper presents a highly scalable and adaptable co-learning framework on multimodal image retrieval and image annotation. The co-learning framework is based on the multiple instance learning theory. While this framework is a general framework that may be used in any specific domains, to evaluate this framework, we apply it to the Berkeley Drosophila ISH embryo image database for the evaluations of the retrieval and annotation performance. In addition, we also apply this framework to across-stage inferencing for the embryo images for knowledge discovery. We have compared the performance of the framework for retrieval, annotation, and inferencing on the Berkeley Drosophila ISH database with a state-of-the-art multimodal image retrieval and annotation method to demonstrate the effectiveness and the promise of the framework.

Current coral reef health monitoring programs rely on biodiversity data obtained through the acquisition and annotation of underwater photographs. Manual annotation of these photographs is a necessary step, but has become problematic due to the high volume of images and the high cost of human resources. While automated and reliable multi-spectral annotation methods exist, coral reef images are often limited to visible light, which makes automation difficult. Much of the previous work has focused on popular texture recognition methods, but the results remain unsatisfactory when compared to human performance for the same task. In this work, we present an improved automatic method for coral image annotation that yields consistent accuracy improvements over existing methods. Our method builds on previous work by combining multiple feature representations. We demonstrate that the aggregation of multiple methods outperforms any single method. Furthermore, our proposed system requires virtually no parameter tuning, and supports rejection for improved results. Firstly, the complex texture diversity of corals is handled by combining multiple feature representations: local binary patterns, hue and opponent angle histograms, textons, and deep convolutional activation feature. Secondly, these multiple representations are aggregated using a score-level fusion of multiple support vector machines. Thirdly, rejection can optionally be applied to enhance classification results, and allows efficient semi-supervised image annotation in collaboration with human experts.

This paper addresses image annotation, i.e. labelling pixels of an image with a class among a finite set of predefined classes. We propose a new method which extracts a sample of subwindows from a set of annotated images in order to train a subwindow annotation model by using the extremely randomized trees ensemble method appropriately extended to handle high-dimensional output spaces. The annotation of a pixel of an unseen image is done by aggregating the annotations of its subwindows containing this pixel. The proposed method is compared to a more basic approach predicting the class of a pixel from a single window centered on that pixel and to other state-of-the-art image annotation methods. In terms of accuracy, the proposed method significantly outperforms the basic method and shows good performances with respect to the state-of-the-art, while being more generic, conceptually simpler, and of higher computational efficiency than these latter.

Even within constrained domains like medicine there are no truly generic methods for automatic image parsing and annotation. Despite the fact that the precision and sophistication of image understanding methods have improved to cope with the increasing amount and complexity of the data, the improvements have not resulted in more flexible or generic image understanding techniques. Instead, the analysis methods are object specific and modality dependent. Consequently, current image search techniques are still dependent on the manual and subjective association of keywords to images for retrieval. Manually annotating the vast numbers of images which are generated and archived in the medical practice is not an option.

Digital imaging has become an essential element in every medical institution. Therefore, medical image retrieval such as chest X-ray (CXR) must be improved via novel feature extraction and annotation activities before they are stored into image databases. To date, many methods have been introduced to annotate medical images using spatial relationships after these features are extracted. However, the annotation performance for each method is inconsistent and does not show promising achievement to retrieve images. It is noticed that each method is still struggling with at least two big problems. Firstly, the recommended annotation model is weak because the method does not consider the object shape and rely on gross object shape estimation. Secondly, the suggested annotation model can only be functional for simple object placement. As a result, it is difficult to determine the spatial relationship feature after they are extracted to annotate images accurately. Hence, this study aims to propose a new model to annotate nodule location within lung zone for CXR image with extracted spatial relationship feature to improve image retrieval. In order to achieve the aim, a methodology that consists of six phases of CXR image annotation using the extracted spatial relationship features is introduced. This comprehensive methodology covers all cycles for image annotation tasks starting from image pre-processing until determination of spatial relationship features for the lung zone in the CXR. The outcome from applying the methodology also enables us to produce a new semi-automatic annotation system named CHEXRIARS which acts as a tool to annotate the extracted spatial relationship features in CXR images. The CHEXRIARS performance is tested using a retrieval test with two common tests namely the precision and recall (PNR). Apart from CHEXRIARS, three other annotation methods that are object slope, object projection and comparison of region boundaries are also included in the retrieval performance test. Overall, the CHEXRIARS interpolated PNR curve shows the best shape because it is the closest curve approaching the value of 1 on the X-axis and Y-axis. Meanwhile the value of area under curve for CHEXRIARS also revealed that this system attained the highest score at 0.856 as compared to the other three annotation methods. The outcome from the retrieval performance test indicated that the proposed annotation model has produced outstanding outcome and improved the image retrieval.

Image annotation is an open and challenging task, especially with large label vocabulary. In this paper, we propose a novel graph learning based method for image annotation, which takes both advantages of the nearest neighbour based and the graph-based methods, by exploiting the graph learning method to propagate the labels on the graph corresponding to the K nearest neighbours of a test image. To acquire more effective graph weights for computing score for each label, besides the similarity of visual features, our method also considers the similarity of two label sets, which is computed based on the label correlation that captures the semantic information between two labels. In addition, we combine the image-to-label distance with the graph learning based score to compute the final decision value for labelling. The proposed method is evaluated on three benchmark datasets for image annotation. The result shows our method substantially outperforms the previous graph learning based methods, and our result matches the current state-of-the-art results in annotation quality.

Although important in practice, manual image annotation and retrieval has rarely been studied by means of formal modeling methods. In this chapter, the authors propose a set of formal models to characterize the annotation times for two commonly-used manual annotation approaches, that is, tagging and browsing. Based on the complementary properties of these models, the authors design new hybrid approaches, called frequency-based annotation and learning-based annotation, to improve the efficiency of manual image annotation as well as retrieval. Both our simulation and experimental results show that the proposed algorithms can achieve up to a 50% reduction in annotation time over baseline methods for manual image annotation, and produce significantly better annotation and retrieval results in the same amount of time.

Web image annotation has become a critical research issue in recent years. It is beneficial to develop effective semisupervised feature selection methods to exploit the labeled data and unlabeled data simultaneously for web image annotation. However, the graph Laplacian based semi-supervised learning methods suffer from the fact that sparse coordinates are biased toward a constant and the Laplacian embedding often cannot preserve local topology well as we expected. In this paper we propose a novel semi-supervised feature selection method based on the second-order Hessian energy, namely Hessian Semisupervised Feature Selection (HSFS), which can overcome the drawbacks of Laplacian based methods. Extensive experiments are performed and the results show that our method is more suitable than Lapacian based methods for large-scale web image annotation.

Region-Based Shape Matching for Automatic Image Annotation and Query-by-Example

With the rapid development of location-based social network, more and more multimedia data are uploaded by users. These data always include large-scale of independent information with both textual and visual contents. To bridge the semantic gap in between, we propose a novel cross-domain learning method for automatic image annotation with geo-location information. First, we propose the topic model-based method for popular concept extraction to adaptively construct cross-domain datasets. Then these concepts are utilized to collect the visual correlation information from Flickr. Finally, we leverage cross-domain learning method for model learning. The comparison experiments on cross-domain datasets are conducted to demonstrate the superiority of the proposed method.