Automatic Image Annotation Based on Co-Training

A Survey on Automatic Image Annotation and Trends of the New Age

Image distance metric learning based on neighborhood sets for automatic image annotation

Adaptive optimized residual convolutional image annotation model with bionic feature selection model

Automatic image annotation is one of the research fields helping to extract the meaning of images, which aims at the production of a set of semantic annotations for an image to help better present the concept. Over the past few decades, researchers have developed a number of approaches for automatic image annotation. Nevertheless, previous studies have not fully accounted for visual features and annotated features. Therefore, it is still possible to achieve a better annotation performance by combining visual and annotated information. In this study, our aim is to associate multiple semantic tags with a given image. In particular, we detect how to obtain the image annotation by utilizing visual and annotated information. To take advantage of visual information, we first designed a modified neural network method to acquire the features of the image content. In addition, to obtain the annotated features, we exploit an aggregated network embedding approach that consists of annotation embedding, social embedding, profile embedding, and semantic embedding. Finally, to produce an accurate image annotation, we integrate the two aforementioned methods, that is, combining the visual and annotated information, to build a unified cooperative training framework. The experimental results on three real-world datasets clarify that our presented method is superior to the current popular image annotation approaches.

The explosive growth of digital images requires effective methods to manage these images. Among various existing methods, automatic image annotation has proved to be an important technique for image management tasks, e.g., image retrieval over large-scale image databases. Automatic image annotation has been widely studied during recent years and a considerable number of approaches have been proposed. However, the performance of these methods is yet to be satisfactory, thus demanding more effort on research of image annotation. In this paper, we propose a novel semi supervised framework built upon feature selection for automatic image annotation. Our method aims to jointly select the most relevant features from all the data points by using a sparsity-based model and exploiting both labeled and unlabeled data to learn the manifold structure. Our framework is able to simultaneously learn a robust classifier for image annotation by selecting the discriminating features related to the semantic concepts. To solve the objective function of our framework, we propose an efficient iterative algorithm. Extensive experiments are performed on different real-world image datasets with the results demonstrating the promising performance of our framework for automatic image annotation.

A survey and analysis on automatic image annotation

Automatic image annotation has been extensively studied, mostly from a content-based approach, whose effectiveness is restricted by the “semantic gap” between low-level image features and semantic annotations, and by the irrelevance of annotations to image content. We propose a social diffusion analysis approach to image annotation, which exploits abundant social diffusion records about how images are disseminated within online social networks. Specifically, we propose a common-interest model to analyze social diffusion records, with the assumption that the diffusion pattern of an image in social networks is highly related to the relevance between image annotations and user preferences. In our proposed model, user preferences are represented as common interests of pairwise users instead of individual user interests. We find the notion of common interests not only facilitates the analysis of social diffusion patterns, but also leads to more accurate profiling of user preferences compared to individual interests. Based on the common-interest model, we design an image annotation framework via social diffusion analysis, which consists of the mining of common interests from social diffusion records, the feature extraction from diffusion graphs and common interests, and the automatic annotation by the learning-to-rank method. Experimental results on real-world data sets show that our proposed common-interest based approach outperforms individual-interest based methods, and also achieves superior performance than state-of-the-art content-based image annotation methods.

This article presents an automatic image level annotation approach that takes advantage of both context and semantics presented in segmented images. The proposed approach is based on the optimization of classes’ scores using particle swarm optimization. In addition, random forest classifier and normalized cuts algorithm have been applied for automatic image classification, annotation, and clustering. For the proposed approach, each input image is segmented using the normalized cuts segmentation algorithm in order to create a descriptor for each segment. Two parameter selection models have been selected for particle swarm optimization algorithm and many voting techniques have been implemented to find the most suitable set of annotation words per image. Experimental results, using Corel5k benchmark annotated images dataset, demonstrate that applying optimization algorithms along with random forest classifier achieved noticeable increase in image annotation performance measures compared to related researches on the same dataset.

Automatic image annotation (AIA) has been adopted in different applications such as image retrieval and classification. Deep Learning is used in AIA to extract image features and then convert these features into text descriptions and labels. However, conventional AIA models that employ deep learning methods suffer from various shortcomings, such as poor annotation performance. This work proposes an AIA model based on convolutional neural networks (CNNs), generative adversarial networks (GANs), and transfer learning. GANs have attracted a lot of interest because of its ability to generate data without explicitly using probability density. Thus, it has proven its usefulness in image annotation and image augmentation. In this work, an Auxiliary classifier-GAN (ACGAN) has been used, where the discriminator predicts the class of an image rather than taking it as a given input; therefore, the stabilization of the training stage is ensured, and the generation of high-quality images is provided. Transfer learning is also used to enhance the performance of the classification. The proposed model outperforms the best state-of-the-art models in terms of MiAP, F-measure and error rate using ImageClef, ESPGame and IAPR-TC12 datasets.

Since most computer vision approaches are now driven by machine learning, the current bottleneck is the annotation of images. This time-consuming task is usually performed manually after the acquisition of images. In this article, we assess the value of various egocentric vision approaches in regard to performing joint acquisition and automatic image annotation rather than the conventional two-step process of acquisition followed by manual annotation. This approach is illustrated with apple detection in challenging field conditions. We demonstrate the possibility of high performance in automatic apple segmentation (Dice 0.85), apple counting (88 percent of probability of good detection, and 0.09 true-negative rate), and apple localization (a shift error of fewer than 3 pixels) with eye-tracking systems. This is obtained by simply applying the areas of interest captured by the egocentric devices to standard, non-supervised image segmentation. We especially stress the importance in terms of time of using such eye-tracking devices on head-mounted systems to jointly perform image acquisition and automatic annotation. A gain of time of over 10-fold by comparison with classical image acquisition followed by manual image annotation is demonstrated.

Nowadays, the number of digital images has increased so that the management of this volume of data needs an efficient system for browsing, categorising and searching. Automatic image annotation is designed for assigning tags to images for more accurate retrieval. Non‐negative matrix factorisation (NMF) is a traditional machine learning technique for decomposing a matrix into a set of basis and coefficients under the non‐negative constraints. In this study, the authors propose a two‐step algorithm for designing an automatic image annotation system that employs the NMF framework for its first step and a variant of K‐nearest neighbourhood as its second step. In the first step, a new multimodal NMF algorithm is proposed to extract the latent factors which reflect the content of images. This is done by jointly factorising the visual and textual data feature matrices so that they have close representation, although not necessarily the same. In the second step, after mapping images to the latent factors space a few tags are predicted for the new images based on a weighted average of similar data. They evaluated the performance of the proposed method and compared it to the state‐of‐the‐art literature. Comparison results demonstrate the effectiveness and potential of the proposed method in image annotation applications.

The explosive increase of image data on Internet has made it an important, yet very challenging task to index and automatically annotate image data. To achieve that end, sophisticated algorithms and models have been proposed to study the correlation between image content and corresponding text description. Despite the success of previous works, however, researchers are still facing two major difficulties that may undermine their effort of providing reliable and accurate annotations for images. The first difficulty is lacking of comprehensive benchmark image dataset with high quality text descriptions. The second difficulty is lacking of effective way to represent the image content and make it associate with the text descriptions. In our paper, we aim to deal with both problems. To deal with the first problem, we utilize Wikipedia as external knowledge source and enrich the ontology structure of ImageNet database with comprehensive and highly-reliable text descriptions from Wikipedia articles. To address the second problem, we develop a Probabilistic Topic-Connection (PTC) model to represent the connection between latent semantic topic in text description and latent patterns from image feature space. We compare the performance of our model with the currently popular Correspondence LDA (Corr-LDA) model under the same automatic image annotation scenario using cross-validation. Experimental results demonstrate that our model is able to well represent the connection between latent semantic topics and latent patterns in image feature space, thus facilitates knowledge organization and understanding of both image and text descriptions.

Automatic image annotation (AIA) has been proved to be an effective and promising solution to automatically deduce the high-level semantics from low-level visual features. Due to the inherent ambiguity of image-label mapping and the scarcity of training examples, it has become a challenge to systematically develop robust annotation models with better performance. In this paper, we try to attack the problem based on 2D CRFs (Conditional Random Fields) and semi-supervised learning which are seamlessly integrated into a unified framework. 2D CRFs can effectively capture the spatial dependency between the neighboring labels, while the semi-supervised learning techniques can exploit the unlabeled data to improve the joint classification performance. We conducted experiments on a medium-sized image collection including about 500 images from Corel Stock Photo CDs. The experimental results demonstrated that the annotation performance of this method outperforms standard CRFs, showing the effectiveness of the proposed unified framework and the feasibility of unlabeled data to help the classification accuracy.

Automatic image annotation techniques are proposed for overcoming the so-called semantic-gap between image low-level feature and high-level concept in content-based image retrieval systems. Due to the limitations of techniques, current state-of-the-art automatic image annotation models still produce some irrelevant concepts to image semantics, which are an obstacle to getting high-quality image retrieval. In this paper we focus on improving image annotation to facilitate web image retrieval. The novelty of our work is to use both WordNet and textual information in web documents to refine original coarse annotations produced by the classic Continuous Relevance Model (CRM). Each keyword in annotations is associated with a certain weight, and larger the weight is, more related to image semantics the corresponding concept is. The experimental results show that the refined annotations improve image retrieval to some extent, compared to the original coarse annotations.

Structural image retrieval using automatic image annotation and region based inverted file