Automatic image annotation and retrieval using weighted feature selection

The development of technology generates huge amounts of non-textual information, such as images. An efficient image annotation and retrieval system is highly desired. Clustering algorithms make it possible to represent visual features of images with finite symbols. Based on this, many statistical models, which analyze correspondence between visual features and words and discover hidden semantics, have been published. These models improve the annotation and retrieval of large image databases. However, image data usually have a large number of dimensions. Traditional clustering algorithms assign equal weights to these dimensions, and become confounded in the process of dealing with these dimensions. In this paper, we propose a top-down, subspace clustering algorithm as a solution to this problem. For a given cluster, we determine relevant features based on histogram analysis and assign greater weight to relevant features as compared to less relevant features. We have implemented four different models to link visual tokens with keywords based on the clustering results of our clustering algorithm and K-means algorithm, and evaluated performance using precision, recall and correspondence accuracy using benchmark dataset. The results show that our algorithm is better than traditional ones for automatic image annotation and retrieval.

Content-based Image retrieval systems extract and retrieve images using their low-level features, such as color, texture, and shape. Nevertheless, these visual contents do not allow a user to formulate semantically meaningful image query. Image annotation systems are a solution to solve the inadequacy of CBIR systems and allow text based image retrieval. There have been several studies on automatic image annotation utilizing machine learning techniques and images' representation with low level features extracted using either global or local methods. However, typically, these approaches suffer from low correlation between the globally assigned annotations and the visual features used to obtain annotations automatically. In this paper, we present an approach to enhance the effectiveness of CBIR using learning based automatic images annotation based on bag of visual word images representation that is created automatically using a set of manually annotated training images. The experimentation is performed with 4,000 annotated images for training, 1000 images for testing from ImageNet. The result has shown 77.5% of performance accuracy. The result of this work is believed to be one step towards enhancing the performance and effectiveness of existing CBIR and minimizing the semantic gap.

The explosive growth of digital images requires effective methods to manage these images. Among various existing methods, automatic image annotation has proved to be an important technique for image management tasks, e.g., image retrieval over large-scale image databases. Automatic image annotation has been widely studied during recent years and a considerable number of approaches have been proposed. However, the performance of these methods is yet to be satisfactory, thus demanding more effort on research of image annotation. In this paper, we propose a novel semi supervised framework built upon feature selection for automatic image annotation. Our method aims to jointly select the most relevant features from all the data points by using a sparsity-based model and exploiting both labeled and unlabeled data to learn the manifold structure. Our framework is able to simultaneously learn a robust classifier for image annotation by selecting the discriminating features related to the semantic concepts. To solve the objective function of our framework, we propose an efficient iterative algorithm. Extensive experiments are performed on different real-world image datasets with the results demonstrating the promising performance of our framework for automatic image annotation.

It has been heavy work that to find the related pictures form Internet without annotation. Therefore, the automatic image annotation was extremely important in image retrieval. The traditional method were translated image visual feature into keywords simply, but it ignored the image similarity problem between the low-level visual features and high-level semantic. That is image "gap" problem, so image annotation was very lower. This paper puts forward a classification of web based image content automatic tagging mixing technology, the first it will map visual feature of image to one or more rough images, then we will preprocess the web page text information, finally we select some keywords similarity as image annotation by using similar semantic processing module. So it realizes the image and text combining the automatic annotation and it achieve high precision of image annotation.

A survey and analysis on automatic image annotation

A Survey on Automatic Image Annotation and Trends of the New Age

Automatic image annotation techniques are proposed for overcoming the so-called semantic-gap between image low-level feature and high-level concept in content-based image retrieval systems. Due to the limitations of techniques, current state-of-the-art automatic image annotation models still produce some irrelevant concepts to image semantics, which are an obstacle to getting high-quality image retrieval. In this paper we focus on improving image annotation to facilitate web image retrieval. The novelty of our work is to use both WordNet and textual information in web documents to refine original coarse annotations produced by the classic Continuous Relevance Model (CRM). Each keyword in annotations is associated with a certain weight, and larger the weight is, more related to image semantics the corresponding concept is. The experimental results show that the refined annotations improve image retrieval to some extent, compared to the original coarse annotations.

Structural image retrieval using automatic image annotation and region based inverted file

Image annotation is the process of assigning significant labels to specific parts of an image, making it easier for systems to categorize, interpret, and retrieve visual data.These annotations are then used in the retrieval process to search for and recover images based on tagged data, improving the accuracy and efficiency of identifying relevant images.However, inadequate or inconsistent annotations lead to poor image retrieval performance, decreasing the overall effectiveness.This research proposes the Fuzzy C-Means Clustering-Gaussian Bare Bones White Shark Optimization (FCM-GBWSO) algorithm for automatic image annotation and retrieval.In WSO, GB is incorporated to prevent premature convergence and avoid local optima issues in generating cluster centroids and clustered images.Various feature extraction techniques, such as ResNet50 and Color Moments, are applied to extract features effectively.Feature transformation and Neighbourhood Component Analysis (NCA) are then used to transform the features into similar significance and select the best features.The proposed FCM-GBWSO achieves an average precision of 0.98, 0.96, and 0.97 on the Corel-10k, Caltech 256 datasets, and Corel 1k respectively, outperforming existing methods like hierarchical clustering and Deep Neural Network-based Deep Search and Rescue (DNN-SAR).

Nowadays, the number of digital images has increased so that the management of this volume of data needs an efficient system for browsing, categorising and searching. Automatic image annotation is designed for assigning tags to images for more accurate retrieval. Non‐negative matrix factorisation (NMF) is a traditional machine learning technique for decomposing a matrix into a set of basis and coefficients under the non‐negative constraints. In this study, the authors propose a two‐step algorithm for designing an automatic image annotation system that employs the NMF framework for its first step and a variant of K‐nearest neighbourhood as its second step. In the first step, a new multimodal NMF algorithm is proposed to extract the latent factors which reflect the content of images. This is done by jointly factorising the visual and textual data feature matrices so that they have close representation, although not necessarily the same. In the second step, after mapping images to the latent factors space a few tags are predicted for the new images based on a weighted average of similar data. They evaluated the performance of the proposed method and compared it to the state‐of‐the‐art literature. Comparison results demonstrate the effectiveness and potential of the proposed method in image annotation applications.

Automatic image annotation (AIA) has been adopted in different applications such as image retrieval and classification. Deep Learning is used in AIA to extract image features and then convert these features into text descriptions and labels. However, conventional AIA models that employ deep learning methods suffer from various shortcomings, such as poor annotation performance. This work proposes an AIA model based on convolutional neural networks (CNNs), generative adversarial networks (GANs), and transfer learning. GANs have attracted a lot of interest because of its ability to generate data without explicitly using probability density. Thus, it has proven its usefulness in image annotation and image augmentation. In this work, an Auxiliary classifier-GAN (ACGAN) has been used, where the discriminator predicts the class of an image rather than taking it as a given input; therefore, the stabilization of the training stage is ensured, and the generation of high-quality images is provided. Transfer learning is also used to enhance the performance of the classification. The proposed model outperforms the best state-of-the-art models in terms of MiAP, F-measure and error rate using ImageClef, ESPGame and IAPR-TC12 datasets.

Automatic image annotation is a critical and challenging problem in pattern recognition and image understanding areas. There are some problems in existing automatic image annotation areas. For example, the size of unlabeled data is much larger than the labeled data. Besides, most image annotation models can only use one kind of image segmentation strategy and certain image description method. According to the above problems, an automatic image annotation model based on Co-training is proposed. In this model, four independent feature properties are constructed and then four corresponding sub-classifiers are built. In this way, different image segmentation strategies and feature representation methods can be integrated into a unified framework. An adaptive algorithm based on vote and consistency is proposed to extend the training dataset. The proposed method use Co-training algorithm and mass unlabeled data to improve the performance of automatic image annotation. Experiments conducted on Corel 5 K dataset verify the effectiveness of proposed method.

Automatic image annotation has been extensively studied, mostly from a content-based approach, whose effectiveness is restricted by the “semantic gap” between low-level image features and semantic annotations, and by the irrelevance of annotations to image content. We propose a social diffusion analysis approach to image annotation, which exploits abundant social diffusion records about how images are disseminated within online social networks. Specifically, we propose a common-interest model to analyze social diffusion records, with the assumption that the diffusion pattern of an image in social networks is highly related to the relevance between image annotations and user preferences. In our proposed model, user preferences are represented as common interests of pairwise users instead of individual user interests. We find the notion of common interests not only facilitates the analysis of social diffusion patterns, but also leads to more accurate profiling of user preferences compared to individual interests. Based on the common-interest model, we design an image annotation framework via social diffusion analysis, which consists of the mining of common interests from social diffusion records, the feature extraction from diffusion graphs and common interests, and the automatic annotation by the learning-to-rank method. Experimental results on real-world data sets show that our proposed common-interest based approach outperforms individual-interest based methods, and also achieves superior performance than state-of-the-art content-based image annotation methods.

In order to investigate the performance of visual feature extraction method for automatic image annotation, three visual feature extraction methods, namely discrete cosine transform, Gabor transform and discrete wavelet transform, are studied in this paper. These three methods are used to extract low-level visual feature vectors from images in a given database separately, then these feature vectors are mapped to high-level semantic words to annotate images with labels in a given semantic label set. As it is more efficient to depict the visual features of an image by the feature distribution than to resort to image segmentation technology for semantic image blocks, this paper is going to find out which of the three feature extraction methods performs better in image annotation based on the distribution of feature vectors from the image. The performance of three different kinds of feature extraction method is fully analyzed, and it is found that discrete cosine transform method is more suitable for Gaussian mixture model in automatic image annotation.

Image classification with the use of radial basis function neural networks and the minimization of the localized generalization error