Semantics-based satellite image retrieval using low-level features

Most current content-based image retrieval systems are still incapable of providing users with their desired results. The major difficulty lies in the gap between low-level image features and high-level image semantics. To address the problem, this study reports a framework for effective image retrieval by employing a novel idea of memory learning. It forms a knowledge memory model to store the semantic information by simply accumulating user-provided interactions. A learning strategy is then applied to predict the semantic relationships among images according to the memorized knowledge. Image queries are finally performed based on a seamless combination of low-level features and learned semantics. One important advantage of our framework is its ability to efficiently annotate images and also propagate the keyword annotation from the labeled images to unlabeled images. The presented algorithm has been integrated into a practical image retrieval system. Experiments on a collection of 10,000 general-purpose images demonstrate the effectiveness of the proposed framework.

Ontologies represent domain concepts and relations in a form of semantic network. Many research works use ontologies in the information matchmaking and retrieval. This trend is further accelerated by the convergence of various information sources supported by ontologies. In this paper, we propose a novel multi-modality ontology model that integrates both the low-level image features and the high-level text information to represent image contents for image retrieval. By embedding this ontology into an image retrieval system, we are able to realize intelligent image retrieval with high precision. Moreover, benefiting from the soft-coded ontology model, this system has good flexibility and can be easily extended to the larger domains. Currently, our experiment is conducted on the animal domain canine. An ontology has been built based on the low-level features and the domain knowledge of canine. A prototype retrieval system is set up to assess the performance. We compare our experiment results with traditional text-based image search engine and prove the advantages of our approach.

Image retrieval is one of the most interesting and fastest growing research areas in all fields. It is an effective and efficient tool for managing large image databases. In most Content-Based Image Retrieval (CBIR) systems, an image is represented by a set of low-level visual features; hence a direct correlation with high-level semantic information will be absent. Therefore, a gap exists between high-level information and low-level features, which is the main reason that hinders the improvement of the image retrieval accuracy. In this work, main focus is on the semantic based image retrieval system using Gray Level Co-occurrence Matrix (GLCM) for texture feature extraction. Based on the texture features, semantic interpretations are given to the extracted textures. The images are retrieved according to user satisfaction and thereby reduce the semantic gap between low level features and high level features.

Based on the mean and variance values of a feature vector, we classify the feature vectors into four classes, obtaining four-class feature spaces. Next, the feature space of each class is vector quantized to obtain several sub-spaces (or clusters), each cluster being represented by a codeword. For each class, we sort the feature vectors in each cluster in the ascending order of their mean values. For any given query image, we first compute its feature vector and then find its class based on the mean and variance values. The online image retrieval for a given query image is then progressively performed in the first several nearest clusters (from the nearest cluster to farther clusters) of this class based on a fast search method named updating enhanced equal-average equal-variance K nearest neighbor search (UEEEKNNS) to find the first several nearest neighbors of the query feature vector as soon as possible.

Advancements in internet accessibility and the affordability of digital picture sensors have led to the proliferation of extensive image databases utilized across a multitude of applications. Addressing the semantic gap between low-level attributes and human visual perception has become pivotal in refining Content-Based Image Retrieval (CBIR) methodologies, especially within this context. As this field is intensely researched, numerous efficient algorithms for CBIR systems have surfaced, precipitating significant progress in the artificial intelligence field. In this study, we propose employing a hard voting ensemble approach on features derived from three robust deep learning architectures: Inception, Exception, and Mobilenet. This is aimed at bridging the divide between low-level image features and human visual perception. The Euclidean method is adopted to determine the similarity metric between the query image and the features database. The outcome was a noticeable improvement in image retrieval accuracy. We applied our approach to a practical dataset named CBIR 50, which encompasses categories such as mobile phones, cars, cameras, and cats. The effectiveness of our method was thereby validated. Our approach outshone existing CBIR algorithms with superior accuracy (ACC), precision (PREC), recall (REC), and F1-score (F1-S), proving to be a noteworthy addition to the field of CBIR. Our proposed methodology could be potentially extended to various other sectors, including medical imaging and surveillance systems, where image retrieval accuracy is of paramount importance.

As the use of digital media increases, effective retrieval and management techniques become more important. In order to extract useful information from this huge amount of data, many content based image retrieval (CBIR) systems have been developed. A typical CBIR system captures image features such as color, texture, or shape of objects in the query image and tries to retrieve images from the database with similar features. The main advantage of CBIR systems with relevance feedback is that these systems take into account the gap between the high-level concepts and low-level features and subjectivity of human perception of visual content. In this paper, a CBIR system based on color histogram has been presented and a new approach for image storage and retrieval called association-based image retrieval (ABIR) has been studied. The results of both systems have been shown.

Content-based image retrieval is a method of retrieving images based on its semantic features. Classical CBIR models in the literature uses low-level features like color, shape, and texture to retrieve similar images from the database. Low-level features work well when the dataset is small; however, it fails to reduce the semantic gap when the image database becomes large and diverse. In the recent past, many researchers have exploited the usage of pretrained neural network as a feature extractor capable of giving a high-level feature representation for images. In this chapter, we use the output vectors of intermediate layers of an Inception Resnet deep learning model as the feature representation for image retrieval. The high-level feature vectors obtained from the deep layers of Inception Resnet are clustered using K-means clustering algorithm to make the retrieval process faster. The second part of the chapter explores various distance metrics to find most similar images corresponding to the query image. Based on the analysis on various distance measures, it is found that Hamming distance performs better than Euclidean distance in retrieving similar images when using deep features owing to the sparse nature of the feature vectors produced by the intermediate deep layers of Inception Resnet.KeywordsContent-based image retrieval Deep learning Hamming distance K-means clustering Deep features for CBIR Image retrieval

In content-based image retrieval (CBIR), a set of low-level features are extracted from an image to represent its visual content. Retrieval is performed by image example where a query image is given as input by the user and an appropriate similarity measure is used to find the best matches in the corresponding feature space. This approach suffers from the fact that there is a large discrepancy between the low-level visual features that one can extract from an image and the semantic interpretation of the image’s content that a particular user may have in a given situation. That is, users seek semantic similarity, but we can only provide similarity based on low-level visual features extracted from the raw pixel data, a situation known as the semantic gap. The selection of an appropriate similarity measure is thus an important problem. Since visual content can be represented by different attributes, the combination and importance of each set of features varies according to the user’s semantic intent. Thus, the retrieval strategy should be adaptive so that it can accommodate the preferences of different users. Relevance feedback (RF) learning has been proposed as a technique aimed at reducing the semantic gap. It works by gathering semantic information from user interaction. Based on the user’s feedback on the retrieval results, the retrieval scheme is adjusted. By providing an image similarity measure under human perception, RF learning can be seen as a form of supervised learning that finds relations between high-level semantic interpretations and low-level visual properties. That is, the feedback obtained within a single query session is used to personalize the retrieval strategy and thus enhance retrieval performance. In this chapter we present an overview of CBIR and related work on RF learning. We also present our own previous work on a RF learning-based probabilistic region relevance learning algorithm for automatically estimating the importance of each region in an image based on the user’s semantic intent.

In this paper, we propose a text extraction method from camera-captured document style images and propose a text segmentation method based on a color clustering method. The proposed extraction method detects text regions from the images using two low-level image features and verifies the regions through a high-level text stroke feature. The two level features are combined hierarchically. The low-level features are intensity variation and color variance. And, we use text strokes as a high-level feature using multi-resolution wavelet transforms on local image areas. The stroke feature vector is an input to a SVM (support vector machine) for verification, when needed. The proposed text segmentation method uses color clustering to the extracted text regions. We improved K-means clustering method and it selects K and initial seed values automatically. We tested the proposed methods with various document style images captured by three different cameras. We confirmed that the extraction rates are good enough to be used in real-life applications.

The classification of images has attracted significant attention due to its applications in several computer vision tasks, including satellite imaging, image analysis, surveillance, object recognition, and image retrieval. The primary objective of image classification is to provide class labels to images based on their content. The applications of imagery classification and analysis in remote sensing are significant since they are utilized in many sectors, including military and civilian areas. In satellite imagery, the challenge of image classification is heightened due to the rotation of objects inside a view and the typically varied background. This review paper discusses satellite image classification through various deep learning approaches, including its historical background and current approaches. It finds the most appropriate classification techniques for different types of satellite images and discusses the impact of preprocessing methods on classification accuracy. Moreover, a comparative analysis of several studies is given based on different parameters such as research objectives, used datasets, methods/algorithms, limitations, etc. Specific case studies that have significantly contributed to the understanding of satellite image classification are also discussed; further, an overview of challenges and future research directions in this field is also provided.

Recently, digital content has become a significant and inevitable asset of or any enterprise and the need for visual content management is on the rise as well. Content-based image retrieval has attracted voluminous research in the last decade paving way for development of numerous techniques and systems besides creating interest on fields that support these systems. CBIR indexes the images based on the features obtained from visual content so as to facilitate speedy retrieval. In this thesis work, we present a steerable pyramid based image retrieval system that uses color, contours and texture as visual features to describe the content of an image region. We have initially used steerable pyramid to extract texture features from query image and database images and store them in feature vectors. Second, to speed up retrieval and similarity computation, the database images are classified and the extracted regions are clustered according to their feature vectors using median vector algorithm. This process is performed before query matching takes place. Therefore to answer a query our system does not need to search the entire database images; instead just a number of candidate images are required to be searched for image similarity. Our proposed system has the advantage of increasing the retrieval accuracy and decreasing the retrieval time. The experimental evaluation of the system is based on a satellite and medical image database. From the experimental results, it is evident that our system performs significantly better and faster compared with other existing systems. In our analysis, we provide a comparison between retrieval results based on features extracted from the whole image using steerable pyramid with median vector and features extracted from same image without median vector. The results demonstrate that each type of feature is effective for a particular type of images according to its semantic contents, and using a combination of them giving better retrieval results for almost all different classes of images in the dataset. Â

Effective World Wide Web image retrieval systems are required to locate relevant images as more and more images are used in HTML documents. There are two general approaches to image retrieval: one is based on text descriptions and the other on image contents (low-level features, such as colour and texture). They have their respective strengths and weaknesses. In this paper, we describe how to improve text-based and content-based image retrieval techniques and to integrate these two types of techniques in one system to take advantage of their complementing strengths. Our experimental results show that the integrated approach has higher retrieval performance than either the text-based or the content-based techniques used by themselves.

With the rapid development of various satellite sensors, automatic and advanced scene classification technique is urgently needed to process a huge amount of satellite image data. Recently, a few of research works start to implant the sparse coding for feature learning in aerial scene classification. However, these previous research works use the single-layer sparse coding in their system and their performances are highly related with multiple low-level features, such as scale-invariant feature transform (SIFT) and saliency. Motivated by the importance of feature learning through multiple layers, we propose a new unsupervised feature learning approach for scene classification on very high resolution satellite imagery. The proposed unsupervised feature learning utilizes multipath sparse coding architecture in order to capture multiple aspects of discriminative structures within complex satellite scene images. In addition, the dense low-level features are extracted from the raw satellite data by using different image patches with varying size at different layers, and this approach is not limited to a particularly designed feature descriptors compared with the other related works. The proposed technique has been evaluated on two challenging high-resolution datasets, including the UC Merced dataset containing 21 different aerial scene categories with a 1 foot resolution and the Singapore dataset containing 5 land-use categories with a 0.5m spatial resolution. Experimental results show that it outperforms the state-of-the-art that uses the single-layer sparse coding. The major contributions of this proposed technique include (1) a new unsupervised feature learning approach to generate feature representation for very high-resolution satellite imagery, (2) the first multipath sparse coding that is used for scene classification in very high-resolution satellite imagery, (3) a simple low-level feature descriptor instead of many particularly designed low-level descriptor, such as SIFT descriptors and saliency, (4) evaluation on two satellite image datasets that come from different sensor sources.

A novel and more effective region-based image retrieval strategy is presented based on semantic ontology. An unsupervised segmentation algorithm splits images into regions that are subsequently used as basis by the ontology-based strategy. The approach comprises three stages, namely automatic region generation, categorization and ontology construction. When receiving a query for a specific object, the search engine will, in addition to conventionally matched images, also find candidates through the semantic ontology using low level features. The proposed approach can thus find a richer set of related candidate images than traditional image retrieval approaches. This strategy is particularly useful for vague queries encountered by inexperienced users that are not trained in searching for images by the means of low-level features. The experimental results demonstrate the effectiveness of the proposed approach.

We propose a computerized framework that, given a region of interest (ROI) circumscribing a lesion, not only predicts radiological observations related to the lesion characteristics with 83.2% average prediction accuracy but also derives explicit association between low-level imaging features and high-level semantic terms by exploiting their statistical correlation. Such direct association between semantic concepts and low-level imaging features can be leveraged to build a powerful annotation system for radiological images that not only allows the computer to infer the semantics from diverse medical images and run automatic reasoning for making diagnostic decision but also provides “human-interpretable explanation” of the system output to facilitate better end user understanding of computer-based diagnostic decisions. The core component of our framework is a radiological observation detection algorithm that maximizes the low-level imaging feature relevancy for each high-level semantic term. On a liver lesion CT dataset, we have implemented our framework by incorporating a large set of state-of-the-art low-level imaging features. Additionally, we included a novel feature that quantifies lesion(s) present within the liver that have a similar appearance as the primary lesion identified by the radiologist. Our framework achieved a high prediction accuracy (83.2%), and the derived association between semantic concepts and imaging features closely correlates with human expectation. The framework has been only tested on liver lesion CT images, but it is capable of being applied to other imaging domains.