Efficient and interactive spatial-semantic image retrieval

Image retrieval is a critical task in many fields such as indexing and Content-Based Image Retrieval is a tried and tested method for image retrieval. With advances in graphics and visualization techniques, image retrieval should be easy, quick and flexible. The aim of this work is to develop an efficient image indexing and retrieval system to collate similar images or images based on conditioned queries in the form of an image itself. The indexing method has been performed using wavelet transform and related mechanisms. Its detailing can easily entail description from a broad range to a narrow range. The indexing algorithm applies a Haar Wavelet-Based Perceptual Similarity Index (HaarPSI) for Image Quality Assessment. The HaarPSI utilizes the coefficients obtained from a Haar wavelet decomposition to assess local similarities between two images, as well as the relative importance of image areas. Results show that the algorithm used is able to retrieve and display images with high retrieval accuracy and medium relevance.The proposed methodology has been tested against known image retrieval techniques and was found to be comparable and in some cases better than most prior known techniques. The testing was performed on images of varied types, from multiple datasets and sizes. The accuracy was found to be 99% in most cases. All in all, this study introduces a simple and convenient way of offline image searches on desktop computers and provides a stepping stone to future content-based image retrieval systems built for similar purposes.

With the proliferation of cloud services, cloud-based image retrieval services enable large-scale image outsourcing and ubiquitous image searching. While enjoying the benefits of the cloud-based image retrieval services, critical privacy concerns may arise in such services since they may contain sensitive personal information. In this paper, we propose an efficient and Privacy-Preserving Image Retrieval scheme with Key Switching Technique (PPIRS). PPIRS utilizes the inner product encryption for measuring Euclidean distances between image feature vectors and query vectors in a privacy-preserving manner. Due to the high dimension of the image feature vectors and the large scale of the image databases, traditional secure Euclidean distance comparison methods provide insufficient search efficiency. To prune the search space of image retrieval, PPIRS tailors key switching technique (KST) for reducing the dimension of the encrypted image feature vectors and further achieves low communication overhead. Meanwhile, by introducing locality sensitive hashing (LSH), PPIRS builds efficient searchable indexes for image retrieval by organizing similar images into a bucket. Security analysis shows that the privacy of both outsourced images and queries are guaranteed. Extensive experiments on a real-world dataset demonstrate that PPIRS achieves efficient image retrieval in terms of computational cost.

With the rapid growing volume of images in medical databases, development of efficient image retrieval systems to retrieve relevant or similar images to a query image has become an active research area. Despite many efforts to improve the performance of techniques for accurate image retrieval, its success in biomedicine thus far has been quite limited. This article presents an adaptive content-based image retrieval (CBIR) system for improving the performance of image retrieval in mammographic databases. In this work, the authors propose a new relevance feedback approach based on incremental learning with support vector machine (SVM) regression. Also, the authors present a new local perturbation method to further improve the performance of the proposed relevance feedback system. The approaches enable efficient online learning by adapting the current trained model to changes prompted by the user's relevance feedback, avoiding the burden of retraining the CBIR system. To demonstrate the proposed image retrieval system, the authors used two mammogram data sets: A set of 76 mammograms scored based on geometrical similarity and a larger set of 200 mammograms scored by expert radiologists based on pathological findings. The experimental results show that the proposed relevance feedback strategy improves the retrieval precision for both data sets while achieving high efficiency compared to offline SVM. For the data set of 200 mammograms, the authors obtained an average precision of 0.48 and an area under the precision-recall curve of 0.79. In addition, using the same database, the authors achieved a high pathology matching rate greater than 80% between the query and the top retrieved images after relevance feedback. Using mammographic databases, the results demonstrate that the proposed approach is more accurate than the model without using relevance feedback not only in image retrieval but also in pathology matching while maintaining its effectiveness for online relevance feedback applications.

Handwritten signature retrieval and identification

‘A visual idea is more powerful than verbal idea’, ‘A picture is worth more than ten thousand words’, ‘No words can convey what a picture speaks’, ‘A picture has to be seen and searched as a picture only’ are few of the well-known sayings that imply the certainty for the widespread availability of images. Common sense evidence suggests that images are required for a variety of reasons, like, illustration of text articles, conveying information or emotions that are difficult to describe in words, display of detailed data for analysis (medical images), formal recording of design data for later use (architectural plans) etc. The advent of digital photography combined with decreasing storage and processing cost, allows more and more people to have their personal collection of photographs and other visual content available on the internet. Organising these digital images into a small number of categories and providing effective indexing is imperative for accessing, browsing and retrieving useful data in “real time”. The process of digitization does not in itself make image collections easier to manage. Some form of indexing (cataloguing) is still necessary. People’s interest to have their own digital libraries has burgeoned and hence requires a data structure to preserve the images for a long time and also provide easy access to the desired images. These requirements have indeed forced the design of specialized imaging systems/ image databases, such that an access to any image is effective and efficient. An efficient image archival and retrieval system is characterized by its ability to retrieve relevant images based on their visual and semantic contents rather than using simple attributes or keywords assigned to them. Thus, it is necessary to support queries based on image semantics rather than mere-pixel-to-pixel matching. An image archival and retrieval system should therefore allow adequate abstraction mechanisms for capturing higher level semantics of images in order to support content addressability as far as possible. That is, for two images to be similar, not only the shape, color and texture properties of individual image regions must be similar, but also they must have the same arrangement (i.e., spatial relationships) in both the images. In fact, this is the strategy, which is generally being employed by our vision system most of the times. An effective method of representing images depends on the perception of knowledge embedded in images in terms of objects/components (generally known as elements) present in them along with their topological relationships. The perception of topological relationships, especially spatial relationships existing among the significant elements of an image, helps in making the image database system more intelligent, fast and flexible. An obvious method to search an image database is sequential scanning. The query is matched with all stored images (i.e., the representation of the query is matched with all representations stored in the image database) one by one. Retrievals may become extremely slow, especially when database search involves time consuming image matching operations. To deal with slow retrieval response times, and high complexity matching, an image database must utilize indexing methods that are faster than sequential scanning methods. In traditional image database systems, the use of indexing to allow database accessing has been well established. Analogously, image indexing techniques have been studied during the last decade to support representation of pictorial information in an image database and also to retrieve information from an image database. The use of significant elements present in images along with their topological relationships as indexes is the basic issue of the indexing methodologies developed to this aim.

Medical imaging is becoming a vital component of war on cancer. Tremendous amounts of medical image data are captured and recorded in a digital format during cancer care and cancer research. Facing such an unprecedented volume of image data with heterogeneous image modalities, it is necessary to develop effective and efficient content-based medical image retrieval systems for cancer clinical practice and research. While substantial progress has been made in different areas of content-based image retrieval (CBIR) research, direct applications of existing CBIR techniques to the medical images produced unsatisfactory results, because of the unique characteristics of medical images. In this paper, we develop a new multimodal medical image retrieval approach based on the recent advances in the statistical graphic model and deep learning. Specifically, we first investigate a new extended probabilistic Latent Semantic Analysis model to integrate the visual and textual information from medical images to bridge the semantic gap. We then develop a new deep Boltzmann machine-based multimodal learning model to learn the joint density model from multimodal information in order to derive the missing modality. Experimental results with large volume of real-world medical images have shown that our new approach is a promising solution for the next-generation medical imaging indexing and retrieval system.

The use of Computer-Aided Diagnosis in dermatology raises the necessity of integrating Content-Based Image Retrieval (CBIR) technologies. The latter could be helpful to untrained users as a decision support system for skin lesion diagnosis. However, classical CBIR systems perform poorly due to semantic gap. To alleviate this problem, we propose in this paper an intelligent Content-Based Dermoscopic Image Retrieval (CBDIR) system with Relevance Feedback (RF) for melanoma diagnosis that exhibits: efficient and accurate image retrieval as well as visual features extraction that is independent of any specific diagnostic method. After submitting a query image, the proposed system uses linear kernel-based active SVM, combined with histogram intersection-based similarity measure to retrieve the K most similar skin lesion images. The dominant (melanoma, benign) class in this set will be identified as the image query diagnosis. Extensive experiments conducted on our system using a 1097 image database show that the proposed scheme is more effective than CBDIR without the assistance of RF.

One of the main challenges in medical image retrieval is the increasing volume of image data, which render it difficult for domain experts to find relevant information from large data sets. Effective and efficient medical image retrieval systems are required to better manage medical image information. Text‐based image retrieval (TBIR) was very successful in retrieving images with textual descriptions. Several TBIR approaches rely on models based on bag‐of‐words approaches, in which the image retrieval problem turns into one of standard text‐based information retrieval; where the meanings and values of specific medical entities in the text and metadata are ignored in the image representation and retrieval process. However, we believe that TBIR should extract specific medical entities and terms and then exploit these elements to achieve better image retrieval results. Therefore, we propose a novel reranking method based on medical‐image‐dependent features. These features are manually selected by a medical expert from imaging modalities and medical terminology. First, we represent queries and images using only medical‐image‐dependent features such as image modality and image scale. Second, we exploit the defined features in a new reranking method for medical image retrieval. Our motivation is the large influence of image modality in medical image retrieval and its impact on image‐relevance scores. To evaluate our approach, we performed a series of experiments on the medical ImageCLEF data sets from 2009 to 2013. The BM25 model, a language model, and an image‐relevance feedback model are used as baselines to evaluate our approach. The experimental results show that compared to the BM25 model, the proposed model significantly enhances image retrieval performance. We also compared our approach with other state‐of‐the‐art approaches and show that our approach performs comparably to those of the top three runs in the official ImageCLEF competition.

Image semantic segmentation contains two sub-tasks, segmenting and labeling. However, the recent fully convolutional network (FCN) based methods often ignore the first sub-task and consider it as a direct labeling one. Even though these methods have achieved competitive performances, they obtained spatially fragmented and disconnected outputs. The reason is that, pixel-level relationships inside the deepest layers become inconsistent since traditional FCNs do not have any explicit pixel grouping mechanism. To address this problem, a multi-step supervised learning method, which contains image-level supervised learning step and pixel-level supervised learning step, is proposed. Specifically, as for the visualized result of image semantic segmentation, it is actually an image-to-image transformation problem, from RGB domain to category label domain. The recent conditional generative adversarial network (cGAN) has achieved significant performance for image-to-image generation task, and the generated image remains good regional connectivity. Therefore, a cGAN supervised by RGB-category label map is used to obtain a coarse segmentation mask, which avoids generating disconnected segmentation results to a certain extent. Furthermore, an interaction information (II) loss term is proposed for cGAN to remain the spatial structure of the segmentation mask. Additionally, dilated convolutional networks (DCNs) have achieved significant performance in object detection field, especially for small objects because of its special receptive field settings. Specific to image semantic segmentation, if each pixel is seen as an object, this task can be transformed to object detection. In this case, combined with the segmentation mask from cGAN, a DCN supervised by the pixel-level label is used to finalize the category recognition of each pixel in the image. The proposed method achieves satisfactory performances on three public and challenging datasets for image semantic segmentation.

Problem statement: Recently, there has been a huge progress in collection of varied image databases in the form of digital. Most of the users found it difficult to search and retrieve required images in large collections. In order to provide an effective and efficient search engine tool, the system has been implemented. In image retrieval system, there is no methodologies have been considered directly to retrieve the images from databases. Instead of that, various visual features that have been considered indirect to retrieve the images from databases. In this system, one of the visual features such as texture that has been considered indirectly into images to extract the feature of the image. That featured images only have been considered for the retrieval process in order to retrieve exact desired images from the databases. Approach: The aim of this study is to construct an efficient image retrieval tool namely, “Content Based Medical Image Retrieval with Texture Content using Gray Level Co-occurrence Matrix (GLCM) and k-Means Clustering algorithms”. This image retrieval tool is capable of retrieving images based on the texture feature of the image and it takes into account the Pre-processing, feature extraction, Classification and retrieval steps in order to construct an efficient retrieval tool. The main feature of this tool is used of GLCM of the extracting texture pattern of the image and k-means clustering algorithm for image classification in order to improve retrieval efficiency. The proposed image retrieval system consists of three stages i.e., segmentation, texture feature extraction and clustering process. In the segmentation process, preprocessing step to segment the image into blocks is carried out. A reduction in an image region to be processed is carried out in the texture feature extraction process and finally, the extracted image is clustered using the k-means algorithm. The proposed system is employed for domain specific based search engine for medical Images such as CT-Scan, MRI-Scan and X-Ray. Results: For retrieval efficiency calculation, conventional measures namely precision and recall were calculated using 1000 real time medical images (100 in each category) from the MATLAB Workspace database. For selected query images from the MATLAB-Image Processing tool Box-Workspace Database, the proposed tool was tested and the precision and recall results were presented. The result indicates that the tool gives better performance in terms of percentage for all the 1000 real time medical images from which the scalable performance of the system has been proved. Conclusion: This study proposed a model for the Content Based Medical Image Retrieval System by using texture feature in calculating the Gray Level Co Occurrence matrix (GLCM) from which various statistical measures were computed in order to increasing similarities between query image and database images for improving the retrieval performance along with the large scalability of the databases.

The development of technology generates huge amounts of non-textual information, such as images. An efficient image annotation and retrieval system is highly desired. Clustering algorithms make it possible to represent visual features of images with finite symbols. Based on this, many statistical models, which analyze correspondence between visual features and words and discover hidden semantics, have been published. These models improve the annotation and retrieval of large image databases. However, image data usually have a large number of dimensions. Traditional clustering algorithms assign equal weights to these dimensions, and become confounded in the process of dealing with these dimensions. In this paper, we propose weighted feature selection algorithm as a solution to this problem. For a given cluster, we determine relevant features based on histogram analysis and assign greater weight to relevant features as compared to less relevant features. We have implemented various different models to link visual tokens with keywords based on the clustering results of K-means algorithm with weighted feature selection and without feature selection, and evaluated performance using precision, recall and correspondence accuracy using benchmark dataset. The results show that weighted feature selection is better than traditional ones for automatic image annotation and retrieval.

Remote visualization of an arbitrary 2-D planar "cut" from a large volumetric dataset with random access has both gained importance and posed significant challenges over the past few years in industrial and medical applications. In this paper, a prediction model is presented that relates transmission efficiency to voxel coverage statistics for a fast random 2-D image retrieval system. This model can be for parameter selection and also provides insights that lead us to propose a new 3D rectangular tiling scheme, which achieves an additional 10% - 30% reduction in average transmission rate as compared to our previously proposed technique, e.g.,a nearly 30%/45% reduction in the average transmission rate at the cost of a factor of ten/fifteen in storage overhead compared to traditional cubic tiling. Furthermore, this approach leads to improved random access, with less storage and run-time memory required at the client.

The development of technology generates huge amounts of non-textual information, such as images. An efficient image annotation and retrieval system is highly desired. Clustering algorithms make it possible to represent visual features of images with finite symbols. Based on this, many statistical models, which analyze correspondence between visual features and words and discover hidden semantics, have been published. These models improve the annotation and retrieval of large image databases. However, image data usually have a large number of dimensions. Traditional clustering algorithms assign equal weights to these dimensions, and become confounded in the process of dealing with these dimensions. In this paper, we propose a top-down, subspace clustering algorithm as a solution to this problem. For a given cluster, we determine relevant features based on histogram analysis and assign greater weight to relevant features as compared to less relevant features. We have implemented four different models to link visual tokens with keywords based on the clustering results of our clustering algorithm and K-means algorithm, and evaluated performance using precision, recall and correspondence accuracy using benchmark dataset. The results show that our algorithm is better than traditional ones for automatic image annotation and retrieval.

Multimodal image retrieval, which involves retrieving images using various modalities such as text, audio, or other images, has significant research importance due to its wide-ranging applications and potential to enhance user experiences across multiple domains. Traditional image retrieval systems, content-based image retrieval (CBIR) systems rely solely on visual features, which can be limiting. By integrating multiple modalities, multimodal retrieval systems can offer more robust and accurate results. A research problem is considered multimodal when it integrates information from more than one type of data source. In multimodal image retrieval (MMIR) systems, one form of data is used to search for outcomes in the same or different modalities. In contrast, cross-modal systems strictly retrieve information from a different modality. A significant challenge in these systems is the effective comparison of input-output queries from different data types, due to their basic forms and the subjective nature of content similarity. Researchers have proposed various techniques to address this challenge and to bridge the semantic gap in information retrieval across different modalities. This article presents comprehensive analysis of various research works pertained in this multimodal image retrieval. The results and comparative analysis of current research works on benchmark datasets have also been discussed. At the end of the paper, some of open issues are presented for future research directions.

In this paper, we design and construct a multi-label image annotation and retrieval system. Various MPEG-7 low level visual features are employed for representing images. For image annotation, a bi-coded genetic algorithm is employed to select optimal feature subsets and corresponding optimal weights for every one vs. one SVM classifiers. After an unlabelled image is segmented into several regions, pre-trained SVMs are used to annotate each region, final label is obtained by merging all the region labels. Based on multi-label of image, image retrieval system provides keyword-based image retrieval service. Multi-labels can provide abundant descriptions for image content in semantic level, high precision annotation algorithm further improve annotation performance.