Connectivity-Based Shape Descriptors

In this paper, we have proposed a method for 2D image retrieval based on object shapes. The method relies on transforming the 2D images into 3D space based on distance transform. Spherical harmonics are obtained for the 3D data and used as descriptors for the underlying 2D images. The proposed method is compared against two existing methods which use spherical harmonics for shape based retrieval of images. MPEG-7 Still Images Content Set is used for performing experiments; this dataset consists of 3621 still images. Experimental results show that the performance of the proposed descriptors is significantly better than other methods in the same category.

With the development of 3D model reconstruction, manufacturing, and 3D model vision technologies, 3D model recognition has attracted much attention recently. To handle the 3D model recognition problem, in this paper, we propose a panorama based on multi-channel-attention (MCA) CNN network for the representation of the 3D model. The proposed method is composed of three parts: extracting views, transform function learning, and generating 3D model descriptor. Concretely, we first extract the 2D panoramic views for each 3D model, and we use the multi-channel-attention neural network to extract the descriptor for each 3D model. Here, the attention model is used to find the unequal weights of each panorama view to generate the more robust 3D model descriptor. Finally, The fusion feature is used to handle the 3D model classification and retrieval problem. The popular data sets ModelNet and ShapeNet are used to demonstrate the performance of our approach. The experiments also demonstrate the superiority of our proposed method over the state-of-art methods.

Due to the huge growth of the World Wide Web, medical images are now available in large numbers in online repositories, and there exists the need to retrieval the images through automatically extracting visual information of the medical images, which is commonly known as content-based image retrieval (CBIR). Since each feature extracted from images just characterizes certain aspect of image content, multiple features are necessarily employed to improve the retrieval performance. Meanwhile, experiments demonstrate that a special feature is not equally important for different image queries. Most of existed feature fusion methods for image retrieval only utilize query independent feature fusion or rely on explicit user weighting. In this paper, we present a novel query dependent feature fusion method for medical image retrieval based on one class support vector machine. Having considered that a special feature is not equally important for different image queries, the proposed query dependent feature fusion method can learn different feature fusion models for different image queries only based on multiply image samples provided by the user, and the learned feature fusion models can reflect the different importances of a special feature for different image queries. The experimental results on the IRMA medical image collection demonstrate that the proposed method can improve the retrieval performance effectively and can outperform existed feature fusion methods for image retrieval.

Content-based object organization for efficient image retrieval in image databases

An effective method for retrieving 3D models is to represent and discriminate them with their 2D images projected from multiple viewpoints. Such view-based methods conform more closely to human visual recognition for 3D model retrieval, since the human retina essentially captures 2D images. However, most of the existing view-based methods do not take into account that different views have different importance even though they belong to the same object. To address this problem, we propose a novel view-based method for 3D CAD model retrieval. First, the PHOG descriptor is employed to describe the 2D images projected from a model. Then, Lagrange multipliers, vector quantization and a Support Vector Machine (SVM) are used to adaptively assign an optimal weight to each projected image. The similarity between a 3D query model and a 3D object in database is determined by the likeness of their corresponding 2D images associated with optimal weights. The effectiveness of the proposed method is shown in the experimental part.

Deep hashing methods utilize an end-to-end framework to mutually learn feature representations and hash codes, thereby achieving a better retrieval performance. Traditional supervised hashing methods adopt handcrafted features for hashing function learning and then generate hash codes through classification and quantization. The lack of adaptability and independence of the quantization procedure leads to low retrieval accuracy of supervised hashing methods with handcrafted features in image retrieval. In this study, a non-relaxation deep hashing method for fast image retrieval is proposed. In this method, a differentiable host thresholding function is used to encourage hash-like codes to approach -1 or 1 non-linearly at the output of the convolutional neural, instead of the symbol function for quantization used in the traditional method. The output of the host thresholding function is directly used to compute the network training error, and a loss function is elaborately designed with the norm to constrain each bit of the hash-like code to be as binary as possible. Finally, a symbol function is added outside the trained network model to generate binary hash codes for image storage and retrieval in a low-dimensional binary space. Extensive experiments on two large-scale public datasets show that our method can effectively learn image features, generate accurate binary hash codes, and outperform state-of-the-art methods in terms of the mean average precision.

One of the main challenges in medical image retrieval is the increasing volume of image data, which render it difficult for domain experts to find relevant information from large data sets. Effective and efficient medical image retrieval systems are required to better manage medical image information. Text‐based image retrieval (TBIR) was very successful in retrieving images with textual descriptions. Several TBIR approaches rely on models based on bag‐of‐words approaches, in which the image retrieval problem turns into one of standard text‐based information retrieval; where the meanings and values of specific medical entities in the text and metadata are ignored in the image representation and retrieval process. However, we believe that TBIR should extract specific medical entities and terms and then exploit these elements to achieve better image retrieval results. Therefore, we propose a novel reranking method based on medical‐image‐dependent features. These features are manually selected by a medical expert from imaging modalities and medical terminology. First, we represent queries and images using only medical‐image‐dependent features such as image modality and image scale. Second, we exploit the defined features in a new reranking method for medical image retrieval. Our motivation is the large influence of image modality in medical image retrieval and its impact on image‐relevance scores. To evaluate our approach, we performed a series of experiments on the medical ImageCLEF data sets from 2009 to 2013. The BM25 model, a language model, and an image‐relevance feedback model are used as baselines to evaluate our approach. The experimental results show that compared to the BM25 model, the proposed model significantly enhances image retrieval performance. We also compared our approach with other state‐of‐the‐art approaches and show that our approach performs comparably to those of the top three runs in the official ImageCLEF competition.

Simple SummaryFollicular lymphoma is an indolent B cell lymphoproliferative disorder of transformed follicular center B cells, which accounts for 20–30 percent of all non-Hodgkin lymphoma (NHL) cases. Although huge efforts have been made in the last 10 years, this pathology is still considered as incurable, leaving open the discovery and testing of new therapeutic targets requiring relevant preclinical models. Here, we report a realistic 3D model of t (14;18)-positive B-NHL cell culture (ultra-low attachment (ULA)-multicellular aggregates of lymphoma cells (MALC)), which monitored by state-of-the-art 2D and 3D imaging, allows more robust drug testing.Follicular lymphoma (FL) is an indolent B cell lymphoproliferative disorder of transformed follicular center B cells, which accounts for 20–30 percent of all non-Hodgkin lymphoma (NHL) cases. Great advances have been made to identify the most relevant targets for precision therapy. However, no relevant models for in vitro studies have been developed or characterized in depth. To this purpose, we generated a 3D cell model from t(14;18)-positive B-NHL cell lines cultured in ultra-low attachment 96-well plates. Morphological features and cell growth behavior were evaluated by classical microscopy (2D imaging) and response to treatment with different drugs was evaluated by a high-content analysis system to determine the robustness of the model. We show that the ultra-low attachment (ULA) method allows the development of regular, spherical and viable ULA-multicellular aggregates of lymphoma cells (MALC). However, discrepancies in the results obtained after 2D imaging analyses on drug-treated ULA-MALC prompted us to develop 3D imaging and specific analyses. We show by using light sheet microscopy and specifically developed 3D imaging algorithms that 3D imaging and dedicated analyses are necessary to characterize morphological properties of 3D models and drug effects. This study proposes a new method, but also imaging tools and informatic solutions, developed for FL necessary for future preclinical studies.

Single image 3D model retrieval has attracted a lot of attentions with the convenience of organizing large‐scale unlabeled 3D models. Existing methods transfer the knowledge from well‐annotated 2D images (i.e., source domain) to unlabeled 3D models (i.e., target domain) to improve the discriminability of 3D models and align the feature distributions of 2D images and 3D models. However, during the alignment, the feature learning target of improving the discriminability of 3D models sometimes confuses the boundaries between 2D image categories, where prior methods ignore keeping the discriminability of 2D images. Motivated by this observation, we propose a source‐enhanced prototypical alignment framework to first remain the discriminability of 2D images and then guide the category‐level cross‐domain alignment with better image representations. Specifically, a novel separation and compactness loss is proposed for images to separate the samples from different categories and compact the samples within the same category. Then we perform prototypical alignment to make 2D image features assist in the discriminative feature learning for 3D models. We evaluate the proposed method on the commonly used cross‐domain 3D model retrieval benchmarks, namely MI3DOR and MI3DOR‐2, and the results demonstrate the effectiveness of the proposed method.

Content Based Image Retrieval (CBIR) is important in computer aided plant species recognition. Texture and shape information have been the primitive image descriptors in content based image retrieval systems. This paper presents a novel framework for combining both the features, texture and shape information, and achieve higher retrieval efficiency. The study provides a methodology for retrieving medicinal plants images from a database of medicinal plant images based on shape and texture features. The shape descriptors include Zernike moment, Fourier descriptor (FD), Generic fourier Descriptor(GFD) and for texture descriptors gabor filters are used. The similarity measures, euclidean distance of each medicinal plant image from the database to query image is used. The images are sorted based on similarity of Euclidean distance. The retrieval experiments are carried on different training and test medicinal plant images. The effectiveness of different descriptors is confirmed by the experimental results. We have investigated shape and texture features for medicinal plant retrieval by successively combining the different transforms. The retrieval efficiency is reported through precision and recall rate. Experimental results by combining Gabor and Zernike transform outperforms the all other methods.

Region-Based Shape Matching for Automatic Image Annotation and Query-by-Example

The structure of a three-dimensional (3D) model will mutate into various shapes when it does deformation. Indeed, various shapes look different from the original 3D model. What is more, the two-dimensional (2D) image of the deformed 3D model is dissimilar to the 2D image of the undeformed model. Therefore, how to find a proper method to analyze the 3D model similarity always be the research hotspot. In these years, with the wide spread of deep learning technology, the research of similarity and retrieval system of 3D models has set off a new technical revolution. However, the data processing method of the 3D model is distinct with the methods of the 2D image, which is more complex. In consequence, the paper presents the similarity analysis method of 3D models based on convolutional neural networks with threshold (t-CNN). The trained and test datasets are multi-view colored 2D images of 3D models which the color is computed by the heat kernel signature (HKS). Meanwhile, in order to improve the accuracy of retrieval, a threshold is added to the CNN before confirming the final category. The experiments show that the colored dataset construction method proposed in the paper makes the data processing easier and improves the classification precision. Similarity analysis of 3D models based on the t-CNN in recall and precision is better than the other methods.

Moldanet: a network distributed molecular graphics and modelling program that integrates secure signed applet and Java 3D technologies

In order to improve the retrieval accuracy of image retrieval, semantic-based image retrieval has become popular in the recent years. It is because this kind of retrieval method could narrow “semantic gap” between the visual features and the high-level semantic features. However, most of the existing methods of the semantic-based image retrieval are limited to fixed number of semantic features. A dynamic hierarchical semantic network method is proposed in this paper to overcome this limitation. The proposed dynamic hierarchical semantic network is constructed by relevance feedback. The number of semantic features can be dynamically changed according to user feedbacks. Moreover, the semantic features are allowed to have different levels of abstraction. In addition, the proposed method also integrates low-level visual feature-based image retrieval style, which could full use of the advantages of visual feature-based image retrieval and semantic-based image retrieval. Experimental results show that the proposed method achieves higher retrieval accuracy than fixed number of semantic feature method and only one level semantic feature method.

A 3D shape descriptor based on spherical harmonics through evolutionary optimization