A New Shape Benchmark for 3D Object Retrieval

Projective analysis is an important solution in three-dimensional (3D) shape retrieval, since human visual perceptions of 3D shapes rely on various 2D observations from different viewpoints. Although multiple informative and discriminative views are utilized, most projection-based retrieval systems suffer from heavy computational cost, and thus cannot satisfy the basic requirement of scalability for search engines. In the past three years, shape retrieval contest (SHREC) pays much attention to the scalability of 3D shape retrieval algorithms, and organizes several large scale tracks accordingly [1] – [3] . However, the experimental results indicate that conventional algorithms cannot be directly applied to large datasets. In this paper, we present a real-time 3D shape search engine based on the projective images of 3D shapes. The real-time property of our search engine results from the following aspects: 1) efficient projection and view feature extraction using GPU acceleration; 2) the first inverted file, called F-IF, is utilized to speed up the procedure of multiview matching; and 3) the second inverted file, which captures a local distribution of 3D shapes in the feature manifold, is adopted for efficient context-based reranking. As a result, for each query the retrieval task can be finished within one second despite the necessary cost of IO overhead. We name the proposed 3D shape search engine, which combines G PU acceleration and i nverted f ile ( t wice), as GIFT . Besides its high efficiency, GIFT also outperforms state-of-the-art methods significantly in retrieval accuracy on various shape benchmarks (ModelNet40 dataset, ModelNet10 dataset, PSB dataset, McGill dataset) and competitions (SHREC14LSGTB, ShapeNet Core55, WM-SHREC07).

Given the wide application of 3D model analysis, covering domains such as medicine, engineering, and virtual reality, the demand for innovative content-based 3D shape retrieval systems capable of handling complex 3D data efficiently have significantly increased. This paper proposes a new 3D shape retrieval method that uses the CatBoost classifier, a machine learning algorithm, to capture a unique descriptor for each 3D mesh. The main idea of our method is to get a specific and a unique signature or descriptor for each 3D model by training the CatBoost classifier with features obtained directly from the 3D models. This idea not only accelerates the training process, but also ensures the consistency and relevance of the data fed to the classifier during the training process. Once fully trained, the classifier generates a descriptor that is used during the indexing and retrieval process. The efficiency of our method is demonstrated by conducting extensive experiments on the Princeton shape benchmark database. The results demonstrate high retrieval accuracy in comparison to various existing methods in the literature. Our method's ability to outperform these methods shows its potential as highly useful tool in the field of content-based 3D shape retrieval.

A spectral graph wavelet approach for nonrigid 3D shape retrieval

Content-based 3D shape retrieval is an important problem in computer vision. Traditional retrieval interfaces require a 2D sketch or a manually designed 3D model as the query, which is difficult to specify and thus not practical in real applications. With the recent advance in low-cost 3D sensors such as Microsoft Kinect and Intel Realsense, capturing depth images that carry 3D information is fairly simple, making shape retrieval more practical and user-friendly. In this paper, we study the problem of cross-domain 3D shape retrieval using a single depth image from low-cost sensors as the query to search for similar human designed CAD models. We propose a novel method using an ensemble of autoencoders in which each autoencoder is trained to learn a compressed representation of depth views synthesize d from each database object. By viewing each autoencoder as a probabilistic model, a likelihood score can be derived as a similarity measure. A domain adaptation layer is built on top of autoencoder outputs to explicitly address the cross-domain issue (between noisy sensory data and clean 3D models) by incorporating training data of sensor depth images and their category labels in a weakly supennsed learning formulation. Experiments using real-world depth images and a large-scale CAD dataset demonstrate the effectiveness of our approach, which offers significant improvements over state-of-the-art 3D shape retrieval methods.

Linear discrimination dictionary learning for shape descriptors

We study the practical task of fine-grained 3D-VR-sketch-based 3D shape retrieval. This task is of particular interest as 2D sketches were shown to be effective queries for 2D images. However, due to the domain gap, it remains hard to achieve strong performance in 3D shape retrieval from 2D sketches. Recent work demonstrated the advantage of 3D VR sketching on this task. In our work, we focus on the challenge caused by inherent inaccuracies in 3D VR sketches. We observe that retrieval results obtained with a triplet loss with a fixed margin value, commonly used for retrieval tasks, contain many irrelevant shapes and often just one or few with a similar structure to the query. To mitigate this problem, we for the first time draw a connection between adaptive margin values and shape similarities. In particular, we propose to use a triplet loss with an adaptive margin value driven by a ‘fitting gap’, which is the similarity of two shapes under structure-preserving deformations. We also conduct a user study which confirms that this fitting gap is indeed a suitable criterion to evaluate the structural similarity of shapes. Furthermore, we introduce a dataset of 202 VR sketches for 202 3D shapes drawn from memory rather than from observation. The code and data are available at https://github.com/Rowl1ng/Structure-Aware-VR-Sketch-Shape-Retrieval.

Recently, feature-learning-based 3D shape retrieval methods have been receiving more and more attention in the 3D shape analysis community. In these methods, the hand-crafted metrics or the learned linear metrics are usually used to compute the distances between shape features. Since there are complex geometric structural variations with 3D shapes, the single hand-crafted metric or learned linear metric cannot characterize the manifold, where 3D shapes lie well. In this paper, by exploring the nonlinearity of the deep neural network and the complementarity among multiple shape features, we propose a novel deep multimetric network for 3D shape retrieval. The developed multimetric network minimizes a discriminative loss function that, for each type of shape feature, the outputs of the network from the same class are encouraged to be as similar as possible and the outputs from different classes are encouraged to be as dissimilar as possible. Meanwhile, the Hilbert-Schmidt independence criterion is employed to enforce the outputs of different types of shape features to be as complementary as possible. Furthermore, the weights of the learned multiple distance metrics can be adaptively determined in our developed deep metric network. The weighted distance metric is then used as the similarity for shape retrieval. We conduct experiments with the proposed method on the four benchmark shape datasets. Experimental results demonstrate that the proposed method can obtain better performance than the learned deep single metric and outperform the state-of-the-art 3D shape retrieval methods.

In this paper, we present an information‐theoretic framework to compute the shape similarity between 3D polygonal models. Given a 3D model, an information channel between a sphere of viewpoints around the model and its polygonal mesh is defined to compute the specific information associated with each viewpoint. The obtained information sphere can be seen as a shape descriptor of the model. Then, given two models, their similarity is obtained by performing a registration process between the corresponding information spheres. The distance between the information histograms is also defined as a coarse measure of similarity, as well as the scalar value given by the mutual information of the channel. The performance of all these measures is tested using the Princeton Shape Benchmark database. Copyright © 2013 John Wiley & Sons, Ltd.

Retrieval of 3D shapes, especially non-rigid shapes, is attracting growing interest in the research community. In this paper, a new robust feature points extraction method based on two major approaches is presented. The proposed method is proven to be robust when applied to non-rigid 3D shapes as well as highly replicable in different scales. In addition, since partial features provide more detailed information regarding non-rigid 3D shape retrieval, a representation combining the global partial descriptors for 3D shape retrieval is developed and a new similarity measurement is applied to the corresponding parts of different shapes. Finally, the proposed method is implemented using global and partial descriptors with different weights. The results indicate that the proposed method efficiently performed non-rigid 3D shape retrieval.

RJAN: Region‐based joint attention network for 3D shape recognition

3D shape retrieval has attracted much research attention due to its wide applications in the fields of computer vision and multimedia. Various approaches have been proposed in recent years for learning 3D shape descriptor from different modalities. The existing works contain the following disadvantages: 1) the vast majority methods rely on the large scale of training data with clear category information; 2) many approaches focus on the fusion of multi-modal information but ignore the guidance of correlations among different modalities for shape representation learning; 3) many methods pay attention to the structural feature learning of 3D shape but ignore the guidance of structural similarity between every two shapes. To solve these problems, we propose a novel multi-graph network (MGN) for unsupervised 3D shape retrieval, which utilizes the correlations among modalities and structural similarity between two models to guide the shape representation learning process without category information. More specifically, we propose two novel loss functions: auto-correlation loss and cross-correlation loss. The auto-correlation loss utilizes information from different modalities to increase the discrimination of shape descriptor. The cross-correlation loss utilizes the structural similarity between two models to strengthen the intra-class similarity and increase the inter-class distinction. Finally, an effective similarity measurement is designed for the shape retrieval task. To validate the effectiveness of our proposed method, we conduct experiments on the ModelNet dataset. Experimental results demonstrate the effectiveness of our proposed method, and significant improvements have been achieved compared with state-of-the-art methods.

The recent advances of low-cost and mobile depth sensors dramatically extend the potential of 3D shape retrieval and analysis. While the traditional research of 3D retrieval mainly focused on searching by a rough 2D sketch or with a high-quality CAD model, we tackle a novel and challenging problem of cross-domain 3D shape retrieval, in which users can use 3D scans from low-cost depth sensors like Kinect as queries to search CAD models in the database. To cope with the imperfection of user-captured models such as model noise and occlusion, we propose a cross-domain shape retrieval framework, which minimizes the potential function of a Conditional Random Field to efficiently generate the retrieval scores. In particular, the potential function consists of two critical components: one unary potential term provides robust cross-domain partial matching and the other pairwise potential term embeds spatial structures to alleviate the instability from model noise. Both potential components are efficiently estimated using random forests with 3D local features, forming a Regression Tree Field framework. We conduct extensive experiments on two recently released user-captured 3D shape datasets and compare with several state-of-the-art approaches on the cross-domain shape retrieval task. The experimental results demonstrate that our proposed method outperforms the competing methods with a significant performance gain.

Searching for relevant 3D models based on hand-drawn sketches is important for many applications, such as sketch-based 3D modeling and recognition. Sketch-based shape retrieval (SBSR) has become a hot research spot in the field of model retrieval, pattern recognition, and computer vision. 3D deep representation based on Convolutional Neural Network (CNN) enables significant performance improvement over state-of-the-arts in task of 3D shape retrieval. Motivated by this, we proposed a sketch-based 3D model retrieval algorithm by utilizing representative views and CNN feature matching. The representative views are obtained by viewpoint entropy. The main idea of the method is that the hand-drawn sketch can be achieved according to a viewpoint of the 3D model. Thus the sketch and the projection of model from same class are similar. Therefore, we filter a certain amount of view as representative view to reduce the computational complexity and improve the accuracy. We extract CNN descriptors as features for representative view of each object. Our experiments on Shape Retrieval Contest (SHREC) 2012 database and SHREC 2013 database demonstrate that our method is better than state-of-the-art approaches.

With the advent of commodity 3D capturing devices and better 3D modeling tools, 3D shape content is becoming increasingly prevalent. Therefore, the need for shape retrieval algorithms to handle large-scale shape repositories is more and more important. This track provides a benchmark to evaluate large-scale 3D shape retrieval based on the ShapeNet dataset. It is a continuation of the SHREC 2016 large-scale shape retrieval challenge with a goal of measuring progress with recent developments in deep learning methods for shape retrieval. We use ShapeNet Core55, which provides more than 50 thousands models over 55 common categories in total for training and evaluating several algorithms. Eight participating teams have submitted a variety of retrieval methods which were evaluated on several standard information retrieval performance metrics. The approaches vary in terms of the 3D representation, using multi-view projections, point sets, volumetric grids, or traditional 3D shape descriptors. Overall performance on the shape retrieval task has improved significantly compared to the iteration of this competition in SHREC 2016. We release all data, results, and evaluation code for the benefit of the community and to catalyze future research into large-scale 3D shape retrieval (website: https://www.shapenet.org/shrec17).

Moment invariants have been extensively studied and widely used in object recognition. The pioneering investigation of moment invariants in pattern recognition was due to Hu, where a set of moment invariants for similarity transformation were developed using the theory of algebraic invariants. This paper details a comparative analysis on several modifications of the original Hu moment invariants which are used to describe and retrieve two-dimensional (2D) shapes with a single closed contour. The main contribution of this paper is that we propose several different weighting functions to calculate the central moment according to human visual processing. The comparative results are detailed through experimental analysis. The results suggest that the moment invariants improved by weighting functions can get a better retrieval performance than the original one does.