GIFT: Towards Scalable 3D Shape Retrieval

Recently, content based 3D shape retrieval has been an active area of research. Benchmarking allows researchers to evaluate the quality of results of different 3D shape retrieval approaches. Here, we propose a new publicly available 3D shape benchmark to advance the state of art in 3D shape retrieval. We provide a review of previous and recent benchmarking efforts and then discuss some of the issues and problems involved in developing a benchmark. A detailed description of the new shape benchmark is provided including some of the salient features of this benchmark. In this benchmark, the 3D models are classified mainly according to visual shape similarity but in contrast to other benchmarks, the geometric structure of each model is modified and normalized, with each class in the benchmark sharing the equal number of models to reduce the possible bias in evaluation results. In the end we evaluate several representative algorithms for 3D shape searching on the new benchmark, and a comparison experiment between different shape benchmarks is also conducted to show the reliability of the new benchmark.

Searching for relevant 3D models based on hand-drawn sketches is important for many applications, such as sketch-based 3D modeling and recognition. Sketch-based shape retrieval (SBSR) has become a hot research spot in the field of model retrieval, pattern recognition, and computer vision. 3D deep representation based on Convolutional Neural Network (CNN) enables significant performance improvement over state-of-the-arts in task of 3D shape retrieval. Motivated by this, we proposed a sketch-based 3D model retrieval algorithm by utilizing representative views and CNN feature matching. The representative views are obtained by viewpoint entropy. The main idea of the method is that the hand-drawn sketch can be achieved according to a viewpoint of the 3D model. Thus the sketch and the projection of model from same class are similar. Therefore, we filter a certain amount of view as representative view to reduce the computational complexity and improve the accuracy. We extract CNN descriptors as features for representative view of each object. Our experiments on Shape Retrieval Contest (SHREC) 2012 database and SHREC 2013 database demonstrate that our method is better than state-of-the-art approaches.

A spectral graph wavelet approach for nonrigid 3D shape retrieval

Recently, feature-learning-based 3D shape retrieval methods have been receiving more and more attention in the 3D shape analysis community. In these methods, the hand-crafted metrics or the learned linear metrics are usually used to compute the distances between shape features. Since there are complex geometric structural variations with 3D shapes, the single hand-crafted metric or learned linear metric cannot characterize the manifold, where 3D shapes lie well. In this paper, by exploring the nonlinearity of the deep neural network and the complementarity among multiple shape features, we propose a novel deep multimetric network for 3D shape retrieval. The developed multimetric network minimizes a discriminative loss function that, for each type of shape feature, the outputs of the network from the same class are encouraged to be as similar as possible and the outputs from different classes are encouraged to be as dissimilar as possible. Meanwhile, the Hilbert-Schmidt independence criterion is employed to enforce the outputs of different types of shape features to be as complementary as possible. Furthermore, the weights of the learned multiple distance metrics can be adaptively determined in our developed deep metric network. The weighted distance metric is then used as the similarity for shape retrieval. We conduct experiments with the proposed method on the four benchmark shape datasets. Experimental results demonstrate that the proposed method can obtain better performance than the learned deep single metric and outperform the state-of-the-art 3D shape retrieval methods.

With the advent of commodity 3D capturing devices and better 3D modeling tools, 3D shape content is becoming increasingly prevalent. Therefore, the need for shape retrieval algorithms to handle large-scale shape repositories is more and more important. This track provides a benchmark to evaluate large-scale 3D shape retrieval based on the ShapeNet dataset. It is a continuation of the SHREC 2016 large-scale shape retrieval challenge with a goal of measuring progress with recent developments in deep learning methods for shape retrieval. We use ShapeNet Core55, which provides more than 50 thousands models over 55 common categories in total for training and evaluating several algorithms. Eight participating teams have submitted a variety of retrieval methods which were evaluated on several standard information retrieval performance metrics. The approaches vary in terms of the 3D representation, using multi-view projections, point sets, volumetric grids, or traditional 3D shape descriptors. Overall performance on the shape retrieval task has improved significantly compared to the iteration of this competition in SHREC 2016. We release all data, results, and evaluation code for the benefit of the community and to catalyze future research into large-scale 3D shape retrieval (website: https://www.shapenet.org/shrec17).

Linear discrimination dictionary learning for shape descriptors

3D shape retrieval has attracted much research attention due to its wide applications in the fields of computer vision and multimedia. Various approaches have been proposed in recent years for learning 3D shape descriptor from different modalities. The existing works contain the following disadvantages: 1) the vast majority methods rely on the large scale of training data with clear category information; 2) many approaches focus on the fusion of multi-modal information but ignore the guidance of correlations among different modalities for shape representation learning; 3) many methods pay attention to the structural feature learning of 3D shape but ignore the guidance of structural similarity between every two shapes. To solve these problems, we propose a novel multi-graph network (MGN) for unsupervised 3D shape retrieval, which utilizes the correlations among modalities and structural similarity between two models to guide the shape representation learning process without category information. More specifically, we propose two novel loss functions: auto-correlation loss and cross-correlation loss. The auto-correlation loss utilizes information from different modalities to increase the discrimination of shape descriptor. The cross-correlation loss utilizes the structural similarity between two models to strengthen the intra-class similarity and increase the inter-class distinction. Finally, an effective similarity measurement is designed for the shape retrieval task. To validate the effectiveness of our proposed method, we conduct experiments on the ModelNet dataset. Experimental results demonstrate the effectiveness of our proposed method, and significant improvements have been achieved compared with state-of-the-art methods.

The recent advances of low-cost and mobile depth sensors dramatically extend the potential of 3D shape retrieval and analysis. While the traditional research of 3D retrieval mainly focused on searching by a rough 2D sketch or with a high-quality CAD model, we tackle a novel and challenging problem of cross-domain 3D shape retrieval, in which users can use 3D scans from low-cost depth sensors like Kinect as queries to search CAD models in the database. To cope with the imperfection of user-captured models such as model noise and occlusion, we propose a cross-domain shape retrieval framework, which minimizes the potential function of a Conditional Random Field to efficiently generate the retrieval scores. In particular, the potential function consists of two critical components: one unary potential term provides robust cross-domain partial matching and the other pairwise potential term embeds spatial structures to alleviate the instability from model noise. Both potential components are efficiently estimated using random forests with 3D local features, forming a Regression Tree Field framework. We conduct extensive experiments on two recently released user-captured 3D shape datasets and compare with several state-of-the-art approaches on the cross-domain shape retrieval task. The experimental results demonstrate that our proposed method outperforms the competing methods with a significant performance gain.

Retrieval of 3D shapes, especially non-rigid shapes, is attracting growing interest in the research community. In this paper, a new robust feature points extraction method based on two major approaches is presented. The proposed method is proven to be robust when applied to non-rigid 3D shapes as well as highly replicable in different scales. In addition, since partial features provide more detailed information regarding non-rigid 3D shape retrieval, a representation combining the global partial descriptors for 3D shape retrieval is developed and a new similarity measurement is applied to the corresponding parts of different shapes. Finally, the proposed method is implemented using global and partial descriptors with different weights. The results indicate that the proposed method efficiently performed non-rigid 3D shape retrieval.

RJAN: Region‐based joint attention network for 3D shape recognition

Given the wide application of 3D model analysis, covering domains such as medicine, engineering, and virtual reality, the demand for innovative content-based 3D shape retrieval systems capable of handling complex 3D data efficiently have significantly increased. This paper proposes a new 3D shape retrieval method that uses the CatBoost classifier, a machine learning algorithm, to capture a unique descriptor for each 3D mesh. The main idea of our method is to get a specific and a unique signature or descriptor for each 3D model by training the CatBoost classifier with features obtained directly from the 3D models. This idea not only accelerates the training process, but also ensures the consistency and relevance of the data fed to the classifier during the training process. Once fully trained, the classifier generates a descriptor that is used during the indexing and retrieval process. The efficiency of our method is demonstrated by conducting extensive experiments on the Princeton shape benchmark database. The results demonstrate high retrieval accuracy in comparison to various existing methods in the literature. Our method's ability to outperform these methods shows its potential as highly useful tool in the field of content-based 3D shape retrieval.

Content-based 3D shape retrieval is an important problem in computer vision. Traditional retrieval interfaces require a 2D sketch or a manually designed 3D model as the query, which is difficult to specify and thus not practical in real applications. With the recent advance in low-cost 3D sensors such as Microsoft Kinect and Intel Realsense, capturing depth images that carry 3D information is fairly simple, making shape retrieval more practical and user-friendly. In this paper, we study the problem of cross-domain 3D shape retrieval using a single depth image from low-cost sensors as the query to search for similar human designed CAD models. We propose a novel method using an ensemble of autoencoders in which each autoencoder is trained to learn a compressed representation of depth views synthesize d from each database object. By viewing each autoencoder as a probabilistic model, a likelihood score can be derived as a similarity measure. A domain adaptation layer is built on top of autoencoder outputs to explicitly address the cross-domain issue (between noisy sensory data and clean 3D models) by incorporating training data of sensor depth images and their category labels in a weakly supennsed learning formulation. Experiments using real-world depth images and a large-scale CAD dataset demonstrate the effectiveness of our approach, which offers significant improvements over state-of-the-art 3D shape retrieval methods.

We study the practical task of fine-grained 3D-VR-sketch-based 3D shape retrieval. This task is of particular interest as 2D sketches were shown to be effective queries for 2D images. However, due to the domain gap, it remains hard to achieve strong performance in 3D shape retrieval from 2D sketches. Recent work demonstrated the advantage of 3D VR sketching on this task. In our work, we focus on the challenge caused by inherent inaccuracies in 3D VR sketches. We observe that retrieval results obtained with a triplet loss with a fixed margin value, commonly used for retrieval tasks, contain many irrelevant shapes and often just one or few with a similar structure to the query. To mitigate this problem, we for the first time draw a connection between adaptive margin values and shape similarities. In particular, we propose to use a triplet loss with an adaptive margin value driven by a ‘fitting gap’, which is the similarity of two shapes under structure-preserving deformations. We also conduct a user study which confirms that this fitting gap is indeed a suitable criterion to evaluate the structural similarity of shapes. Furthermore, we introduce a dataset of 202 VR sketches for 202 3D shapes drawn from memory rather than from observation. The code and data are available at https://github.com/Rowl1ng/Structure-Aware-VR-Sketch-Shape-Retrieval.

Rapidly spreading 3D shape applications have led to the development of content-based 3D shape retrieval research. In this paper, we propose a new retrieval method using Spherical Healpix. Spherical Healpix is a new framework for efficient discretization and fast analysis or synthesis of functions defined on the sphere. We analyzed the construction process of this structure and defined a new Spherical Healpix Extent Function. We then analyzed this Spherical Healpix Extent Function using an inverse-construction process from the sphere to the Euclidean plane. We transformed the result of inverse-construction to the frequency domain using a 2D Fourier transform, instead of spherical harmonics, a well-known tool in spherical analysis. We obtained the low-frequency component in the frequency domain by using a Butterworth low-pass filter. The power spectrum of the low frequency component can be used as the feature vector to describe a 3D shape. This descriptor is extracted in the canonical coordinate frame; that is, each 3D-model is first normalized. We have examined this method on the Konstanz Shape Benchmark and SHREC data set, and confirmed its efficiency. We also compared this method with other methods on the same Konstanz Shape Benchmark and SHREC data set and evaluated the shape retrieval performance.

Triplanar convolution with shared 2D kernels for 3D classification and shape retrieval