Instance and prototype contrastive learning for multi-view 3D model retrieval and classification

3D model classification has many applications in CAD, 3D object retrieval, and so on. The description of 3D model is crucial but difficult, which leads to the difficulty of classification. The traditional classifier has its limitation in classification of 3D model description. In this paper, we present 3D model classification-based nonparametric discriminant analysis with kernels combined with geometry projection-based histogram model for invariable feature extraction. Firstly, we present nonparametric discriminant analysis with kernels, and secondly, we proposed the invariable feature extraction method with geometry projection-based histogram model. Thirdly, we present the framework of 3D model classification using the proposed nonparametric discriminant analysis with kernels and geometry projection-based histogram model. Finally, we testify the feasibility of the proposed algorithm and performance on 3D model classification. The experimental results show that the proposed scheme is feasible and effective on 3D model classification on the public datasets.

IntroductionExisting multi-view-based 3D model classification methods have the problems of insufficient view refinement feature extraction and poor generalization ability of the network model, which makes it difficult to further improve the classification accuracy. To this end, this paper proposes a multi-view SoftPool attention convolutional network for 3D model classification tasks.MethodsThis method extracts multi-view features through ResNest and adaptive pooling modules, and the extracted features can better represent 3D models. Then, the results of the multi-view feature extraction processed using SoftPool are used as the Query for the self-attentive calculation, which enables the subsequent refinement extraction. We then input the attention scores calculated by Query and Key in the self-attention calculation into the mobile inverted bottleneck convolution, which effectively improves the generalization of the network model. Based on our proposed method, a compact 3D global descriptor is finally generated, achieving a high-accuracy 3D model classification performance.ResultsExperimental results showed that our method achieves 96.96% OA and 95.68% AA on ModelNet40 and 98.57% OA and 98.42% AA on ModelNet10.DiscussionCompared with a multitude of popular methods, our algorithm model achieves the state-of-the-art classification accuracy.

Deep learning based 3D model classification methods have poor effectiveness in fine-grained 3D model classification. Aiming at the problem, an end-to-end fine-grained 3D model classification framework is proposed, and a network based on deep ensemble learning network and context detail awareness module (CDAM) is constructed. Inputting the multiply views of a 3D model, the global shape features are extracted through the deep ensemble learning sub-network. And the local detail features are obtained through the auxiliary sub-network based on CDAM. Based on above two sub-networks, an end-to-end weakly supervision fine-grained 3D model classification network is constructed. Experiments are conducted on three sub-datasets with different levels of difficulties, Airplane, Chair and Car, from the public dataset FG3D. The classification accuracies for above three sub-datasets are 96.31%, 85.44% and 79.62% respectively, which demonstrate the fine classification performance and more generalization of the proposed method.

With the rapid advancement of 3D acquisition technologies, 3D sensors such as LiDARs, 3D scanners, and RGB-D cameras have become increasingly accessible and cost-effective. These sensors generate 3D point cloud data that require efficient algorithms for tasks such as 3D model classification and segmentation. While deep learning techniques have proven effective in these areas, existing models often rely on complex architectures, leading to high computational costs that are impractical for real-time applications like augmented reality and robotics. In this work, we propose the Multi-level Graph Convolutional Neural Network (MLGCN), an ultra-efficient model for 3D point cloud analysis. The MLGCN model utilizes shallow Graph Neural Network (GNN) blocks to extract features at various spatial locality levels, leveraging precomputed KNN graphs shared across GCN blocks. This approach significantly reduces computational overhead and memory usage, making the model well-suited for deployment on low-memory and low-CPU devices. Despite its efficiency, MLGCN achieves competitive performance in object classification and part segmentation tasks, demonstrating results comparable to state-of-the-art models while requiring up to a thousand times fewer floating-point operations and significantly less storage. The contributions of this paper include the introduction of a lightweight, multi-branch graph-based network for 3D shape analysis, the demonstration of the model's efficiency in both computation and storage, and a thorough theoretical and experimental evaluation of the model's performance. We also conduct ablation studies to assess the impact of different branches within the model, providing valuable insights into the role of specific components.

Existing class-incremental learning methods have achieved significant progress in unimodal domains, but they face severe challenges of catastrophic forgetting and cross-modal interference when handling multimodal data. We propose a causal intervention-based multimodal class-incremental learning framework to address the stability-plasticity trade-off in 3D model classification and retrieval tasks. By employing backdoor adjustment-based causal intervention, our approach decouples historical task biases from new knowledge acquisition. It introduces an alternating modality freezing strategy to isolate parameter updates between geometric (point cloud) and visual (multi-view) modalities, thereby suppressing cross-modal gradient interference. Furthermore, a dynamic feature anchoring loss is utilized to constrain feature space drift and maintain cross-task consistency. Experimental results on the ModelNet40 and ShapeNet-Core55 datasets demonstrate that our method achieves state-of-the-art performance compared to competitive baselines, confirming the effectiveness of multimodal redundancy in mitigating unimodal degradation. This work provides theoretical support and technical implementation pathways for lifelong learning systems in complex 3D scenarios.

Recently, 3D model retrieval based on views has become a research hotspot. In this method, 3D models are represented as a collection of 2D projective views, which allows deep learning techniques to be used for 3D model classification and retrieval. However, current methods need improvements in both accuracy and efficiency. To solve these problems, we propose a new 3D model retrieval method, which includes index building and model retrieval. In the index building stage, 3D models in library are projected to generate a large number of views, and then representative views are selected and input into a well-learned convolutional neural network (CNN) to extract features. Next, the features are organized according to their labels to build indexes. In this stage, the views used for representing 3D models are reduced substantially on the premise of keeping enough information of 3D models. This method reduces the number of similarity matching by 87.8%. In retrieval, the 2D views of the input model are classified into a category with the CNN and voting algorithm, and then only the features of one category rather than all categories are chosen to perform similarity matching. In this way, the searching space for retrieval is reduced. In addition, the number of used views for retrieval is gradually increased. Once there is enough evidence to determine a 3D model, the retrieval process will be terminated ahead of time. The variable view matching method further reduces the number of similarity matching by 21.4%. Experiments on the rigid 3D model datasets ModelNet10 and ModelNet40 and the nonrigid 3D model dataset McGill10 show that the proposed method has achieved retrieval accuracy rates of 94%, 92%, and 100%, respectively.

In order to improve the classification effect of the 3D CAD model, this paper combines the knowledge recognition algorithm of convolutional neural network to construct the 3D CAD model classification model. Moreover, this paper analyzes the application of the convolutional neural network algorithm in the classification of 3D CAD models, improves the knowledge recognition algorithm, and builds the basic structure of the convolutional neural network. In addition, this paper builds an intelligent recognition model with the support of the improved algorithm, designs experiments to verify the effect of this paper, and draws a visual display statistical graph based on the statistical test results. Through experimental research, it can be known that the 3D CAD classification model based on the knowledge recognition algorithm of convolutional neural network proposed in this paper can effectively improve the classification effect of the CAD model.

Recently, the researches on 3D model analysis begin to come into focus. With the development of 3D data capturing devices, people can obtain 3D data much easier than before. At present, many existing models for 3D classification are constructed with convolution operator which can benefit from improvements on convolution. Meanwhile, attention mechanisms have demonstrated their utility across quite a few tasks including sequence learning, understanding in images. Some lightweight and general modules based on the attention mechanism are designed to improve the performance of network based on CNN architectures. Inspired by the research on image representations based on the attention mechanism, this paper applies the mechanism to address the problem of 3D shape classification. In this work, a channel attention module named 3DSE block is used to improve the performance of some representative models for 3D point cloud classification. The dependency relationship between the different channels is explicitly modeled to obtain the channel descriptor of the feature map, and it is used to refine the original feature map, so as to sift through the information from the feature and enhance the ability to feature extraction. This paper uses 3DSE module to improve the performance of the existing typical 3D classification networks. Then some experiments are performed on popular shape classification benchmark to test our ideas. The result demonstrates the effectiveness of our method.

Category-specific upright orientation estimation for 3D model classification and retrieval

With the development of 3D model reconstruction, manufacturing, and 3D model vision technologies, 3D model recognition has attracted much attention recently. To handle the 3D model recognition problem, in this paper, we propose a panorama based on multi-channel-attention (MCA) CNN network for the representation of the 3D model. The proposed method is composed of three parts: extracting views, transform function learning, and generating 3D model descriptor. Concretely, we first extract the 2D panoramic views for each 3D model, and we use the multi-channel-attention neural network to extract the descriptor for each 3D model. Here, the attention model is used to find the unequal weights of each panorama view to generate the more robust 3D model descriptor. Finally, The fusion feature is used to handle the 3D model classification and retrieval problem. The popular data sets ModelNet and ShapeNet are used to demonstrate the performance of our approach. The experiments also demonstrate the superiority of our proposed method over the state-of-art methods.

This paper presents an alternative approach on physical human action classification implemented by mobile robots. In contrast with other action recognition methods, this research indicates the best configuration topology of a number of dynamic neural networks to be used in 31) time series classification by showing several comparison performances. In this action recognition investigation we demonstrate high level network granularity on dynamic classification and class discrimination of normal and aggressive action recognition. An interconnection between an ubiquitous 31) sensory tracker system and a mobile robot is set to create a perception to action architecture capable to perceive, process, and classify physical human actions. The robot is used as a process-to-action unit to process the 31) data taken by the tracker and to eventually generate surveillance assessment reports pointing towards action-class matchings as well as generating evaluation statistics which signify the quality of the actions recognized.

A development of 3D construction technology has led to 3D models being applied in many fields, and the number of 3D models has exploded in recent years. Thus, 3D model retrieval has become a popular topic with many proposed approaches. However, all of the methods focus on the 3D model’s global structural descriptor design based on various deep learning networks and ignore the local structural information of the 3D model and the correlation of the local structures. In this paper, we propose a novel 3D model retrieval method based on a 3D shape knowledge graph. We first introduce the concept of a geometric word that can be utilized to assemble other 3D model. Second, we construct a 3D shape knowledge graph based on the geometric words, models and their relations. Additionally, we propose a novel graph embedding method to generate embeddings of nodes. Finally, an effective multiple entities’ retrieval method is used to handle the 3D model retrieval problem. More specifically, the 3D shape knowledge graph retains the basic structural information and saves these as a set of triples. Any 3D model can find its geometric words in a rich enough knowledge graph. It is reasonable that our approach can solve the cross-domain model retrieval problem. Our approach focuses on the structural information of 3D model and is not restricted by the database. We evaluate the proposed method on the ModelNet40 dataset for the 3D model retrieval task. Meanwhile, we also utilize the ShapeNet dataset to evaluate the performance of cross-domain retrieval task. Experimental results and comparisons with state-of-the-art methods demonstrate that our framework can achieve superior performance.

We propose an initial exploration of an interactive machine-learning (IML) dialogue in immersive, interactive Virtual Reality (VR) for the classification of points in 3D point clouds. We contribute ImmersiveIML, an Immersive Analytics tool which builds on human-machine learning trial-and-error dialogue to support an iterative classification process of points in the 3D point cloud. The interactions in ImmersiveIML are designed to be both expressive and minimal; we designed the iterative process to be supported by (1) 6-DOF controllers to allow direct interaction in large 3D point clouds via a minimal brushing technique, (2) a fast trainable machine learning model, and (3) the direct visual feedback of classification results. This constitutes an iterative human-in-the-loop process that eventually converges to a classification according to human intent. We argue that this approach is a novel contribution that supports a constant improvement of the classification model and fast tracks classification tasks with this type of data, in an immersive scenario. We report on the design and implementation of ImmersiveIML and demonstrate its capabilities with two emblematic application scenarios: edge detection in 3D and classification of trees in a city LiDAR dataset.

3D point cloud analysis and processing find numerous applications in computer-aided design, 3D printing, autonomous driving, etc. Most state-of-the-art point cloud processing methods are based on convolutional neural networks (CNNs). Although they outperform traditional methods in terms of accuracy, they demand heavy supervision and higher training complexity. Besides, they lack mathematical transparency. In this talk, I will present three interpretable and effective machine learning methods for 3D point cloud registration, classification and segmentation, respectively. First, an unsupervised registration method that extracts salient points for matching is presented. Second, an unambiguous way to order points sequentially in a point cloud set is developed. Then, their spatial coordinates can be treated as geometric attributes of 1 D data array. This idea facilitates the classification task. Third, for the segmentation task, we show how to leverage prior knowledge on point clouds to derive an intuitive and effective segmentation method. Extensive experiments are conducted to demonstrate the performance of the three new methods. I will also provide performance benchmarking between these interpretable methods and deep learning methods.

An automatic image segmentation for MR brain images based on the gray level characteristics of the images is developed. The method analyses a sequence of MR brain images to provide region information as well as boundary data for classification and eventual creation of 3D models. The system incorporates global information from the image set through an analysis of the statistics of the cooccurrence matrices. Local consistency is then applied with the use of a relaxation algorithm on individual images. The cooccurrence matrices provide conditional probabilities for the classification of pixels into specific regions or boundaries based on the matrix distribution. A constrained stochastic relaxation is then used to refine the probabilistic labels using local image information. Results of the technique are presented for MR brain images.