Cross Diffusion on Multi-hypergraph for Multi-modal 3D Object Recognition

Proceedings of the 2011 joint ACM workshop on Human gesture and behavior understanding

Recognizing 3D objects in the presence of noise, varying mesh resolution, occlusion and clutter is a very challenging task. This paper presents a novel method named Rotational Projection Statistics (RoPS). It has three major modules: local reference frame (LRF) definition, RoPS feature description and 3D object recognition. We propose a novel technique to define the LRF by calculating the scatter matrix of all points lying on the local surface. RoPS feature descriptors are obtained by rotationally projecting the neighboring points of a feature point onto 2D planes and calculating a set of statistics (including low-order central moments and entropy) of the distribution of these projected points. Using the proposed LRF and RoPS descriptor, we present a hierarchical 3D object recognition algorithm. The performance of the proposed LRF, RoPS descriptor and object recognition algorithm was rigorously tested on a number of popular and publicly available datasets. Our proposed techniques exhibited superior performance compared to existing techniques. We also showed that our method is robust with respect to noise and varying mesh resolution. Our RoPS based algorithm achieved recognition rates of 100, 98.9, 95.4 and 96.0 % respectively when tested on the Bologna, UWA, Queen’s and Ca’ Foscari Venezia Datasets.

Abstract3D object recognition has great research and application value in the fields of automatic drive, virtual reality, and commercial manufacturing. Although various deep models have been exploited and achieved remarkable results for 3D object recognition, their computational cost is too high for most mobile applications. This paper combines edge computing and 3D object recognition into a powerful and efficient framework. It consists of a cloud-based rendering stage and a terminal-based recognition stage. In the first stage, inspired by the cloud-based rendering technique, we upload the 3D object data from the mobile device to the edge cloud server for multi-view rendering. The rendering stage utilizes the powerful computing resource in the edge cloud server to generate multiple view images of the given 3D object from different views by parallel high-quality rendering. During the terminal-based recognition stage, we integrate a lightweight CNN architecture and a neural network quantization technique into a 3D object recognition model based on the multiple images rendered in the edge cloud server, which can be executed fast in the mobile device. To reduce the cost of network training, we propose a novel semi-supervised 3D deep learning method with fewer labeled samples. Experiments demonstrate that our method achieves competitive performance compared to the state-of-the-art methods with low latency running in the mobile edge environment.

This paper proposes a method for recognition and classification of 3D objects using 2D moments and HMLP network. The 2D moments are calculated based on 2D intensity images taken from multiple cameras that have been arranged using multiple views technique. 2D moments are commonly used for 2D pattern recognition. However, the current study proves that with some adaptation to multiple views technique, 2D moments are sufficient to model 3D objects. In addition, the simplicity of 2D moment's calculation reduces the processing time for feature extraction, thus decreases the recognition time. The 2D moments were then fed into a neural network for classification of the 3D objects. In the current study, hybrid multi-layered perceptron (HMLP) network is proposed to perform the classification. Two distinct groups of objects that are polyhedral and free-form objects were used to access the performance of the proposed method. The recognition results show that the proposed method has successfully classified the 3D object with the accuracy of up to 100%.

A linha de pesquisa em multimodos e múltiplas representações vem atualmente sendo inspiradora de ações instrucionais na educação científica. Partindo dos fundamentos que justificam um encaminhamento didático à luz dessas referências, este trabalho procura mostrar que há compatibilidade dos seus fundamentos com a teoria da aprendizagem significativa de Ausubel e com as questões levantadas pelas pesquisas que indicam a necessidade de se considerar a subjetividade dos alunos presentes numa sala de aula. Essencialmente, procuramos argumentar que a promoção de um ensino por meio de multimodos e múltiplas representações é consistente com o ambiente plural das subjetividades existentes numa sala de aula e com uma aprendizagem significativa.

Recognizing 3D object has attracted plenty of attention recently, and view-based methods have achieved best results until now. However, previous view-based methods ignore the region-to-region and view-to-view relationships between different view images, which are crucial for multi-view 3D object representation. To tackle this problem, we propose a Relation Network to effectively connect corresponding regions from different viewpoints, and therefore reinforce the information of individual view image. In addition, the Relation Network exploits the inter-relationships over a group of views, and integrates those views to obtain a discriminative 3D object representation. Systematic experiments conducted on ModelNet dataset demonstrate the effectiveness of our proposed methods for both 3D object recognition and retrieval tasks.

For many practical applications in industrial and medical fields, 3D object recognition has become an actively investigative field. In general, 3D object recognition system can be completed through two stages. Firstly, we use 3D reconstruction of input data to get object expression. Then we use 3D invariant to identify objects. In order to promote further development of this field, this paper summarizes the 3D object recognition technology research, and introduces typical methods of modern 3D object recognition, the concepts of 3D reconstruction, 3D invariants. In the process of 3D object recognition, feature line extraction, 3D invariant calculation, matching strategy, 3D object recognition fields are reviewed. The first section presents the significance of 3D object recognition. The second section presents the development of object recognition. The third section presents the double viewpoint of 3D reconstruction. The fourth section presents the 3D reconstruction of image processing. The fifth section presents the 3D invariant technology. The sixth section presents the problems and development direction.

Local features with invariant descriptions are important for many tasks in image processing and computer vision. This paper presents a new local feature descriptor for 3D object and scene representation. The new descriptor, named 3DSSIM, explores the internal geometric property of layout similarity of 3D objects to produce efficient feature representation. The 3D-SSIM is highly distinctive, quick to compute, and shows superior advantages in terms of robustness to noises, invariance to viewpoints, and tolerance to geometric distortions. We extensively evaluated performance of the new descriptor with various datasets.

Owing to the wide spread of the 3D objects technologies, learning 3D objects labeling and segmentation is becoming one of the most provocative tasks in computer vision and digital multimedia. Recently, convolutional neural network (CNN) has proved itself as a powerful model in segmentation and classification by giving excellent performances. The use of a graphical model such as conditional random field (CRF) contributes further in capturing contextual information and thus improving the segmentation performance. In this paper, we implement a deep architecture in order to segment and label 3D shape parts by considering both spectral and geometric features via a combined framework consisting of a CNN and CRF models. First, low-level features are used to learn deep features using a CNN model, and then formulate the deep CRF model with a CNN-based unary and pairwise potential functions to effectively extract the semantic correlations between adjacent triangles on the mesh. By comparing our results with those from several state-of-the-art, our method shows promising potentials.

Object detection is a major branch and fundamental task in computer vision, aiming to localize, identify and classify even the smallest objects of interest in images. Features can be extracted efficiently by deep convolutional neural networks (CNNs) as the backbone for real-time or near real-time object detection performance than the hand-crafted-based traditional methods. In the past few years, the advent of transformer-based models with robust self-attention mechanisms has not only raised object detection performance to a higher level but has also enabled it to produce excellent results. Many object detection tasks in the real world require that 3D information about the object be obtained, thus strengthening active research in 3D object detection. However, the algorithms for detecting 3D objects are not easy to propagate in real-world applications due to many factors, making reconstruction of 2D object detection algorithms to 3D object detection algorithms the suitable alternative. Therefore, we review the evolution of 2D object detection algorithms for digital imaging applications, focusing on their developments, models, applications, datasets, evaluation metrics, strengths and weaknesses, for better understanding of their landmarks and contributions to the advancement of the field.

Occluded objects detection is a challenge task in computer vision. To address this problem, this paper proposes an effective light field imaging system for occluded objects 3D detection, which integrates digital refocus methods to imaging occluded objects and deep learning based method to located objects position with defocus clues. Camera arrays based integral imaging system could provide focal stacks images, which makes occluded objects more clear and attenuates foreground occlusion. With observation that recognition probability are related to objects clarity, as well as focal length of images, recognition probability based defocus clues are proposed to located objects depth. Hierarchical object localization process is applied on refocus images stacks to coarsely located object depth by detected probabilities, following gradient based fine-grained defocus response process could further refine the depth accuracy. With the depths from defocus clues and detected locations from neural model, proposed algorithm could achieve 3D object detection under partial occlusion. Furthermore, a parallel computation framework is proposed to accelerate whole detection process. Real experiments show the robust performance of proposed 3D occluded objects detection algorithm.

This paper proposes an effective method for recognition and classification of 3D objects using multiple views technique and neural networks system. In the processing stage, we propose to use 2D moment invariants as the features for modeling 3D objects. 2D moments have been commonly used for 2D object recognition. However, we have proved that with some adaptation to multiple views technique, 2D moments are sufficient to model 3D objects. In addition, the simplicity of 2D moments calculation reduces the processing time for feature extraction, hence increases the system efficiency. In the recognition stage, we propose a cascaded multilayered perceptron (c-MLP) network for matching and classification. The c-MLP contains two MLP networks which are arranged in a serial combination. This proposed method has been tested using two groups of object, polyhedral and free-form objects. We also compare our method with standard MLP network. Our results show that the proposed method can successfully be applied to 3D object recognition. In addition, the proposed network also achieved better performance and faster convergence rate compared to the than standard MLP

In order to develop human friendly robots, we develop a robotic system with a vision system, robot arm and voice recognition system, capable of grasping an object for which a human asks by voice. In this paper we present the 3D object recognition method by silhouette images in the first place. We propose the method to construct 3D object models by silhouette images and use them for making templates in order to undertake the template matching for 3D object recognition by silhouette images. We find that very high correct recognition rate is achieved by this method. We then employ a 4-DOF robot arm with a gripper for grasping the object. We present the method to approach and grasp the object by cooperating with the movement of arm and image obtained from the camera mounted on the tip of the arm. The experiment shows that recognition of the object, approach and grasping it are well undertaken.

Three-Dimensional (3D) object recognition has been receiving more and more attention in the field of Computer Vision, as the development of AI algorithms and advancements of hardware components allow for better results nowadays more than ever before. Neural Networks are used to better classify different types of objects, these are being fed with pre-processed 3D objects represented in a 2D domain for better highlighting the important features and for a better performance. In this paper, we propose a strategy of feeding 2D representations of 3D meshes made by using different types of slicing to a CNN. In this way, we want to minimize the information loss and the redundancy the data suffers going from a 3D representation to a 2D one. To assess the results, we used Princeton's state-of-the-art ModelNet10 dataset, consisting of 4899 3D objects divided into 10 categories.

To improve the performance of appearance-based three-dimensional object recognition system, we propose to extract Hupsilas moment invariants, affine moment invariants and color moments from the 2D images of 3D objects. Hupsilas and affine moment invariants have the properties of rotation, scale, translation invariance and affine transformation invariance respectively for the objects in images, and color moments are used to distinguish objects of similar shape and different color. Then these moments compose a 1-dimensional feature vector of 11 components for each 2D image of 3D objects and presented to the BP neural network for training. The trained network can be used to recognize 3D objects when provided the feature vectors of unseen views. We assessed our method based on both the original and noise corrupted COIL-100 3D objects dataset and achieved 100% correct rate of recognition when training images were presented every 10 degrees.