Efficient 3D Point Cloud Feature Learning for Large-Scale Place Recognition.

This paper concerns on overcoming the challenges caused by drastic appearance change in visual tracking, especially the long-term appearance variation due to occlusion or large object deformation. We aim to build a long-term appearance model for robust tracking against large appearance change in two new respects: using historical and distinguishing cues to model target representation and extracting effective spatial objectness features from each frame to distinguish outliers. For the first purpose, an adaptive superpixel-based appearance model is formulated. Different from previous superpixel-based trackers, a complementary feature set is defined for the update model to preserve the features of those temporally disappeared object parts especially under occlusion and large deformation. For the second purpose, three new spatial objectness cues specially designed for tracking are defined, including surrounding comparison, edge density change and weighted superpixel straddling. With these spatial objectness cues, our method facilitates target object localization and ensures the target has similar edge distribution between adjacent frames. These cues greatly improve the ability of our method to distinguish the target from its surrounding background. The adaptive appearance model retains valuable features of historical results, and the spatial objectness cues are extracted from the current frame, and thus they are finally combined to complement with each other to solve large appearance changes. The extensive evaluations on the CVPR 2013 online object tracking benchmark and VOT 2014 datasets demonstrate the effectiveness of our method as compared with related trackers.

As a fundamental yet challenging problem in intelligent transportation systems, point cloud registration attracts vast attention and has been attained with various deep learning-based algorithms. The unsupervised registration algorithms take advantage of deep neural network-enabled novel representation learning while requiring no human annotations, making them applicable to industrial applications. However, unsupervised methods mainly depend on global descriptors, which ignore the high-level representations of local geometries. In this paper, we propose to jointly use both global and local descriptors to register point clouds in a self-supervised manner, which is motivated by a critical observation that all local geometries of point clouds are transformed consistently under the same transformation. Therefore, local geometries can be employed to enhance the representation ability of the feature extraction module. Moreover, the proposed local descriptor is flexible and can be integrated into most existing registration methods and improve their performance. Besides, we also utilize point cloud reconstruction and normal estimation to enhance the transformation awareness of global and local descriptors. Lastly, extensive experimental results on one synthetic and three real-world datasets demonstrate that our method outperforms existing state-of-art unsupervised registration methods and even surpasses supervised ones in some cases. Robustness and computational efficiency evaluations also indicate that the proposed method applies to intelligent vehicles.

The current research aimed to study the strategies that people employ in order to become more desirable as mates in different cultural settings. More specifically, using a closed‐ended questionnaire on a sample of 7181 participants from 14 different countries, we identified 10 different strategies that people employ to become more appealing as mates. Participants indicated that they had more frequently used the “Enhance looks,” followed by the “Show off abilities and talents,” and the “Demonstrate similarity” strategies. On the other hand, they had less frequently used the “Keep undesirable things hidden,” the “Show off and exaggerate wealth and abilities,” and the “Drastic appearance changes” strategies. Female participants indicated that they had more extensive used the “Enhance looks” strategy than male participants, while male participants indicated that they had more extensive used the “Increase income and social status” and the “Show off and exaggerate wealth and abilities” strategies than female participants. The sex effects, as well as the extent of use, were generally consistent across the different cultures. The identified strategies were classified further into two main strategies, namely the “Develop and demonstrate desirable traits” and the “Deceive about undesirable traits,” which was generally consistent across the different countries.

Existing Siamese network based trackers are easily disturbed by large deformation, occlusion and distractor objects in the background. By comparing these trackers, we observe that the monotonous positive pairs usually have limited challenging factors (Occlusion, Deformation, etc.), which may make the learned features less robust. In addition, the foreground information of the substantial training data is utilized directly without deeper exploration. Thus, the trackers cannot effectively discriminate the foreground from the semantic backgrounds. In this paper, we focus on modifying the Siamese tracker by enriching the positive pairs and taking further advantage of the foreground information. During the offline training phase, a simple sampling strategy is adopted to enrich the challenging factors in positive pairs, which can effectively enhance the robustness of the tracker. At the same time, we highlight the foreground information by padding the background, and the information is utilized to generate a novel padding loss, which guides the tracker to pay less attention to the distractors in the background. Moreover, an improved feature information fusion is adopted to update the template, so that the tracker can adapt to the drastic appearance changes. Comprehensive experiments on the OTB and the VOT benchmarks demonstrate that our proposed tracker can achieve outstanding performance in both accuracy and robustness.

SC-GAN: Subspace Clustering based GAN for Automatic Expression Manipulation

This study proposes a framework for describing a scene change using natural language text based on indoor scene observations conducted before and after a scene change. The recognition of scene changes plays an essential role in a variety of real-world applications, such as scene anomaly detection. Most scene understanding research has focused on static scenes. Most existing scene change captioning methods detect scene changes from single-view RGB images, neglecting the underlying three-dimensional structures. Previous three-dimensional scene change captioning methods use simulated scenes consisting of geometry primitives, making it unsuitable for real-world applications. To solve these problems, we automatically generated large-scale indoor scene change caption datasets. We propose an end-to-end framework for describing scene changes from various input modalities, namely, RGB images, depth images, and point cloud data, which are available in most robot applications. We conducted experiments with various input modalities and models and evaluated model performance using datasets with various levels of complexity. Experimental results show that the models that combine RGB images and point cloud data as input achieve high performance in sentence generation and caption correctness and are robust for change type understanding for datasets with high complexity. The developed datasets and models contribute to the study of indoor scene change understanding.

Surface reconstruction from point clouds is vital for 3D computer vision. State-of-the-art methods leverage large datasets to first learn local context priors that are represented as neural network-based signed distance functions (SDFs) with some parameters encoding the local contexts. To reconstruct a surface at a specific query location at inference time, these methods then match the local reconstruction target by searching for the best match in the local prior space (by optimizing the parameters encoding the local context) at the given query location. However, this requires the local context prior to generalize to a wide variety of unseen target regions, which is hard to achieve. To resolve this issue, we introduce Predictive Context Priors by learning Predictive Queries for each specific point cloud at inference time. Specifically, we first train a local context prior using a large point cloud dataset similar to previous techniques. For surface reconstruction at inference time, however, we specialize the local context prior into our Predictive Context Prior by learning Predictive Queries, which predict adjusted spatial query locations as displacements of the original locations. This leads to a global SDF that fits the specific point cloud the best. Intuitively, the query prediction enables us to flexibly search the learned local context prior over the entire prior space, rather than being restricted to the fixed query locations, and this improves the generalizability. Our method does not require ground truth signed distances, normals, or any additional procedure of signed distance fusion across overlapping regions. Our experimental results in surface reconstruction for single shapes or complex scenes show significant improvements over the state-of-the-art under widely used benchmark-s. Code and data are available at https://github.com/mabaorui/PredictableContextPrior.

SAANet: Spatial adaptive alignment network for object detection in automatic driving

Place recognition gives a SLAM system the ability to correct cumulative errors. Unlike images that contain rich texture features, point clouds are almost pure geometric information which makes place recognition based on point clouds challenging. Existing works usually encode low-level features such as coordinate, normal, reflection intensity, etc., as local or global descriptors to represent scenes. Besides, they often ignore the translation between point clouds when matching descriptors. Different from most existing methods, we explore the use of high-level features, namely semantics, to improve the descriptor’s representation ability. Also, when matching descriptors, we try to correct the translation between point clouds to improve accuracy. Concretely, we propose a novel global descriptor, Semantic Scan Context, which explores semantic information to represent scenes more effectively. We also present a two-step global semantic ICP to obtain the 3D pose (x, y, yaw) used to align the point cloud to improve matching performance. Our experiments on the KITTI dataset show that our approach outperforms the state-of-the- art methods with a large margin. Our code is available at: https://github.com/lilin-hitcrt/SSC.

3D object detection from raw and sparse point clouds has been far less treated to date, compared with its 2D counterpart. In this paper, we propose a novel framework called FVNet for 3D front-view proposal generation and object detection from point clouds. It consists of two stages: generation of front-view proposals and estimation of 3D bounding box parameters. Instead of generating proposals from camera images or bird's-eye-view maps, we first project point clouds onto a cylindrical surface to generate front-view feature maps which retains rich information. We then introduce a proposal generation network to predict 3D region proposals from the generated maps and further extrude objects of interest from the whole point cloud. Finally, we present another network to extract the point-wise features from the extruded object points and regress the final 3D bounding box parameters in the canonical coordinates. Our framework achieves real-time performance with 12ms per point cloud sample. Extensive experiments on the 3D detection benchmark KITTI show that the proposed architecture outperforms state-of-the-art techniques which take either camera images or point clouds as input, in terms of accuracy and inference time.

Familiar face recognition is robust to subtle and drastic changes in appearance. Knowing which conditions harm our recognition highlights underlying processes that have prominent roles in face learning. Here, we focused on two image deformations that studies suggest independently harm recognition: contrast negation and stretching of top or bottom halves of a face orthogonal to the unstretched half (nonglobal stretching). Participants were asked to categorize self-reported familiar or unfamiliar faces presented in photographic positive and negative in a fully within-subjects design. In Experiments 1 and 2, recognition of contrast-positive faces was robust to global and nonglobal stretching, suggesting iso-dimension ratios do not have a role in familiar face recognition. However, performance was consistently impaired by contrast negation in all configurational conditions. Further reductions in categorization accuracy when top halves of contrast-negated faces are stretched suggest some limited role for configuration under these image conditions. In Experiment 3, presenting the top or bottom half of nonglobal stretch conditions suggested categorization of nonglobal stretch faces did not require perception of the whole face, in the research design reported here. These results highlight further limits to configurational accounts of face recognition and indicate a relatively important role for surface reflectance cues.

Robust object tracking using semi-supervised appearance dictionary learning

Point cloud is the popular researches in the industry, and the research on partial point cloud is favored by many researchers. In the acquisition process of original 3D point cloud data, there are partial or sparse problems due to occlusion, lighting and other reasons, which will lead to deviation in downstream tasks. We propose a network MDPCN using multiple decoders, which can make the partial point cloud complete. The network uses deep learning to get multiple features of the partial point cloud, and uses multiple identical decoders to decode. Each decoder uses the self-attention module and the Folding-Net to generate smooth point clouds from the features, and finally integrates the point clouds generated by each decoder to obtain dense point cloud. Compared with other mainstream point cloud completion network methods and ablation experiments, it can be proved that this network model can generate more accurate point cloud and effectively complete the 3D reconstruction task.

Nowadays, point clouds are frequently gathered by 3D scanners such as Lidar and Kinect, which produces thousands of point cloud models. Point cloud processing is vital to 3D vision, especially 3D object recognition, positioning, and navigation technology. Addressing uneven data density caused by coordinate frame transformations and the inherent problem of insufficient context connection in point clouds, the DiM-PCNet (Multi-scale and Multi-level Point Clouds Classification Network, and the Di is a prefix to represent double M in Multi-scale and Multi-level) is proposed in this paper. DiM-PCNet is provided for object classification with multi-scale and multi-level features. We encode the point cloud in multi-scale and fully fuse the features with the raw point cloud for keeping the context relationship. In DiM-PCNet, we sample the point clouds from eight parts for multi-scales feature extraction. The multi-scales features are fully fused by multi-level pyramid models. The multi-scale and multi-level strategies are applied in DiM-PCNet, in which the abundant and important features of point clouds are extracted and utilized in the 3D object classification. It is worth noting that the DiM-PCNet feature block can be embedded into the segmentation net, where the accuracy achieved is 87.1%. We conducted experiments on ShapeNet and ModelNet40 and the experimental results show that DiM-PCNet achieves state-of-the-art performance in 3D object classification. The experiment shows competitive performance on robustness and segmentation tasks.

The paper presents a learning-based method for computing a discriminative 3D point cloud descriptor for place recognition purposes. Existing methods, such as Point-NetVLAD, are based on unordered point cloud representation. They use PointNet as the first processing step to extract local features, which are later aggregated into a global descriptor. The PointNet architecture is not well suited to capture local geometric structures. Thus, state-of-the-art methods enhance vanilla PointNet architecture by adding different mechanism to capture local contextual information, such as graph convolutional networks or using hand-crafted features. We present an alternative approach, dubbed Min-kLoc3D, to compute a discriminative 3D point cloud descriptor, based on a sparse voxelized point cloud representation and sparse 3D convolutions. The proposed method has a simple and efficient architecture. Evaluation on standard benchmarks proves that MinkLoc3D outperforms current state-of-the-art. Our code is publicly available on the project website. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>