Manually labelling point cloud scenes for use as training data in machine learning applications is a time- and labour-intensive task. In this paper, we aim to reduce the effort associated with learning semantic segmentation tasks by introducing a semi-supervised method that operates on scenes with only a small number of labelled points. For this task, we advocate the use of pseudo-labelling in combination with PointNet, a neural network architecture for point cloud classification and segmentation. We also introduce a method for incorporating information derived from spatial relationships to aid in the pseudo-labelling process. This approach has practical advantages over current methods by working directly on point clouds and not being reliant on predefined features. Moreover, we demonstrate competitive performance on scenes from three publicly available datasets and provide studies on parameter sensitivity.

Commercial Unmanned aerial vehicle (UAV) industry, which is publicly known as drone, has seen a tremendous increase in last few years, making these devices highly accessible to public. This phenomenon has immediately raised security concerns due to fact that these devices can intentionally or unintentionally cause serious hazards. In order to protect critical locations, the academia and industry have proposed several solutions in recent years. Computer vision is extensively used to detect drones autonomously compared to other proposed solutions such as RADAR, acoustics and RF signal analysis thanks to its robustness. Among these computer vision-based approaches, we see the preference of deep learning algorithms thanks to their effectiveness. In this paper, we are presenting an autonomous drone detection and tracking system which uses a static wide-angle camera and a lower-angle camera mounted on a rotating turret. In order to use memory and time efficiently, we propose a combined multi-frame deep learning detection technique, where the frame coming from the zoomed camera on the turret is overlaid on the wide-angle static camera’s frame. With this approach, we are able to build an efficient pipeline where the initial detection of small sized aerial intruders on the main image plane and their detection on the zoomed image plane is performed simultaneously, minimizing the cost of resource exhaustive detection algorithm. In addition to this, we present the integral system including tracking algorithms, deep learning classification architectures and the protocols.

Motion information can be important for detecting objects, but it has been used less for pedestrian detection, particularly with deep-learning-based methods. We propose a method that uses deep motion features as well as deep still-image features, following the success of two-stream convolutional networks, each of which are trained separately for spatial and temporal streams. To extract motion clues for detection differentiated from other background motions, the temporal stream takes as input the difference in frames that are weakly stabilized by optical flow. To make the networks applicable to bounding-box-level detection, the mid-level features are concatenated and combined with a sliding-window detector. We also introduce transfer learning from multiple sources in the two-stream networks, which can transfer still image and motion features from ImageNet and an action recognition dataset respectively, to overcome the insufficiency of training data for convolutional neural networks in pedestrian datasets. We conducted an evaluation on two popular large-scale pedestrian benchmarks, namely the Caltech Pedestrian Detection Benchmark and Daimler Mono Pedestrian Detection Benchmark. We observed 10% improvement compared to the same method but without motion features.

We present a simple multi-scale learning network for image classification that is inspired by the MobileNet. The proposed method has two advantages: (1) It uses the multi-scale block with depthwise separable convolutions, which forms multiple sub-networks by increasing the width of the network while keeping the computational resources constant. (2) It combines the multi-scale block with residual connections and that accelerates the training of networks significantly. The experimental results show that the proposed method has strong performance compared to other popular models on different datasets.

The past 20 years has seen a progressive evolution of computer vision algorithms for unsupervised 2D image segmentation. While earlier efforts relied on Markov random fields and efficient optimization (graph cuts, etc.), the next wave of methods beginning in the early part of this century were, in the main, stovepiped. Of these 2D segmentation efforts, one of the most popular and, indeed, one that comes close to being a state of the art method is the ultrametric contour map (UCM). The pipelined methodology consists of (i) computing local, oriented responses, (ii) graph creation, (iii) eigenvector computation (globalization), (iv) integration of local and global information, (v) contour extraction, and (vi) superpixel hierarchy construction. UCM performs well on a range of 2D tasks. Consequently, it is somewhat surprising that no 3D version of UCM exists at the present time. To address that lack, we present a novel 3D supervoxel segmentation method, dubbed 3D UCM, which closely follows its 2D counterpart while adding 3D relevant features. The methodology, driven by supervoxel extraction, combines local and global gradient-based features together to first produce a low-level supervoxel graph. Subsequently, an agglomerative approach is used to group supervoxel structures into a segmentation hierarchy with explicitly imposed containment of lower-level supervoxels in higher-level supervoxels. Comparisons are conducted against state of the art 3D segmentation algorithms. The considered applications are 3D spatial and 2D spatiotemporal segmentation scenarios. For the latter comparisons, we present results of 3D UCM with and without optical flow video pre-processing. As expected, when motion correction beyond a certain range is required, we demonstrate that 3D UCM in conjunction with optical flow is a very useful addition to the pantheon of video segmentation methods.

In this paper, we describe the world’s largest gait database with real-life carried objects (COs), which has been made publicly available for research purposes, and its application to the performance evaluation of vision-based gait recognition. Whereas existing databases for gait recognition include at most 4007 subjects, we constructed an extremely large-scale gait database that includes 62,528 subjects, with an equal distribution of males and females, and ages ranging from 2 to 95 years old. Moreover, whereas existing gait databases consider a few predefined CO positions on a subject’s body, we constructed a database that contained unconstrained variations of COs being carried in unconstrained positions. Additionally, gait samples were manually classified into seven carrying status (CS) labels. The extremely large-scale gait database enabled us to evaluate recognition performance under cooperative and uncooperative settings, the impact of the training data size, the recognition difficulty level of the CS labels, and the possibility of the classification of CS labels. Particularly, the latter two performance evaluations have not been investigated in previous gait recognition studies.

This paper describes the world’s largest gait database with wide view variation, the “OU-ISIR gait database, multi-view large population dataset (OU-MVLP)”, and its application to a statistically reliable performance evaluation of vision-based cross-view gait recognition. Specifically, we construct a gait dataset that includes 10,307 subjects (5114 males and 5193 females) from 14 view angles ranging 0° −90°, 180° −270°.In addition, we evaluate various approaches to gait recognition which are robust against view angles. By using our dataset, we can fully exploit a state-of-the-art method requiring a large number of training samples, e.g., CNN-based cross-view gait recognition method, and we validate effectiveness of such a family of the methods.

In this paper, we propose a saliency detection model for RGB-D images based on the deep features of RGB images and depth images within a Bayesian framework. By analysing 3D saliency in the case of RGB images and depth images, the class-conditional mutual information is computed for measuring the dependence of deep features extracted using a convolutional neural network; then, the posterior probability of the RGB-D saliency is formulated by applying Bayes’ theorem. By assuming that deep features are Gaussian distributions, a discriminative mixed-membership naive Bayes (DMNB) model is used to calculate the final saliency map. The Gaussian distribution parameters can be estimated in the DMNB model by using a variational inference-based expectation maximization algorithm. The experimental results on RGB-D images from the NLPR dataset and NJU-DS400 dataset show that the proposed model performs better than other existing models.

In this paper, we describe the world’s largest gait database, the “OU-ISIR Gait Database, Large Population Dataset with Age (OULP-Age)” and its application to a statistically reliable performance evaluation of gait-based age estimation. Whereas existing gait databases include only 4016 subjects at most, we constructed an extremely large-scale gait database that includes 63,846 subjects (31,093 males and 32,753 females) with ages ranging from 2 to 90 years old. Benchmark algorithms of gait-based age estimation were then implemented to evaluate statistically significant performance differences. Additionally, the dependence of gait-based age estimation performance on gender and age group, in addition to the number of training subjects, was investigated to provide several insights for future research on the topic.

This paper presents a novel method for pair-wise range image registration, a backbone task in world modeling, parts inspection and manufacture, object recognition, pose estimation, robotic navigation, and reverse engineering. The method finds the most suitable homogeneous transformation matrix between two constructed range images to create a more complete 3D view of a scene. The proposed solution integrates a ray casting-based fitness estimation with a global optimization method called improved self-adaptive differential evolution. This method eliminates the fine registration steps of the well-known iterative closest point (ICP) algorithm used in previously proposed methods, and thus, is the first direct global registration algorithm. With its parallel implementation potential, the ray casting-based algorithm speeds up the fitness calculation for the global optimization method, which effectively exploits the search space to find the best transformation solution. The integration was successfully implemented in a parallel paradigm on a multi-core computer processor to solve a simultaneous 3D localization problem. The fast, accurate, and robust results show that the proposed algorithm significantly improves on the registration problem over state-of-the-art algorithms.