SSM-HPC: Front View Gait Recognition Using Spherical Space Model with Human Point Clouds

In complex industrial human-robot collaboration (HRC) environment, obstacles in the shared working space will occlude the operator, and the industrial robot will threaten the safety of the operator if it is unable to get the complete human spatial point cloud. This paper proposes a real-time human point cloud inpainting method based on the deep generative model. The method can recover the human point cloud occluded by obstacles in the shared working space to ensure the safety of the operator. The method proposed in this paper can be mainly divided into three parts: (i) real-time obstacles detection. This process can detect obstacle locations in real time and generate the image of obstacles. (ii) the application of the deep generative model algorithm. It is a complete convolutional neural network (CNN) structure and introduces advanced generative adversarial loss. The model can generate the missing depth data of operators at arbitrary position in the human depth image. (iii) spatial mapping of the depth image. The depth image will be mapped to point cloud by coordinate system conversion. The effectiveness of the method is verified by filling hole of the human point cloud occluded by obstacles in industrial HRC environment. The experiment results show that the proposed method can accurately generate the occluded human point cloud in real time and ensure the safety of the operator.

Rapidly development in the realm of biometrics in recent years results in the unprecedented growth with the number of researchers and interesters. As one of the biometrics, gait recognition has many advantages such as from a distance, lower quality video, hard to disguised comparing with others. However, nearly all studies on gait recognition are 2D methods based on analysis of image sequences captured by a monocular camera. This paper presents a novel 3D method for automatic gait recognition by analyzing the changes of a 1D silhouette signal. Binarized silhouette of a moving individual is firstly achieved. Secondly contour is extracted for next stereo matching. Afterward 3D contour is obtained through a step of contour based stereo matchin. Then, stereo silhouette vector (SSV) is extracted from 3D contour and transformed to a 1D silhouette signal as the stereo gait feature (SGF). K-L transform is applied for reducing the dimensionality of feature and the complexity of computing. Finally, kNN and NN are applied to gait classification and recognition. A stereo gait database named PRLAB is established for gait recognition based on stereo vision. Experimental results on PRLAB showed the efficiency and robustness of our algorithm.

Most existing gait recognition approaches adopt a two-step procedure: a preprocessing step to extract silhouettes or skeletons followed by recognition. In this paper, we propose an end-to-end model-based gait recognition method. Specifically, we employ a skinned multi-person linear (SMPL) model for human modeling, and estimate its parameters using a pre-trained human mesh recovery (HMR) network. As the pre-trained HMR is not recognition-oriented, we fine-tune it in an end-to-end gait recognition framework. To cope with differences between gait datasets and those used for pre-training the HMR, we introduce a reconstruction loss between the silhouette masks in the gait datasets and the rendered silhouettes from the estimated SMPL model produced by a differentiable renderer. This enables us to adapt the HMR to the gait dataset without supervision using the ground-truth joint locations. Experimental results with the OU-MVLP and CASIA-B datasets demonstrate the state-of-the-art performance of the proposed method for both gait identification and verification scenarios, a direct consequence of the explicitly disentangled pose and shape features produced by the proposed end-to-end model-based framework.

Gait recognition is one of the most important biometric technologies and has been applied in many fields. Recent gait recognition frameworks represent each gait frame by descriptors extracted from either global appearances or local regions of humans. However, the representations based on global information often neglect the details of the gait frame, while local region based descriptors cannot capture the relations among neighboring regions, thus reducing their discriminativeness. In this paper, we propose a novel feature extraction and fusion framework to achieve discriminative feature representations for gait recognition. Towards this goal, we take advantage of both global visual information and local region details and develop a Global and Local Feature Extractor (GLFE). Specifically, our GLFE module is composed of our newly designed multiple global and local convolutional layers (GLConv) to ensemble global and local features in a principle manner. Furthermore, we present a novel operation, namely Local Temporal Aggregation (LTA), to further preserve the spatial information by reducing the temporal resolution to obtain higher spatial resolution. With the help of our GLFE and LTA, our method significantly improves the discriminativeness of our visual features, thus improving the gait recognition performance. Extensive experiments demonstrate that our proposed method outperforms state-of-the-art gait recognition methods on two popular datasets.

In recent years, due to the significant advancements in hardware sensors and software technologies, 3D environmental point cloud modeling has gradually been applied in the automation industry, autonomous vehicles, and construction engineering. With the high-precision measurements of 3D LiDAR, its point clouds can clearly reflect the geometric structure and features of the environment, thus enabling the creation of high-density 3D environmental point cloud models. However, due to the enormous quantity of high-density 3D point clouds, storing and processing these 3D data requires a considerable amount of memory and computing time. In light of this, this paper proposes a real-time 3D point cloud environmental contour modeling technique. The study uses the point cloud distribution from the 3D LiDAR body frame point cloud to establish structured edge features, thereby creating a 3D environmental contour point cloud map. Additionally, unstable objects such as vehicles will appear during the mapping process; these specific objects will be regarded as not part of the stable environmental model in this study. To address this issue, the study will further remove these objects from the 3D point cloud through image recognition and LiDAR heterogeneous matching, resulting in a higher quality 3D environmental contour point cloud map. This 3D environmental contour point cloud not only retains the recognizability of the environmental structure but also solves the problems of massive data storage and processing. Moreover, the method proposed in this study can achieve real-time realization without requiring the 3D point cloud to be organized in a structured order, making it applicable to unorganized 3D point cloud LiDAR sensors. Finally, the feasibility of the proposed method in practical applications is also verified through actual experimental data.

Abstract. Nowadays, Terrestrial Laser Scanning (TLS) technology is gaining popularity in monitoring and predicting the movement of landslide due to the capability of high-speed data capture without requiring a direct contact with the monitored surface. It offers very high density of point cloud data in high resolution and also can be an effective tool in detecting the surface movement of the landslide area. The aim of this research is to determine the optimal level of scanning resolution for landslide monitoring using TLS. The Topcon Geodetic Laser Scanner (GLS) 2000 was used in this research to obtain the three dimensional (3D) point cloud data of the landslide area. Four types of resolution were used during scanning operation which were consist of very high, high, medium and low resolutions. After done with the data collection, the point clouds datasets were undergone the process of registration and filtering using ScanMaster software. After that, the registered point clouds datasets were analyzed using CloudCompare software. Based on the results obtained, the accuracy of TLS point cloud data between picking point manually and computed automatically by ScanMaster software shows the maximum Root Mean Square (RMS) value of coordinate differences were 0.013 m in very high resolution, 0.017 m in high resolution, 0.031 m in medium resolution and 0.052 m in low resolution respectively. Meanwhile, the accuracy of TLS point cloud data between picking point manually and total station data using intersection method shows the maximum RMS values of coordinate differences were 0.013 m in very high resolution, 0.018 m in high resolution, 0.033 m in medium resolution and 0.054 m in low resolution respectively. Hence, it can be concluded that the high or very high resolution is needed for landslide monitoring using Topcon GLS-2000 which can provide more accurate data in slope result, while the low and medium resolutions is not suitable for landslide monitoring due to the accuracy of TLS point cloud data that will decreased when the resolution value is increased.

Gait recognition is an appealing biometric modality which aims to identify individuals based on the way they walk. Deep learning has reshaped the research landscape in this area since 2015 through the ability to automatically learn discriminative representations. Gait recognition methods based on deep learning now dominate the state-of-the-art in the field and have fostered real-world applications. In this paper, we present a comprehensive overview of breakthroughs and recent developments in gait recognition with deep learning, and cover broad topics including datasets, test protocols, state-of-the-art solutions, challenges, and future research directions. We first review the commonly used gait datasets along with the principles designed for evaluating them. We then propose a novel taxonomy made up of four separate dimensions namely body representation, temporal representation, feature representation, and neural architecture, to help characterize and organize the research landscape and literature in this area. Following our proposed taxonomy, a comprehensive survey of gait recognition methods using deep learning is presented with discussions on their performances, characteristics, advantages, and limitations. We conclude this survey with a discussion on current challenges and mention a number of promising directions for future research in gait recognition.

Surface defect identification is a crucial task in many manufacturing systems, including automotive, aircraft, steel rolling, and precast concrete. Although image-based surface defect identification methods have been proposed, these methods usually have two limitations: images may lose partial information, such as depths of surface defects, and their precision is vulnerable to many factors, such as the inspection angle, light, color, noise, etc. Given that a three-dimensional (3D) point cloud can precisely represent the multidimensional structure of surface defects, we aim to detect and classify surface defects using a 3D point cloud. This has two major challenges: (i) the defects are often sparsely distributed over the surface, which makes their features prone to be hidden by the normal surface and (ii) different permutations and transformations of 3D point cloud may represent the same surface, so the proposed model needs to be permutation and transformation invariant. In this paper, a two-step surface defect identification approach is developed to investigate the defects’ patterns in 3D point cloud data. The proposed approach consists of an unsupervised method for defect detection and a multi-view deep learning model for defect classification, which can keep track of the features from both defective and non-defective regions. We prove that the proposed approach is invariant to different permutations and transformations. Two case studies are conducted for defect identification on the surfaces of synthetic aircraft fuselage and the real precast concrete specimen, respectively. The results show that our approach receives the best defect detection and classification accuracy compared with other benchmark methods.

Gait Recognition is a non-invasive biometric technique for identifying persons through the way they walk. Currently there are many Gait Recognition methods, most of them based on a similarity function. In this paper, we propose two new methods for Gait Recognition based on silhouette and contour, using a classifier ensemble. Experimental results on a public standard database are shown and compared against others Gait Recognition methods.

Converting three-dimensional (3D) point cloud data to two-dimensional (2D) image based on virtual structured-light system is a popular method in 3D data compression because the image format is easier to process by the exiting storing and transmitting methods. When this method is used in the 3D point cloud compression, the quantization error is introduced during the coordinate mapping. To solve this problem, a virtual structured-light 3D point cloud compression algorithm based on geometric reshaping is presented. And a least-square system parameter optimization method is proposed to further improve the data decoding accuracy. In the proposed method, the 3D spatial coordinates are reshaped to a 2D matrix first, and then the 2D matrix is stored as a “Holoimage” by using the parameter optimized virtual structured-light system. The quantization error introduced by coordinate mapping between X and Y coordinates of the 3D point cloud and the 2D image pixel coordinates is suppressed, so the decoding accuracy is improved. In addition, the geometric information of the 3D point cloud is hidden synchronously when the 3D point cloud is compressed, which is of great significance in the copyright protection of the point cloud data. Experiments verify the effectiveness of the proposed algorithm. The decoding root mean square error (RMSE) of the proposed method is decreased by 79.86% on average compared with the traditional one under the same compression ratio, and the compression ratio of the proposed method is 1.96 times bigger than the traditional one under the similar decoding accuracy.

Deep learning on a three dimensional (3D) point cloud has attracted a great deal of attention. In this paper, we propose a binarization method for latent representation in a 3D point cloud auto-encoder. The proposed method is based on z-score normalization and deterministic binarization. In the experiment, the proposed method increased data compression efficiency 32-fold, while slightly decreasing the generalization ability of the point cloud reconstruction.

Gait recognition is one of the key technologies for exoskeleton robot control. The existing gait recognition methods cannot meet the needs of real-time control well in terms of recognition accuracy and robustness. In this paper, a gait recognition method based on the recurrent neural network and fully convolutional network (RNN-FCN) algorithm is proposed. In this paper, a human lower limb gait information acquisition device is developed, ten types of human lower limb gait data are collected, and a human gait recognition model is constructed using the RNN-FCN algorithm. The experimental results demonstrate that the RNN-FCN algorithm achieves an average recognition classification accuracy of 94.43% in the experiments, outperforming the other four algorithms.

Gait energy image (GEI) is composed of static body silhouette and dynamic frequency information of human gait. To achieve fast and efficient gait recognition, combined with the accurate description of the information of details and directions in image by Curvelet transform, a gait recognition method using GEI and Curvelet (GEIC) is presented. Firstly, to gain the gait energy images, the gait cycle is selected according to the aspect ratio. Secondly, Curvelet energy coefficients of the GEI, which are used as gait feature vector, are extracted by Curvelet transform in different scales and different directions. Finally, the gait recognition is accomplished by the K nearest neighbor (KNN) classifier. The experimental results demonstrate that GEIC performs well on CASIA(B) database, with the average accuracy of 86.83%. Compared with GEI+KPCA, GEI+W(2D)2PCA and GEI+(2D)2PCA, the algorithm GEIC achieves better robustness in the condition of the person wearing or packaging.

Gait recognition aims to recognize one subject by the way she/he walks without alerting the subject, which has drawn increasing attention. Recently, gait recognition can be represented using various data modalities, such as RGB, skeleton, depth, infrared data, acceleration, gyroscope, .etc., which have various advantages depending on the application scenarios. In this paper, we present a comprehensive survey of recent progress in gait recognition methods based on the type of input data modality. Specifically, we review commonly-used gait datasets with different gait data modalities, following with effective gait recognition methods both for single data modality and multiple data modalities. We also present comparative results of effective gait recognition approaches, together with insightful observations and discussions.KeywordsGait recognitionSensorDeep learningData modality

The paper explores the challenge of human identification through gait recognition within biometric identification systems. Itoutlines the essential criteria for human biometric features, discusses primary biometric characteristics, and their application in biometric identification systems. The paper also examines the feasibility of utilizing gait as a biometric identifier, emphasizing its advantages, such as not requiring the upfront provision of personal biometric information and specialized equipment.The authors conduct an analysis of existing scientific literature in the field of gait recognition, categorizing gait recognition methodsinto template-based and non-template-based approaches. Throughout their research, they identify the key issues and challenges that researchers face in this domain, along with the prevailing trends in human gait recognition within biometric identification systems.Additionally, the paper introduces a method for person identification based on gait, utilizing the Histogram of Oriented Gradients and the Sum Variance Haralick texture features. It involves transforming input video into a series of images depicting the gait silhouette, creating a Gait Energy Image (GEI) by combining these gait silhouettes throughout a gait cycle, and translating the GEI into the Gait Gradient Magnitude Image (GGMI). The subsequent step involves extracting recommended gait characteristics from the GGMIs of participants included in a dataset.To preprocess the collected characteristics, Principal Component Analysis (PCA) is applied, reducing the dimensions that may negatively impact classification robustness, thereby enhancing overall performance. In the final step, a K-Nearest Neighbors (KNN) classifier is employed to categorize the characteristics obtained from a specific dataset.The proposed novel feature vector in the paper demonstrates increased reliability and effectively captures spatial variations in gait patterns. Notably, it reduces the dimensionality of the feature vector from 3780×1 to 63×1, resulting in decreased computational complexity in the gait recognition system. Experimental evaluations on the CASIA A and CASIA B datasets reveal that the proposed approach outperforms other HOG-based methods in most scenarios, with the exception of situations involving frontal images.