PGGait: Gait Recognition Based on Millimeter-Wave Radar Spatio-Temporal Sensing of Multidimensional Point Clouds

During walking, the human lower limbs primarily support the body and drive forward motion, while the arms exhibit greater variability and flexibility without bearing such loads. In gait-based target recognition, collecting exhaustive arm-motion data for training is challenging, and unseen arm movements during testing may degrade the performance. This paper investigates the impact of arm movements on radar-based gait recognition and proposes a gait recognition method using extracted lower limb motion data to mitigate interference from different arm motions. Gait data is collected via a millimeter-wave radar sensor encompassing four kinds of common arm movements, including natural arm swings, object-holding states, and irregular arm motions, from 11 volunteers. Using extracted lower limb motion data, millimeter wave point-cloud gait datasets covering diverse arm motions are generated. Three gait recognition experiments are conducted for comparing the performances of our proposed method using only lower limb data and existing method using all limb data, both based the on PointNet++ model. And the experimental results show that our method consistently outperforms existing methods, with a 22.9-percent improvement in accuracy. Results also show that the proposed method can enhance feature extraction, accelerate convergence, and achieve higher accuracy, especially with limited samples, and the highest recognition accuracy reaches 96.9%. In addition, in unseen arm movement cases, our method significantly outperforms existing methods, demonstrating superior robustness and recognition accuracy.

This paper presents a novel approach to construct spatially-referenced, multidimensional virtual city models from remotely generated point clouds for areas that lack reliable geographical reference data. A multidimensional point cloud is an unstructured array of single, irregular points in a spatial 3D coordinate system plus time stamp. If geospatial reference points are available, a point cloud is geo-referenced. Geo-referenced point clouds contain a high-precision reference dataset. Point clouds can be utilised in a variety of applications. They are particularly suitable for the representation of surfaces, structures, terrain and objects. Point clouds are used here to generate a virtual 3D city model representing the complex, granular cityscape of Jerusalem and its centre, the Old City. The generation of point clouds is based on two data acquisition methods: active data capture by laser scanning and passive data collection by photogrammetric methods. In our case, very high-resolution stereo imagery in visible light and near infrared bands have been systematically acquired an aerial flight campaign. The spatio-temporal data gathered necessitate further processing to extract the geographical reference and semantic features required in a specific resolution and scale. An insight is given into the processing of an unstructured point cloud to extract and classify the 3D urban fabric and reconstruct its objects. Eventually, customised, precise and up-to-date geographical datasets can be made available for a defined region at a defined resolution and scale.

In this paper, we propose a novel gait recognition framework - Spherical Space Model with Human Point Clouds (SSM-HPC) to recognize front view of human gait. A new gait representation - Marching in Place (MIP) gait is also introduced which preserves the spatiotemporal characteristics of individual gait manner. In comparison with the previous studies on gait recognition which usually use human silhouette images from image sequences, this research applies three dimensional (3D) point clouds data of human body obtained from stereo camera. The proposed framework exhibits gait recognition rates superior to those of other gait recognition methods.

Location-based services and identification hold promise for future smart home applications. Through them, we can provide customized services for specific users in current locations. Recent studies have demonstrated that Wi-Fi signals can be leveraged to achieve device-free tracking and gait recognition. Despite their good performance, these two technologies are not effectively integrated for the following reasons: First, the device-free tracking method might yield tracking results that conflict with human gait. Second, extracting gait features relies on knowing or accurately estimating the user's trajectory. Consequently, gait recognition and tracking are inherently linked, but there has been no effective approach to integrate these two techniques. In this paper, we present STAGR, a system capable of <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">S</u>imultaneous <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">T</u>racking <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</u>nd <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G</u>ait <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R</u>ecognition. The main contribution of our technique is that we establish a theoretical model that reveals how to transform path-dependent spectra into path-independent spectra directly. Specifically, we conduct a preliminary study to demonstrate the need for simultaneous tracking and gait recognition. Second, we propose a novel method to extract path-independent gait features, which can significantly save execution time compared with the learning-based method. Third, we design a polar-coordinate filtering method to retain the gait features while correcting the trajectory. We implement a prototype STAGR system and conduct extensive experiments to verify the proposed mechanism. The experimental results show that we can realize simultaneous tracking and gait recognition. The median tracking error is <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$ 0.45m$</tex-math></inline-formula>, while the recognition accuracy is 95.3% for 6 users.

Gait recognition stands as a pivotal biometric technology in individual identification, yet its real-world implementation faces challenges stemming from intra-subject disparities. The task of extracting consistent features to distinguish among various subjects becomes onerous due to factors such as image noise and magnitude divergence, significantly impacting recognition accuracy. In addressing this hurdle, we introduce a groundbreaking approach known as the Spline Magnitude Disparity Cross-Correlated Deep Network, designed to optimize gait recognition efficiency. Our method, the Spline Magnitude Disparity Cross-Correlated Deep Network, operates through two key steps: B-Spline magnitude disparity deformation (BS-MDD) registration and cross-correlated long-short gait recognition modeling. The BS-MDD algorithm employs free-form deformation to approximate the magnitude divergence in gait input, enhancing viewpoint optimization and contributing to the development of the cross-correlated model. By focusing on preserving high-output recognition gates while eliminating forget gates, our approach achieves a heightened recognition rate. Evaluation on the widely utilized CASIA B dataset showcases the superiority of our proposed method over state-of-the-art alternatives in terms of the true positive rate, false-positive rate, recognition time, and overall recognition rate. Notably, our approach elevates the true positive rate by 5% and reduces the false-positive rate by 4%. These results underscore the high effectiveness of our method, demonstrating its capacity to substantially improve the accuracy of gait recognition in practical applications.”

Gait recognition is one of the new biometric techniques. However, gait recognition is easily affected by external factors, resulting in a decrease in the accuracy. Therefore, this paper proposes a gait recognition method based on spatial and temporal multi grained information. This method uses hierarchical network to obtain different receptive fields, focusing on both shallow and deep features of gait. In addition, more spatial and temporal information can be saved to improve the accuracy of gait recognition. Finally, the proposed method is experimentally verified on the CASIA-B dataset. the experimental results show that the accuracy of the proposed method is greatly improved under normal walking, carrying bags and wearing coats.

Gait recognition with wearable sensors is an effective approach to identifying people by recognizing their distinctive walking patterns. Deep learning-based networks have recently emerged as a promising technique in gait recognition, yielding better performance than template matching and traditional machine learning methods. However, most recent studies have focused on improving gait detection accuracy while neglecting model complexity in the deep learning domain, making them unsuitable for low-power wearable devices. Therefore, inference from these models results in latency due to calculation overhead. This study proposes an efficient network suitable for wearable devices without sacrificing prediction performance. We have modified the residual block and accumulated it in shallow convolutional neural networks with five weighted layers only for gait recognition and proved the efficacy of all the architectural components with extensive experiments over publicly available IMU-based datasets: whuGait and OU-ISIR. Our proposed model outperforms all the state-of-the-art methods regarding recognition accuracy and is more than 85 percent efficient on average in terms of model parameters and memory consumption.

Surveillance scenes and pedestrian retrieval and recognition in videos gradually play an increasingly important role. Due to the long distance between the camera and pedestrians, the commonly used biometric identification methods such as face recognition and iris recognition cannot be applied to actual scenes. middle. The gait is a biological feature that is not easy to hide, not easy to camouflage, and can be obtained from a long distance, so it can be effectively used in surveillance and pedestrian retrieval and recognition in videos. Can be used as a feature to distinguish individuals. This paper proposes a framework for gait recognition based on Autoencoder, which can be uniformly converted to an angle that is conducive to retrieval and recognition without knowing the angle between the pedestrian and the camera in advance. The accuracy of gait recognition in the case of angle. The framework takes the characteristic gait energy map of commonly used gait recognition as input, and extracts better features to express the pedestrian's gait. This article is tested on the gait database dataset B collected by the Institute of Dynamics of the Chinese Academy of Sciences, that is, the public database of CAS worker A GaitDatabase B. The experimental results show that the framework can effectively solve the three common challenges in gait recognition, which are multi-view, carrying items and clothing changes, and achieve good results.

We present an investigation of how topological data analysis (TDA) can be applied to condition-based monitoring (CBM) of wind turbines for energy generation.TDA is a branch of data analysis focusing on extracting mean- ingful information from complex datasets by analyzing their structure in state space and computing their underlying topo- logical features. By representing data in a high-dimensional state space, TDA enables the identification of patterns, anoma- lies, and trends in the data that may not be apparent through traditional signal processing methods. For this study, wind turbine data was acquired from a wind park in Norway via standard vibration sensors at different lo- cations of the turbine’s gearbox. Both the vibration acceler- ation data and its frequency spectra were recorded at infre- quent intervals for a few seconds at high frequency and fail- ure events were labelled as either gear-tooth or ball-bearing failures. The data processing and analysis are based on a pipeline where the time series data is first split into intervals and then transformed into multi-dimensional point clouds via a time-delay embedding. The shape of the point cloud is an- alyzed with topological methods such as persistent homol- ogy to generate topology-based key health indicators based on Betti numbers, information entropy and signal persistence. Such indicators are tested for CBM and diagnosis (fault de- tection) to identify faults in wind turbines and classify them accordingly. Topological indicators are shown to be an in- teresting alternative for failure identification and diagnosis of operational failures in wind turbines.

Human gait recognition (HGR) has been employed as a biometric technique for security purposes over the last decade. Various factors, including clothing, carrying items, and walking surfaces, can influence the performance of gait recognition. Additionally, identifying individuals from different viewpoints presents a significant challenge in HGR. Numerous conventional and deep learning techniques have been introduced in the literature for HGR, but traditional methods are not well suited to handling large datasets. This research explores the effectiveness of four deep learning models for gait identification in the CASIA B dataset: the convolutional neural network (CNN), multi-layer perceptron (MLP), self-organizing map (SOMs), and transfer learning with EfficientNet. The selected deep learning techniques offer robust feature extraction and the efficient handling of large datasets, making them ideal in enhancing the accuracy of gait recognition. The collection includes gait sequences from 10 individuals, with a total of 92,596 images that have been reduced to 64 × 64 pixels for uniformity. A modified model was developed by integrating sequential convolutional layers for detailed spatial feature extraction, followed by dense layers for classification, optimized through rigorous hyperparameter tuning and regularization techniques, resulting in an accuracy of 97.12% for the test set. This work enhances our understanding of deep learning methods in gait analysis, offering significant insights for choosing optimal models in security and surveillance applications.

In computer-vision-based human computer interaction (HCI), higher-quality signal leads to better system performance. In this paper, we develop a real-time high-resolution 3D object scanning system based on structured light illumination (SLI). Our system fuses depth information with RGB texture to reconstruct high-resolution 3D point cloud. The point cloud preserves accurate surface geometry of the object (e.g., finger postures of hands, facial expressions, etc). Respectively, for a 640 × 480 video stream, our system can generate phase and texture video at 1500 frames per second (fps) and produce full 3D point clouds at 300 fps. For gesture recognition, we propose to combine the module of robust face recognition with the module of 3D point cloud classification. Moreover, rather than extracting sophisticated features, we leverage the accurate reconstruction and classify each point cloud by directly matching the whole 3D surface geometry with the templates of different classes. The proposed recognition system is robust to the scaling, translation, rotation and texture of objects. Finally, utilizing the system, we contribute to the research community two large-scale high-resolution 3D point cloud databases, i.e., SLI 3D Hand Gesture Database and SLI 3D Face Database. The proposed point cloud recognition approach achieves recognition rates up to 98.0% over the gesture database and 88.2% over the face database in our pilot study.

TS[formula omitted]HGRNet: A paradigm of two stream best deep learning feature fusion assisted framework for human gait analysis using controlled environment in smart cities

Gait recognition provides a more convenient way for human identification, as it can identify person with less cooperation and intrusion compared to other biometric features. Current gait recognition frameworks utilize a template to extract temporal feature or regard the whole person as a unit, and they obtain limited temporal information and fine-grained features. To overcome this problem, we propose a network consisting of two parts: Temporal Feature Fusion (TFF) and Fine-grained Feature Extraction (FFE). First, we extract the most representative temporal information from raw gait sequences by TFF. Next, we use the idea of partial features on fused temporal features to extract more fine-grained spatial block features. It is worth mentioning that the proposed algorithm provides an effective feature extraction framework for complex gait recognition, as it focuses on the temporal fusion for representative information, and the extraction of the fine-grained spatial features. Extensive experiments illustrated that we have an outstanding performance on CASIA-B and mini-OUMVLP compared to other state-of-the-art methods including GaitSet and GaitNet. In particularly, the average rank-1 accuracy of all probe views on normal walking condition (NM) achieve 95.7%.

As a unique biometric feature that can be recognized at a distance, gait has broad applications in crime prevention, forensic identification and social security. To portray a gait, existing gait recognition methods utilize either a gait template, where temporal information is hard to preserve, or a gait sequence, which must keep unnecessary sequential constraints and thus loses the flexibility of gait recognition. In this paper we present a novel perspective, where a gait is regarded as a set consisting of independent frames. We propose a new network named GaitSet to learn identity information from the set. Based on the set perspective, our method is immune to permutation of frames, and can naturally integrate frames from different videos which have been filmed under different scenarios, such as diverse viewing angles, different clothes/carrying conditions. Experiments show that under normal walking conditions, our single-model method achieves an average rank-1 accuracy of 95.0% on the CASIA-B gait dataset and an 87.1% accuracy on the OU-MVLP gait dataset. These results represent new state-of-the-art recognition accuracy. On various complex scenarios, our model exhibits a significant level of robustness. It achieves accuracies of 87.2% and 70.4% on CASIA-B under bag-carrying and coat-wearing walking conditions, respectively. These outperform the existing best methods by a large margin. The method presented can also achieve a satisfactory accuracy with a small number of frames in a test sample, e.g., 82.5% on CASIA-B with only 7 frames. The source code has been released at https://github.com/AbnerHqC/GaitSet.

Piezoelectric energy harvester (PEH), which generates electricity from stress or vibrations, is attracting tremendous attention as a viable solution to extend battery life of wearable devices. More interestingly, besides the energy harvesting capability, recent research has demonstrated the feasibility of leveraging PEH as an power-free sensor for gait recognition as its stress or vibration patters are significantly influenced by the gait. However, as PEHs are not designed for precise motion sensing, the gait recognition accuracy remains low with conventional classification algorithms. The accuracy deteriorates further when the generated electricity is stored simultaneously. In this work, to achieve high performance gait recognition and efficient energy harvesting at the same time, we make two distinct contributions. First, we propose a preprocessing algorithm to filter out the effect of energy storage on PEH electricity signals. Second, we propose long short-term memory (LSTM) network-based classifiers to accurately capture temporal information in gait-induced electricity generation. We prototype the proposed gait recognition architecture in the form factor of an insole and evaluate its gait recognition as well as energy harvesting performance with 20 subjects. Our results show that the proposed architecture detects human gait with 12 percent higher recall and harvests up to 127 percent more energy while consuming 38 percent less power compared to the state-of-the-art.