OpenGait: A Comprehensive Benchmark Study for Gait Recognition Toward Better Practicality.

Gait recognition is an appealing biometric modality which aims to identify individuals based on the way they walk. Deep learning has reshaped the research landscape in this area since 2015 through the ability to automatically learn discriminative representations. Gait recognition methods based on deep learning now dominate the state-of-the-art in the field and have fostered real-world applications. In this paper, we present a comprehensive overview of breakthroughs and recent developments in gait recognition with deep learning, and cover broad topics including datasets, test protocols, state-of-the-art solutions, challenges, and future research directions. We first review the commonly used gait datasets along with the principles designed for evaluating them. We then propose a novel taxonomy made up of four separate dimensions namely body representation, temporal representation, feature representation, and neural architecture, to help characterize and organize the research landscape and literature in this area. Following our proposed taxonomy, a comprehensive survey of gait recognition methods using deep learning is presented with discussions on their performances, characteristics, advantages, and limitations. We conclude this survey with a discussion on current challenges and mention a number of promising directions for future research in gait recognition.

Most existing gait recognition approaches adopt a two-step procedure: a preprocessing step to extract silhouettes or skeletons followed by recognition. In this paper, we propose an end-to-end model-based gait recognition method. Specifically, we employ a skinned multi-person linear (SMPL) model for human modeling, and estimate its parameters using a pre-trained human mesh recovery (HMR) network. As the pre-trained HMR is not recognition-oriented, we fine-tune it in an end-to-end gait recognition framework. To cope with differences between gait datasets and those used for pre-training the HMR, we introduce a reconstruction loss between the silhouette masks in the gait datasets and the rendered silhouettes from the estimated SMPL model produced by a differentiable renderer. This enables us to adapt the HMR to the gait dataset without supervision using the ground-truth joint locations. Experimental results with the OU-MVLP and CASIA-B datasets demonstrate the state-of-the-art performance of the proposed method for both gait identification and verification scenarios, a direct consequence of the explicitly disentangled pose and shape features produced by the proposed end-to-end model-based framework.

Today, many different biometric features are used for human identification. Unlike biometric features, such as eye, iris, ear, and fingerprint, gait biometrics enables recognition from long distance and low resolution images. In this paper, different design choices for a deep learning-based gait recognition system are investigated in detail. Some preprocessing steps, such as human silhouette extraction and gait cycle calculation are eliminated to make the system suitable for practical applications. To assess different input types’ effect on the gait recognition performance, both binary silhouettes and RGB images are given as input to the network. To observe the contribution of transfer learning, we fine-tuned a pre-trained generic object recognition model with the CASIA-B gait dataset and performed experiments on the OU-ISIR Large Population gait dataset. To observe the effect of pose variations, we conducted experiments for both identical-view and cross-view conditions. Successful results are obtained, especially for cross-view gait recognition, compared to different approaches for gait recognition.

Gait recognition appears to be a valuable asset when conventional biometrics cannot be employed. Nonetheless, recognizing human by gait is not a trivial task due to the complex human kinematic structure and other external factors affecting human locomotion. A major challenge in gait recognition is view variation. A large difference between the views in the query and reference sets often leads to performance deterioration. In this paper, we show how to generate virtual views to compensate the view difference in the query and reference sets, making it possible to match the query and reference sets using standardized views. The proposed method, which combines multiview matrix representation and a novel randomized kernel extreme learning machine, is an end-to-end solution for view change problem under Grassmann manifold treatment. Under the right condition, the view-tagging problem can be eliminated. Since the recording angle and walking direction of the subject are not always available, this is particularly valuable for a practical gait recognition system. We present several working scenarios for multiview recognition that have not be considered before. Rigorous experiments have been conducted on two challenging benchmark databases containing multiview gait datasets. Experiments show that the proposed approach outperforms several state-of-the-arts methods.

Gait energy image (GEI) is composed of static body silhouette and dynamic frequency information of human gait. To achieve fast and efficient gait recognition, combined with the accurate description of the information of details and directions in image by Curvelet transform, a gait recognition method using GEI and Curvelet (GEIC) is presented. Firstly, to gain the gait energy images, the gait cycle is selected according to the aspect ratio. Secondly, Curvelet energy coefficients of the GEI, which are used as gait feature vector, are extracted by Curvelet transform in different scales and different directions. Finally, the gait recognition is accomplished by the K nearest neighbor (KNN) classifier. The experimental results demonstrate that GEIC performs well on CASIA(B) database, with the average accuracy of 86.83%. Compared with GEI+KPCA, GEI+W(2D)2PCA and GEI+(2D)2PCA, the algorithm GEIC achieves better robustness in the condition of the person wearing or packaging.

As a unique biometric feature that can be recognized at a distance, gait has broad applications in crime prevention, forensic identification and social security. To portray a gait, existing gait recognition methods utilize either a gait template, where temporal information is hard to preserve, or a gait sequence, which must keep unnecessary sequential constraints and thus loses the flexibility of gait recognition. In this paper we present a novel perspective, where a gait is regarded as a set consisting of independent frames. We propose a new network named GaitSet to learn identity information from the set. Based on the set perspective, our method is immune to permutation of frames, and can naturally integrate frames from different videos which have been filmed under different scenarios, such as diverse viewing angles, different clothes/carrying conditions. Experiments show that under normal walking conditions, our single-model method achieves an average rank-1 accuracy of 95.0% on the CASIA-B gait dataset and an 87.1% accuracy on the OU-MVLP gait dataset. These results represent new state-of-the-art recognition accuracy. On various complex scenarios, our model exhibits a significant level of robustness. It achieves accuracies of 87.2% and 70.4% on CASIA-B under bag-carrying and coat-wearing walking conditions, respectively. These outperform the existing best methods by a large margin. The method presented can also achieve a satisfactory accuracy with a small number of frames in a test sample, e.g., 82.5% on CASIA-B with only 7 frames. The source code has been released at https://github.com/AbnerHqC/GaitSet.

Gait recognition stands as a pivotal biometric technology in individual identification, yet its real-world implementation faces challenges stemming from intra-subject disparities. The task of extracting consistent features to distinguish among various subjects becomes onerous due to factors such as image noise and magnitude divergence, significantly impacting recognition accuracy. In addressing this hurdle, we introduce a groundbreaking approach known as the Spline Magnitude Disparity Cross-Correlated Deep Network, designed to optimize gait recognition efficiency. Our method, the Spline Magnitude Disparity Cross-Correlated Deep Network, operates through two key steps: B-Spline magnitude disparity deformation (BS-MDD) registration and cross-correlated long-short gait recognition modeling. The BS-MDD algorithm employs free-form deformation to approximate the magnitude divergence in gait input, enhancing viewpoint optimization and contributing to the development of the cross-correlated model. By focusing on preserving high-output recognition gates while eliminating forget gates, our approach achieves a heightened recognition rate. Evaluation on the widely utilized CASIA B dataset showcases the superiority of our proposed method over state-of-the-art alternatives in terms of the true positive rate, false-positive rate, recognition time, and overall recognition rate. Notably, our approach elevates the true positive rate by 5% and reduces the false-positive rate by 4%. These results underscore the high effectiveness of our method, demonstrating its capacity to substantially improve the accuracy of gait recognition in practical applications.”

Gait recognition is a biometric recognition technology that supports long-distance, multi-target recognition with resistance to partial occlusions and does not require active user cooperation; thus, it is more suitable than other technologies for individual identification in mass video surveillance systems. Gait recognition based on deep learning has become the mainstream technology in this field because of its strong self-learning and model prediction abilities. However, there is still a lack of research focusing on actual scenes and application requirements for gait recognition, such as multi-target, real-time, and robust recognition. Therefore, this paper analyzes the basic tasks of deep gait recognition methods and encapsulates the application scope of deep gait recognition. Subsequently, this paper investigates the methods of large-space deep gait recognition from three aspects: image preprocessing, gait feature extraction with deep learning, and classifier and evaluation. In particular, the study investigated and analyzed the gait input templates often used in mass surveillance, auto encoder with deep learning, and performance evaluation indexes for the first time. Finally, the unresolved issues in deep gait recognition are summarized, and suggestions and directions for future research are presented.

Surveillance scenes and pedestrian retrieval and recognition in videos gradually play an increasingly important role. Due to the long distance between the camera and pedestrians, the commonly used biometric identification methods such as face recognition and iris recognition cannot be applied to actual scenes. middle. The gait is a biological feature that is not easy to hide, not easy to camouflage, and can be obtained from a long distance, so it can be effectively used in surveillance and pedestrian retrieval and recognition in videos. Can be used as a feature to distinguish individuals. This paper proposes a framework for gait recognition based on Autoencoder, which can be uniformly converted to an angle that is conducive to retrieval and recognition without knowing the angle between the pedestrian and the camera in advance. The accuracy of gait recognition in the case of angle. The framework takes the characteristic gait energy map of commonly used gait recognition as input, and extracts better features to express the pedestrian's gait. This article is tested on the gait database dataset B collected by the Institute of Dynamics of the Chinese Academy of Sciences, that is, the public database of CAS worker A GaitDatabase B. The experimental results show that the framework can effectively solve the three common challenges in gait recognition, which are multi-view, carrying items and clothing changes, and achieve good results.

Gait recognition is a distinctive biometric technique that can identify pedestrians by their walking patterns from considerable distances. A critical challenge in gait recognition lies in effectively acquiring discriminative spatial-temporal representations from silhouettes that exhibit invariance to disturbances. In this paper, we present a novel gait recognition network by aggregating features in the spatial-temporal and view domains, which consists of two-path spatial-temporal feature fusion module and view embedding module. Specifically, two-path spatial-temporal feature fusion module firstly utilizes multi-scale feature extraction (MSFE) to enrich the input features with multiple convolution kernels of various sizes. Then, frame-level spatial feature extraction (FLSFE) and multi-scale temporal feature extraction (MSTFE) are parallelly constructed to capture spatial and temporal gait features of different granularities and these features are fused together to obtain muti-scale spatial-temporal features. FLSFE is designed to extract both global and local gait features by employing a specially designed residual operation. Simultaneously, MSTFE is applied to adaptively interact multi-scale temporal features and produce suitable motion representations in temporal domain. Taking into account the view information, we introduce a view embedding module to reduce the impact of differing viewpoints. Through the extensive experimentation over CASIA-B and OU-MVLP datasets, the proposed method has achieved superior performance to the other state-of-the-art gait recognition approaches.

Gait recognition of camouflaged people based on UAV infrared imaging

Gait recognition is an emerging biometric technology that can be used to protect the privacy of wearable device owners. To improve the performance of the existing gait recognition method based on wearable devices and to reduce the memory size of the model and increase its robustness, a new identification method based on multimodal fusion of gait cycle data is proposed. In addition, to preserve the time-dependence and correlation of the data, we convert the time-series data into two-dimensional images using the Gramian angular field (GAF) algorithm. To address the problem of high model complexity in existing methods, we propose a lightweight double-channel depthwise separable convolutional neural network (DC-DSCNN) model for gait recognition for wearable devices. Specifically, the time series data of gait cycles and GAF images are first transferred to the upper and lower layers of the DC-DSCNN model. The gait features are then extracted with a three-layer depthwise separable convolutional neural network (DSCNN) module. Next, the extracted features are transferred to a softmax classifier to implement gait recognition. To evaluate the performance of the proposed method, the gait dataset of 24 subjects were collected. Experimental results show that the recognition accuracy of the DC-DSCNN algorithm is 99.58%, and the memory usage of the model is only 972 KB, which verifies that the proposed method can enable gait recognition for wearable devices with lower power consumption and higher real-time performance.

With the advancement of technology and the demanding requirements for security-related applications, automatic human identification at a far distance has generated a considerable amount of interest in the research community. The gait recognition has an important advantage of identifying people from a long distance with minimal cooperation from the subject. In recent days, gait recognition has emerged as one of the most noticeable biometric technology for the identification of individuals by looking at the manner in which they walk. One of the most commonly faced problems in gait recognition is the presence of various intraclass variations like clothing and carrying conditions, which drastically reduces the recognition accuracy. The main intent of this article is to present an extensive review of existing contemporary gait recognition methods. KeywordsGait recognitionSilhouette extractionFeature extractionClassificationCovariate factorsCASIA Gait Dataset

Silhouette-based gait recognition is a popular biometric modality because of its non-intrusive nature and its ability to capture identity information from a distance and at low resolution. However, cross-view gait recognition presents significant challenges because the appearance changes drastically across different viewpoints. Existing gait datasets are sparse in view coverage, limiting the generalization of models to novel views. The evaluation of current state-of-the-art gait recognition models is confined to the views present in the training dataset, posing challenges for practical application scenarios. In this paper, we propose a three-dimensional (3D) human-model-guided view-morphing framework to generate high-quality gait silhouettes for arbitrary views, enhancing gait recognition performance for novel views unseen during training. By leveraging the body topology of the 3D human model, unlike existing GAN-based or deformation-based methods that rely only on two-dimensional images, the proposed method enables fine control over the output views and achieves plausible deformations, even for self-occluded body parts, through the supervision of the pseudo ground-truth deformation field computed from the 3D model. To enhance the realism of silhouette generation, we use clothed silhouettes alongside skin silhouettes rendered from the 3D model (e.g., the SMPL) to train the deformation estimation network to account for clothing and accessories in the final output. We evaluated our approach on the OU-MVLP, CASIA-B, and CASIA-E datasets, demonstrating significant improvements in gait recognition performance, particularly in view-limited scenarios. For example, recovering 6.47% rank-1 recognition rate on OU-MVLP using three close views only and an average of 24.36% and 8.34% on CASIA-B and CASIA-E normal clothing, respectively, using a single view. Our framework increases the robustness of the cross-view problem, enhancing the applicability of gait recognition in real-world settings such as surveillance and security.

Gait recognition has become a mainstream technology for identification, as it can recognize the identity of subjects from a distance without any cooperation. However, when subjects wear coats (CL) or backpacks (BG), their gait silhouette will be occluded, which will lose some gait information and bring great difficulties to the identification. Another important challenge in gait recognition is that the gait silhouette of the same subject captured by different camera angles varies greatly, which will cause the same subject to be misidentified as different individuals under different camera angles. In this article, we try to overcome these problems from three aspects: data augmentation, feature extraction, and feature refinement. Correspondingly, we propose gait sequence mixing (GSM), multigranularity feature extraction (MFE), and feature distance alignment (FDA). GSM is a method that belongs to data enhancement, which uses the gait sequences in NM to assist in learning the gait sequences in BG or CL, thus reducing the influence of lost gait information in abnormal gait sequences (BG or CL). MFE explores and fuses different granularity features of gait sequences from different scales, and it can learn as much useful information as possible from incomplete gait silhouettes. FDA refines the extracted gait features with the help of the distribution of gait features in real world and makes them more discriminative, thus reducing the influence of various camera angles. Extensive experiments demonstrate that our method has better results than some state-of-the-art methods on CASIA-B and mini-OUMVLP. We also embed the GSM module and FDA module into some state-of-the-art methods, and the recognition accuracy of these methods is greatly improved.