Deep Large Margin Nearest Neighbor for Gait Recognition

Gait recognition is an appealing biometric modality which aims to identify individuals based on the way they walk. Deep learning has reshaped the research landscape in this area since 2015 through the ability to automatically learn discriminative representations. Gait recognition methods based on deep learning now dominate the state-of-the-art in the field and have fostered real-world applications. In this paper, we present a comprehensive overview of breakthroughs and recent developments in gait recognition with deep learning, and cover broad topics including datasets, test protocols, state-of-the-art solutions, challenges, and future research directions. We first review the commonly used gait datasets along with the principles designed for evaluating them. We then propose a novel taxonomy made up of four separate dimensions namely body representation, temporal representation, feature representation, and neural architecture, to help characterize and organize the research landscape and literature in this area. Following our proposed taxonomy, a comprehensive survey of gait recognition methods using deep learning is presented with discussions on their performances, characteristics, advantages, and limitations. We conclude this survey with a discussion on current challenges and mention a number of promising directions for future research in gait recognition.

Rapidly development in the realm of biometrics in recent years results in the unprecedented growth with the number of researchers and interesters. As one of the biometrics, gait recognition has many advantages such as from a distance, lower quality video, hard to disguised comparing with others. However, nearly all studies on gait recognition are 2D methods based on analysis of image sequences captured by a monocular camera. This paper presents a novel 3D method for automatic gait recognition by analyzing the changes of a 1D silhouette signal. Binarized silhouette of a moving individual is firstly achieved. Secondly contour is extracted for next stereo matching. Afterward 3D contour is obtained through a step of contour based stereo matchin. Then, stereo silhouette vector (SSV) is extracted from 3D contour and transformed to a 1D silhouette signal as the stereo gait feature (SGF). K-L transform is applied for reducing the dimensionality of feature and the complexity of computing. Finally, kNN and NN are applied to gait classification and recognition. A stereo gait database named PRLAB is established for gait recognition based on stereo vision. Experimental results on PRLAB showed the efficiency and robustness of our algorithm.

As a unique biometric feature that can be recognized at a distance, gait has broad applications in crime prevention, forensic identification and social security. To portray a gait, existing gait recognition methods utilize either a gait template, where temporal information is hard to preserve, or a gait sequence, which must keep unnecessary sequential constraints and thus loses the flexibility of gait recognition. In this paper we present a novel perspective, where a gait is regarded as a set consisting of independent frames. We propose a new network named GaitSet to learn identity information from the set. Based on the set perspective, our method is immune to permutation of frames, and can naturally integrate frames from different videos which have been filmed under different scenarios, such as diverse viewing angles, different clothes/carrying conditions. Experiments show that under normal walking conditions, our single-model method achieves an average rank-1 accuracy of 95.0% on the CASIA-B gait dataset and an 87.1% accuracy on the OU-MVLP gait dataset. These results represent new state-of-the-art recognition accuracy. On various complex scenarios, our model exhibits a significant level of robustness. It achieves accuracies of 87.2% and 70.4% on CASIA-B under bag-carrying and coat-wearing walking conditions, respectively. These outperform the existing best methods by a large margin. The method presented can also achieve a satisfactory accuracy with a small number of frames in a test sample, e.g., 82.5% on CASIA-B with only 7 frames. The source code has been released at https://github.com/AbnerHqC/GaitSet.

Compared with other biometrics, biometric based on gait features can be collected under long-distance and contactless conditions to achieve identity recognition under contactless and long-distance conditions. At present, gait recognition methods are still sensitive to illumination and background changes and are susceptible to noise in feature extraction, the gait template approach suffers from inflexibility and neglect of timing information in recognition tasks. In this paper, Mask R-CNN, a deep learning detection and segmentation model, is used to extract gait silhouettes and achieve effective and real-time segmentation of human gait silhouettes. We propose an improved GaitSet algorithm with a vertical-horizontal pyramid pooling module, and introduce a Softmax loss function for joint training to address the problem that the triplet loss function does not consider intra-class compactness. The proposed algorithm achieves the current more advanced recognition performance on the gait dataset CASIAB, and for gait recognition under jacket walking conditions, the improvement in accuracy is more obvious.

We propose a new gait feature extraction method for gait recognition that uses the dyadic wavelet transform (DYWT) and structural similarity (SSIM). Gait recognition is a person authentication technique that uses the gait features obtained from a monitoring camera video. Most existing gait recognition methods use gait features based on the silhouette image sequence. However, the gait recognition accuracy decreases if the resolution of the silhouette image is low. We developed a new method to extract gait features from silhouette images having a low-resolution gait period sequence that uses the DYWT and SSIM. For our experiment, we prepared two types of silhouette image sequences of 100 subjects for use as the probe and gallery images, respectively. The sizes of the silhouette images were 64 × 44, 32 × 22, and 16 × 11. We describe our proposed method in detail and present our experimental results demonstrating that the extracted gait features are effective.

Gait recognition is a biometric recognition technology that supports long-distance, multi-target recognition with resistance to partial occlusions and does not require active user cooperation; thus, it is more suitable than other technologies for individual identification in mass video surveillance systems. Gait recognition based on deep learning has become the mainstream technology in this field because of its strong self-learning and model prediction abilities. However, there is still a lack of research focusing on actual scenes and application requirements for gait recognition, such as multi-target, real-time, and robust recognition. Therefore, this paper analyzes the basic tasks of deep gait recognition methods and encapsulates the application scope of deep gait recognition. Subsequently, this paper investigates the methods of large-space deep gait recognition from three aspects: image preprocessing, gait feature extraction with deep learning, and classifier and evaluation. In particular, the study investigated and analyzed the gait input templates often used in mass surveillance, auto encoder with deep learning, and performance evaluation indexes for the first time. Finally, the unresolved issues in deep gait recognition are summarized, and suggestions and directions for future research are presented.

Gait Recognition is a non-invasive biometric technique for identifying persons through the way they walk. Currently there are many Gait Recognition methods, most of them based on a similarity function. In this paper, we propose two new methods for Gait Recognition based on silhouette and contour, using a classifier ensemble. Experimental results on a public standard database are shown and compared against others Gait Recognition methods.

Gait energy image (GEI) is composed of static body silhouette and dynamic frequency information of human gait. To achieve fast and efficient gait recognition, combined with the accurate description of the information of details and directions in image by Curvelet transform, a gait recognition method using GEI and Curvelet (GEIC) is presented. Firstly, to gain the gait energy images, the gait cycle is selected according to the aspect ratio. Secondly, Curvelet energy coefficients of the GEI, which are used as gait feature vector, are extracted by Curvelet transform in different scales and different directions. Finally, the gait recognition is accomplished by the K nearest neighbor (KNN) classifier. The experimental results demonstrate that GEIC performs well on CASIA(B) database, with the average accuracy of 86.83%. Compared with GEI+KPCA, GEI+W(2D)2PCA and GEI+(2D)2PCA, the algorithm GEIC achieves better robustness in the condition of the person wearing or packaging.

Bootstrapping search margin-based nearest neighbor method for qualitative spectroscopic analysis.

With the advancement of technology and the demanding requirements for security-related applications, automatic human identification at a far distance has generated a considerable amount of interest in the research community. The gait recognition has an important advantage of identifying people from a long distance with minimal cooperation from the subject. In recent days, gait recognition has emerged as one of the most noticeable biometric technology for the identification of individuals by looking at the manner in which they walk. One of the most commonly faced problems in gait recognition is the presence of various intraclass variations like clothing and carrying conditions, which drastically reduces the recognition accuracy. The main intent of this article is to present an extensive review of existing contemporary gait recognition methods. KeywordsGait recognitionSilhouette extractionFeature extractionClassificationCovariate factorsCASIA Gait Dataset

The paper explores the challenge of human identification through gait recognition within biometric identification systems. Itoutlines the essential criteria for human biometric features, discusses primary biometric characteristics, and their application in biometric identification systems. The paper also examines the feasibility of utilizing gait as a biometric identifier, emphasizing its advantages, such as not requiring the upfront provision of personal biometric information and specialized equipment.The authors conduct an analysis of existing scientific literature in the field of gait recognition, categorizing gait recognition methodsinto template-based and non-template-based approaches. Throughout their research, they identify the key issues and challenges that researchers face in this domain, along with the prevailing trends in human gait recognition within biometric identification systems.Additionally, the paper introduces a method for person identification based on gait, utilizing the Histogram of Oriented Gradients and the Sum Variance Haralick texture features. It involves transforming input video into a series of images depicting the gait silhouette, creating a Gait Energy Image (GEI) by combining these gait silhouettes throughout a gait cycle, and translating the GEI into the Gait Gradient Magnitude Image (GGMI). The subsequent step involves extracting recommended gait characteristics from the GGMIs of participants included in a dataset.To preprocess the collected characteristics, Principal Component Analysis (PCA) is applied, reducing the dimensions that may negatively impact classification robustness, thereby enhancing overall performance. In the final step, a K-Nearest Neighbors (KNN) classifier is employed to categorize the characteristics obtained from a specific dataset.The proposed novel feature vector in the paper demonstrates increased reliability and effectively captures spatial variations in gait patterns. Notably, it reduces the dimensionality of the feature vector from 3780×1 to 63×1, resulting in decreased computational complexity in the gait recognition system. Experimental evaluations on the CASIA A and CASIA B datasets reveal that the proposed approach outperforms other HOG-based methods in most scenarios, with the exception of situations involving frontal images.

Gait recognition aims to recognize one subject by the way she/he walks without alerting the subject, which has drawn increasing attention. Recently, gait recognition can be represented using various data modalities, such as RGB, skeleton, depth, infrared data, acceleration, gyroscope, .etc., which have various advantages depending on the application scenarios. In this paper, we present a comprehensive survey of recent progress in gait recognition methods based on the type of input data modality. Specifically, we review commonly-used gait datasets with different gait data modalities, following with effective gait recognition methods both for single data modality and multiple data modalities. We also present comparative results of effective gait recognition approaches, together with insightful observations and discussions.KeywordsGait recognitionSensorDeep learningData modality

Gait recognition is one of the most important biometric technologies and has been applied in many fields. Recent gait recognition frameworks represent each gait frame by descriptors extracted from either global appearances or local regions of humans. However, the representations based on global information often neglect the details of the gait frame, while local region based descriptors cannot capture the relations among neighboring regions, thus reducing their discriminativeness. In this paper, we propose a novel feature extraction and fusion framework to achieve discriminative feature representations for gait recognition. Towards this goal, we take advantage of both global visual information and local region details and develop a Global and Local Feature Extractor (GLFE). Specifically, our GLFE module is composed of our newly designed multiple global and local convolutional layers (GLConv) to ensemble global and local features in a principle manner. Furthermore, we present a novel operation, namely Local Temporal Aggregation (LTA), to further preserve the spatial information by reducing the temporal resolution to obtain higher spatial resolution. With the help of our GLFE and LTA, our method significantly improves the discriminativeness of our visual features, thus improving the gait recognition performance. Extensive experiments demonstrate that our proposed method outperforms state-of-the-art gait recognition methods on two popular datasets.

Gait recognition is an emerging biometric technology that can be used to protect the privacy of wearable device owners. To improve the performance of the existing gait recognition method based on wearable devices and to reduce the memory size of the model and increase its robustness, a new identification method based on multimodal fusion of gait cycle data is proposed. In addition, to preserve the time-dependence and correlation of the data, we convert the time-series data into two-dimensional images using the Gramian angular field (GAF) algorithm. To address the problem of high model complexity in existing methods, we propose a lightweight double-channel depthwise separable convolutional neural network (DC-DSCNN) model for gait recognition for wearable devices. Specifically, the time series data of gait cycles and GAF images are first transferred to the upper and lower layers of the DC-DSCNN model. The gait features are then extracted with a three-layer depthwise separable convolutional neural network (DSCNN) module. Next, the extracted features are transferred to a softmax classifier to implement gait recognition. To evaluate the performance of the proposed method, the gait dataset of 24 subjects were collected. Experimental results show that the recognition accuracy of the DC-DSCNN algorithm is 99.58%, and the memory usage of the model is only 972 KB, which verifies that the proposed method can enable gait recognition for wearable devices with lower power consumption and higher real-time performance.

Most existing gait recognition approaches adopt a two-step procedure: a preprocessing step to extract silhouettes or skeletons followed by recognition. In this paper, we propose an end-to-end model-based gait recognition method. Specifically, we employ a skinned multi-person linear (SMPL) model for human modeling, and estimate its parameters using a pre-trained human mesh recovery (HMR) network. As the pre-trained HMR is not recognition-oriented, we fine-tune it in an end-to-end gait recognition framework. To cope with differences between gait datasets and those used for pre-training the HMR, we introduce a reconstruction loss between the silhouette masks in the gait datasets and the rendered silhouettes from the estimated SMPL model produced by a differentiable renderer. This enables us to adapt the HMR to the gait dataset without supervision using the ground-truth joint locations. Experimental results with the OU-MVLP and CASIA-B datasets demonstrate the state-of-the-art performance of the proposed method for both gait identification and verification scenarios, a direct consequence of the explicitly disentangled pose and shape features produced by the proposed end-to-end model-based framework.