Local derivative pattern for action recognition in depth images

This paper proposes a novel high-order local pattern descriptor, local derivative pattern (LDP), for face recognition. LDP is a general framework to encode directional pattern features based on local derivative variations. The n(th)-order LDP is proposed to encode the (n-1)(th) -order local derivative direction variations, which can capture more detailed information than the first-order local pattern used in local binary pattern (LBP). Different from LBP encoding the relationship between the central point and its neighbors, the LDP templates extract high-order local information by encoding various distinctive spatial relationships contained in a given local region. Both gray-level images and Gabor feature images are used to evaluate the comparative performances of LDP and LBP. Extensive experimental results on FERET, CAS-PEAL, CMU-PIE, Extended Yale B, and FRGC databases show that the high-order LDP consistently performs much better than LBP for both face identification and face verification under various conditions.

In this paper, a novel local pattern descriptor generated by the proposed local vector pattern (LVP) in high-order derivative space is presented for face recognition. The proposed vector representation of the referenced pixel is generated to provide the one-dimensional structure of micropatterns. To effectively extract more detailed discriminative information in a given sub-region, the vector of LVP is refined by varying local derivative directions from the nth-order LVP in (n−1)th-order derivative space. The proposed LVP is compared with the existing local pattern descriptors including local binary pattern (LBP), local derivative pattern (LDP), and local tetra pattern (LTrP) to evaluate the performances from input grayscale face images. Extensive experiments conducting on benchmark face image databases, FERET and Extended Yale B, demonstrate that the proposed LVP in high-order derivative space indeed performs much better than LBP, LDP and LTrP for face recognition.

With development of RGB-D sensors, high-quality depth images are obtained easily. In this paper, we investigate the depth and skeleton information obtained from Kinect sensor for person reidentification and consider using inexpensive depth camera device known as Kinect camera. Using depth and skeleton information, some challenging problems in person reidentification as illumination and computation complexity are considered and new solutions are specified for the issues. In this paper, histograms of local binary patterns, local derivative patterns, and local tetra patterns are computed as features for person reidentification. Then, these histograms are fused with anthropometric features using score-level fusion. The proposed methods are applied on two database: RGBD-ID database and KinectREID database. Finally, experimental results demonstrate the validity of the proposed methods.

Texture is the surface property that is used to identify and recognise objects. This property is widely used in many applications including texture-based face recognition systems, surveillance, identity verification and so on. The Local binary pattern (LBP) texture method is most successful for face recognition. Owing to the great success of LBP, recently many models, which are variants of LBP have been proposed for texture analysis. Some of the derivatives of LBPs are multivariate local binary pattern, centre symmetric local binary pattern, local binary pattern variance, dominant local binary pattern, advanced local binary pattern, local texture pattern (LTP) and local derivative pattern (LDP). In this scenario, it is essential to review, whether LBP or their derivatives perform better for face recognition. The real-time challenges such as illumination changes, rotations, angle variations and facial expression variations are evaluated by different LBP-based models. Experiments were conducted on the Japanese female facial expression, YALE and FRGC version2 databases. The results show that LDP and LTP perform much better than the other LBP-based models.

Palmprint is a reliable and unique biometric trait with high acceptability. In this paper, we propose a new Local Composition Derivative Pattern (LCDP) for palmprint recognition. LCDP extracts first order derivative information of images along radial and directional directions which can capture more detailed information than the non-directional local binary pattern (LBP). Different from LBP encoding just the binary result of the radial derivative among central pixel and local neighbors by using a simple threshold function with static threshold value, the LCDP extracts discriminative local information by composition of both radial and directional derivatives information by using a threshold function with dynamic threshold value which obtains by first-order derivative information among local neighbors. Experimental evaluation through palmprint recognition on the Hong Kong Polytechnic University 2D_3D_palmprint database demonstrates the LCDP performs much better than LBP for palmprint identification.

Histogram refinement for texture descriptor based image retrieval

Face recognition has lot of challenges in biometrics. The challenges are addressed effectively by local pattern descriptors. The idea of local descriptors is to determine the feature vector and then compute the difference between test images with training images by using similarity measure. Based on the observation, local approaches attained better performance rate than other approaches in face recognition. Due to that, researchers made a significant attention on local descriptors for face recognition. For nonlinear subspace, the local descriptors will achieve better result than holistic approach. Local pattern descriptor follows simple procedure to extract the facial features. The steps are face alignment, face representation, and matching. Face alignment is the first step of local descriptor, which is used to divide the image into several blocks. Face representation is used to extract the meaningful information from each region. This local feature extraction method carries discriminant information of the region; it will improve the classification rate and matching rate. Local descriptors extract the discriminant information from the neighbors by setting a threshold value as center pixel value, and it is not capable of extracting the detailed information from microstructure. Finally, matching by classification techniques or distance measure is used to identify or verify the person. The local pattern descriptors are more robust against pose, lighting, and scale variations. This chapter describes the various local pattern descriptors and shows the effectiveness of the descriptor. The results of local pattern descriptors are experimented on standard benchmark databases such as FERET, Extended Yale-B, ORL, CAS-PEAL, LFW, JAFFE, and Cohn–Kannade.

A new image feature descriptor for content based image retrieval using scale invariant feature transform and local derivative pattern

Action recognition in static image is challenging. The authors propose mutually incoherent pose bases which are implicit poselet co‐occurrences and are learned by dictionary training to describe body pose. Poselets in a pose basis are not constrained in space and quantity, thus pose basis can describe body pose more flexibly than k ‐poselet. In their method, body pose in an image is represented by a sparse linear combination of pose bases because pose in an action varies while each image only captures a snapshot from a single viewpoint. In dictionary training, the challenge is how to stabilise the sparse representation which is the input of Support Vector Machine (SVM) for action recognition, because the original pose signal is ambiguous while dictionary is an over complete matrix. Their solution is to add cumulative coherence as penalty in objective function and induce pose bases become mutually incoherent. They evaluate the method on two popular datasets and experiment results show the pose representation has encouraging performance in action recognition. Furthermore, they empirically exploit the complementary role of the local pose feature with deep convolutional neural network features from holistic image. Experiment results demonstrate aggressive performance improvement by concatenating the two features.

Local feature descriptor plays a significant role in texture classification. However, in the traditional local binary pattern method, image pixels are converted into a binary pattern based on the relationship between center and neighborhood pixels. This paper introduces a novel feature extraction method named local neighbourhood edge responsive binary pattern to extract and categorize reliable texture features from images for achieving an automated classification of different types of ocean bottom sediments. Initially, the basic magnitude variance pixel values are derived depending on an odd and even pixel value of a 3 × 3 image patch. Further, the edge information is extracted using the local directional pattern method from all images. The edge response of the image is obtained using a kirsch mask in all eight directions. The encoding condition is then applied to both the local intensity and the edge information to create a unique descriptor value. Finally, a new learning algorithm called GMJAYA-ELM combines the Gaussian mutated JAYA (GMJAYA) with an extreme learning machine (ELM) for texture classification. The GMJAYA is used to optimize the input weights and hidden biases of single-hidden-layer feed-forward neural networks. Experimental findings indicate that the suggested method yields better accuracy and sensitivity efficiency among various groups. The proposed algorithm is tested by contrasting outcomes with standard learning schemes such as PSO-ELM, GA-ELM, ABC-ELM, Birdswarm-ELM, and JAYA-ELM, suggesting the dominance of GMJAYA-ELM.

Gender recognition for interactive functions becomes essential topic in terms of service robotics applications. Ensemble learning which combines multiple classifiers prediction is now an active area of research in Machine Learning and Pattern Recognition. We propose an ensemble learning to facilitate gender classification. The features which we use are raw data (image pixels as input), Local binary pattern (LBP), Local derivative pattern (LDP), Gabor, and Weber local descriptors (WLD). We carry out comparative experimental studies of various gender recognition schemes, including Eigenfaces, any individual classifiers, Rotation Forest, and Adaboost. Specifically, not only individual classifiers but also ensemble classifiers are based on Support Vector Machine, namely using SVM as component classifier in ensemble learning. Furthermore, we evaluate the effect of image size on classification rate. In conclusion, we find that the best classification rate is achieved with Discrete Adaboost with SVM as component classifier using the aforementioned features. Another finding is that the classification rates will increase when face image size increases.

Automatic image annotation is the process of providing tags to salient objects in the image. The aim is achieved by first identifying salient objects. For this, the traditional gradient vector flow (GVF) model is modified to incorporate saliency by adding minimum directional contrast to the data part in the energy functional of GVF. To provide tags, three features: color, local binary pattern and local direction pattern are used. Classification is done by modifying cluster-based multi-label learning with feature-induced labeling information enrichment (C-MLFE) and is termed as C’-MLFE. This involves clustering the training data into two sets. For each cluster, a squared weight matrix records the influence of each instance on the other. This relationship among the training instance is used to enrich the labeling information of the test set. The result is compared with six state-of-the-art algorithms.

In this paper, we propose an end-to-end deep learning framework that could extract the global spatial-temporal features from the videos based on the NetVladframework optimized with data augmentation. Given keyframes extracted from the original videos, we have presented a three-step action recognition framework: the first step of our framework is given by a data augmentation based on central crop, random crop and keyframe scaling. Then the second step is given by a local feature descriptor of each frame with Two-Stream Inflated 3D ConvNet (I3D) that is based on 2D ConvNet inflation, which enables the network to capture spatial and temporal features simultaneously. The third step is given by aggregating global features for action recognition through the new generalized “Vector of Locally Aggregated Descriptors” (NetVLAD) layer optimized with a novel pooling strategy, avoiding the misjudgment caused by local features. The whole framework is trained and fine-tuned via an end-to-end way. It's demonstrated that the performance of our framework outperforms the state-of-the-art algorithms on UCF10l dataset. The competitive results clearly reveal an efficient action recognition of high accuracy (up to 91.25%) in fast time (close to 1.8s), which will significantly improve performance and application of video action recognition.

An improved shifted phase-encoded fringe-adjusted joint transform correlation technique is proposed in this paper for face recognition which can accommodate the detrimental effects of noise, illumination, and other 3D distortions such as expression and rotation variations. This technique utilizes a third order local derivative pattern operator (LDP3) followed by a shifted phase-encoded fringe-adjusted joint transform correlation (SPFJTC) operation. The local derivative pattern operator ensures better facial feature extraction in a variable environment while the SPFJTC yields robust correlation output for the desired signals. The performance of the proposed method is determined by using the Yale Face Database, Yale Face Database B, and Georgia Institute of Technology Face Database. This technique has been found to yield better face recognition rate compared to alternate JTC based techniques.

Video-based human action recognition is one of the most important and challenging areas of research in the field of computer vision. Human action recognition has found many pragmatic applications in video surveillance, human-computer interaction, entertainment, autonomous driving, etc. Owing to the recent development of deep learning methods for human action recognition, the performance of action recognition has significantly enhanced for challenging datasets. Deep learning techniques are mainly used for recognizing actions in images and videos comprising of Euclidean data. A recent development in deep learning methods is the extension of these techniques to non-Euclidean data or graph data with many nodes and edges. Human body skeleton resembles a graph, therefore, the graph convolutional network (GCN) is applicable to the non-Euclidean body skeleton. In the past few years, GCN has emerged as an important tool for skeleton-based action recognition. Therefore, we conduct a survey using GCN methods for action recognition. Herein, we present a comprehensive overview of recent GCN techniques for action recognition, propose a taxonomy for the categorization of GCN techniques for action recognition, carry out a detailed study of the benchmark datasets, enlist relevant resources and open-source codes, and finally provide an outline for future research directions and trends. To the best of authors' knowledge, this is the first survey for action recognition using GCN techniques.