Robust facial landmark detection by probability‐guided hourglass network

In martial arts action recognition, complex poses, occlusions, and dynamic changes can lead to insufficient estimation accuracy, and traditional methods suffer from inaccurate joint localization and poor recognition of continuous pose jumps. In response to this, the research proposes the Dilated Convolution-Attention-Stacked Hourglass Network Pose Estimation (MS-DConv-Att-SHN) to achieve martial arts action recognition. Firstly, multiple dilated convolutions and mixed attention are proposed to improve the computational complexity and loss of joint information in stacked hourglass networks, enhancing the ability to capture subtle joint displacements. This is crucial for accurate estimation of complex poses such as movement, flicker, and virtual real transformations in martial arts. Secondly, in response to the strong coherence of martial arts movements, local feature refinement and channel fusion techniques are used to enhance the correlation analysis between consecutive action frames, solving the problem of traditional methods’ fragmented recognition of action chains such as “exertion contraction.” A martial arts action recognition system based on the improved MS-DConv-Att-SHN method has been developed to better identify individual movements and capture the intrinsic relationships between movements in routines. This provides key technical support for the digital inheritance, intelligent evaluation, and standardized promotion of martial arts, making it more closely aligned with the movement characteristics of martial arts that combine form and spirit. The results indicate that structural improvements to the stacked hourglass network can effectively increase its percentage of correct keypoints (PCK) and mean average precision (mAP) for both datasets, with PCK and mAP values exceeding 92% and 85%, respectively. The average recognition accuracy of attitude keypoints in the MS-DConv-Att-SHN model is superior to other comparison models, with a difference of over 1.2% compared to other models. The improved MS-DConv-Att-SHN model achieves recognition accuracy of over 90% for different martial arts movements, showing smaller parameter counts and PCK values compared to other comparative models. The research method can effectively provide technical support for the automation analysis of martial arts movements, sports training assistance systems, and intelligent martial arts teaching.

Human pose estimation, a research hotspot of computer vision, has been a key and hard task. Recently, significant progress has been achieved using the newly emergent deep convolutional neural network technique. Many methods first apply the bottom-up and top-down structure, which uses first downsampling and then upsampling. This structure can capture multi-scale feature but lead to loss of information due to downsampling. In this work, we propose a Dilated Convolutional Module (DCM) based on dilated convolution and skip connections. The proposed module, which aims to reduce the loss of information, can expand receptive field and reduce the times of downsampling. Using DCMs as building blocks, this work proposes a novel convolutional network named Dilated Hourglass Network, based on Stacked Hourglass Network. The proposed network is compared with the Stacked Hourglass Network on the standard benchmarks MPII dataset for human pose estimation, which shows that the proposed network achieves a notable progress.

The Hausdorff Distance (HD) is widely used in evaluating medical image segmentation methods. However, the existing segmentation methods do not attempt to reduce HD directly. In this paper, we present novel loss functions for training convolutional neural network (CNN)-based segmentation methods with the goal of reducing HD directly. We propose three methods to estimate HD from the segmentation probability map produced by a CNN. One method makes use of the distance transform of the segmentation boundary. Another method is based on applying morphological erosion on the difference between the true and estimated segmentation maps. The third method works by applying circular/spherical convolution kernels of different radii on the segmentation probability maps. Based on these three methods for estimating HD, we suggest three loss functions that can be used for training to reduce HD. We use these loss functions to train CNNs for segmentation of the prostate, liver, and pancreas in ultrasound, magnetic resonance, and computed tomography images and compare the results with commonly-used loss functions. Our results show that the proposed loss functions can lead to approximately 18-45% reduction in HD without degrading other segmentation performance criteria such as the Dice similarity coefficient. The proposed loss functions can be used for training medical image segmentation methods in order to reduce the large segmentation errors.

Decision letter: Visual and motor signatures of locomotion dynamically shape a population code for feature detection in Drosophila

Cascade regression framework has been shown to be effective for facial landmark detection. It starts from an initial face shape and gradually predicts the face shape update from the local appearance features to generate the facial landmark locations in the next iteration until convergence. In this paper, we improve upon the cascade regression framework and propose the Constrained Joint Cascade Regression Framework (CJCRF) for simultaneous facial action unit recognition and facial landmark detection, which are two related face analysis tasks, but are seldomly exploited together. In particular, we first learn the relationships among facial action units and face shapes as a constraint. Then, in the proposed constrained joint cascade regression framework, with the help from the constraint, we iteratively update the facial landmark locations and the action unit activation probabilities until convergence. Experimental results demonstrate that the intertwined relationships of facial action units and face shapes boost the performances of both facial action unit recognition and facial landmark detection. The experimental results also demonstrate the effectiveness of the proposed method comparing to the state-of-the-art works.

Facial feature detection algorithms have seen great progress over the recent years. However, they still struggle in poor lighting conditions and in the presence of extreme pose or occlusions. We present the Constrained Local Neural Field model for facial landmark detection. Our model includes two main novelties. First, we introduce a probabilistic patch expert (landmark detector) that can learn non-linear and spatial relationships between the input pixels and the probability of a landmark being aligned. Secondly, our model is optimised using a novel Non-uniform Regularised Landmark Mean-Shift optimisation technique, which takes into account the reliabilities of each patch expert. We demonstrate the benefit of our approach on a number of publicly available datasets over other state-of-the-art approaches when performing landmark detection in unseen lighting conditions and in the wild.

Achieving a balance between accuracy and efficiency is a critical challenge in facial landmark detection (FLD). This paper introduces Parallel Optimal Position Search (POPoS), a high-precision encoding-decoding framework designed to address the limitations of traditional FLD methods. POPoS employs three key contributions: (1) Pseudo-range multilateration is utilized to correct heatmap errors, improving landmark localization accuracy. By integrating multiple anchor points, it reduces the impact of individual heatmap inaccuracies, leading to robust overall positioning. (2) To enhance the pseudo-range accuracy of selected anchor points, a new loss function, named multilateration anchor loss, is proposed. This loss function enhances the accuracy of the distance map, mitigates the risk of local optima, and ensures optimal solutions. (3) A single-step parallel computation algorithm is introduced, boosting computational efficiency and reducing processing time. Extensive evaluations across five benchmark datasets demonstrate that POPoS consistently outperforms existing methods, particularly excelling in low-resolution heatmaps scenarios with minimal computational overhead. These advantages make POPoS as a highly efficient and accurate tool for FLD, with broad applicability in real-world scenarios.

Locating facial landmarks like eyes, eyebrows, nose, lips, facial outline, etc. can be used as the foundation for generating facial deformations caused by expressions and head movements, which is very crucial in 3D animation, the VFX industry, the gaming industry, VR & AR. This project intends to focus on building a robust and real-time Facial Landmark detection model by improving upon existing models. Over the past few years, a lot of research has been done for detecting facial landmarks without the use of markers. Yet, there is a huge scope for improvement in terms of accuracy and speed. There are three major types of facial Landmark detection algorithms: Holistic methods, Constrained Local Model (CLM), and regression-based methods. Holistic methods try to build a Global representation of facial appearance accurately but require a lot of processing power and time. CLM Build a local representation of facial appearance whereas, regression-based methods simply capture the facial shape and appearance information. Classical regression techniques are fast but have many limitations, for instance, they fail to detect landmarks in occluded faces. Neural networks and deep learning have opened up new possibilities in building models that are efficient, robust, and dependable. Though deep learning models are processor-intensive tasks, recent improvements in network compression and pruning have made it possible to use deep learning models in real-time on fairly low specification computers. This study intends to examine new combinations of previous implementations, new ideas, and heuristic information. The proposed model will be better in terms of speed than other models and on par if not better in terms of accuracy.KeywordsFacial landmark detectionCNNDeep learningMachine learningAI

Due to the COVID-19 pandemic, the necessity for a contactless biometric system able to recognize masked faces drew attention to the periocular region as a valuable biometric trait. However, periocular recognition remains challenging for deployments in the wild or in unconstrained environments where images are captured under non-ideal conditions with large variations in illumination, occlusion, pose, and resolution. These variations increase within-class variability and between-class similarity, which degrades the discriminative power of the features extracted from the periocular trait. Despite the remarkable success of convolutional neural network (CNN) training, CNN requires a huge volume of data, which is not available for periocular recognition. In addition, the focus is on reducing the loss between the actual class and the predicted class but not on learning the discriminative features. To address these problems, in this paper we used a pre-trained CNN model as a backbone and introduced an effective deep CNN periocular recognition model, called linear discriminant analysis CNN (LDA-CNN), where an LDA layer was incorporated after the last convolution layer of the backbone model. The LDA layer enforced the model to learn features so that the within-class variation was small, and the between-class separation was large. Finally, a new fully connected (FC) layer with softmax activation was added after the LDA layer, and it was fine-tuned in an end-to-end manner. Our proposed model was extensively evaluated using the following four benchmark unconstrained periocular datasets: UFPR, UBIRIS.v2, VISOB, and UBIPr. The experimental results indicated that LDA-CNN outperformed the state-of-the-art methods for periocular recognition in unconstrained environments. To interpret the performance, we visualized the discriminative power of the features extracted from different layers of the LDA-CNN model using the t-distributed Stochastic Neighboring Embedding (t-SNE) visualization technique. Moreover, we conducted cross-condition experiments (cross-light, cross-sensor, cross-eye, cross-pose, and cross-database) that proved the ability of the proposed model to generalize well to different unconstrained conditions.

Attentive occlusion-adaptive deep network for facial landmark detection

Convolutional neural networks (CNN) have been attracting attention for accurate scene parsing including a human pose estimation. However, CNN requires a massive amount of floating-point operations, so it is difficult to realize CNN on low-cost devices. Thus, we propose RegionDrop, an annotation-aware spatially sparse network, which skips computations of unnecessary spatial regions in activations. We present a novel loss that directly uses annotations so that important activation regions are retained. We also developed an efficient sparse GPU kernel to accelerate processing speed of both depthwise and general \(K\times K\) convolutional layers. Our RegionDrop is evaluated by using two pose estimation networks, a modified stacked hourglass network, and an HRNet. RegionDrop using an hourglass network archived 3.2 times faster processing speed compared with a non-sparse network, and 1.8 times faster processing speed than a prior spatially sparse network, with no accuracy degradation. Moreover, the processing speed of RegionDrop using HRNet is increased by a factor of 2.0 with negligible accuracy loss.KeywordsConvolutional neural networksPose estimationSpatially sparse networkGPU

Face recognition in real world applications is a very difficult task because of image misalignments, pose and illumination variations, or occlusions. Many researchers in this field have investigated both face representation and classification techniques able to deal with these drawbacks. However, none of them is free from limitations. Early proposed algorithms were generally holistic, in the sense they consider the face object as a whole. Recently, challenging benchmarks demonstrated that they are not adequate to be applied in unconstrained environments, despite of their good performances in more controlled conditions. Therefore, the researchers' attention is now turning on local features that have been demonstrated to be more robust to a large set of non-monotonic distortions. Nevertheless, though local operators partially overcome some drawbacks, they are still opening new questions (e.g., Which criteria should be used to select the most representative features?). This is the reason why, among all the others, hybrid approaches are showing a high potential in terms of recognition accuracy when applied in uncontrolled settings, as they integrate complementary information from both local and global features. This chapter explores local, global, and hybrid approaches.

In order to make the elderly care service system intelligent, this paper proposes a human posture recognition algorithm based on big data analysis and artificial intelligence technology. The algorithm framework includes three parts: human target detection, human posture estimation and human action recognition. Firstly, this paper adopts the obscured human body detection algorithm based on the mixed attention mechanism to detect the human bounding box. Then, to solve the problem that the detected human bounding box may have inaccurate positioning leading to the human body position deviation, the space transformation network is used to affine the coordinates of the human bounding box to correct its position. Finally, the stacked hourglass network is used to predict the joints of the detected human body, so as to realize the recognition of human posture. In this paper, the algorithm is trained and tested on the open pedestrian test set. The experimental results show that the algorithm in this paper can effectively and correctly estimate the visible parts of human posture, and reduce the influence of obstacle occlusion on human posture estimation and human fall detection. The evaluation index of human posture recognition (Kps AP) reached 68.7%, indicating that the posture recognition algorithm proposed in this paper can realize the rapid recognition and estimation of human posture without shelter, and the algorithm can complete the fall recognition of the elderly in the elderly service system, and can effectively prevent the elderly from falling after the subsequent big data statistics and analysis of the elderly's posture. It has good practicability.

Pose estimation has been a hot topic in the field of machine vision in recent years. Animals exist widely in nature, and the analysis of their shape and movement is important in many fields and industries. In the pose estimation task, to improve the detection accuracy, the existing models often need to consume a lot of computing and memory resources. Therefore, it is a key problem for the pose estimation methods to carry out a lightweight model and reduce the computational overhead on the premise of ensuring model accuracy. In this paper, we focus on the structure of the convolutional neural network in animal pose estimation, construct a lightweight and efficient stacked hourglass network model oriented to optimize the balance of model computation and accuracy, and implement the application algorithm design based on it. Aiming at the problem of large parameters in depthwise convolutional neural networks, a lightweight residual module is proposed, that is, based on the lightweight efficient channel attention improved conditional channel-weighted method (ICCW-Bottle), thereby reducing the weight of the network and obtaining the feature information of different scales. Given the problem that a large amount of feature information is easily lost after the network pooling operation, a lightweight dual-branch fusion module is proposed that fully integrates high-level semantic information and low-level detailed features under the condition of a small number of parameters. Finally, the same as the CC-SSL method: the model is trained jointly using synthetic and real animal datasets, but the CC-SSL method does not take into account the computational power of the model, which consumes a lot of time and memory to run. Through experiments, it is known that compared with the CC-SSL method, the PCK@0.05 of this method is increased by 5.5% on the TigDog dataset. The model in this paper reduces the number of parameters and calculations of the network while ensuring less information loss and model accuracy. The ablation experiment verifies the advancement and effectiveness of the overall network.

For accurate and fast detection of facial landmarks, we propose a new facial landmark detection method. Previous facial landmark detection models generally perform a face detection step before landmark detection. This greatly affects landmark detection performance depending on which face detection model is used. Therefore, we propose a model that can simultaneously detect a face region and a landmark without performing the face detection step before landmark detection. The proposed single-shot detection model is based on the framework of YOLOv3, a one-stage object detection method, and the loss function and structure are altered to learn faces and landmarks at the same time. In addition, EfficientNet-B0 was utilized as the backbone network to increase processing speed and accuracy. The learned database used 300W-LP with 64 facial landmarks. The average normalized error of the proposed model was 2.32 pixels. The processing time per frame was about 15 milliseconds, and the average precision of face detection was about 99%. As a result of the evaluation, it was confirmed that the single-shot detection model has better performance and speed than the previous methods. In addition, as a result of using the COFW database, which has 29 landmarks instead of 64 to verify the proposed method, the average normalization error was 2.56 pixels, which was also confirmed to show promising performance.