Head Pose Estimation on Low-Quality Images

The existing denoising methods for restricted Boltzmann machines (RBM) models are entirely based on their being undirected graphical models, when multiple RBM combination models are used, the distribution of the lower layer depends on the distribution of the higher layer. This leads to high computational complexity and subpar denoising performance and makes it difficult to generalize applications. To solve this problem, we propose a method for random noise removal from image data based on deep belief nets (DBN) with RBM models. The advantage of this method lies in the fact that the distribution of the lower layers does not depend on the distribution of the higher layers. Moreover, during the denoising process, it does not require training sets or pre-training, making it efficient and highly effective in denoising. The original image is added with random noise, and the noisy image is divided into several small blocks, which are pulled into vectors one by one, and then input into DBN model with two hidden layers to learn in batches. The original image is used as a label for fine-tuning, and finally the output is restored to the image, so as to eliminate the random noise. The DBN model algorithm is used to evaluate the effectiveness of removing random noise in the image data, simulated seismic data and real seismic data. The experimental results show that the proposed denoising method using DBN model is feasible in image data and seismic data. When the standard deviation of noise is 50 dB, the peak signal-to-noise ratio of the denoised images is improved by at least 2.08 dB and at least 6.99% on the Set12 dataset, and under different noise standard deviations, the proposed method outperforms other unsupervised learning algorithms and deep learning methods such as convolutional neural networks in removing random noise. This demonstrates that the RBM model has a strong performance in image feature learning and essential feature extraction. It also provides a new research idea and reference for image denoising methods in the engineering fields.

Comunicació presentada a la 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), celebrada els dies 30 de maig a 3 de juny de 2017 a Washington DC, EUA.

Restricted Boltzmann Machines (RBMs) have been developed for a lot of applications in the past few years, and many of its variants have also appeared. In this paper, RBM model and its learning algorithm with contrastive divergence algorithm will be introduced firstly. Then three important variants of RBM are presented in details, which are sparse RBM, discriminative RBM, and the Deep Boltzmann Machines (DBM). All the variants including original RBM are tested on MNIST handwriting digit dataset for classification task. Our empirical results demonstrate the advantage of RBM models and show that compared with other variants, the DBM is the best one in terms of the classification accuracy.

Restricted Boltzmann Machine (RBM) is a two layer undirected graph model that capable to represent complex distributions. Recent research has shown RBM-based approach has comparable performance with or even better performance than previous models on many collaborative filtering (CF) tasks. However, the intractable inference makes the training of RBM sophisticated, which prevents it from practical application. We present a novel feedforward neural framework for collaborative filtering called NACF, which is extended from the Neural Autoregressive Distribution Estimator (NADE). Because of the autoregressive feed-forward architecture, NACF can be trained with efficient stochastic gradient descent, instead of slow and hard-to-tune truncated Gibbs sampling for RBM. By introducing linear visible units and dual reversed ordering, NACF show faster convergence and better results than Probabilistic Matrix Factorization (PMF) and corresponding RBM models on MovieLens dataset. Besides, by combining NACF results, the rating prediction of efficientsignificantly improved, showing NACF is an effective and efficient model for collaborative filtering.

Riboswitches are structured allosteric RNA molecules that change conformation upon metabolite binding, triggering a regulatory response. Here we focus on the de novo design of riboswitch-like aptamers, the core part of the riboswitch undergoing structural changes. We use Restricted Boltzmann machines (RBM) to learn generative models from homologous sequence data. We first verify, on four different riboswitch families, that RBM-generated sequences correctly capture the conservation, covariation and diversity of natural aptamers. The RBM model is then used to design new SAM-I riboswitch aptamers. To experimentally validate the properties of the structural switch in designed molecules, we resort to chemical probing (SHAPE and DMS), and develop a tailored analysis pipeline adequate for high-throughput tests of diverse sequences. We probe a total of 476 RBM-designed and 201 natural sequences. Designed molecules with high RBM scores, with 20% to 40% divergence from any natural sequence, display ≈ 30% success rate of responding to SAM with a structural switch similar to their natural counterparts. We show how the capability of the designed molecules to switch conformation is connected to fine energetic features of their structural components.

The existing traditional neural network reconstruction models have some questions, including the high training epochs, low recognition rate, and complex structure. The restricted Boltzmann machine (RBM) is an excellent generative learning model for feature extraction, which is a simple model compared to other deep neural networks. The Discriminative Fuzzy Restricted Boltzmann Machine (DFRBM) is proposed by extending its parameters from natural numbers to fuzzy ones. Then we introduced the random permutation (RP) algorithm, with the hidden units random permutation, Discriminative Infinite Fuzzy Restricted Boltzmann Machine (Dis-iFRBM) is proposed. Dis-iFRBM is a better RBM model than DFRBM and Classical RBMs.We further investigate and compare the generative ability of the Dis-iFRBM on image reconstruction. First, we transform the MSTAR SAR piece to HRRPs images. Then the Dis-iFRBM, DFRBM, and Classical RBMs are compared in detail under optimal conditions on the HRRPs images that transformed from MSTAR data sets. Specifically, they can be trained by relatively limited datasets into excellent stand alone classifiers and retain satisfactory generative capability simultaneously. The comparison of experimental images shows that the Dis-iFRBM possesses better generative capability than the Discriminative Fuzzy Restricted Boltzmann Machine (DFRBM). Meanwhile, surveillance images in the city, license plate number recognition, and other scenarios need damaged image restoration. Dis-iFRBM as a model can save computational resources of terminal image devices that deploy in the Urban Internet of Things. The experiment results indicate that the Dis-iFRBM outperforms image restoration. It can achieve smaller average reconstruction errors (AREs) while given a small number of hidden units. Finally, experimentation over several classical RBMs revealed the proposed approach's preferable reconstruction capability.

Quantitative analysis of facial soft tissue using weighted cascade regression model applicable for facial plastic surgery

We present a new approach to localize extensive facial landmarks with a coarse-to-fine convolutional network cascade. Deep convolutional neural networks (DCNN) have been successfully utilized in facial landmark localization for two-fold advantages: 1) geometric constraints among facial points are implicitly utilized, 2) huge amount of training data can be leveraged. However, in the task of extensive facial landmark localization, a large number of facial landmarks (more than 50 points) are required to be located in a unified system, which poses great difficulty in the structure design and training process of traditional convolutional networks. In this paper, we design a four-level convolutional network cascade, which tackles the problem in a coarse-to-fine manner. In our system, each network level is trained to locally refine a subset of facial landmarks generated by previous network levels. In addition, each level predicts explicit geometric constraints (the position and rotation angles of a specific facial component) to rectify the inputs of the current network level. The combination of coarse-to-fine cascade and geometric refinement enables our system to locate extensive facial landmarks (68 points) accurately in the 300-W facial landmark localization challenge.

Facial landmark localization plays a critical role in face recognition and analysis. In this paper, we propose a novel cascaded Backbone-Branches Fully Convolutional Neural Network (BB-FCN) for rapidly and accurately localizing facial landmarks in unconstrained and cluttered settings. Our proposed BB-FCN generates facial landmark response maps directly from raw images without any pre-processing. It follows a coarse-to-fine cascaded pipeline, which consists of a backbone network for roughly detecting the locations of all facial landmarks and one branch network for each type of detected landmarks for further refining their locations. Extensive experimental evaluations demonstrate that our proposed BB-FCN can significantly outperform the state of the art under both constrained (i.e. within detected facial regions only) and unconstrained settings.

Facial landmark localization is well known as one of the bottlenecks in face recognition. This paper proposes a novel facial landmark localization method, which introduces facial context constrains into cascaded AdaBoost framework. The motivation of our method lies in the basic human physiology observation that not only the local texture information but also the global context information is used together for human to realize the landmark location task. Therefore, in our solution, a novel type of Haar-like feature, called discontinuous Haar-like feature, is proposed to characterize the facial context, i.e. the cooccurrence relationship between target facial landmark and other local texture patterns within face region (including other landmarks, facial organs and also smoothing regions). For the locating task, traditional Haar-like features (characterizing local texture information) and discontinuous Haar-like features (characterizing context constrains in global sense) are combined together to form more powerful representations. Through Real AdaBoost learning, distinctive features are selected automatically and used for facial landmark detection. Our experiments on BioID and Cohn-Kanade databases have validated the proposed method by comparing with other state-of-the-art results.

The next generation D-Wave quantum computer (QC) having more than 2000 qubits was demonstrated to be adequate for embedding graphs of restricted Boltzmann machines (RBMs) having the connectivity between RBM units (though still limited) already sufficient for successful training by classical algorithms. As a verification of the validity and quality of the employed embedding approach, the RBM was trained by classical contrastive divergence (CD), while the D-Wave was initially used only in the recognition step for reconstruction of incomplete test images (8 × 8 bars-and-stripes), and for their classification. The label classification errors by the D-Wave compared favorably to those obtained by classical Gibbs sampling, indicating that the QC was successfully finding the most energetically favorable combinations of the unknown visible RBM units/labels for the given fixed incomplete input image. The valid RBM embedding was further applied to investigate opportunities for using QCs in the RBM training, and specifically for replacing the classical Gibbs technique in generating a representative sample from the RBM model distribution. Statistical comparison of samples obtained by the Gibbs technique versus 10000 sample states generated from the D-Wave revealed significant differences in the observed outcomes. The D-Wave samples were found insufficiently representative of the model distribution, specifically by missing many local valley found by the Gibbs sampling. On the other hand, the D-Wave possessed an ability to find local valleys of the configuration space that were consistently missed by the classical sampling algorithms, which could potentially be very interesting for sampling applications.

3D facial landmarking is becoming a fundamental part of clinical and biological applications. Manual landmarking is time consuming and prone to cumulative errors, so attempts have been made to automate 3D facial landmarking. However, data in the literature are sparse. The objectives of this study are to investigate current evidence for the accuracy and reliability of various 3D facial automated landmarking methods used in medical and biological studies and evaluate their performance against the manual annotation method. Electronic and manual searches of the literature were performed in April 2021. Only studies that were published in English and evaluated the accuracy of automated landmarking algorithms in 3D facial images for medical or biological settings were included. Two authors independently screened the articles for eligibility. The QUADAS-2 tool was used for the quality analysis of the included studies. Due to the heterogeneity of the selected studies, a meta-analysis was not possible, so a narrative synthesis of the findings was performed. From 1002 identified records, after applying the inclusion and exclusion criteria, 14 articles were ultimately selected, read, and critically analysed. Different algorithms were used for the automated 3D landmarking of various numbers of facial landmarks ranging from 10 to 29 landmarks. The average difference between the manual and automated methods ranged from 0.67 to 4.73 mm, and the best performance was achieved using deep learning models. Poor study design and inadequate reporting were found in the implementation of the reference standards and population selection for the intended studies, which could have led to overfitting of the tested algorithm. This systematic review was limited by the quality of the included studies and uncovered several methodological limitations evident in the corresponding literature. Compared to manual landmarking, automated Landmark localization of individual facial landmarks reported in the literature is not accurate enough to allow their use for clinical purposes. This result indicates that automatic facial landmarking is still developing, and further studies are required to develop a system that could match or exceed the performance of the current gold standard. PROSPERO: CRD42021241531.

For accurate and fast detection of facial landmarks, we propose a new facial landmark detection method. Previous facial landmark detection models generally perform a face detection step before landmark detection. This greatly affects landmark detection performance depending on which face detection model is used. Therefore, we propose a model that can simultaneously detect a face region and a landmark without performing the face detection step before landmark detection. The proposed single-shot detection model is based on the framework of YOLOv3, a one-stage object detection method, and the loss function and structure are altered to learn faces and landmarks at the same time. In addition, EfficientNet-B0 was utilized as the backbone network to increase processing speed and accuracy. The learned database used 300W-LP with 64 facial landmarks. The average normalized error of the proposed model was 2.32 pixels. The processing time per frame was about 15 milliseconds, and the average precision of face detection was about 99%. As a result of the evaluation, it was confirmed that the single-shot detection model has better performance and speed than the previous methods. In addition, as a result of using the COFW database, which has 29 landmarks instead of 64 to verify the proposed method, the average normalization error was 2.56 pixels, which was also confirmed to show promising performance.

Facial landmark localization is an essential component for facial recognition which is a vital technology for patients tracking in the situation of a global pandemic of COVID-19. Parameter amount and computation cost are two key factors affecting the application of facial landmark localization. In addition, the intrinsic imbalance of head pose in existing datasets has a great impact on the model generalization. To address these problems, this paper proposed a lightweight model LiteFace with multi-knowledge distillation and a pose-aware resampling strategy. Specifically, we proposed to extract multi-scale features and enhance the receptive filed of network by delicate model design to obtain an efficient architecture and improved the performance of the lightweight model by multi-knowledge distillation which combines knowledge distillation and mutual learning in a two-stage manner. To further improve the model performance and boost the generalization, we proposed a pose-aware re-sampling strategy that generates samples with different head poses and utilized the LaPa dataset to generate masked face images to increase data diversity. We conducted extensive ablation studies on the model design, multi-knowledge distillation and resampling strategy. The proposed method achieves 1.93% and 3.04% NME on the JD-landmark-mask val and test dataset respectively. Finally, LiteFace wins the third place in the 3rd Grand Challenge of 106-Point Facial Landmark Localization.

Facial landmark localization on images with occlusions is an important and challenging task in many visual applications. Recently, the cascaded pose regression has attracted increasing attention, since it achieved superior performance in terms of facial landmark localization under occlusions. However, such approach is sensitive to initialization, where an improper initialization will decrease the performance sharply. In this paper, we propose a novel initialization method to get a robust initial shape by analysing correlation of Local Binary Patterns (LBP) histograms between the estimated face and training faces. The shape of the training face that is most correlated with the estimated face, will be selected as the initialization for the regression. The selected shape is closer to the real shape of the estimated face, which makes the landmark localization more accurate. Besides, in order to make the initial shape more robust to occlusions, we propose a boosted smart restarts technique by checking location and occlusion jointly instead of checking location only. We show that the proposed method significantly improves performance over existing landmark localization methods on the challenging dataset of COFW. The experimental results demonstrate that the proposed method reduces error by 11.9% and failure cases by 20.8% on COFW dataset. Moreover, it detects face occlusions with 85/40% precision/recall.