Face alignment by minimizing the closest classification distance

In this paper we present a new method for accurate real-time facial feature detection. Our method is based on local feature detection and enhancement. Previous work in this area, such as that of Viola and Jones, require looking at the face as a whole. Consequently, such approaches have increased chances of reporting negative hits. Furthermore, such algorithms require greater processing power and hence they are especially not attractive for real-time applications. Through our recent work, we have devised a method to identify the face from real-time images and divide it into regions of interest (ROI). Firstly, based on a face detection algorithm, we identify the face and divide it into four main regions. Then, we undertake a local search within those ROI, looking for specific facial features. This enables us to locate the desired facial features more efficiently and accurately. We have tested our approach using the Cohn-Kanade’s Extended Facial Expression (CK+) database. The results show that applying the ROI has a relatively low false positive rate as well as provides a marked gain in the overall computational efficiency. In particular, we show that our method has a 4-fold increase in accuracy when compared to existing algorithms for facial feature detection.

Robust face alignment and tracking by combining local search and global fitting

Head pose and facial feature detection are important for face analysis. However, many studies reported good results in constrained environment, the performance could be decreased due to the high variations in facial appearance, poses, illumination, occlusion, expression and make-up. In this paper, we propose a hierarchical regression approach, Dirichlet-tree enhanced random forests (D-RF) for face analysis in unconstrained environment. D-RF introduces Dirichlet-tree probabilistic model into regression RF framework in the hierarchical way to achieve the efficiency and robustness. To eliminate noise influence of unconstrained environment, facial patches extracted from face area are classified as positive or negative facial patches, only positive facial patches are used for face analysis. The proposed hierarchical D-RF works in two iterative procedures. First, coarse head pose is estimated to constrain the facial features detection, then the head pose is updated based on the estimated facial features. Second, the facial feature localization is refined based on the updated head pose. In order to further improve the efficiency and robustness, multiple probabilitic models are learned in leaves of the D-RF, i.e. the patch’s classification, the head pose probabilities, the locations of facial points and face deformation models (FDM). Moreover, our algorithm takes a composite weight voting method, where each patch extracted from the image can directly cast a vote for the head pose or each of the facial features. Extensive experiments have been done with different publicly available databases. The experimental results demonstrate that the proposed approach is robust and efficient for head pose and facial feature detection.

We propose an effective facial features detection method for human– robot interaction (HRI) with indoor mobile robot. In case of human-robot interaction, its vision system has to manage such difficult problems as pose variations, illumination changes, and complex backgrounds, which is caused by the mobility of robot. In this paper, in order to overcome such difficult problems, we suggest a facial feature detection method based on local image area and direct pixel-intensity distributions, in which we propose two novel concepts; the directional template for evaluating intensity distributions and the edge-like blob map with multiple strength intensity. Using this proposed blob map, we show that the locations of major facial features – two eyes and a mouth – can be successfully estimated. Without the information of facial area boundary, final candidate face region is determined by both obtained locations of facial features and weighted correlations with stored facial templates. Our approach is also a flexible algorithm that can be applicable to both color and gray image. In case of color image, faster detection of both facial features and face is feasible by using the chromatic property of facial color. Experimental results from many color images and well-known gray level face database images prove the usefulness of proposed algorithm in human– robot interaction applications.

At present, the correct recognition rate of face recognition algorithm is limited under unconstrained conditions. To solve this problem, a face recognition algorithm based on deep learning under unconstrained conditions is proposed in this paper. The algorithm takes LBP texture feature as the input data of deep network, and trains the network layer by layer greedily to obtain optimized parameters of network, and then uses the trained network to predict the test samples. Experimental results on the face database LFW show that the proposed algorithm has higher correct recognition rate than some traditional algorithms under unconstrained conditions. In order to further verify its effectiveness and universality, this algorithm was also tested in YALE and YALE-B, and achieved a high correct recognition rate as well, which indicated that the deep learning method using LBP texture feature as input data is effective and robust to face recognition.

Facial feature detection is a well-studied field. Efficient facial feature detection is significant in face analysis based applications, especially on mobile devices. Balance between accuracy and time efficiency is a practical problem in real time applications. This paper aims at proposing a real-time and accurate algorithm for facial feature detection. It is based on the assumption that classifiers may improve performance by limiting searching region. We propose a simplified Active Shape Model (ASM) to speed up such searching process. To ensure accuracy, several facial feature detectors are compared, such as the Adaboost classifiers with the Haar-feature, and the random forest classifiers. Since the simplified ASM provides a good constraint to different facial features, the detected results are promoted as well. We also design multiple experiments to verify our hypothesis by varying searching region. Experiments on MBGC databases prove the effect of the proposed simplified ASM model (sASM).

In order to develop human friendly robots, we develop a robotic system with a vision system, robot arm and voice recognition system, capable of grasping an object for which a human asks by voice. In this paper we present the 3D object recognition method by silhouette images in the first place. We propose the method to construct 3D object models by silhouette images and use them for making templates in order to undertake the template matching for 3D object recognition by silhouette images. We find that very high correct recognition rate is achieved by this method. We then employ a 4-DOF robot arm with a gripper for grasping the object. We present the method to approach and grasp the object by cooperating with the movement of arm and image obtained from the camera mounted on the tip of the arm. The experiment shows that recognition of the object, approach and grasping it are well undertaken.

Automatic assessment and understanding of facial skin condition have several applications, including the early detection of underlying health problems, lifestyle and dietary treatment, skin-care product recommendation, etc. S Selfies in the wild serve as an excellent data resource to democratize skin quality assessment, but suffer from several data collection challenges. The key to guaranteeing an accurate assessment is accurate detection of different skin features. We present an automatic facial skin feature detection method that works across a variety of skin tones and age groups for selfies in the wild. To be specific, we annotate the locations of acne, pigmentation, and wrinkle for selfie images with different skin tone colors, severity levels, and lighting conditions. The annotation is conducted in a two-phase scheme with the help of a dermatologist to train volunteers for annotation. We employ Unet++ as the network architecture for feature detection. This work shows that the two-phase annotation scheme can robustly detect the accurate locations of acne, pigmentation, and wrinkle for selfie images with different ethnicities, skin tone colors, severity levels, age groups, and lighting conditions.

Detection of facial landmarks and their feature points plays an important role in many facial image-related applications such as face recognition/verification, facial expression analysis, pose normalization, and 3D face reconstruction. Generally, detection of facial features is easy for persons; however, for machines it is not an easy task at all. The difficulty comes from high inter-personal variation (e.g., gender, race), intra-personal changes (e.g., pose, expression), and from acquisition conditions (e.g., lighting, image resolution). This chapter discusses basic concepts related to the problem of facial landmarks detection and overviews the successes and failures of exiting solutions. Also, it explores the difficulties that hinders the path of progress in the topic and the challenges involved in the adaptation of existing approaches to build successful systems that can be utilized in real-world facial images-related applications. Additionally, it discusses the performance evaluation metrics and the available benchmarking datasets. Finally, it suggests some possible future directions for research in the topic.

This paper proposes a fully automated post image processing scheme based on facial feature detection to correct the horizontal temporal shear or rolling shutter distortion. This distortion occurs when obtaining images or video sequences from a CMOS camera with a rolling shutter whenever there is relative horizontal movement between the sensor and the object being imaged during the integration time of the image frame. Unlike CCD sensors, such as the interline CCD, which provides an electronic shutter mechanism called a global shutter in which the light collection starts and ends at exactly the same time for all pixels, CMOS sensors can not hold and store all the pixels at the same time. Each scanline is exposed, sampled, and stored in sequence, resulting in the rolling shutter effect or temporal distortion of the image that will cause inaccurate facial recognition results. Facial feature detection is performed using correlation based methods with low computational complexity. The location of key facial feature points is then used to calculate the temporal horizontal shear or the distortion of the image. This information can then be used to remove the temporal horizontal shear distortion from the detected face or the entire image. We present experimental results on controlled data sets and real scenes to show that the proposed method yields excellent results in reversing the temporal horizontal shear caused by the CMOS rolling shutter sensor and significantly improves the accuracy of our facial recognition algorithm.

Human face detection is concerned with finding the location and size of every human face in a given image. Face detection plays a very important role in human computer interaction field. It represents the first step in a fully automatic face re cognition, facial features detection, and expression recognition. There are many techniques used in face detection, each one has its advantages and disadvantages. The face detection system presented in this paper is a hybrid of known algorithms. First stag e of the proposed method is applying skin detection algorithm to specify all skin locations in the image. Second, extract face features like eyes, mouth and nose. At the last, a verification step is applied to ensure that the extracted features are facial features. In experiments on images having upright frontal faces with any background our system has achieved high detection rates and low false positives.

Summary form only given. Facial feature detection and segmentation have gained many interests in recent technologies. There are many variations on facial images, such as shadow, occultation, and uneven illumination. All these side effects can influence the segmentation results. In this paper, we propose an efficient algorithm to effectively and accurately extract the desired facial features from the uneven illumination images.

Students are the main body of the classroom, and their learning status reflects the quality of classroom teaching to a certain extent. This paper aims to manage students’ learning status in industrial education classrooms by designing a student learning status assessment system. The collected video data from industrial education classrooms are processed by binarization and histogram equalization. The key technologies, such as face detection and eye movement analysis, are used to detect the learning status of students in the industrial education classroom, and the functional modules, such as data acquisition and statistical analysis, are combined to form this paper’s student learning status assessment system based on facial feature detection. It is found that the face detection algorithm of this paper’s system improves the detection accuracy by 11.35% compared with the baseline algorithm when the standard difficulty is difficult, and this paper’s algorithm is able to successfully detect and display the students’ facial features through 68 key feature points. The system in this paper is able to detect students’ learning states, such as concentration, fatigue, and doubt, by analyzing their eye movement frequency and other indicators. After applying the system discussed in this paper, the final grades of students in industrial education have significantly improved.

This paper, firstly, presents a new signature representation for point in range data, called curgram. Curgram serves to describe the structural neighborhood of a point and establish a signature for each of the given 3D data points rather than information of this point position. It yields invariance under rigid motions and mirror imaging. Secondly, its application to detection of facial features from range data is performed by incorporating the metric for curgram into the conventional appearance-based face detection framework. Experimental results with fifteen face range images have demonstrated the validity and effectiveness of the proposed method.

Detection of facial landmark features is an initial step in facial expression recognition. Detection of eyebrows can aid the detection of the remaining facial features as eyebrows are relatively more stable across changing facial expressions. Existing eyebrow detection algorithms in literature involve complex computations and are not suitable for direct porting on to embedded platforms. In this paper, a compute-efficient eyebrow detection algorithm has been proposed and tested on three standard databases with an average detection rate of 96%. The method has also been shown to be computationally less complex compared to the state of the art.