Incremental Sequential Three-Way Decision Using a Deep Stacked Autoencoder

Illumination variation significantly affects the performance of face recognition algorithms. However, it is difficult to construct a database that meets the demand of illumination analysis with current face image database construction technology. The influence of illumination variation on deep face recognition based on 3D face models is investigated. First a new method is presented to produce face images under arbitrary illuminations by using a 3D face model based on face basis image representation theory proposed. Then the database used for illumination analysis is constructed. Afterwards, an optimal way is explored to select illuminations used for constructing a high quality face image database by analyzing the impacts of different illumination sampling strategies on the performances of face recognition models. With the advantage of accurate illumination labeling of virtual data, the performances of different illumination labeling methods on the training of face recognition network are tested on a multi-task learning framework. Through experiments on the effects of different lighting sampling schemes and labeling methods on the performances of face recognition models carried out on virtual data and real data, it is concluded that using an appropriate amount of lightings of basis images to build a database is an effective lighting sampling scheme, and accurate lighting annotation can help further improve the face recognition rate, the corresponding recognition models have face recognition accuracy rates of more than 98% on virtual and real monitoring datasets which contain face images captured under very challenging lighting conditions. Proposed work improve the performance of deep face recognition models, and facilitate the construction of high quality face image databases with various illuminations.

Text-guided face recognition (TGFR) aims to improve the performance of state-of-the-art face recognition (FR) algorithms by incorporating auxiliary information, such as distinct facial marks and attributes, provided as natural language descriptions. Current TGFR algorithms have been proven to be highly effective in addressing performance drops in state-of-the-art FR models, particularly in scenarios involving sensor noise, low resolution, and turbulence effects. Although existing methods explore various algorithms using different cross-modal alignment and fusion techniques, they encounter practical limitations in real-world applications. For example, during inference, textual descriptions associated with face images may be missing, lacking crucial details, or incorrect. Furthermore, the presence of inherent modality heterogeneity poses a significant challenge in achieving effective cross-modal alignment. To address these challenges, we introduce CaptionFace, a TGFR framework that integrates GPTFace, a face image captioning model designed to generate context-rich natural language descriptions from low-resolution facial images. By leveraging GPTFace, we overcome the issue of missing textual descriptions, expanding the applicability of CaptionFace to single-modal FR datasets. Additionally, we introduce a multi-scale feature alignment (MSFA) module to ensure semantic alignment between face-caption pairs at different granularities. Furthermore, we introduce an attribute-aware loss and perform knowledge adaptation to specifically adapt textual knowledge from facial features. Extensive experiments on three face-caption datasets and various unconstrained single-modal benchmark datasets demonstrate that CaptionFace significantly outperforms state-of-the-art FR models and existing TGFR approaches.

Face recognition systems extract embedding vectors from face images and use these embeddings to verify or identify individuals. Face reconstruction attack (also known as template inversion) refers to reconstructing face images from embeddings and using the reconstructed image to enter a face recognition system. In this paper, we propose to use a face foundation model to reconstruct face images from the embeddings of a blackbox face recognition model. The foundation model is trained with 42M images to generate face images from the facial embeddings of a fixed face recognition model. We propose to use an adapter (called Face Adapter) to translate target embeddings into the embedding space of the foundation model. The generated images are evaluated on different face recognition models and different datasets, demonstrating the effectiveness of our method to translate embeddings of different face recognition models. We also evaluate the transferability of reconstructed face images when attacking different face recognition models. Our experimental results show that our reconstructed face images outperform previous reconstruction attacks against face recognition models. Project Page

Sequential three-way decision is a kind of dynamic three way decision model, which is suitable to deal with dynamic, complex and uncertain questions. Classical sequential three-way decisions are based on a granular structure with a single threshold under multiple levels of granularity, which make the traditional models incapable of adapting to the case of multiview granular structures with multiple thresholds. To overcome this disadvantage, Qian et al. proposed a generalized multigranulation sequential three-way decision model based on multiple different thresholds. We have observed that multiple different thresholds will determine the same positive and negative regions in sequential three-way decision model. That is, there exists a many-to-one relationship between parameter thresholds and positive/negative regions. Therefore, it is essential to investigate parameter threshold selection strategies. First, since parameter thresholds taken from the same interval may result in identical positive and negative regions for single-granularity sequential three-way decision model based on multiple different thresholds. Therefore, some effective strategies of determining the selection interval of thresholds are put forward, which can greatly reduce the repeated operations and improve the decision efficiency. Second, the parameter selection strategy in the single-granularity sequential three-way decision model proposed in this paper is extended to the multi-granularity sequential three-way decision model, which can further improve decision-making efficiency.

Facial emotion recognition using modified HOG and LBP features with deep stacked autoencoders

Sequential three-way decision and granulation for cost-sensitive face recognition

Facial images are of critical importance in many real-world applications from gaming to surveillance. The current literature on facial image analysis, from face detection to face and facial expression recognition, are mainly performed in either RGB, Depth (D), or both of these modalities. But, such analyzes have rarely included Thermal (T) modality. This paper paves the way for performing such facial analyzes using synchronized RGB-D-T facial images by introducing a database of 51 persons including facial images of different rotations, illuminations, and expressions. Furthermore, a face recognition algorithm has been developed to use these images. The experimental results show that face recognition using such three modalities provides better results compared to face recognition in any of such modalities in most of the cases.

The objective of this project was to assess the performance of the leading commercial-off-the-shelf (COTS) facial recognition software package when used as a laptop application. We performed the assessment to determine the system's usefulness for enrolling facial images in a database from remote locations and conducting real-time searches against a database of previously enrolled images. The assessment involved creating a database of 40 images and conducting 2 series of tests to determine the product's ability to recognize and match subject faces under varying conditions. This report describes the test results and includes a description of the factors affecting the results. After an extensive market survey, we selected Visionics' FaceIt{reg_sign} software package for evaluation and a review of the Facial Recognition Vendor Test 2000 (FRVT 2000). This test was co-sponsored by the US Department of Defense (DOD) Counterdrug Technology Development Program Office, the National Institute of Justice, and the Defense Advanced Research Projects Agency (DARPA). Administered in May-June 2000, the FRVT 2000 assessed the capabilities of facial recognition systems that were currently available for purchase on the US market. Our selection of this Visionics product does not indicate that it is the ''best'' facial recognition software package for all uses. It was the most appropriate package based on the specific applications and requirements for this specific application. In this assessment, the system configuration was evaluated for effectiveness in identifying individuals by searching for facial images captured from video displays against those stored in a facial image database. An additional criterion was that the system be capable of operating discretely. For this application, an operational facial recognition system would consist of one central computer hosting the master image database with multiple standalone systems configured with duplicates of the master operating in remote locations. Remote users could perform real-time searches where network connectivity is not available. As images are enrolled at the remote locations, periodic database synchronization is necessary.

Dangerous persons can be monitored accurately and quickly using the high-precision recognition of the relative facial images via the image sensors. To increase the recognition rate of face recognition by improving the immune algorithm, the immune computation was redesigned for better face recognition to decrease the facial disturbances of the pose, illumination and expression (PIE), in this paper. The clonal selection algorithm was improved with the modification of the affinity and the algorithm workflow. The improved clonal selection algorithm searches the most similar antibody sample against the antigen of an unknown facial image, according to the affinity between the antigen and the antibody. The unknown facial images were recognized with this improved affinity and the uncertainty-based reasoning, so the affinity matching of the antibody with the unknown antigen was also improved. Experimental results show that this immune algorithm outperforms some state-of-the-art algorithms in the face recognition accuracy tests with such facial image databases as AR, Yale and CMU-PIE. So the proposed immune algorithm is useful and effective to improve the performance of face recognition in the image sensor network.

Face quality evaluation can filter out low quality face image to save computational resources and improve the system performance, labeling the face image quality score by manual consume too much manpower. To solve this problem, an unsupervised face image evaluation based on face recognition is proposed. We use the face recognition model to calculate the features of faces and label the images quality score. The face recognition model is compressed by knowledge distillation method to obtain efficient quality assessment model. Experimental results show that this method can effectively evaluate the quality of face image and improve the performance of face recognition.

Changes in illumination condition, pose, facial expression and others is not an easy task in face recognition. Solving these problems requires not only a feature extraction method that can generate distinct features for each class of image but requires other additional technique that able to improve the overall classification accuracy. This paper presents the face recognition using combined features of original and pre-processed face images. This technique is experimented using orthogonal moments namely Zernike moments (ZMs) and Krawtchouk moments (KMs). The classification technique used in the recognition stage is Euclidean square distance or Nearest Neighbour (NN) classifier. Database face images from Olivetti research laboratory (ORL) consisting of 40 subjects of 10 images each where none of them are identical, is used in the experiments. The face images vary in position, rotation, scale and expression, with and without spectacles. From the experiments, the new technique is able to improve the classification accuracy significantly.

Mental health assessments that combine patients' facial expressions and behaviors have been proven effective, but screening large-scale student populations for mental health problems is time-consuming and labor-intensive. This study aims to provide an efficient and accurate intelligent method for further psychological diagnosis and treatment, which combines artificial intelligence technologies to assist in evaluating the mental health problems of college students. We propose a mixed-method study of mental health assessment that combines psychological questionnaires with facial emotion analysis to comprehensively evaluate the mental health of students on a large scale. The Depression Anxiety and Stress Scale-21(DASS-21) is used for the psychological questionnaire. The facial emotion recognition model is implemented by transfer learning based on neural networks, and the model is pre-trained using FER2013 and CFEE datasets. Among them, the FER2013 dataset consists of 48 × 48-pixel face gray images, a total of 35,887 face images. The CFEE dataset contains 950,000 facial images with annotated action units (au). Using a random sampling strategy, we sent online questionnaires to 400 college students and received 374 responses, and the response rate was 93.5%. After pre-processing, 350 results were available, including 187 male and 153 female students. First, the facial emotion data of students were collected in an online questionnaire test. Then, a pre-trained model was used for emotion recognition. Finally, the online psychological questionnaire scores and the facial emotion recognition model scores were collated to give a comprehensive psychological evaluation score. The experimental results of the facial emotion recognition model proposed to show that its classification results are broadly consistent with the mental health survey results. This model can be used to improve efficiency. In particular, the accuracy of the facial emotion recognition model proposed in this paper is higher than that of the general mental health model, which only uses the traditional single questionnaire. Furthermore, the absolute errors of this study in the three symptoms of depression, anxiety, and stress are lower than other mental health survey results and are only 0.8%, 8.1%, 3.5%, and 1.8%, respectively. The mixed method combining intelligent methods and scales for mental health assessment has high recognition accuracy. Therefore, it can support efficient large-scale screening of students' psychological problems.

In this paper, a person-specific facial expression recognition method which is based on Personal Facial Expression Space (PFES) is presented. The multidimensional scaling maps facial images as points in lower dimensions in PFES. It reflects personality of facial expressions as it is based on the peak instant of facial expression images of a specific person. In constructing PFES for a person, his/her whole normalized facial image is considered as a single pattern without block segmentation and differences of 2-D DCT coefficients from neutral facial image of the same person are used as features. Therefore, in the early part of the paper, separation characteristics of facial expressions in the frequency domain are analyzed using a still facial image database which consists of neutral, smile, anger, surprise and sadness facial images for each of 60 Japanese males (300 facial images). Results show that facial expression categories are well separated in the low frequency domain. PFES is constructed using multidimensional scaling by taking these low frequency domain of differences of 2-D DCT coefficients as features. On the PFES, trajectory of a facial image sequence of a person can be calculated in real time. Based on this trajectory, facial expressions can be recognized. Experimental results show the effectiveness of this method.© (2000) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Short videos on the Internet are growing exponentially, and the number of videos uploaded every day is huge; people also involve a lot of video data in real life. People can retrieve and view all kinds of videos, but it also brings a lot of problems. On the one hand, the accumulation of a large number of videos makes people unable to find the videos they want quickly, and the repeated scenes in the videos will also waste people's time and energy; on the other hand, a large amount of video data also brings enormous pressure to storage. Aiming at the problems of inaccurate selection of key frames and how to select video frame features in existing video summarization models, this paper proposes a multi-feature-based video summarization generation model (DME-VSNet), which extracts multiple features of video frames. Including importance score, image memory strength and image entropy. Aiming at the problem of inaccurate video shot segmentation, this model proposes a video shot segmentation algorithm based on TransNet network, which divides the original video into several short shots through shot boundaries; the model inputs the above three features into the proposed The video frame score is obtained in the MLP architecture, and the key frame is selected by the score to generate a video summary. The effectiveness of the video shot segmentation method based on TransNet network and the overall model based on convolutional neural network is verified by comparative experiments. The experimental results show that the evaluation results of the video summaries generated by the three features are better.

Watermarking enables provable rights over content, which has been successfully applied in multimedia applications. However, it is not trivial to apply the known effective watermarking schemes to XML data, since noisy data may not be acceptable due to its structures and node extents. In this paper, we present two different watermarking schemes on XML data: the selective approach and the compression approach. The former allows us to embed non-destructive hidden information content over XML data. The latter takes verbosity and the need in updating XML data in real life into account. We conduct experiments on the efficiency and robustness of both approaches against different forms of attack, which shows that our proposed watermarking schemes are reasonably efficient and effective.