Metric Learning with Dynamically Generated Pairwise Constraints for Ear Recognition

Ear recognition is a complex research domain within biometrics, aiming to identify individuals using their ears in uncontrolled conditions. Despite the exceptional performance of convolutional neural networks (CNNs) in various applications, the efficacy of deep ear recognition systems is nascent. This paper proposes a two-step ear recognition approach. The initial step employs deep convolutional generative adversarial networks (DCGANs) to enhance ear images. This involves the colorization of grayscale images and the enhancement of dark shades, addressing visual imperfections. Subsequently, a feature extraction and classification technique, referred to as Mean-CAM-CNN, is introduced. This technique leverages mean-class activation maps in conjunction with CNNs. The Mean-CAM approach directs the CNN to focus specifically on relevant information, extracting and assessing only significant regions within the entire image. The process involves the implementation of a mask to selectively crop the pertinent area of the image. The cropped region is then utilized to train a CNN for discriminative classification. Extensive evaluations were conducted using two ear recognition datasets: mathematical analysis of images (MAI) and annotated web ears (AWEs). The experimental results indicate that the proposed approach shows notable improvements and competitive performance: the Rank-1 recognition rates are 100.00% and 76.25% for MAI and AWE datasets, respectively.

PurposeTo differentiate complicated appendicitis (CA) from uncomplicated appendicitis (UA) in children, we developed and validated an artificial intelligence (AI) model using a multimodal approach integrating ultrasound images and clinical data.MethodsA retrospective analysis was performed on 372 pathologically confirmed pediatric appendicitis cases (230 male, 142 female) from three centers, all with preoperative abdominal ultrasound. Deep learning (DL) features and radiomic features were extracted from appendiceal ultrasound images using deep transfer learning (DTL) and conventional radiomic methods, respectively. We selected 12 radiomic features, 9 DL features, and 3 clinical features—namely, white blood cell count (WBC), neutrophil count (NEU), and C-reactive protein (CRP)—for building the machine learning classification model. Based on these features, four distinct models were constructed: the Rad model (radiomic features only), the DL model (DL features only), the DTL model (combined radiomic and DL features), and the Combine model (integrating all three feature types: radiomic, DL, and clinical features). Model performance was evaluated using receiver operating characteristic (ROC) curves, decision curve analysis (DCA), and the DeLong test. Finally, the combined model was compared to the performance of clinicians with varying levels of experience.ResultsThe combined model demonstrated consistently favorable performance across all cohorts (AUC: 0.940, 0.895, 0.866, and 0.783 for training and validation sets, respectively). The model's accuracy (0.862) and positive predictive value (0.896) were comparable to senior surgeons (0.741, 0.970) and superior to junior surgeons (0.672, 0.865) in the internal validation cohort. DCA confirmed the clinical utility of the combined model over conventional strategies.ConclusionOur ultrasound-based AI model provides reliable differentiation between CA and UA in children, offering potential value as a diagnostic support tool for clinical decision making.

The ear’s relatively stable structure makes it suitable for recognition. In common identification applications, only one sample per person (OSPP) is registered in a gallery; consequently, effectively training deep-learning-based ear recognition approach is difficult. The state-of-the-art (SOA) 3D ear recognition using the OSPP approach bottlenecks when large occluding objects are close to the ear. Hence, we propose a system that combines PointNet++ and three layers of features that are capable of extracting rich identification information from a 3D ear. Our goal is to correctly recognize a 3D ear affected by a large nearby occlusion using one sample per person (OSPP) registered in a gallery. The system comprises four primary components: (1) segmentation; (2) local and local joint structural (LJS) feature extraction; (3) holistic feature extraction; and (4) fusion. We use PointNet++ for ear segmentation. For local and LJS feature extraction, we propose an LJS feature descriptor–pairwise surface patch cropped using a symmetrical hemisphere cut-structured histogram with an indexed shape (PSPHIS) descriptor. Furthermore, we propose a local and LJS matching engine based on the proposed LJS feature descriptor and SOA surface patch histogram indexed shape (SPHIS) local feature descriptor. For holistic feature extraction, we use a voxelization method for global matching. For the fusion component, we use a weighted fusion method to recognize the 3D ear. The experimental results demonstrate that the proposed system outperforms the SOA normalization-free 3D ear recognition methods using OSPP when the ear surface is influenced by a large nearby occlusion.

Recently biometric systems have shown improved capabilities because of the remarkable success of deep learning in solving various computer vision tasks. In ear recognition, the use of deep learning techniques is seldom due to training data scarcity. The existing work has shown poor performance as the majority of techniques are based on either handcraft features or pre-trained models. Besides this, transfer-learning has also shown poor performance because of the diversity among the tasks. To circumvent the existing issues, in this work, we have presented an end-to-end framework for ear recognition. It consist of the Ear Mask Extraction (EME) network to segment the ear, a normalization algorithm to align the ear, and a novel siamese-based CNN (CG-ERNet) for deep ear feature learning. CG-ERNet exploits domain-specific knowledge by using Curvature Gabor filters and uses triplet loss, triplet selection, and adaptive margin for better convergence of the loss. For comparative analysis, we trained state-of-the-art deep learning models like Face-Net, VGG19, ResNet50, Inception, Exception, and Mobile-Net for ear-recognition. The performance is assessed using five well-known evaluation metrics. In the extensive experimentation, our proposed model (CG-ERNet) outperformed the deep learning models and handcrafted feature based methods on four different, publicly available, benchmark datasets. To make the results more interpretable, we employ the t-SNE visualization of learned features. Additionally, our proposed method has shown robustness to various environmental challenges like Gaussian noise, Gaussian blur, up to ± 30 degrees of rotation, and 20% of occlusion.

Cayley-Klein metric is a specific kind of non-Euclidean metric in projective space. Recently, it has been introduced into metric learning with encouraging performance when dealing with computer vision tasks. However, the original Cayley-Klein metric learning methods with conventional pairwise and triplet-wise constraints, which are fixed bound based constraints, may not perform well when the intra-and inter-class variations of data distribution become complex. Pairwise constraints restrict the distance between samples of a similar pair to be lower than a fixed upper bound, and the distance between samples of a dissimilar pair higher than a fixed lower bound. Triplet-wise constraints restrict the distance between samples of a similar pair to be smaller than that between a pair of samples from different classes. In this paper, we propose a novel Cayley-Klein metric learning method (CKseML) with adaptive shrinkage-expansion pairwise constraints. CKseML is very effective in learning metric from data with complex distributions. Our experimental results demonstrate that CKseML achieves better performance than the original Cayley-Klein metric learning methods.

In this paper we describe a multi-modal ear and face biometric system. The system is comprised of two components: a 3D ear recognition component and a 2D face recognition component. For the 3D ear recognition, a series of frames is extracted from a video clip and the region of interest (i.e., ear) in each frame is independently reconstructed in 3D using Shape From Shading. The resulting 3D models are then registered using the iterative closest point algorithm. We iteratively consider each model in the series as a reference model and calculate the similarity between the reference model and every model in the series using a similarity cost function. Cross validation is performed to assess the relative fidelity of each 3D model. The model that demonstrates the greatest overall similarity is determined to be the most stable 3D model and is subsequently enrolled in the database. For the 2D face recognition, a set of facial landmarks is extracted from frontal facial images using the Active Shape Model. Then, the response of facial images to a series of Gabor filters at the locations of facial landmarks are calculated. The Gabor features (attributes) are stored in the database as the face model for recognition. The similarity between the Gabor features of a probe facial image and the reference models are utilized to determine the best match. The match scores of the ear recognition and face recognition modalities are fused to boost the overall recognition rate of the system. Experiments are conducted using a gallery set of 402 video clips and a probe of 60 video clips (images). As a result, a rank-one identification rate of 100% was achieved using the weighted sum technique for fusion.

Symmetrical null space LDA for face and ear recognition

Nowadays, Biometrics has become a popular tool in personal identification as it can use physiological or behavioral characteristics to identify individuals. Recent advances in information technology has increased the accuracy of biometric to another level, there is still a slew of problems existed, such as complex environment, aging and unique problems. Among many classes of identifications, recognizing twins is one of the most difficult tasks as they resemble each other. This affects the use of biometrics in general cases and raises potential risks of biometrics in access control. In this paper, the authors manage to distinguish twins using four different models, namely, face recognition, ear recognition, voice recognition and lip movement recognition. Their results show that voice recognition has the best performance in twin recognition with 100% accuracy. This is much higher than that of face recognition and ear recognition (with 58% and 53% respectively); and lip movement recognition that yields 76% accuracy.

Metallocene and Single Site Polymers (#1624)

When only one sample per person (OSPP) is registered in the gallery, it is difficult for ear recognition methods to sufficiently and effectively reduce the search range of the matching features, thus resulting in low computational efficiency and mismatch problems. A 3D ear biometric system using OSPP is proposed to solve this problem. By categorising ear images by shape and establishing the corresponding relationship between keypoints from ear images and regions (regional cluster) on the directional proposals that can be arranged to roughly face the ear image, the corresponding keypoints are obtained. Then, ear recognition is performed by combining corresponding keypoints and a multi-keypoint descriptor sparse representation classification method. The experimental results conducted on the University of Notre Dame Collection J2 dataset yielded a rank-1 recognition rate of 98.84%; furthermore, the time for one identification operation shared by each gallery subject was 0.047 ms.

Sentiment analysis is an important task in the field of big data and artificial intelligence and has a wide range of real-world applications. The majority of current approaches, however, attempt to detect the overall polarity of a sentence, or text span, irrespective of the entities mentioned and their aspects. A typical way for solve this is to employ a classifier that learns to identify the polarity labels given input text. Such methods require amount of training examples which were manually constructed. While most work in text mining in the field of sentiment analysis domain focus on the use of supervised machine learning technologies, in this paper, we investigate on aspect-based sentiment analysis for mobile game reviews. Our method integrates both deep learning learned features and support vector machines. The deep learning feature is mainly derived from inducting word relations in Wikipedia and the collected in-domain text. To prevent from assessing unknown missing words, we design a fuzzy-based approach to capture word-level sentiment scores. By using a fuzzy-based soft computing strategy with a small lexicon allows us to produce a system with better accuracy and precision than pure machine learning such as Naive Bayes and SVMs. To demonstrate the effectiveness of the proposed approach, we conduct the experimental results on the collected Mobile game reviews. These results show that our method outperforms supervised systems. One good property of this method is that it does not need to perform Chinese word segmentation in testing time.

In the current scenario face identification and recognition is an important technique in surveillance. The face is a necessary biometric in humans. Therefore face detection plays a major job in computer vision applications. Several face recognition and emotions classification approaches have been presented throughout the last few decades of research to improve the rate of face recognition for thermal pictures. However, in real-time, lighting conditions might change due to several factors, such as the different times of capture, weather, etc. Due to variations in lighting intensity, the performance of the facial expression recognition system is not good. This paper proposed a model for human thermal face detection and expression classification. Four main steps were involved in this research. Initially, the Difference of the Gaussian (DOG) filter is utilized to crop the input thermal images and then normalize the images using the median filter in pre-processing step. Then, Efficient Net is used for extracting features such as shape, location, and occurrences from thermal face images. After that, detect human faces utilized by the YOLOv4 technique to better emotions classification. Finally, classify the emotions on faces by using the DenseNet technique into seven emotions such as happy, sad, disgust, surprise, anger, fear, and neutral. The proposed method outperforms state-of-art techniques for face recognition on thermal pictures, and classifies the expressions, according to experimentations on the RGB-D-T database. The accuracy, precision, recall, and f1-score metrics will be utilized with the database to assess the efficacy of the proposed methodology. The proposed models achieve a high classification accuracy of 95.97% on the RGB-D-T database. Furthermore, the outcomes show good precision for various face recognition tasks.

Xu Yang, 2 Bo Wen, Chao-nan Tong *1, School of Automation & Electrical Engineering, Key Laboratory of Advanced Control of Iron and Steel Process (Ministry of Education), University of Science and Technology Beijing, Beijing 100083, China, yangxu@ustb.edu.cn 2, School of Automation & Electrical Engineering, University of Science and Technology Beijing, Beijing 100083, China, gzwebber@qq.com 3, School of Automation & Electrical Engineering, Key Laboratory of Advanced Control of Iron and Steel Process (Ministry of Education), University of Science and Technology Beijing, Beijing 100083, China, tcn@ies.ustb.edu.cn

Subspace methods have been successfully applied to face recognition tasks. It is well-studied in both unsupervised learning and supervised learning, such as Eigenface and Fisherface. In practice, besides abundant unlabeled examples, domain knowledge in the form of pairwise constraints is commonly available, which specifies whether a pair of instances belong to the same class or different classes. In this study, we propose a face recognition method based on semi-supervised locality preserving learning together with pairwise constraints and unlabeled data, called Semi-supervised Laplacianface (S-Laplacianface). It tries to preserve the local geometric structure of the face manifold as Laplacianface, also requires the subspace to satisfy the pairwise constraints defined by the user. Experimental results on two face databases demonstrate the effectiveness of proposed algorithm.

Face image classification is an important branch of computer vision and artificial intelligence, commonly applied in various fields such as facial recognition and facial attribute analysis. One facial attribute that is particularly interesting to classify is the use of eyeglasses, as it can affect the overall accuracy of facial recognition systems. This study aims to develop an eyeglass-use classification system based on a Convolutional Neural Network (CNN) and implement it in a web application using Flask to enable real-time prediction results. The research methodology includes collecting facial image datasets from Kaggle, performing preprocessing steps such as resizing, augmentation, and normalization, designing the CNN architecture, training the model, and evaluating its performance using a confusion matrix and classification report. The designed CNN model consists of three convolutional layers, max pooling, a flattening process, two fully connected layers, and a dropout layer to reduce the risk of overfitting. During the training phase, the model achieved 90% training accuracy and 96% validation accuracy, while testing on the test dataset resulted in an overall accuracy of 82%. The Flask-based system is capable of displaying real-time predictions, including the input image, classification label, accuracy percentage, and inference time. In the detection process, the accuracy of the model implementation reached 93% and the time required was in the range of a few milliseconds. The results demonstrate that the CNN can effectively classify faces with and without eyeglasses, and its implementation through a web interface offers broad potential for visual identification applications.