Deep Ear Recognition Pipeline

Ear Recognition has recently received significant attention in the literature. Even though current ear recognition systems have reached a certain level of maturity, their success is still limited. This paper presents an efficient complete ear-based biometric system that can process five frames/sec; Hence it can be used for surveillance applications. The ear detection is achieved using Haar features arranged in a cascaded Adaboost classifier. The feature extraction is based on dividing the ear image into several blocks from which Local Binary Pattern feature distributions are extracted. These feature distributions are then fused at the feature level to represent the original ear texture in the classification stage. The contribution of this paper is three fold: (i) Applying a new technique for ear feature extraction, and studying various optimization parameters for that technique; (ii) Presenting a practical ear recognition system and a detailed analysis about error propagation in that system; (iii) Studying the occlusion effect of several ear parts. Detailed experiments show that the proposed ear recognition system achieved better performance (94:34%) compared to other shape-based systems as Scale-invariant feature transform (67:92%). The proposed approach can also handle efficiently hair occlusion. Experimental results show that the proposed system can achieve about (78%) rank-1 identification, even in presence of 60% occlusion.

Ear recognition is a complex research domain within biometrics, aiming to identify individuals using their ears in uncontrolled conditions. Despite the exceptional performance of convolutional neural networks (CNNs) in various applications, the efficacy of deep ear recognition systems is nascent. This paper proposes a two-step ear recognition approach. The initial step employs deep convolutional generative adversarial networks (DCGANs) to enhance ear images. This involves the colorization of grayscale images and the enhancement of dark shades, addressing visual imperfections. Subsequently, a feature extraction and classification technique, referred to as Mean-CAM-CNN, is introduced. This technique leverages mean-class activation maps in conjunction with CNNs. The Mean-CAM approach directs the CNN to focus specifically on relevant information, extracting and assessing only significant regions within the entire image. The process involves the implementation of a mask to selectively crop the pertinent area of the image. The cropped region is then utilized to train a CNN for discriminative classification. Extensive evaluations were conducted using two ear recognition datasets: mathematical analysis of images (MAI) and annotated web ears (AWEs). The experimental results indicate that the proposed approach shows notable improvements and competitive performance: the Rank-1 recognition rates are 100.00% and 76.25% for MAI and AWE datasets, respectively.

Biometric technology is fast gaining pace as a veritable developmental tool. So far, biometric procedures have been predominantly used to ensure identity and ear recognition techniques continue to provide very robust research prospects. This paper proposes to identify and review present techniques for ear biometrics using certain parameters: machine learning methods, and procedures and provide directions for future research. Ten databases were accessed, including ACM, Wiley, IEEE, Springer, Emerald, Elsevier, Sage, MIT, Taylor & Francis, and Science Direct, and 1121 publications were retrieved. In order to obtain relevant materials, some articles were excused using certain criteria such as abstract eligibility, duplicity, and uncertainty (indeterminate method). As a result, 73 papers were selected for in-depth assessment and significance. A quantitative analysis was carried out on the identified works using search strategies: source, technique, datasets, status, and architecture. A Quantitative Analysis (QA) of feature extraction methods was carried out on the selected studies with a geometric approach indicating the highest value at 36%, followed by the local method at 27%. Several architectures, such as Convolutional Neural Network, restricted Boltzmann machine, auto-encoder, deep belief network, and other unspecified architectures, showed 38%, 28%, 21%, 5%, and 4%, respectively. Essentially, this survey also provides the various status of existing methods used in classifying related studies. A taxonomy of the current methodologies of ear recognition system was presented along with a publicly available occlussion and pose sensitive black ear image dataset of 970 images. The study concludes with the need for researchers to consider improvements in the speed and security of available feature extraction algorithms.

A novel method of feature-level fusion based on kernel Fisher discriminant analysis (KFDA) is proposed and applied to fusion of ear and profile face biometrics in this paper. Ear recognition is proved to be a new and promising authentication technique. Because of ear's special physiological structure and location, it is reasonable to combine ear with profile face for recognition in such scenarios as frontal face images are not available. First, only the face profile-view images are captured for recognition. Then based on KFDA, three feature fusion rules are presented. With the rules, the fusion discriminant vectors of ear and profile face are established and nonlinear feature fusion projection could be implemented. The experimental results show that the method is efficient for feature-level fusion, and the multimodal recognition based on ear and profile face performs better than ear or profile face unimodal biometric recognition. The work provides a new effective approach of non-intrusive biometric recognition.

Ear detection is an important step in ear recognition pipeline as it makes or breaks the system. However, in the literature there is arguably the lack of ear detection approaches available. This poses a problem for opening ear recognition system to wider use and applications in commercial systems. To tackle this problem we present the use of Mask R-CNN for pixel-wise ear detection. Furthermore, we directly compare our approach to one of the previous best performing pixel-wise ear detection approach by using the same dataset and protocol. Our results with intersection over union score of 79.24% on AWE dataset show the superiority of our approach and present a viable approach for future use in ear recognition pipelines.

The advancement of Artificial Intelligence (AI) significantly impacts biometric identification methods, with ear biometrics becoming a standout approach due to the unique and consistent features of the 1) human ear.This study uses two datasets, UERC and IIT-D and develops three custom ear recognition models.The primary focus is on employing ensemble learning techniques to enhance the accuracy and robustness of ear recognition systems.Ensemble methods are particularly beneficial as they reduce errors and improve overall performance by combining multiple models.The individual custom models show promising results, with Model 1, Model 2, and Model 3 achieving accuracies of 97.1%, 96.9%, and 97.3% on the IIT-D dataset, and 84%, 91.1%, and 91.6% on the UERC dataset.Combining these models, the average ensemble achieves an accuracy of 97.8% on the IIT-D dataset and 95.2% on the UERC dataset.The weighted ensemble further improves the results, achieving an accuracy of 98.99% on the IIT-D dataset and demonstrating similarly enhanced performance on the UERC dataset with an accuracy of 97.8.These findings highlight the effectiveness of ensemble learning in significantly boosting the performance of ear recognition systems.Future research could delve into more advanced ensemble techniques, employ larger and more diverse datasets, and evaluate the practical applicability of ear biometrics in various conditions.

Recognizing humans by their ear have recently received significant attention in the field of research. Ear is the rich in characteristics. The ear recognition does not suffer from some problems associated with other passive biometrics, such as face recognition and the ear shape very stable with respect to age. Further, the ear can be used for human recognition in surveillance videos where the face may be occluded completely or in part. This paper provides a detailed survey of research done in ear detection and recognition on 2D and 3D images respectively. This survey paper is very useful in the current state-of- art for those who are working in this area and also for those who might exploit this new approach.

Automatic ear recognition is gaining popularity within the research community due to numerous desirable properties, such as high recognition performance, the possibility of capturing ear images at a distance and in a covert manner, etc. Despite this popularity and the corresponding research effort that is being directed towards ear recognition technology, open problems still remain. One of the most important issues stopping ear recognition systems from being widely available are ear occlusions and accessories. Ear accessories not only mask biometric features and by this reduce the overall recognition performance, but also introduce new non-biometric features that can be exploited for spoofing purposes. Ignoring ear accessories during recognition can, therefore, present a security threat to ear recognition and also adversely affect performance. Despite the importance of this topic there has been, to the best of our knowledge, no ear recognition studies that would address these problems. In this work we try to close this gap and study the impact of ear accessories on the recognition performance of several state-of-the-art ear recognition techniques. We consider ear accessories as a tool for spoofing attacks and show that CNN-based recognition approaches are more susceptible to spoofing attacks than traditional descriptor-based approaches. Furthermore, we demonstrate that using inpainting techniques or average coloring can mitigate the problems caused by ear accessories and slightly outperforms (standard) black color to mask ear accessories.

An ear biometric system based on artificial bees and the scale invariant feature transform

In deep convolutional neural networks (DCNNs), model size and computation complexity are two important factors governing throughput and energy efficiency when deployed to hardware for inference. Recent works on compact DCNNs as well as pruning methods are effective, yet with drawbacks. For instance, more than half the size of all MobileNet models lies in their last two layers, mainly because compact separable convolution (CONV) layers are not applicable to their last fully connected (FC) layers. Also, in pruning methods, the compression is gained at the expense of irregularity in the DCNN architecture, which necessitates additional indexing memory to address nonzero weights, thereby increasing memory footprint, decompression delays, and energy consumption. In this article, we propose cyclic sparsely connected (CSC) architectures, with memory/computation complexity of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathcal {O}(N\log {}N)$ </tex-math></inline-formula> , where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$N$ </tex-math></inline-formula> is the number of nodes/channels given a DCNN layer that, contrary to compact depthwise separable layers, can be used as an overlay for both FC and CONV layers of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathcal {O}(N^{2})$ </tex-math></inline-formula> . Also, contrary to pruning methods, CSC architectures are structurally sparse and require no indexing due to their cyclic nature. We show that both standard convolution and depthwise convolution layers are special cases of the CSC layers, whose mathematical function, along with FC layers, can be unified into one single formulation and whose hardware implementation can be carried out under one arithmetic logic component. We examine the efficacy of the CSC architectures for compression of LeNet, AlexNet, and MobileNet DCNNs with precision ranging from 2 to 32 bits. More specifically, we surge upon the compact 8-bit quantized 0.5 MobileNet V1 and show that by compressing its last two layers with CSC architectures, the model is compressed by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\sim 1.5\times $ </tex-math></inline-formula> with a size of only 873 kB and little accuracy loss. Finally, we design a configurable hardware that implements all types of DCNN layers including FC, CONV, depthwise, CSC-FC, and CSC-CONV indistinguishably within a unified pipeline. We implement the hardware on a tiny Xilinx field-programmable gate array (FPGA) for total on-chip processing of the compressed MobileNet that, compared to the related work, has the highest Inference/J while utilizing the smallest FPGA.

A lightweight real-time unified detection model for rice and wheat ears in complex agricultural environments

The ear recognition task is known as predicting whether two ear images belong to the same person or not. More recently, most ear recognition methods have started based on deep learning features that can achieve a good accuracy, but it requires more resources in the training phase and suffer from time-consuming computational complexity. On the other hand, descriptor features and metric learning play a vital role and also provide excellent performance in many computer vision applications, such as face recognition and image classification. Therefore, in this paper, we adopt the descriptor features and present a novel metric learning method that is efficient in matching real-time for ear recognition system. This method is formulated as a pairwise constrained optimization problem. In each training cycle, this method selects the nearest similar and dissimilar neighbors of each sample to construct the pairwise constraints and then solves the optimization problem by the iterated Bregman projections. Experiments are conducted on Annotated Web Ears (AWE) database, West Pommeranian University of Technology (WPUT), the University of Science and Technology Beijing II (USTB II), and Mathematical Analysis of Images (AMI) databases.. The results show that the proposed approach can achieve promising recognition rates in ear recognition, and its training process is much more efficient than the other competing metric learning methods.

Conventional geographic object-based image analysis (GEOBIA) land cover classification methods by using very high resolution images are hardly applicable due to their complex ground truth and manually selected features, while convolutional neural networks (CNNs) with many hidden layers provide the possibility of extracting deep features from very high resolution images. Compared with pixel-based CNNs, superpixel-based CNN classification, carrying on the idea of GEOBIA, is more efficient. However, superpixel-based CNNs are still problematic in terms of their processing units and accuracies. Firstly, the limitations of salt and pepper errors and low boundary adherence caused by superpixel segmentation still exist; secondly, this method uses the central point of the superpixel as the classification benchmark in identifying the category of the superpixel, which does not allow classification accuracy to be ensured. To solve such problems, this paper proposes a region-based majority voting CNN which combines the idea of GEOBIA and the deep learning technique. Firstly, training data was manually labeled and trained; secondly, images were segmented under multiresolution and the segmented regions were taken as basic processing units; then, point voters were generated within each segmented region and the perceptive fields of points voters were put into the multi-scale CNN to determine their categories. Eventually, the final category of each region was determined in the region majority voting system. The experiments and analyses indicate the following: 1. region-based majority voting CNNs can fully utilize their exclusive nature to extract abstract deep features from images; 2. compared with the pixel-based CNN and superpixel-based CNN, the region-based majority voting CNN is not only efficient but also capable of keeping better segmentation accuracy and boundary fit; 3. to a certain extent, region-based majority voting CNNs reduce the impact of the scale effect upon large objects; and 4. multi-scales containing small scales are more applicable for very high resolution image classification than the single scale.

Following recent advancements in deep learning and artificial intelligence, spoken language identification applications are playing an increasingly significant role in our day-to-day lives, especially in the domain of multi-lingual speech recognition. In this article, we propose a spoken language identification system that depends on the sequence of feature vectors. The proposed system uses a hybrid Convolutional Recurrent Neural Network (CRNN), which combines a Convolutional Neural Network (CNN) with a Recurrent Neural Network (RNN) network, for spoken language identification on seven languages, including Arabic, chosen from subsets of the Mozilla Common Voice (MCV) corpus. The proposed system exploits the advantages of both CNN and RNN architectures to construct the CRNN architecture. At the feature extraction stage, it compares the Gammatone Cepstral Coefficient (GTCC) feature and Mel Frequency Cepstral Coefficient (MFCC) feature, as well as a combination of both. Finally, the speech signals were represented as frames and used as the input for the CRNN architecture. After conducting experiments, the results of the proposed system indicate higher performance with combined GTCC and MFCC features compared to GTCC or MFCC features used individually. The average accuracy of the proposed system was 92.81% in the best experiment for spoken language identification. Furthermore, the system can learn language-specific patterns in various filter size representations of speech files.

Ear is a relatively new biometric among others. Many methods have been used for ear recognition to improve the performance of ear recognition systems. In continuation of these efforts, we propose a new ear recognition method based on curvelet transform. Features of the ear are computed by applying Fast Discrete Curvelet Transform via the wrapping technique. Feature vector of each image is composed of an approximate curvelet coefficient and second coarsest level curvelet coefficients at eight different angles. k-NN (k-nearest neighbour) is utilized as a classifier. The proposed method is experimented on two ear databases from IIT Delhi. Results achieved using the proposed method on publicly available ear database are up to 97.77% which show encouraging performance.