SecureSense: Enhancing Person Verification through Multimodal Biometrics for Robust Authentication

The primary factor driving the growing interest in novel biometric features is the vulnerability of traditional methods like fingerprint and facial recognition to forgery. This study focuses on a multimodal biometric identification system that integrates data from electroencephalograms (EEG) and facial features. To derive valuable insights from EEG data, we apply signal processing techniques such as filtering, segmentation, and feature extraction, alongside Daubechies-4 (DB4) wavelet analysis with five decomposition levels. The enhanced facial video features include entropy calculations and tracking of facial measurements. Six classifiers Gaussian Naïve Bayes, K-Nearest Neighbour, Random Forest, AdaBoost, Support Vector Machine, and Multilayer Perceptron were trained utilizing the combined EEG and facial data. Findings reveal that AdaBoost and Random Forest emerged as the most effective classifiers for this application, achieving accuracies of 99.87±0.13% and requiring EEG recording times of 2s and 1.5s, respectively, showcasing excellent precision.

In the field of data security, biometric security is a significant emerging concern. The multimodal biometrics system with enhanced accuracy and detection rate for smart environments is still a significant challenge. The fusion of an electrocardiogram (ECG) signal with a fingerprint is an effective multimodal recognition system. In this work, unimodal and multimodal biometric systems using Convolutional Neural Network (CNN) are conducted and compared with traditional methods using different levels of fusion of fingerprint and ECG signal. This study is concerned with the evaluation of the effectiveness of proposed parallel and sequential multimodal biometric systems with various feature extraction and classification methods. Additionally, the performance of unimodal biometrics of ECG and fingerprint utilizing deep learning and traditional classification technique is examined. The suggested biometric systems were evaluated utilizing ECG (MIT-BIH) and fingerprint (FVC2004) databases. Additional tests are conducted to examine the suggested models with:1) virtual dataset without augmentation (ODB) and 2) virtual dataset with augmentation (VDB). The findings show that the optimum performance of the parallel multimodal achieved 0.96 Area Under the ROC Curve (AUC) and sequential multimodal achieved 0.99 AUC, in comparison to unimodal biometrics which achieved 0.87 and 0.99 AUCs, for the fingerprint and ECG biometrics, respectively. The overall performance of the proposed multimodal biometrics outperformed unimodal biometrics using CNN. Moreover, the performance of the suggested CNN model for ECG signal and sequential multimodal system based on neural network outperformed other systems. Lastly, the performance of the proposed systems is compared with previously existing works.

Biometrics are increasingly used for access control, fraud detection, and authentication systems. Nevertheless, attackers can deceive such systems using forged biometrics. This research proposes a novel method that makes biometric security systems more resilient to such attacks. The proposed method transforms the user’s biometric data into an irreversible code to protect the original data. This code combines data from multiple biometric modalities, making fabricating a false biometric harder. Additionally, the proposed method does not depend on any secret keys, which helps avoid cases of stolen tokens. The proposed method utilizes the generative adversarial network (GAN) to generate synthetic biometric templates from multiple modalities, which is considered a transformation function for biometric data. Three fusion levels are presented; features from multiple biometric modalities are extracted first in each fusion level. Subsequently, the features train a generative adversarial network to produce synthesized biometric templates. These synthesized templates serve as secure substitutes for the original biometrics during authentication, preventing direct exposure of raw biometric data. We evaluated our methods on the CASIA-V3-Internal and MMU1 iris datasets and the AT&T (ORL) and FERET face datasets. The results showed that our proposed methods can achieve higher accuracy, usability, and improved security compared to a single biometric modality. The proposed feature-level, GAN-based, and decision-level fusion schemes achieved 2.03%, 0.82%, and 0.0297% error rates, respectively, for CASIA and ORL datasets and 1.53%, 0.80%, and 0.0313% error rates, respectively, for MMU1 and FERET datasets. Moreover, we have demonstrated that our method resists pre-image and correlation attacks.

Ear based identity recognition subject to uncontrolled conditions such as illumination changes, pose variation, low contrast, partial occlusion and noise, is an active research area in the field of biometrics. Meanwhile, multimodal biometrics is becoming increasingly popular and offers improved performance due to the use of multiple sources of information. In this paper, a multimodal biometrics system is proposed based on the ear and profile face that not only alleviates the short-comings of ear biometrics but also improves the overall recognition rate. The ear and profile face modalities are first represented individually using the combination of two efficient local feature descriptors namely, local phase quantization (LPQ) and local directional patterns (LDP). These histogram-based local descriptors are then combined into a high-dimensional feature vector that preserves complementary information in both frequency and spatial domains. The PCA along with the z-score normalization technique is independently applied on each feature vector and the resultant reduced feature vectors are combined at the feature level. The kernel discriminative common vector (KDCV) approach is finally exploited over the combined feature set to derive more discriminative and non-linear features for the identification of individuals using kNN classifier. The effectiveness of the proposed model has been verified with the deep features derived from three popular pre-trained CNN models such as AlexNet, VGG16 and GoogleNet. Experimental results on two benchmark databases clearly show that the proposed approach achieves better performance than individual modality and other state-of-the-art methods.

Biometric systems based solely on one-modal biometrics are often not able to meet the desired performance requirements for large user population applications, due to problems such as noisy data, intra-class variations, restricted degrees of freedom, nonuniversity, spoof attacks, and unacceptable error rates. Multimodal biometrics refers to the use of a combination of two or more biometric modalities in a single identification system. The most compelling reason to combine different modalities is to improve the recognition accuracy. This can be done when features of different biometrics are statistically independent. This paper overviews and discusses the various scenarios that are possible in multimodal biometric systems using fingerprint, face and iris recognition, the levels of fusion that are possible and the integration strategies that can be adopted to fuse information and improve overall system accuracy. This paper also discusses how the image quality of fingerprint, face and iris used in the multimodal biometric systems affects the overall identification accuracy and the need of staffing for the secondary human validation. For a large user population identification system, which often has more than tens or hundreds of millions of subject images already enrolled in the matcher databases and has to process more than hundreds of thousands of identification requests, the system's identification accuracy and the need of staffing levels to properly operate the system are two of the most important factors in determining whether a system is properly designed and integrated

Human identification is essential for proper functioning of society. Human identification through multimodal biometrics is becoming an emerging trend, and one of the reasons is to improve recognition accuracy. Unimodal biometric systems are affected by various problemssuch as noisy sensor data,non-universality, lack of individuality, lack of invariant representation and susceptibility to circumvention.A unimodal system has limited accuracy. Hence, Multimodal biometric systems by combining more than one biometric feature in different levels are proposed in order to enhance the performance of the system. A supervisor module combines the different opinions or decisions delivered by each subsystem and then make a final decision. In this paper, a multimodal biometrics authentication is proposed by combining face, iris and finger features. Biometric features are extracted by Local Derivative Ternary Pattern (LDTP) in Contourlet domain and an extensive evaluation of LDTP is done using Support Vector Machine and Nearest Neighborhood Classifier. The experimental evaluations are performed on a public dataset demonstrating the accuracy of the proposed system compared with the existing systems. It is observed that, the combination of face, fingerprint and iris gives better performance in terms of accuracy, False Acceptance Rate, False Rejection Rate with minimum computation time.

In multimodal biometric systems, the fusion of information is an important step. Information can be fused at the different levels, i.e., at the feature, matching or decision level, of the recognition system. In this research feature fusion of three biometric features viz. face, profile face, and ear has fused using discriminant correlation analysis. Discriminant correlation analysis performs an effective functional fusion through maximizing the pair correlations of the two feature sets and simultaneously eliminating correlations between classes and limiting correlations to the classes. In pattern recognition applications, the proposed method can be used to fuse features extracted from multiple modalities or combine different vectors extracted from a single modality. Many sets of experiments carried out on different biometric databases and using various techniques for extraction of the features demonstrate the effectiveness of the proposed method, which results in more advanced approaches.

A biometric system is an amalgamation of algorithms that uses an individual’s unique biological data innate physical features (fingerprints, retina, face, ears, etc.) or behavioural features (signature, voice, keystroke dynamics, etc.) or both to authenticate the individual. This essentially makes a biometric system function as a pattern recognition system that helps an individual in any kind of adversarial environment. Hence, like any conventional authentication system, it is vulnerable to the attacks of malicious adversaries that try to manipulate the data and thus decline the security of the system by compromising its robustness. These possible attacks have not been considered while developing most biometric authentication systems. Our work describes multimodal biometrics, its feature extraction methods, research challenges and focuses on the major security issues in both unimodal and multimodal biometric systems. This paper summarizes some robust multimodal systems, their countermeasures and algorithms that have been devised to tackle these security challenges.KeywordsBiometricsMultimodal biometricsBiometric fusionSecurityAttacksCountermeasuresEncryption

In the case of most of the biometric recognition systems, the type of features extracted from the biometric template depends on the biometric modality. A generalized feature representation is convenient and efficient in multimodal biometric system as the complexity involved in fusing or protecting the heterogeneous features corresponding to different biometric modalities can be avoided. The dimensionality of the features can also be reduced effectively if we can extract generalized feature vectors without compromising the matching performance. This work proposes two schemes: (i) a generalized yet efficient matching technique based on the cosine similarity of the principal component subspaces (ii) a protection scheme by fusing the feature vectors of multiple modalities in principal components domain and matching in that protected domain. The key feature of the proposed protection scheme over existing schemes is that it does not require a user specific key for ensuring the irreversibility. User specific key is used only for ensuring the revocability of the protected template. Results show that the proposed method is able to achieve a matching performance of $$98.57\%$$ for a multimodal system with face, fingerprint, and finger vein templates in the fused domain.Ragendhu, S. P. Thomas, Tony

"This paper explores the application of neural networks in multimodal biometric authentication systems, emphasizing the integration of multiple biometric modalities to enhance security, accuracy, and robustness against various attacks. Traditional authentication methods, such as passwords and single-modal biometrics, often suffer from vulnerabilities, including spoofing, environmental factors, and data breaches. To address these challenges, multimodal authentication systems combine several biometric traits, such as facial recognition, fingerprint scanning, voice recognition, and keystroke dynamics, to achieve higher reliability and resistance to security threats. The study provides an overview of public datasets used for training neural network-based biometric authentication models, including VoxCeleb, RAVDESS, MOBIO, and SDUMLA-HMT. These datasets contain diverse biometric information necessary for developing robust multimodal authentication systems. The paper evaluates the effectiveness of existing approaches using key performance metrics such as accuracy, false acceptance rate (FAR), false rejection rate (FRR), and area under the curve (AUC). Additionally, specialized metrics are considered, including failure to enroll rate (FTE), failure to acquire rate (FTA), and template stability (TS), which are crucial for real-world applications. The role of neural networks in multimodal biometric authentication is analyzed by examining state-of-the-art architectures, including convolutional neural networks (CNNs) and deep learning-based feature fusion methods. Various fusion levels—feature-level, score-level, and decision-level—are discussed to determine the optimal integration strategy for improving authentication performance. The results indicate that multimodal systems significantly outperform unimodal authentication methods by reducing vulnerability to spoofing and environmental variations. Experimental findings suggest that integrating multiple biometric traits enhances the system’s adaptability to dynamic conditions, reducing both false acceptance and false rejection rates. Despite these advantages, several challenges remain, including computational complexity, data privacy concerns, and the need for real-time processing capabilities. Future research should focus on optimizing multimodal fusion techniques, improving generalization across different datasets, and enhancing the security of biometric templates against adversarial attacks. Additionally, developing lightweight neural network architectures suitable for mobile and embedded systems is essential for the practical deployment of multimodal authentication technologies"

A multimodal biometric system is applied to recognize individuals for authentication using neural networks. In this paper multimodal biometric algorithm is designed by integrating iris, finger vein, palm print and face biometric traits. Normalized score level fusion approach is applied and optimized, encoded for matching decision. It is a multilevel wavelet, phase based fusion algorithm. This robust multimodal biometric algorithm increases the security level, accuracy, reduces memory size and equal error rate and eliminates unimodal biometric algorithm vulnerabilities.

Biometric authentication systems operating in real world environments using a single modality are found to be insecure and unreliable due to numerous limitations. Multimodal biometric systems have better accuracy and reliability due to the use of multiple biometric traits to authenticate a claimed identity or perform identification. In this paper a novel method for person identification using multimodal biometrics with hand geometry and palmprint biometric traits is proposed. The geometrical information embedded in the user hand and palmprint images are brought out through the graph representations. The topological characterization of the image moments, represented as the virtual nodes of the palmprint image graph is a novel feature of this work. The user hand and palmprint images are represented as weighted undirected graphs and spectral characteristics of the graphs are extracted as features vectors. The feature vectors of the hand geometry and palmprint are fused at feature level to obtain a graph spectral feature vector to represent the person. User identification is performed by using a multiclass support vector machine (SVM) classifier. The experimental results demonstrate, an appreciable performance giving identification rate of 99.19% for multimodal biometric after feature level fusion of hand geometry and palmprint modalities. The performance is investigated by conducting the experiments separately for handgeometry, palmprint and fused feature vectors for person identification. Experimental results show that the proposed multimodal system achieves better performance than the unimodal cues, and can be used in high security applications. Further comparison show that it is better than similar other multimodal techniques.

Digital Rights Management (DRM) system is the effective schemes for digital transactions. A novel authentication scheme of DRM system for remote users based on multimodal biometrics (such as iris and face feature) verification and watermarking and smart cards is described in this paper, which include two authentication phases, i.e. the client server authentication and the server authentication. For the client server authentication, we integrate watermarking technique and multimodal biometric system which is based on highly secure iris recognition and face recognition to provide more secure and reliable personal recognition. In watermarking algorithm, face image is chosen to be the host image, iris feature is selected to use as watermark hidden in the host face image. For the server authentication, the proposed scheme is an extended and generalized form of ElGamal signature scheme whose security is based on discrete logarithm problem, which is not yet forged. So the proposed scheme can achieve the rights management of digital content exactly through the illegal user access control. Meanwhile, the bimodal biometrics (iris and face) recognition provides the improvement in the accuracy performance of the system.

Review Paper on Applications of Principal Component Analysis in Multimodal Biometrics System

In this research paper I discuss about the challenges to implement secure personal identification protocols with biometric technology are increasing and the need for accurate human identification is higher than ever. Single modality biometric systems have to contend with a variety of problems such as noisy data, intra class variations, non-universality, spoof attacks, and distinctiveness. Some of these limitations can be addressed by deploying multimodal biometric systems that integrate multiple biometric modalities in a single scan to alleviate the challenges of a unimodal system. Performance in biometric verification is often affected by external conditions and variabilities. These are often related to mismatched conditions between enrolment and verification sessions, e.g. handsets/microphones for recording speech, cameras for capturing facial images and fingerprint readers. In addition, the user’s speech may vary according to ambient noise conditions, the speaker’s health (e.g. contracting a cold) or speaking styles. The user’s facial images may vary due to changes in backgrounds, illumination, head positions and expressions. While none of the biometrics alone can guarantee absolute reliability, they can reinforce one another when used jointly to maximize verification performance. This motivates multi-biometric (multimodal) authentication.