Multimodal Feature-Level Fusion for Biometrics Identification System on IoMT Platform

Compared with single biometric recognition, multimodal biometric recognition based on fingerprint and finger vein has been widely considered because of its convenient sample collection, high security and accurate recognition. However, according to our investigation, there is no public dataset of fingerprint and finger vein collected at the same time. The existing work uses fingerprint datasets and finger vein datasets from different sources for research, besides the researchers data from building their own equipment, which lacks consideration of practical applications. This is not conducive to the promotion of multibiometric technology based on finger. To promote research on multimodal biometric recognition based on fingerprint and finger vein, we design a finger collection device and introduce a new dataset, NUPT-FPV. It is the first public dataset to collect fingerprint and finger vein simultaneously in real-world applications. NUPT-FPV obtained 840 finger information from 140 volunteers, each finger was collected 20 times (collected in two sessions), and 33600 fingerprint and finger vein images were obtained. In addition, we propose a novel multimodal fusion method based on a convolutional neural network as a benchmark. Extensive experiments were conducted to verify the necessity of our dataset. Through the released dataset and benchmark, we hope to further promote the development of multimodal biometrics based on fingerprint and finger vein.

Recent years have seen a plenty of personal identification methods with different biometrics such as finger pattern, face, palm-print and vein. The majority of these methods focus on complex image data projections and transforms in Fourier space, wavelet space or other domains, which usually bring heavy load in computation and difficult understanding in perceptual intuition. Moreover, these methods, oriented to multiple samples learning, are constricted usually in application. Among so much biometrics, vein, as a living feature with high anti-counterfeiting capability, has attracted considerable attention. In this paper, we propose a structured personal identification approach using finger vein Location and Direction Coding(LDC). First of all, we design a finger vein imaging device with near-infrared(NIR) light source, by which a database for finger vein images is established. Subsequently, we make use of the brightness difference in the finger vein image to extract the vein pattern. Furthermore, finger vein LDC is proposed and performed, which creates a structured feature image for each finger vein. Finally, the structured feature image is utilized to conduct the personal identification on our image database for finger vein, which includes 440 vein images from 220 different fingers. The equal error rate of our method for this database is 0.44%.

In this study, the authors propose a multimodal biometric system by combining the finger knuckle and finger vein images at feature‐level fusion using fractional firefly (FFF) optimisation. Biometric characteristics, like finger knuckle and finger vein are unique and secure. Initially, the features are extracted from the finger knuckle and finger vein images using repeated line tracking method. Then, a newly developed method of feature‐level fusion using FFF optimisation is used. This method is utilised to find out the optimal weight score to fuse the extracted feature sets of finger knuckle and finger vein images. Thus, the recognition is carried out by the fused feature set using layered k‐SVM (k‐support vector machine) which is newly developed by combining the layered SVM classifier and k‐neural network classifier. The experimental results are evaluated and the performance is analysed with false acceptance ratio, false rejection ratio and accuracy. The outcome of the proposed FFF optimisation system obtains a higher accuracy of 96%.

Recently, finger vein sensors have been embedded in all kinds of electronic devices for personal identification, such as intelligent door locks and attendance machines. The embedded sensors are generally small, thus capturing only part of the finger vein. However, prior studies have focused on near-full finger vein recognition, without considering the partial finger vein image caused by the small imaging window of the finger vein sensor. This paper aims to study personal identification based on partial finger vein images, known as small-area finger vein recognition. The effect of the small-area finger vein on recognition performance is first analyzed by cutting out the local part from the near-full finger vein image to model a small-area finger vein image. Second, a small-area finger vein database is built using a commercial finger vein imaging device, in which the vein pattern from approximately one-third of one adult finger is captured. To explore more discriminative information from small-area finger vein images, we propose a locality-constrained consistent dictionary learning (LCDL) method to fuse multiple features for small-area finger vein recognition. Finally, the proposed method is evaluated on the self-built small-area finger vein database and four synthetic small-area finger vein databases. Experimental results show the promising recognition performance of the proposed method.

In the field of data security, biometric security is a significant emerging concern. The multimodal biometrics system with enhanced accuracy and detection rate for smart environments is still a significant challenge. The fusion of an electrocardiogram (ECG) signal with a fingerprint is an effective multimodal recognition system. In this work, unimodal and multimodal biometric systems using Convolutional Neural Network (CNN) are conducted and compared with traditional methods using different levels of fusion of fingerprint and ECG signal. This study is concerned with the evaluation of the effectiveness of proposed parallel and sequential multimodal biometric systems with various feature extraction and classification methods. Additionally, the performance of unimodal biometrics of ECG and fingerprint utilizing deep learning and traditional classification technique is examined. The suggested biometric systems were evaluated utilizing ECG (MIT-BIH) and fingerprint (FVC2004) databases. Additional tests are conducted to examine the suggested models with:1) virtual dataset without augmentation (ODB) and 2) virtual dataset with augmentation (VDB). The findings show that the optimum performance of the parallel multimodal achieved 0.96 Area Under the ROC Curve (AUC) and sequential multimodal achieved 0.99 AUC, in comparison to unimodal biometrics which achieved 0.87 and 0.99 AUCs, for the fingerprint and ECG biometrics, respectively. The overall performance of the proposed multimodal biometrics outperformed unimodal biometrics using CNN. Moreover, the performance of the suggested CNN model for ECG signal and sequential multimodal system based on neural network outperformed other systems. Lastly, the performance of the proposed systems is compared with previously existing works.

Recent advancements in finger vein recognition technology: Methodology, challenges and opportunities

Finger vein verification is a promising biometric pattern for personal identification in terms of security and convenience. The recognition performance of this technology heavily relies on the quality of finger vein images and on the recognition algorithm. To achieve efficient recognition performance, a special finger vein imaging device is developed, and a finger vein recognition method based on sparse representation is proposed. The motivation for the proposed method is that finger vein images exhibit a sparse property. In the proposed system, the regions of interest (ROIs) in the finger vein images are segmented and enhanced. Sparse representation and sparsity preserving projection on ROIs are performed to obtain the features. Finally, the features are measured for recognition. An equal error rate of 0.017% was achieved based on the finger vein image database, which contains images that were captured by using the near-IR imaging device that was developed in this study. The experimental results demonstrate that the proposed method is faster and more robust than previous methods.

Traditional identity verification of students based on the human proctoring approach can cause a scam identity verification and ineffective processing time, particularly among vast groups of students. Most student identification cards outdated personal information. Several biometric recognition approaches have been proposed to strengthen students’ identity verification. Most educational adoption technologies struggle with evaluation and validation techniques to ensure that biometric recognition systems are unsuitable for utilization and implementation for student identity verification. This study presents the internet of things to develop flexible biometric recognition systems and an approach to assess the quality of biometric systems for educational use by investigating the effectiveness of identity verification of various biometric recognition technologies compared to the traditional verification method. The unimodal, multimodal, and semi-multimodal biometric technologies were tested using the developed internet of things-base biometric recognition systems examined by applying the proposed quality metrics of scoring factors based on accuracy, error rate, processing time, and cost. Hundreds of undergraduate exam takers were a sample group. Key findings indicate that the designed and presented systems suitably attain identity verification of exam students using a unimodal biometric. The unimodal facial biometric system promises excellent support. A unimodal fingerprint biometric system ensures second excellent aid for student identity verification. However, multimodal and semi-multimodal biometric systems provide better accuracy with fewer handling times and higher costs. This study contributes significantly to the knowledge of utilizing biometric recognition for identity verification in smart educational applications.

Exploring finger vein based personal authentication for secure IoT

The biometrics is an alternative for password-based recognition systems in recent days. But biometric recognition systems are venerable to spoof attacks. Hence, anti-spoofing techniques are attracting growing interest in this field. Liveness detection is the anti-spoofing technique, which has the aim to identify the dead and living modalities presented at the time of authentication. This paper presents an efficient and robust multimodal biometric system integrated with liveness detection technique. In the proposed system, face and fingerprint modalities are used at the enrolment time and only face modality is used to authenticate a person. Here, fingerprint is used to generate a fingerprint key using K-means clustering algorithm and then encrypt the face feature set with asymmetric encryption standard (AES) using fingerprint key. At the time of authentication, liveness of a person is checked using challenge--response method at the sensor level. If in any case, an intruder has passed liveness test, he/she may be detected at matching score level as template of system is secured with AES encryption. Multimodal biometric system integrated with liveness detection technique can increase the speed of the system. Matrix Laboratory (MATLAB) 2017b and multi-biometric integration (MUBI) tool are used for the experimental work, and results show that the proposed system gives better performance as compared to others.KeywordsFace recognitionEncrypted face templateLiveness detectionMultimodal biometric system

multimodal biometric systems have been widely used to overcome the limitation of unimodal biometric systems and to achieve high recognition accuracy. However, users feel inconvenience because most of the multimodal systems require several steps in order to acquire multimodal biometric data, which also requires the specific behaviors of users. In this research, we propose a new multimodal biometric recognition of touched fingerprint and finger-vein. This paper is novel in the following four ways. First, we can get a fingerprint and a finger-vein image at the same time by the proposed device, which acquires the fingerprint and finger-vein images from the first and second knuckles of finger, respectively. Second, the device's size is so small that we can adopt it on a mobile device, easily. Third, fingerprint recognition is done based on the minutia points of ridge area and finger-vein recognition is performed based on local binary pattern (LBP) with appearance information of finger area. Fourth, based on decision level fusion, we combined two results of fingerprint and finger-vein recognition. Experimental results confirmed the efficiency and usefulness of the proposed method.

The use of personal identity authentication systems with multi-modal biometrics has been proposed in order to increase the performance and robustness against environmental variation and fraudulent attacks. This paper presents a novel fingerprint and finger vein identity authentication system based on multi-route detection. Firstly, two classifiers are designed for fingerprint image and finger vein image respectively. Then extracted feature vectors from the first stage are then concatenated to make the third classifier. The final result is achieved by the fusion of the three classifiers' recognition results at the decision level. Experimental results show that this algorithm not only overcomes the limitations of single-modal biometrics, but also effectively improves the recognition performance of the system.

Biometric Systems (BS) are commonly used to recognize individuals based on biological characteristics such as fingerprints, iris, ears, face, signatures, veins, typing styles gaits, and voices. Uni-modal BS does not have improved protection or recognition accuracy, so multi-modal BS is implemented. However, multi-modal BS has several disadvantages, including intra-class differences, non-universality, Spoof attacks, and distinctiveness. To address these problems, this paper proposes a multimodal biometric image watermarking scheme based on Particle Swarm Optimization (PSO). The sharpening filter is used in this method for image enhancement, resulting in an effective input image for authentication. For face and finger vein feature value extraction, Empirical Mode Decomposition (EMD) technique is used. For fingerprint feature value extraction, minutiae extraction methodology is used. For combining the feature, feature level fusion is used. The main concept of proposed technique is to use a fingerprint image and demographic data to watermark an individual's face and finger vein image. For embedding the watermark, PSO is used to select the best Discrete Cosine Transform (DCT) coefficients in the face and finger vein image. PSO's objective feature is focused on capacity of human visual perception and the watermarked image's ability to withstand image processing attacks. When the watermark is removed on the receiver's hand, the functional utility of biometrics is preserved, resulting in a more secure.KeywordsBiometricsMultimodalPSOWatermarkingSecurityAuthentication

Finger vein recognition has become a research hotspot in the field of biometrics due to its advantages of non-contact acquisition, unique information, and difficulty in terms of forging or pirating. However, in the real-world application process, the extraction of image features for the biometric remains a significant challenge when the captured finger vein images suffer from blur, noise, or missing feature information. To address the above challenges, we propose a novel deep reinforcement learning-based finger vein image recovery method, DRL-FVRestore, which trained an agent that adaptively selects the appropriate restoration behavior according to the state of the finger vein image, enabling continuous restoration of the image. The behaviors of image restoration are divided into three tasks: deblurring restoration, defect restoration, and denoising and enhancement restoration. Specifically, a DeblurGAN-v2 based on the Inception-Resnet-v2 backbone is proposed to achieve deblurring restoration of finger vein images. A finger vein feature-guided restoration network is proposed to achieve defect image restoration. The DRL-FVRestore is proposed to deal with multi-image problems in complex situations. In this paper, extensive experimental results are conducted based on using four publicly accessible datasets. The experimental results show that for restoration with single image problems, the EER values of the deblurring network and damage restoration network are reduced by an average of 4.31% and 1.71%, respectively, compared to other methods. For images with multiple vision problems, the EER value of the proposed DRL-FVRestore is reduced by an average of 3.98%.

The biometric recognition on contactless multi-spectrum finger images