PulseID: Multi-scale photoplethysmographic identification using a deep convolutional neural network

Signature representations that are extracted by convolutional neural networks (CNN) can achieve low error rates. However, a trade-off exists between such models’ complexities and hand-crafted features’ slightly higher error rates. A novel writer-dependent (WD) recurrent binary pattern (RBP) network, and a novel signer identification CNN is proposed. RBP network is a recurrent neural network (RNN) to learn the sequential relation between binary pattern histograms over image windows. A novel histogram selection method is introduced to remove the stop-word codes. Dimensionality is reduced by more than 25% while improving the results. This work is the first to combine binary patterns and RNNs for static signature verification. Several test sets, derived from large-scale and popular databases (GPDS-960 and GPDS-Synthetic-10000) are used. Without training any global classifier, RBP network provides competitive equal error rates (EER). The proposed architectures are compared and integrated with other recent CNN models. Score-level integration of WD classifiers trained with different representations are investigated. Cross-validation tests demonstrate the EERs reduced compared to the best single classifier. A state-of-the-art EER of 1.11% is reported with a global decision threshold (0.57% EER with user-based thresholds) on GPDS-160 database.

For the internet of medical things (IoMT) enabled long-term health monitoring and disease prediction applications, there is a demand for an automatic photoplethysmogram (PPG) signal quality assessment (SQA) for reducing false alarms and energy consumption. This paper presents a PPG-SQA method by using raw PPG signal and convolutional neural network (CNN) with optimal parameters. The main focus of this paper is to find an optimal number of filters (16, 32, 64) and number of layers (2 and 4 layers) with rectified linear unit (ReLU) activation function and to study robustness of trained CNN models by using unseen PPG datasets and different kinds of noise sources, which are not addressed in the past studies on the CNN-based PPG-SQA methods. Evaluation results showed that the 4 layer CNN-based method had the higher accuracy of 99.58% for noise-free PPG (NF-PPG) versus wrist-cup noisy PPG signal database (MA-DB01), 99.99% for NF versus random noises added PPG (RN-PPG) signals, and 75.80% for NF-PPG versus acceleration corrupted PPG signals (MA-DB02). For the unknown dataset, the 4-layer CNN model had the higher accuracy of 96.71% for NF-PPG versus MA-DB01 and 99.04% for NF versus RN-PPG, and the 2-layer CNN model had the higher accuracy of 76.16% for NF-PPG versus MA-DB02 PPG segments. Results demonstrate that the CNN-based PPG-SQA method with optimal parameters is not only improve the accuracy but can also reduce the computational load as compared with other existing methods.

Accurate segmentation and delineation of the sub-tumor regions are very challenging tasks due to the nature of the tumor. Traditionally, convolutional neural networks (CNNs) have succeeded in achieving most promising performance for the segmentation of brain tumor; however, handcrafted features remain very important in identification of tumor's boundary regions accurately. The present work proposes a robust deep learning-based model with three different CNN architectures along with pre-defined handcrafted features for brain tumor segmentation, mainly to find out more prominent boundaries of the core and enhanced tumor regions. Generally, automatic CNN architecture does not use the pre-defined handcrafted features because it extracts the features automatically. In this present work, several pre-defined handcrafted features are computed from four MRI modalities (T2, FLAIR, T1c, and T1) with the help of additional handcrafted masks according to user interest and fed to the convolutional features (automatic features) to improve the overall performance of the proposed CNN model for tumor segmentation. Multi-pathway CNN is explored in this present work along with single-pathway CNN, which extracts simultaneously both local and global features to identify the accurate sub-regions of the tumor with the help of handcrafted features. The present work uses a cascaded CNN architecture, where the outcome of a CNN is considered as an additional input information to next subsequent CNNs. To extract the handcrafted features, convolutional operation was applied on the four MRI modalities with the help of several pre-defined masks to produce a predefined set of handcrafted features. The present work also investigates the usefulness of intensity normalization and data augmentation in pre-processing stage in order to handle the difficulties related to the imbalance of tumor labels. The proposed method is experimented on the BraST 2018 datasets and achieved promising results than the existing (currently published) methods with respect to different metrics such as specificity, sensitivity, and dice similarity coefficient (DSC) for complete, core, and enhanced tumor regions. Quantitatively, a notable gain is achieved around the boundaries of the sub-tumor regions using the proposed two-pathway CNN along with the handcrafted features. Graphical Abstract This data is mandatory. Please provide.

Palm print scanning is a widespread method for biometric identity detection which has some advantages over other methods including its simplicity and relatively lower cost. In this study, a novel methods for biometric verification and identification by contactless palm scanning technique is proposed. In the study, Ripplet-I Transform (R-IT) which is a generalized form of Curvelet Transform (CuT), have been used in addition to multi-resolution transforms which were previously used in the literature as palm print verification and identification methods such as Discrete Cosine Transform (DCT), Discrete Wavelet Transform (DWT), Contourlet Transform (CoT). In addition, Principal Component Analysis (PCA) and Local Binary Pattern (LBP) have been utilized to increase the algorithm diversity. In order to investigate the effect of classification methods on the study results and the processing times, Artificial Neural Network (ANN), Euclidean Distance (ED) and Support Vector Machine (SVM) have been used separately for matching in the verification part of study. The performance of Convolutional Neural Network (CNN) as a classifier has also been examined. Verification and identification algorithms proposed in the study have been tested using palm print images of Hong Kong Polytechnic University Contact-free 3D/2D Hand Images Database (Version 1.0). The studies, that were carried out under two main sections yielded interesting results. At the end of the study, AUC (Area Under the ROC Curve) values ranging from 0.550 (Equal Error Rate (EER)= 0.4594) to 0.9875 (EER= 0.0336) were obtained for palm print verification. The highest AUC value without using LBP was obtained as 0.9563 (EER= 0.1096) using R-IT/CuT+DCT+CNN. Study results were showed that CNN is more successful than other classifiers without using LBP. It also has pointed out that the R-IT/CuT provides better results than the DWT and CoT. Using LBP in algorithms has increased success for ED, SVM and ANN. However, it has reduced overall for CNN. The highest AUC value (0.9875 and EER= 0.0336) was provided by the LBP+DWT+ED algorithm for palm print verification. The highest Identification Rate (IR) was achieved by using the LBP+CoT+ED algorithm with 84.444% for for palm print identification.

Multi-objective optimization sampling based on Pareto optimality for soil mapping

Convolutional neural networks (CNN) have achieved outstanding performance in image recognition tasks, but many hand-crafted features still play important roles in some areas. Hand-crafted features are designed to describe image content from specific aspects, which may provide complementary information for CNN in image classification tasks. This paper explores feature fusion methods and proposes a novel framework for combining CNN with hand-crafted features. The framework has two main advantages. First, feature encoder can encode non-normalized features in CNN, which takes advantage of some good edge, texture and local features. Second, joint training strategy makes features fuse better in CNN. We validate that many handcrafted features help to improve the performance of origin CNN. Experiments show our method outperforms the origin CaffeNet on Cifar10 dataset with 79.16% accuracy.

Stroke is a major reason for disability and mortality in most of the developing nations. Early detection of stroke is highly significant in bio-medical research. Research illustrates that signs of stroke are reflected in the eye and may be analyzed from fundus images. A custom dataset of fundus images has been compiled for formulating an automated stroke detection algorithm. In this paper, a comparative study of hand-crafted texture features and convolutional neural network (CNN) has been recommended for stroke diagnosis. The custom CNN model has also been compared with five pre-trained models from ImageNet. Experimental results reveal that the recommended custom CNN model gives the best performance by achieving an accuracy of 95.8 %.

Abstract. The recent success of deep convolutional neural networks (CNN) on a large number of applications can be attributed to large amounts of available training data and increasing computing power. In this paper, a semantic pixel labelling scheme for urban areas using multi-resolution CNN and hand-crafted spatial-spectral features of airborne remotely sensed data is presented. Both CNN and hand-crafted features are applied to image/DSM patches to produce per-pixel class probabilities with a L1-norm regularized logistical regression classifier. The evidence theory infers a degree of belief for pixel labelling from different sources to smooth regions by handling the conflicts present in the both classifiers while reducing the uncertainty. The aerial data used in this study were provided by ISPRS as benchmark datasets for 2D semantic labelling tasks in urban areas, which consists of two data sources from LiDAR and color infrared camera. The test sites are parts of a city in Germany which is assumed to consist of typical object classes including impervious surfaces, trees, buildings, low vegetation, vehicles and clutter. The evaluation is based on the computation of pixel-based confusion matrices by random sampling. The performance of the strategy with respect to scene characteristics and method combination strategies is analyzed and discussed. The competitive classification accuracy could be not only explained by the nature of input data sources: e.g. the above-ground height of nDSM highlight the vertical dimension of houses, trees even cars and the nearinfrared spectrum indicates vegetation, but also attributed to decision-level fusion of CNN’s texture-based approach with multichannel spatial-spectral hand-crafted features based on the evidence combination theory.

Signature verification task needs relevant signature representations to achieve low error rates. Many signature representations have been proposed so far. In this work we propose a hybrid user-independent/dependent offline signature verification technique with a two-channel convolutional neural network (CNN) both for verification and feature extraction. Signature pairs are input to the CNN as two channels of one image, where the first channel always represents a reference signature and the second channel represents a query signature. We decrease the image size through the network by keeping the convolution stride parameter large enough. Global average pooling is applied to decrease the dimensionality to 200 at the end of locally connected layers. We utilize the CNN as a feature extractor and report 4.13% equal error rate (EER) considering 12 reference signatures with the proposed 200-dimensional representation, compared to 3.66% of a recently proposed technique with 2048-dimensional representation using the same experimental protocol. When the two methods are combined at score level, more than 50% improvement (1.76% EER) is achieved demonstrating the complementarity of them. Sensitivity of the model to gray-level and binary images is investigated in detail. One model is trained using gray-level images and the other is trained using binary images. It is shown that the availability of gray-level information in train and test data decreases the EER e.g. from 11.86% to 4.13%. information about the claimed identity and they can usually be detected with very small error rates. Our focus in this work is offline signature verification where the query is either genuine signature or skilled forgery. Format of the signature image can be binary, gray-level or color. To the best of our knowledge, no public color signature database is available. However the distinction between binary and gray-level image is critical, as a gray-level image carries more information than the binary one such as the pressure of the stroke. Performance of signature verification systems is generally measured by equal error rate (EER) which is the error rate when the false accept (FA) and false reject (FR) rates are equal. If the EER is not reported, distinguishing error rate (DER) can be calculated which is simply the average of FA and FR.

Face anti-spoofing is the practice of using an authorized person's face to escape false face verification with a picture, video, mask, or another substitute. Latest innovations in face recognition and deep learning technology have made it possible to identify people from a distance using images acquired by a camera. The existing methods used different training sets and classifiers to recognize the face, which was difficult to classify and increase the time. A novel technology 3D Convolutional Restricted Basis Neural Network (CRBNN) is proposed in this paper to overcome this drawback. The proposed system is initialized by testing based on cross-entropy loss with augmented facial samples and further reinforced with an explicitly built generalization loss. The 3D-Convolutional layer is used for mapping the feature of the input image. Next, the Boltzmann technique is used for translation to form the reconstructed parameters. The Radial Basis Function Network (RBFN) is used to obtain an accurate classification. The experimental result proved the proposed CRBNN is given a better accuracy rate, less Equal Error Rate, and less time duration than existing methods. Our method is compared with spatial pyramid coding micro-texture (SPMT), Convolutional Neural Network (CNN), and radial basis function network (RBFN) and outperformed.

The success of Convolutional Neural Network (CNN) is attributed to their ability to learn rich midlevel image representations as opposed to hand-crafted low-level features used in many natural image classification methods. Learning CNN, however, amounts to estimating millions of parameters and requires a very large number of annotated image samples. In this paper, we explored transfer learning for gastrointestinal bleeding detection on small-size imbalanced endoscopy images, and showed how image representations learned with CNN on large-scale annotated datasets can be efficiently transferred to other tasks with limited amount of training data. We first transferred pre-trained Inception V3 model trained on the ImageNet dataset to compute mid-level image representation, and then fine-tuned the trained model with labeled endoscopy images, and resumed training from already learned weights. Additionally, we introduce both data augmentation and image resampling to increase the size of the training database and the positive sample rate to perform the Transfer Learning. Our results showed that our transfer learning method produces the best performance on AUC (the area under the receiver operating curve), Precision, Recall and Accuracy as compared to both the hand-crafted feature based method and training CNN model from-scratch method.

Content-Based large-scale image retrieval has recently attracted considerable attention because of the explosive increase of online images. Inspired by recent advances in convolutional neural networks, we propose a hierarchical deep semantic method for learning similarity function that solves the problems of precision and speed of retrieval in the setting of large-scale environments. The distinctive contribution of our work is a novel approach that can utilize previous knowledge of the semantic hierarchy. When semantic information and a related hierarchy structure are available, significant improvements can be attained. Exploiting hierarchical relationships is the most important thing for large-scale issues. Binary code can be learned from deep neural network for representing the latent concepts that dominate the semantic labels. Different from other supervised methods that require learning an explicit hashing function to map the binary code features from the images, our method learns Hierarchical Deep Semantic Hashing code (HDSH-code) and image representations in an implicit manner, making it suitable for large-scale datasets. An additional contribution is a novel hashing scheme (generated at the same time with semantic information) that is able to reduce the computational cost of retrieval. Comprehensive experiments were conducted on Holidays, Oxford5k/105k, Caltech256 retrieval datasets, our HDSH performs competitively even when the convolutional neural network has been pre-trained for a surrogate unrelated task. We further demonstrates its efficacy and scalability on a large-scale dataset Imagenet with millions of images. With deep hierarchical semantic hashing, we report retrieval times are 0.15ms and 53.92ms on Holidays and Imagenet dataset, respectively.

Text recognition in natural scene images is a challenging problem in computer vision. Different than the optical character recognition (OCR), text recognition in natural scene images is more complex due to variations in text size, colors, fonts, orientations, complex backgrounds, occlusion, illuminations and uneven lighting conditions. In this paper, we propose a segmentation-free method based on a deep convolutional recurrent neural network to solve the problem of cursive text recognition, particularly focusing on Urdu text in natural scenes. Compared to the non-cursive scripts, Urdu text recognition is more complex due to variations in the writing styles, several shapes of the same character, connected text, ligature overlapping, stretched, diagonal and condensed text. The proposed model gets a whole word image as an input without pre-segmenting into individual characters, and then transforms into the sequence of the relevant features. Our model is based on three components: a deep convolutional neural network (CNN) with shortcut connections to extract and encode the features, a recurrent neural network (RNN) to decode the convolutional features, and a connectionist temporal classification (CTC) to map the predicted sequences into the target labels. To increase the text recognition accuracy further, we explore deeper CNN architectures like VGG-16, VGG-19, ResNet-18 and ResNet-34 to extract more appropriate Urdu text features, and compare the recognition results. To conduct the experiments, a new large-scale benchmark dataset of cropped Urdu word images in natural scenes is developed. The experimental results show that the proposed deep CRNN network with shortcut connections outperform than other network architectures. The dataset is publicly available and can be downloaded from <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://data.mendeley.com/datasets/k5fz57zd9z/1</uri> .

Microscopic laser engraving surface defect classification plays an important role in the industrial quality inspection field. The key challenges of accurate surface defect classification are the complete description of the defect and the correct distinction into categories in the feature space. Traditional classification methods focus on the terms of feature extraction and independent classification; therefore, feed handcrafted features may result in useful feature loss. In recent years, convolutional neural networks (CNNs) have achieved excellent results in image classification tasks with the development of deep learning. Deep convolutional networks integrate feature extraction and classification into self-learning, but require large datasets. The training datasets for microscopic laser engraving image classification are small; therefore, we used pre-trained CNN models and applied two fine-tuning strategies. Transfer learning proved to perform well even on small future datasets. The proposed method was evaluated on the datasets consisting of 1986 laser engraving images captured by a metallographic microscope and annotated by experienced staff. Because handcrafted features were not used, our method is more robust and achieves better results than traditional classification methods. Under five-fold-validation, the average accuracy of the best model based on DenseNet121 is 96.72%.

Sleep Apnea (SA) is a prevalent sleep disorder with multifaceted etiologies that can have severe consequences for patients. Diagnosing SA traditionally relies on the in-laboratory polysomnogram (PSG), which records various human physiological activities overnight. SA diagnosis involves manual scoring by qualified physicians. Traditional machine learning methods for SA detection depend on hand-crafted features, making feature selection pivotal for downstream classification tasks. In recent years, deep learning has gained popularity in SA detection due to its capability for automatic feature extraction and superior classification accuracy. This study introduces a Deep Attention Network with Multi-Temporal Information Fusion (DAN-MTIF) for SA detection using single-lead electrocardiogram (ECG) signals. This framework utilizes three 1D convolutional neural network (CNN) blocks to extract features from R-R intervals and R-peak amplitudes using segments of varying lengths. Recognizing that features derived from different temporal scales vary in their contribution to classification, we integrate a multi-head attention module with a self-attention mechanism to learn the weights for each feature vector. Comprehensive experiments and comparisons between two paradigms of classical machine learning approaches and deep learning approaches are conducted. Our experiment results demonstrate that (1) compared with benchmark methods, the proposed DAN-MTIF exhibits excellent performance with 0.9106 accuracy, 0.9396 precision, 0.8470 sensitivity, 0.9588 specificity, and 0.8909 [Formula: see text] score at per-segment level; (2) DAN-MTIF can effectively extract features with a higher degree of discrimination from ECG segments of multiple timescales than those with a single time scale, ensuring a better SA detection performance; (3) the overall performance of deep learning methods is better than the classical machine learning algorithms, highlighting the superior performance of deep learning approaches for SA detection.