Recurrent Binary Patterns and CNNs for Offline Signature Verification

Signature verification task needs relevant signature representations to achieve low error rates. Many signature representations have been proposed so far. In this work we propose a hybrid user-independent/dependent offline signature verification technique with a two-channel convolutional neural network (CNN) both for verification and feature extraction. Signature pairs are input to the CNN as two channels of one image, where the first channel always represents a reference signature and the second channel represents a query signature. We decrease the image size through the network by keeping the convolution stride parameter large enough. Global average pooling is applied to decrease the dimensionality to 200 at the end of locally connected layers. We utilize the CNN as a feature extractor and report 4.13% equal error rate (EER) considering 12 reference signatures with the proposed 200-dimensional representation, compared to 3.66% of a recently proposed technique with 2048-dimensional representation using the same experimental protocol. When the two methods are combined at score level, more than 50% improvement (1.76% EER) is achieved demonstrating the complementarity of them. Sensitivity of the model to gray-level and binary images is investigated in detail. One model is trained using gray-level images and the other is trained using binary images. It is shown that the availability of gray-level information in train and test data decreases the EER e.g. from 11.86% to 4.13%. information about the claimed identity and they can usually be detected with very small error rates. Our focus in this work is offline signature verification where the query is either genuine signature or skilled forgery. Format of the signature image can be binary, gray-level or color. To the best of our knowledge, no public color signature database is available. However the distinction between binary and gray-level image is critical, as a gray-level image carries more information than the binary one such as the pressure of the stroke. Performance of signature verification systems is generally measured by equal error rate (EER) which is the error rate when the false accept (FA) and false reject (FR) rates are equal. If the EER is not reported, distinguishing error rate (DER) can be calculated which is simply the average of FA and FR.

In this paper, alternative direction finding methods have been proposed using deep learning techniques. Firstly, Regeression and Classification models have created by using Convolutional Neural Networks (CNNs). In the second Convolutional Neural Networks and Recurrent Neural Networks (RNNs) have been utilized in the proposed methods. Despite having vast amount of direction finding studies, utilization of neural networks is scarce in literature and past works mostly only includes usage of CNNs. In this study, direction finding is performed by learning signals reaching multiple antenna arrays by networks. Created neural networks have been fed with different data formats and their performances against noisy and no-noise data have been shown. In addition, comparative analysis of the developed models were made in the similar Signal-to-Noise Ratio (SNR) range with the subspace based MUSIC algorithm, which is frequently used in direction finding.

The study presents the results of the development and testing of deep learning neural network architectures, which demonstrate high accuracy rates in classifying neurophysiological data, in particular hemodynamic brain activation patterns obtained by functional near-infrared spectroscopy, during solving mathematical problems on spatial-numerical associations. The analyzed signal represents a multidimensional time series of oxyhemoglobin and deoxyhemoglobin dynamics. Taking the specificity of the fNIRS signal into account, a comparative analysis of 2 types of neural network architectures was carried out: (1) architectures based on recurrent neural networks: recurrent neural network with long short-term memory, recurrent neural network with long short-term memory with fully connected layers, bidirectional recurrent neural network with long short-term memory, convolutional recurrent neural network with long short-term memory; (2) architectures based on convolutional neural networks with 1D convolutions: convolutional neural network, fully convolutional neural network, residual neural network. Trained long short-term memory recurrent neural network architectures showed worse results in accuracy in comparison with 1D convolutional neural network architectures. Residual neural network (model_Resnet) demonstrated the highest accuracy rates in three experimental conditions more than 88% in detecting age-related differences in brain activation during spatial-numerical association tasks considering the individual characteristics of the respondents’ signal.

The presented paper proposes a new speech command recognition model for novel engineering applications with limited resources. We built the proposed model with the help of a Convolutional Recurrent Neural Network (CRNN). The use of CRNN instead of Convolutional Neural Network (CNN) helps us to reduce the model parameters and memory requirement as per resource constraints. Furthermore, we insert transmute and curtailment layer between the layers of CRNN. By doing this we further reduce model parameters and float number of operations to half of the CRNN requirement. The proposed model is tested on Google’s speech command dataset. The obtained result shows that the proposed CRNN model requires 1/3 parameters as compared to the CNN model. The number of parameters of the CRNN model is further reduced by 45% and the float numbers of operations between 2% to 12 % in different recognition tasks. The recognition accuracy of the proposed model is 96% on Google’s speech command dataset, and on laboratory recording, its recognition accuracy is 89%.

Automatic sleep stage monitoring is an essential tool for the diagnosis and treatment of sleep-related disorders effectively. Although extensive studies focusing on single source data or information, such as single channel electroencephalogram (EEG), electrooculogram (EOG), and electromyogram (EMG), with machine learning (ML) and deep learning (DL) have been presented in this field of automatic sleep staging, there are very limited literature's that have investigated the impact of multisource information fusion on ML or DL based schemes for the sleep stage classification. In addition, exploiting recurrent neural network (RNN) to learn temporal information from the sequences of multi-source inputs can bring significant improvement in classifying the sleep stage. Therefore, we aim to develop an multi-source data fusion, more specifically two source's signal fusion, enabled end-to-end convolutional recurrent neural network (CRNN) in order to perform automatic sleep stage classification in this article. In the proposed scheme, no hand crafted features is used to train the model and hence the proposed model can be called an end-to-end approach. The proposed CRNN model can extract time-invariant features from raw data of two input sources and can fuse them to learn temporally correlated features. Sleep-EDF expanded benchmark dataset is used in this study and we have employed Fpz-Cz EEG channel signal and EOG signal as input with 3 variants of RNN layer in model architecture separately in order to evaluate and investigate the performance. The proposed CRNN model with standard RNN layers can classify the Sleep-EDF dataset with maximum average accuracy of 90.30% and maximum Cohen's kappa coefficient of 86.86 compared other two RNN variant's layer (e.g., gated recurrent unit and long short term memory) based models which is very promising in the studies on classification of sleep stages as well as can be regarded as an alternative to conventional sleep staging.

This paper proposes a novel deep learning architecture involving combinations of Convolutional Neural Networks (CNN) layers and Recurrent neural networks (RNN) layers that can be used to perform segmentation and classification of 5 cardiac rhythms based on ECG recordings. The algorithm is developed in a sequence to sequence setting where the input is a sequence of five second ECG signal sliding windows and the output is a sequence of cardiac rhythm labels. The novel architecture processes as input both the spectrograms of the ECG signal as well as the heartbeats' signal waveform. Additionally, we are able to train the model in the presence of label noise. The model's performance and generalizability is verified on an external database different from the one we used to train. Experimental result shows this approach can achieve an average F1 scores of 0.89 (averaged across 5 classes). The proposed model also achieves comparable classification performance to existing state-of-the-art approach with considerably less number of training parameters.

A hybrid model based on neural networks for biomedical relation extraction

Palm print scanning is a widespread method for biometric identity detection which has some advantages over other methods including its simplicity and relatively lower cost. In this study, a novel methods for biometric verification and identification by contactless palm scanning technique is proposed. In the study, Ripplet-I Transform (R-IT) which is a generalized form of Curvelet Transform (CuT), have been used in addition to multi-resolution transforms which were previously used in the literature as palm print verification and identification methods such as Discrete Cosine Transform (DCT), Discrete Wavelet Transform (DWT), Contourlet Transform (CoT). In addition, Principal Component Analysis (PCA) and Local Binary Pattern (LBP) have been utilized to increase the algorithm diversity. In order to investigate the effect of classification methods on the study results and the processing times, Artificial Neural Network (ANN), Euclidean Distance (ED) and Support Vector Machine (SVM) have been used separately for matching in the verification part of study. The performance of Convolutional Neural Network (CNN) as a classifier has also been examined. Verification and identification algorithms proposed in the study have been tested using palm print images of Hong Kong Polytechnic University Contact-free 3D/2D Hand Images Database (Version 1.0). The studies, that were carried out under two main sections yielded interesting results. At the end of the study, AUC (Area Under the ROC Curve) values ranging from 0.550 (Equal Error Rate (EER)= 0.4594) to 0.9875 (EER= 0.0336) were obtained for palm print verification. The highest AUC value without using LBP was obtained as 0.9563 (EER= 0.1096) using R-IT/CuT+DCT+CNN. Study results were showed that CNN is more successful than other classifiers without using LBP. It also has pointed out that the R-IT/CuT provides better results than the DWT and CoT. Using LBP in algorithms has increased success for ED, SVM and ANN. However, it has reduced overall for CNN. The highest AUC value (0.9875 and EER= 0.0336) was provided by the LBP+DWT+ED algorithm for palm print verification. The highest Identification Rate (IR) was achieved by using the LBP+CoT+ED algorithm with 84.444% for for palm print identification.

Objective. Event-related potential (ERP) sensitivity to faces is predominantly characterized by an N170 peak that has greater amplitude and shorter latency when elicited by human faces than images of other objects. We aimed to develop a computational model of visual ERP generation to study this phenomenon which consisted of a three-dimensional convolutional neural network (CNN) connected to a recurrent neural network (RNN). Approach. The CNN provided image representation learning, complimenting sequence learning of the RNN for modeling visually-evoked potentials. We used open-access data from ERP Compendium of Open Resources and Experiments (40 subjects) to develop the model, generated synthetic images for simulating experiments with a generative adversarial network, then collected additional data (16 subjects) to validate predictions of these simulations. For modeling, visual stimuli presented during ERP experiments were represented as sequences of images (time x pixels). These were provided as inputs to the model. By filtering and pooling over spatial dimensions, the CNN transformed these inputs into sequences of vectors that were passed to the RNN. The ERP waveforms evoked by visual stimuli were provided to the RNN as labels for supervised learning. The whole model was trained end-to-end using data from the open-access dataset to reproduce ERP waveforms evoked by visual events. Main results. Cross-validation model outputs strongly correlated with open-access (r = 0.98) and validation study data (r = 0.78). Open-access and validation study data correlated similarly (r = 0.81). Some aspects of model behavior were consistent with neural recordings while others were not, suggesting promising albeit limited capacity for modeling the neurophysiology of face-sensitive ERP generation. Significance. The approach developed in this work is potentially of significant value for visual neuroscience research, where it may be adapted for multiple contexts to study computational relationships between visual stimuli and evoked neural activity.

In existing convolutional neural networks (CNNs), both convolution and pooling are locally performed for image regions separately, no contextual dependencies between different image regions have been taken into consideration. Such dependencies represent useful spatial structure information in images. Whereas recurrent neural networks (RNNs) are designed for learning contextual dependencies among sequential data by using the recurrent (feedback) connections. In this work, we propose the convolutional recurrent neural network (C-RNN), which learns the spatial dependencies between image regions to enhance the discriminative power of image representation. The C-RNN is trained in an end-to-end manner from raw pixel images. CNN layers are firstly processed to generate middle level features. RNN layer is then learned to encode spatial dependencies. The C-RNN can learn better image representation, especially for images with obvious spatial contextual dependencies. Our method achieves competitive performance on ILSVRC 2012, SUN 397, and MIT indoor.

Sound-event recognition often utilizes time-frequency analysis to produce an image-like spectrogram that provides a rich visual representation of original signal in time and frequency. Convolutional Neural Networks (CNN) with the ability of learning discriminative spectrogram patterns are suitable for sound-event recognition. However, there is relatively little effort that CNN makes full use of the important temporal information. In this paper, we propose MCRNN, a Convolutional Recurrent Neural Networks (CRNN) architecture for sound-event recognition, the letter “M” in the name “MCRNN” of our model denotes the multi-sized convolution filters. Richer features are extracted by using several different convolution filter sizes at the last convolution layer. In addition, cochleagram images are used as the input layer of the network, instead of the traditional spectrogram image of a sound signal. Experiments on the RWCP dataset shows that the recognition rate of the proposed method achieved 98.4% in clean conditions, and it robustly outperforms the existing methods, the recognition rate increased by 0.9%, 1.9% and 10.3% in 20 dB, 10 dB and 0 dB signal-to-noise ratios (SNR), respectively.

Recent experience in applying recurrent neural networks (RNNs) to interpreting Permannt Downhole Gauge (PDG) records has highlighted the potential utility of machine learning algorithms to learn reservoir behavior from data. The power of the RNN resides in its ability to retain information in a form of memory of previous patterns and information contained in the previous behavior of phenomena being modeled. This memory plays a role of informing the decision at the present time by using what happened in the past. This property suggests RNNs as a suitable choice to model sequences of reservoir information, even when the reservoir modeler is faced with incomplete knowledge of the underlying physical system. Convolutional Neural Networks (CNNs) are another variant of machine learning algorithms that have shown promise in sequence modeling domains, such as audio synthesis and machine translation. In this study, RNN and CNN were applied to tasks that traditionally would be modeled by a reservoir simulator. This was achieved by formulating the relationship between physical quantities of interest from subsurface reservoirs as a sequence mapping problem. In addition, the performance of a CNN layer as compared to RNN was evaluated systematically to investigate their capabilities in a variety of tasks of interest to the reservoir engineer. Preliminary results suggest that CNNs, with specific design modifications, are as capable as RNNs in modeling sequences of information and as reliable when making inferences to cases that has not been seen by the algorithm during training. Design details and reasons pertaining to the way these two seemingly different architectures process information, and handle memory are also discussed.

ABSTRACTIn this paper, the convolutional recurrent neural network (ConvLSTM) architecture is proposed to predict individual stock prices. The characteristics of stock data are automatically extracted through convolutional neural network (CNN). Afterward, the extracted features are inputted into a long short-term memory (LSTM) model with memory characteristics for prediction. Since CNN has been a representation learning model, it is quite appropriate for automatic feature extraction. Besides, the LSTM architecture of the recurrent neural network could effectively surmount the issues of gradient disappearance and expansion of the time series data. Ten stock historical data had been collected in the experimental data set. Furthermore, many commonly used technical indicators are calculated in advance for the expansion of the dimension of the training samples. The experimental results obtain a 3.449 RMSE (root-mean-square error) in average.

Following recent advancements in deep learning and artificial intelligence, spoken language identification applications are playing an increasingly significant role in our day-to-day lives, especially in the domain of multi-lingual speech recognition. In this article, we propose a spoken language identification system that depends on the sequence of feature vectors. The proposed system uses a hybrid Convolutional Recurrent Neural Network (CRNN), which combines a Convolutional Neural Network (CNN) with a Recurrent Neural Network (RNN) network, for spoken language identification on seven languages, including Arabic, chosen from subsets of the Mozilla Common Voice (MCV) corpus. The proposed system exploits the advantages of both CNN and RNN architectures to construct the CRNN architecture. At the feature extraction stage, it compares the Gammatone Cepstral Coefficient (GTCC) feature and Mel Frequency Cepstral Coefficient (MFCC) feature, as well as a combination of both. Finally, the speech signals were represented as frames and used as the input for the CRNN architecture. After conducting experiments, the results of the proposed system indicate higher performance with combined GTCC and MFCC features compared to GTCC or MFCC features used individually. The average accuracy of the proposed system was 92.81% in the best experiment for spoken language identification. Furthermore, the system can learn language-specific patterns in various filter size representations of speech files.

Analysis of spectrum sensing using deep learning algorithms: CNNs and RNNs