A Neuronal Morphology Classification Approach Based on Deep Residual Neural Networks

This paper proposes a method to evaluate the degradation stages of stability of the screw with data fusion technology and deep residual neural network. Firstly, the data provided by multi-sensors are fused, and then the time domain images of the signals are input into the deep residual neural network for training and testing. The effectiveness of the proposed method is verified using data sets collected from the degradation test bench of ball screw. The data sets contain massive samples involving 3 degradation stages under 9 working conditions obtained by 3 accelerometers. From the results and comparison, it can be found that the deep residual neural network can automatically extract the features from the original data layer by layer and achieve a better evaluation rate when it is used to evaluate the performance stage of the screw, and the multi-sensor data fusion vibration data can better reflect the degradation stages of the screw's performance. It is showed that this method enhances the intelligence of the evaluation compared with the traditional method.

Automatic modulation classification (AMC) is a signal processing technology used to identify the modulation type of unknown signals without prior information such as modulation parameters for drone communications. In recent years, deep learning (DL) has been widely used in AMC methods due to its powerful feature extraction ability. The significant performance of DL-based AMC methods is highly dependent on large amount of data. However, with the increasingly complex signal environment and the emergence of new signals, several recognition tasks have difficulty obtaining sufficient high-quality signals. To address this problem, we propose an AMC method based on a deep residual neural network with masked modeling (DRMM). Specifically, masked modeling is adopted to improve the performance of a deep neural network with limited signal samples. Both complex-valued and real-valued residual neural networks (ResNet) play an important role in extracting signal features for identification. Several typical experiments are conducted to evaluate our proposed DRMM-based AMC method on the RadioML 2016.10A dataset and a simulated dataset, and comparison experiments with existing AMC methods are also conducted. The simulation results illustrate that our proposed DRMM-based AMC method achieves better performance in the case of limited signal samples with low signal-to-noise ratio (SNR) than other existing methods.

A key challenge in radar micro-Doppler classification is the difficulty in obtaining a large amount of training data due to costs in time and human resources. Small training datasets limit the depth of deep neural networks (DNNs), and, hence, attainable classification accuracy. In this work, a novel method for diversifying Kinect-based motion capture (MOCAP) simulations of human micro-Doppler to span a wider range of potential observations, e.g. speed, body size, and style, is proposed. By applying three transformations, a small set of MOCAP measurements is expanded to generate a large training dataset for network initialization of a 30-layer deep residual neural network. Results show that the proposed training methodology and residual DNN yield improved bottleneck feature performance and the highest overall classification accuracy among other DNN architectures, including transfer learning from the 1.5 million sample ImageNet database.

Image acquisition is an essential step in image processing. When the image acquisition is done the image that is generated is subjected to Impulse noise, Gaussian noise etc. We have performed the image denoising on images inflicted with impulse noise. Image denoising is an essential step in all types of image processing. Traditional techniques reduce the noise in the image but it also reduces the quality of the image. Traditional filters like gaussian filter, median filter is analyzed which work in the spatial domain and filters working in the frequency domain are also considered like Butterworth filters, Weiner filter. A Deep residual Neural Network filter is proposed which is compared with the Fuzzy Neural Network denoiser. Their performance is compared on the metrics PSNR and SSIM. The Fuzzy Neural Network system improves the SSIM significantly compared to a deep residual neural network and a comparison is made with traditional image denoising methods. We also compare the performance of the deep residual neural network, Fuzzy Neural Network system and Median denoising algorithm on impulse noise has been compared. The performance of deep neural networks depends on the total number of examples used and the performance can be improved if we have more image pairs.

Data-driven rolling-bearing fault diagnosis methods are mostly based on deep-learning models, and their multilayer nonlinear mapping capability can improve the accuracy of intelligent fault diagnosis. However, problems such as gradient disappearance occur as the number of network layers increases. Moreover, directly taking the raw vibration signals of rolling bearings as the network input results in incomplete feature extraction. In order to efficiently represent the state characteristics of vibration signals in image form and improve the feature learning capability of the network, this paper proposes fault diagnosis model MTF-ResNet based on a Markov transition field and deep residual network. First, the data of raw vibration signals are augmented by using a sliding window. Then, vibration signal samples are converted into two-dimensional images by MTF, which retains the time dependence and frequency structure of time-series signals, and a deep residual neural network is established to perform feature extraction, and identify the severity and location of the bearing faults through image classification. Lastly, experiments were conducted on a bearing dataset to verify the effectiveness and superiority of the MTF-ResNet model. Features learned by the model are visualized by t-SNE, and experimental results indicate that MTF-ResNet showed better average accuracy compared with several widely used diagnostic methods.

As an enabling technology for emerging and future generations of wireless networks, sparse code multiple access (SCMA) offers major improvements in terms of spectral efficiency and massive connectivity. Although the message passing algorithm (MPA) for SCMA decoding at the receiver side can achieve near optimum performance, it entails high computational complexity. In this paper, to address this issue, we propose a novel SCMA decoder based on deep residual neural network (ResNet), wherein the decoder is trained to predict the transmit codewords. In our approach, residual blocks are employed to tackle the problems of accuracy saturation and vanishing gradients with deep learning based decoder, while batch normalization is utilized to enhance the stability and robustness of the decoder. The performance of the proposed ResNet decoder for SCMA is validated by means of simulations over AWGN and Rayleigh fading channels. The results show that besides a much reduced complexity, the proposed decoder leads to improvements in term of bit error rate (BER) over competing deep neural network (DNN) based decoders.

The aim of this paper is to provide a deep learning based method that can solve high-dimensional Fredholm integral equations. A deep residual neural network is constructed at a fixed number of collocation points selected randomly in the integration domain. The loss function of the deep residual neural network is defined as a linear least-square problem using the integral equation at the collocation points in the training set. The training iteration is done for the same set of parameters for different training sets. The numerical experiments show that the deep learning method is efficient with a moderate generalization error at all points. And the computational cost does not suffer from “curse of dimensionality” problem.

We present PyMerger, a Python tool for detecting binary black hole (BBH) mergers from the Einstein Telescope (ET), based on a deep residual neural network (ResNet) model. ResNet was trained on data combined from all three proposed subdetectors of ET (TSDCD) to detect BBH mergers. Five different lower-frequency cutoffs (F low)—5 Hz, 10 Hz, 15 Hz, 20 Hz, and 30 Hz—with the match-filter signal-to-noise ratio (MSNR) ranges 4–5, 5–6, 6–7, 7–8, and >8 were employed in the data simulation. Compared to previous work that utilized data from a single subdetector, the detection accuracy from TSDCD has shown substantial improvements, increasing from 60%, 60.5%, 84.5%, 94.5%, and 98.5% to 78.5%, 84%, 99.5%, 100%, and 100% for sources with MSNRs of 4–5, 5–6, 6–7, 7–8, and >8, respectively. The ResNet model is evaluated on the first ET mock data challenge (ET-MDC1) data set, where the model demonstrates strong performance in detecting BBH mergers, identifying 5566 out of 6578 BBH events, with optimal SNRs starting from 1.2 and a minimum and maximum D L of 0.5 Gpc and 148.95 Gpc, respectively. Despite being trained only on BBH mergers without overlapping sources, the model achieves high BBH detection rates. Notably, even though the model was not trained on binary neutron star (BNS) and black hole-neutron star (BHNS) mergers, it successfully detected 11,477 BNS and 323BHNS mergers in ET-MDC1, with optimal SNRs starting from 0.2 and 1, respectively, indicating its potential for broader applicability.

This paper makes a research for the speed control of wind turbine system under multiple operating regimes, which also studied the sleep residual recurrent neural network method in this work. We aim at designing deep residual recurrent neural network robust controllers, which guarantee the existence of the multiple regime system poles in some predefined zone and wind speed precise tracking. Moreover, the feedback gains which guarantee desired speed tracking performance are obtained by solving the Lyapunov stability functions. The results are applied to a directly driven wind energy conversion experiment systems and the numerical experiment, comparing with the existing results, shows the satisfactory performance of the proposed method.

Contact-map of a protein sequence dictates the global topology of structural fold. Accurate prediction of the contact-map is thus essential to protein 3D structure prediction, which is particularly useful for the protein sequences that do not have close homology templates in the Protein Data Bank. We developed a new method, ResPRE, to predict residue-level protein contacts using inverse covariance matrix (or precision matrix) of multiple sequence alignments (MSAs) through deep residual convolutional neural network training. The approach was tested on a set of 158 non-homologous proteins collected from the CASP experiments and achieved an average accuracy of 50.6% in the top-L long-range contact prediction with L being the sequence length, which is 11.7% higher than the best of other state-of-the-art approaches ranging from coevolution coupling analysis to deep neural network training. Detailed data analyses show that the major advantage of ResPRE lies at the utilization of precision matrix that helps rule out transitional noises of contact-maps compared with the previously used covariance matrix. Meanwhile, the residual network with parallel shortcut layer connections increases the learning ability of deep neural network training. It was also found that appropriate collection of MSAs can further improve the accuracy of final contact-map predictions. The standalone package and online server of ResPRE are made freely available, which should bring important impact on protein structure and function modeling studies in particular for the distant- and non-homology protein targets. https://zhanglab.ccmb.med.umich.edu/ResPRE and https://github.com/leeyang/ResPRE. Supplementary data are available at Bioinformatics online.

A deep residual neural network for semiconductor defect classification in imbalanced scanning electron microscope datasets

MotivationAntibody structure is largely conserved, except for a complementarity-determining region featuring six variable loops. Five of these loops adopt canonical folds which can typically be predicted with existing methods, while the remaining loop (CDR H3) remains a challenge due to its highly diverse set of observed conformations. In recent years, deep neural networks have proven to be effective at capturing the complex patterns of protein structure. This work proposes DeepH3, a deep residual neural network that learns to predict inter-residue distances and orientations from antibody heavy and light chain sequence. The output of DeepH3 is a set of probability distributions over distances and orientation angles between pairs of residues. These distributions are converted to geometric potentials and used to discriminate between decoy structures produced by RosettaAntibody and predict new CDR H3 loop structures de novo.ResultsWhen evaluated on the Rosetta antibody benchmark dataset of 49 targets, DeepH3-predicted potentials identified better, same and worse structures [measured by root-mean-squared distance (RMSD) from the experimental CDR H3 loop structure] than the standard Rosetta energy function for 33, 6 and 10 targets, respectively, and improved the average RMSD of predictions by 32.1% (1.4 Å). Analysis of individual geometric potentials revealed that inter-residue orientations were more effective than inter-residue distances for discriminating near-native CDR H3 loops. When applied to de novo prediction of CDR H3 loop structures, DeepH3 achieves an average RMSD of 2.2 ± 1.1 Å on the Rosetta antibody benchmark.Availability and ImplementationDeepH3 source code and pre-trained model parameters are freely available at https://github.com/Graylab/deepH3-distances-orientations.Supplementary information Supplementary data are available at Bioinformatics online.

An electrocardiograph (ECG) is widely used in diagnosis and prediction of cardiovascular diseases (CVDs). The traditional ECG classification methods have complex signal processing phases that leads to expensive designs. This paper provides a deep learning (DL) based system that employs the convolutional neural networks (CNNs) for classification of ECG signals present in PhysioNet MIT-BIH Arrhythmia database. The proposed system implements 1-D convolutional deep residual neural network (ResNet) model that performs feature extraction by directly using the input heartbeats. We have used synthetic minority oversampling technique (SMOTE) that process class-imbalance problem in the training dataset and effectively classifies the five heartbeat types in the test dataset. The classifier’s performance is evaluated with ten-fold cross validation (CV) using accuracy, precision, sensitivity, F1-score, and kappa. We have obtained an average accuracy of 98.63%, precision of 92.86%, sensitivity of 92.41%, and specificity of 99.06%. The average F1-score and Kappa obtained were 92.63% and 95.5% respectively. The study shows that proposed ResNet performs well with deep layers compared to other 1-D CNNs.

We have developed an automatic sleep stage classification algorithm based on deep residual neural networks and raw polysomnogram signals. Briefly, the raw data is passed through 50 convolutional layers before subsequent classification into one of five sleep stages. Three model configurations were trained on 1850 polysomnogram recordings and subsequently tested on 230 independent recordings. Our best performing model yielded an accuracy of 84.1% and a Cohen's kappa of 0.746, improving on previous reported results by other groups also using only raw polysomnogram data. Most errors were made on non-REM stage 1 and 3 decisions, errors likely resulting from the definition of these stages. Further testing on independent cohorts is needed to verify performance for clinical use.

Modeling green recycled aggregate concrete using machine learning and variance-based sensitivity analysis