Deep residual networks for automatic sleep stage classification of raw polysomnographic waveforms.

Data-driven rolling-bearing fault diagnosis methods are mostly based on deep-learning models, and their multilayer nonlinear mapping capability can improve the accuracy of intelligent fault diagnosis. However, problems such as gradient disappearance occur as the number of network layers increases. Moreover, directly taking the raw vibration signals of rolling bearings as the network input results in incomplete feature extraction. In order to efficiently represent the state characteristics of vibration signals in image form and improve the feature learning capability of the network, this paper proposes fault diagnosis model MTF-ResNet based on a Markov transition field and deep residual network. First, the data of raw vibration signals are augmented by using a sliding window. Then, vibration signal samples are converted into two-dimensional images by MTF, which retains the time dependence and frequency structure of time-series signals, and a deep residual neural network is established to perform feature extraction, and identify the severity and location of the bearing faults through image classification. Lastly, experiments were conducted on a bearing dataset to verify the effectiveness and superiority of the MTF-ResNet model. Features learned by the model are visualized by t-SNE, and experimental results indicate that MTF-ResNet showed better average accuracy compared with several widely used diagnostic methods.

This paper proposes a method to evaluate the degradation stages of stability of the screw with data fusion technology and deep residual neural network. Firstly, the data provided by multi-sensors are fused, and then the time domain images of the signals are input into the deep residual neural network for training and testing. The effectiveness of the proposed method is verified using data sets collected from the degradation test bench of ball screw. The data sets contain massive samples involving 3 degradation stages under 9 working conditions obtained by 3 accelerometers. From the results and comparison, it can be found that the deep residual neural network can automatically extract the features from the original data layer by layer and achieve a better evaluation rate when it is used to evaluate the performance stage of the screw, and the multi-sensor data fusion vibration data can better reflect the degradation stages of the screw's performance. It is showed that this method enhances the intelligence of the evaluation compared with the traditional method.

Automatic modulation classification (AMC) is a signal processing technology used to identify the modulation type of unknown signals without prior information such as modulation parameters for drone communications. In recent years, deep learning (DL) has been widely used in AMC methods due to its powerful feature extraction ability. The significant performance of DL-based AMC methods is highly dependent on large amount of data. However, with the increasingly complex signal environment and the emergence of new signals, several recognition tasks have difficulty obtaining sufficient high-quality signals. To address this problem, we propose an AMC method based on a deep residual neural network with masked modeling (DRMM). Specifically, masked modeling is adopted to improve the performance of a deep neural network with limited signal samples. Both complex-valued and real-valued residual neural networks (ResNet) play an important role in extracting signal features for identification. Several typical experiments are conducted to evaluate our proposed DRMM-based AMC method on the RadioML 2016.10A dataset and a simulated dataset, and comparison experiments with existing AMC methods are also conducted. The simulation results illustrate that our proposed DRMM-based AMC method achieves better performance in the case of limited signal samples with low signal-to-noise ratio (SNR) than other existing methods.

Background Frequent sleep disruption in preterm infants in neonatal intensive care units (NICU) is suspected to be associated with adverse neurodevelopmental outcomes. A continuous, noninvasive sleep monitoring solution could optimise care by aligning interventions with natural sleep–wake cycles. The algorithm for automatic sleep stage classification by Demme et al. [5] offers a promising approach, with an accuracy of 92.2 ± 0.01% compared to sleep stages measured in the sleep laboratory. Objective This study aims to evaluate the applicability of the sleep classification algorithm with regards to two main questions: (1) Does the algorithm’s performance vary with factors like pre-existing medical conditions or other patient characteristics in preterm infants? (2) How well does the algorithm capture sleep stage transitions compared to manual annotations? Materials and methods This study is a secondary analysis of data previously used by Demme et al. [5] to train models for automatic sleep stage classification in preterm infants. The models, which were trained on manual annotations of polysomnography (PSG) data, were applied to a diverse cohort of preterm infants. The performance was analysed in relation to medical conditions, sex, gestational age (GA), apnoea–hypopnea index (AHI), bodyweight at measurement, and Fleiss’ kappa (a measure of interrater variability). Sleep stage transition frequencies were compared between algorithmic and manual scoring. Results No significant performance differences could be observed with respect to any covariates, except for a reduction in performance among patients with low Fleiss’ kappa values, suggesting classification difficulties in cases in which manual raters have come to diverging assessments. Transition analysis revealed significant differences, likely due to the algorithm’s more sensitive ability to detect short sleep stage transitions. Conclusion The algorithm shows strong performance and generalisability. Despite some discrepancies in sleep stage transition detection, it appears suitable for noninvasive, continuous sleep monitoring in the NICU, supporting sleep-aware clinical care.

Image acquisition is an essential step in image processing. When the image acquisition is done the image that is generated is subjected to Impulse noise, Gaussian noise etc. We have performed the image denoising on images inflicted with impulse noise. Image denoising is an essential step in all types of image processing. Traditional techniques reduce the noise in the image but it also reduces the quality of the image. Traditional filters like gaussian filter, median filter is analyzed which work in the spatial domain and filters working in the frequency domain are also considered like Butterworth filters, Weiner filter. A Deep residual Neural Network filter is proposed which is compared with the Fuzzy Neural Network denoiser. Their performance is compared on the metrics PSNR and SSIM. The Fuzzy Neural Network system improves the SSIM significantly compared to a deep residual neural network and a comparison is made with traditional image denoising methods. We also compare the performance of the deep residual neural network, Fuzzy Neural Network system and Median denoising algorithm on impulse noise has been compared. The performance of deep neural networks depends on the total number of examples used and the performance can be improved if we have more image pairs.

Automatic classification of sleep stages based on the time-frequency image of EEG signals

Sleep stage classification is closely related to human health and plays an effective role in sleep disorders diagnose. At present, the accuracy rate of conventional automatic sleep staging system is still not high, and there is still a large space for improvement. To solve the above problems, this paper proposed an automatic sleep stage classification method based on the Long Short-Term Memory (LSTM) and fuzzy entropy. We firstly adopt fuzzy entropy to extract the feature from electroencephalography (EEG) signals, then these features are put into the designed LSTM to carry out the automatic sleep stage classification and attain the final sleep stages. To verify the validity of our experiment, we tested it on public datasets called ISRUC_Sleep. The results demonstrate that this method improves the classification accuracy of sleep stages.

The neuron classification problem is significant for understanding structure-function relationships in computational neuroscience. Advances in recent years have accelerated the speed of data collection, resulting in a large amount of data on the geometric, morphological, physiological, and molecular characteristics of neurons. These data encourage researchers to strive for automated neuron classification through powerful machine learning techniques. This paper extracts a statistical dataset of 43 geometrical features obtained from 116 human neurons, and proposes a neuronal morphology classification approach based on deep residual neural networks with feature scaling. The approach is applied to classify 18 types of human neurons and compares the accuracy of different number of residual block. Then, we also compare the accuracy between the proposed approach and other mainstream machine learning approaches, the classification accuracy of our approach is 100% in the training set and the testing set accuracy is 76.96%. The experimental results show that the deep residual neural network model has better classification accuracy for human neurons.

The vibration signal is easily interfered by noise due to the influence of environment and other factors, which can lead to the poor adaptability, low accuracy of remaining useful life (RUL) prediction, and other problems. To solve this problem, this paper proposes a novel RUL prediction method, which is based on multiscale stacking deep residual shrinkage network (MSDRSN). MSDRSN combines the ability of stacking in improving prediction accuracy and the advantages of deep residual shrinkage network (DRSN) in denoising. First, cumulative sum (CUSUM) from statistics is used to divide the full life cycle of the rolling bearings and discover the points of failure. Second, stacking is used for feature learning on the raw data, multiple convolutional kernels of different scales are selected as base‐learners, and fully connected neural networks are selected as meta‐learners for feature fusion and learning. Then, DRSN is used to do prediction, and the acquired results are fitted with Savitzky–Golay (SG) smoothing. Finally, the effectiveness of the proposed method is proved by the IEEE PHM 2012 data challenge dataset. Compared with the multiscale convolutional neural network with fully connected layer (MSCNN‐FC) and the bidirectional long short‐term memory (BiLSTM) for RUL prediction under the noise. Using the proposed method, the mean absolute error (MSE) of the best result is 0.002 and the mean square error (MSE) is 0.014; meanwhile, the coefficient of determination (R2) of the best prediction result can reach 97.6%. It is also compared with other machine learning methods, and all the results prove the accuracy and effectiveness of the proposed method for RUL prediction applications.

Background: This study aims to explore the circadian rhythm of interictal epileptiform discharges (IEDs) and changes of spindles in patients with temporal lobe epilepsy (TLE). Methods: Subjects were divided into two groups: research group (patients diagnosed with TLE, n=50) and control group (n=30). The whole-night natural sleep of all subjects was monitored, analysis and statistics were performed on IED, and the number, amplitude and the average duration of spindles were compared between these two groups. Results: (1) In TLE patients, the number of IED was larger in non-rapid eye movement (NREM) stages 1+2 and NREM stages 3+4 than in the waking stage and rapid eye movement (REM) stage, and the differences were statistically significant (P<0.05). (2) TLE patients had bilateral asymmetric sleep spindles, and their amplitude was lower in the affected side than in the unaffected side. Furthermore, the number, amplitude and mean duration of sleep spindles in the affected side in NREM stage 2 was significantly reduced compared with that in the control group (P<0.05). Conclusions: (1) TLE patients have the highest frequency of IED in slow wave sleep, followed by NREM stages 1+2, waking and REM stages. Slow wave sleep easily induces IED. (2) The number, amplitude and mean duration of sleep spindles in the affected side in NREM stage 2 were significantly reduced, compared with that in the control group. This suggests that changes in spindles have a lateralization value.

The automatic sleep stage classification technique can facilitate the diagnosis of sleep disorders and release the medical expert from labor-consumption work. In this paper, novel improved model based essence features (IMBEFs) were proposed combining locality energy (LE) and dual state space models (DSSMs) for automatic sleep stage detection on single-channel electroencephalograph (EEG) signals. Firstly, each EEG epoch is decomposed into low-level sub-bands (LSBs) and high-level sub-bands (HSBs) by wavelet packet decomposition (WPD), separately. Then, the DSSMs are estimated by the LSBs and the LE calculation is carried out on HSBs. Thirdly, the IMBEFs extracted from the DSSM and LE are fed into the appropriate classifier for sleep stage classification. The performance of the proposed method was evaluated on three public sleep databases. The experimental results show that under the Rechtschaffen’s and Kale’s (R&K) standard, the sleep stage classification accuracies of six classes on the Sleep EDF database and the Dreams Subjects database are 92.04% and 78.92%, respectively. Under the American Academy of Sleep Medicine (AASM) standard, the classification accuracies of five classes in the Dreams Subjects database and the ISRUC database reached 79.90% and 81.65%. The proposed method can be used for reliable sleep stage classification with high accuracy compared with state-of-the-art methods.

Contact-map of a protein sequence dictates the global topology of structural fold. Accurate prediction of the contact-map is thus essential to protein 3D structure prediction, which is particularly useful for the protein sequences that do not have close homology templates in the Protein Data Bank. We developed a new method, ResPRE, to predict residue-level protein contacts using inverse covariance matrix (or precision matrix) of multiple sequence alignments (MSAs) through deep residual convolutional neural network training. The approach was tested on a set of 158 non-homologous proteins collected from the CASP experiments and achieved an average accuracy of 50.6% in the top-L long-range contact prediction with L being the sequence length, which is 11.7% higher than the best of other state-of-the-art approaches ranging from coevolution coupling analysis to deep neural network training. Detailed data analyses show that the major advantage of ResPRE lies at the utilization of precision matrix that helps rule out transitional noises of contact-maps compared with the previously used covariance matrix. Meanwhile, the residual network with parallel shortcut layer connections increases the learning ability of deep neural network training. It was also found that appropriate collection of MSAs can further improve the accuracy of final contact-map predictions. The standalone package and online server of ResPRE are made freely available, which should bring important impact on protein structure and function modeling studies in particular for the distant- and non-homology protein targets. https://zhanglab.ccmb.med.umich.edu/ResPRE and https://github.com/leeyang/ResPRE. Supplementary data are available at Bioinformatics online.

This paper makes a research for the speed control of wind turbine system under multiple operating regimes, which also studied the sleep residual recurrent neural network method in this work. We aim at designing deep residual recurrent neural network robust controllers, which guarantee the existence of the multiple regime system poles in some predefined zone and wind speed precise tracking. Moreover, the feedback gains which guarantee desired speed tracking performance are obtained by solving the Lyapunov stability functions. The results are applied to a directly driven wind energy conversion experiment systems and the numerical experiment, comparing with the existing results, shows the satisfactory performance of the proposed method.

The work considers automatic sleep stage classification, based on heart rate variability (HRV) analysis, with a focus on the distinction of wakefulness (WAKE) from sleep and rapid eye movement (REM) from non-REM (NREM) sleep. A set of 20 automatically annotated one-night polysomnographic recordings was considered, and artificial neural networks were selected for classification. For each inter-heartbeat (RR) series, beside features previously presented in literature, we introduced a set of four parameters related to signal regularity. RR series of three different lengths were considered (corresponding to 2, 6, and 10 successive epochs, 30 s each, in the same sleep stage). Two sets of only four features captured 99 % of the data variance in each classification problem, and both of them contained one of the new regularity features proposed. The accuracy of classification for REM versus NREM (68.4 %, 2 epochs; 83.8 %, 10 epochs) was higher than when distinguishing WAKE versus SLEEP (67.6 %, 2 epochs; 71.3 %, 10 epochs). Also, the reliability parameter (Cohens's Kappa) was higher (0.68 and 0.45, respectively). Sleep staging classification based on HRV was still less precise than other staging methods, employing a larger variety of signals collected during polysomnographic studies. However, cheap and unobtrusive HRV-only sleep classification proved sufficiently precise for a wide range of applications.

The development of the Internet-of-Things (IoT) security is comparatively slower than the pace of the IoT innovations. The seamless IoT network operates in an untrusted environment and is exposed to many malicious active attacks. As the process of identifying the modulation format of signals is corrupted by noise and fading, automatic modulation classification (AMC) can be viewed as an effective approach to counter physical-layer threats for IoT as it can detect and identify the pilot jamming, deceptive jamming, and Sybil attacks. Nowadays, data-driven deep learning (DL) techniques, which are capable of extracting discriminative features and perform better robustness to channel and noise conditions, have drawn widespread attention. The deep residual network (ResNet) has a strong representative ability, which can learn latent information repeatedly from the received signals and improve the classification accuracy. Meanwhile, the gated recurrent unit (GRU), which is capable of exploiting temporal information of the received signal can expand the dimension of the signal features for satisfactory classification performance. Considering the advantages of the above networks, this article proposes a novel gated recurrent residual neural network (GrrNet) for feature-based AMC, where the amplitude and phase of the received signal are utilized as the inputs of GrrNet. In GrrNet, a ResNet extractor module is first designed to extract the highly representative features and then temporal information is obtained by the subsequent GRU module which is capable of processing the representative features with the arbitrary length for modulation classification. Moreover, extensive simulations are conducted to verify the classification performance and robustness of the proposed GrrNet and it is shown that GrrNet outperforms other recent DL-based AMC methods. Moreover, the influence of the network parameters, symbol length, and frequency offset on performance is also explored.