Solving Fredholm Integral Equations Using Deep Learning

This paper proposes a method to evaluate the degradation stages of stability of the screw with data fusion technology and deep residual neural network. Firstly, the data provided by multi-sensors are fused, and then the time domain images of the signals are input into the deep residual neural network for training and testing. The effectiveness of the proposed method is verified using data sets collected from the degradation test bench of ball screw. The data sets contain massive samples involving 3 degradation stages under 9 working conditions obtained by 3 accelerometers. From the results and comparison, it can be found that the deep residual neural network can automatically extract the features from the original data layer by layer and achieve a better evaluation rate when it is used to evaluate the performance stage of the screw, and the multi-sensor data fusion vibration data can better reflect the degradation stages of the screw's performance. It is showed that this method enhances the intelligence of the evaluation compared with the traditional method.

Automatic modulation classification (AMC) is a signal processing technology used to identify the modulation type of unknown signals without prior information such as modulation parameters for drone communications. In recent years, deep learning (DL) has been widely used in AMC methods due to its powerful feature extraction ability. The significant performance of DL-based AMC methods is highly dependent on large amount of data. However, with the increasingly complex signal environment and the emergence of new signals, several recognition tasks have difficulty obtaining sufficient high-quality signals. To address this problem, we propose an AMC method based on a deep residual neural network with masked modeling (DRMM). Specifically, masked modeling is adopted to improve the performance of a deep neural network with limited signal samples. Both complex-valued and real-valued residual neural networks (ResNet) play an important role in extracting signal features for identification. Several typical experiments are conducted to evaluate our proposed DRMM-based AMC method on the RadioML 2016.10A dataset and a simulated dataset, and comparison experiments with existing AMC methods are also conducted. The simulation results illustrate that our proposed DRMM-based AMC method achieves better performance in the case of limited signal samples with low signal-to-noise ratio (SNR) than other existing methods.

Image acquisition is an essential step in image processing. When the image acquisition is done the image that is generated is subjected to Impulse noise, Gaussian noise etc. We have performed the image denoising on images inflicted with impulse noise. Image denoising is an essential step in all types of image processing. Traditional techniques reduce the noise in the image but it also reduces the quality of the image. Traditional filters like gaussian filter, median filter is analyzed which work in the spatial domain and filters working in the frequency domain are also considered like Butterworth filters, Weiner filter. A Deep residual Neural Network filter is proposed which is compared with the Fuzzy Neural Network denoiser. Their performance is compared on the metrics PSNR and SSIM. The Fuzzy Neural Network system improves the SSIM significantly compared to a deep residual neural network and a comparison is made with traditional image denoising methods. We also compare the performance of the deep residual neural network, Fuzzy Neural Network system and Median denoising algorithm on impulse noise has been compared. The performance of deep neural networks depends on the total number of examples used and the performance can be improved if we have more image pairs.

The neuron classification problem is significant for understanding structure-function relationships in computational neuroscience. Advances in recent years have accelerated the speed of data collection, resulting in a large amount of data on the geometric, morphological, physiological, and molecular characteristics of neurons. These data encourage researchers to strive for automated neuron classification through powerful machine learning techniques. This paper extracts a statistical dataset of 43 geometrical features obtained from 116 human neurons, and proposes a neuronal morphology classification approach based on deep residual neural networks with feature scaling. The approach is applied to classify 18 types of human neurons and compares the accuracy of different number of residual block. Then, we also compare the accuracy between the proposed approach and other mainstream machine learning approaches, the classification accuracy of our approach is 100% in the training set and the testing set accuracy is 76.96%. The experimental results show that the deep residual neural network model has better classification accuracy for human neurons.

Data-driven rolling-bearing fault diagnosis methods are mostly based on deep-learning models, and their multilayer nonlinear mapping capability can improve the accuracy of intelligent fault diagnosis. However, problems such as gradient disappearance occur as the number of network layers increases. Moreover, directly taking the raw vibration signals of rolling bearings as the network input results in incomplete feature extraction. In order to efficiently represent the state characteristics of vibration signals in image form and improve the feature learning capability of the network, this paper proposes fault diagnosis model MTF-ResNet based on a Markov transition field and deep residual network. First, the data of raw vibration signals are augmented by using a sliding window. Then, vibration signal samples are converted into two-dimensional images by MTF, which retains the time dependence and frequency structure of time-series signals, and a deep residual neural network is established to perform feature extraction, and identify the severity and location of the bearing faults through image classification. Lastly, experiments were conducted on a bearing dataset to verify the effectiveness and superiority of the MTF-ResNet model. Features learned by the model are visualized by t-SNE, and experimental results indicate that MTF-ResNet showed better average accuracy compared with several widely used diagnostic methods.

As an enabling technology for emerging and future generations of wireless networks, sparse code multiple access (SCMA) offers major improvements in terms of spectral efficiency and massive connectivity. Although the message passing algorithm (MPA) for SCMA decoding at the receiver side can achieve near optimum performance, it entails high computational complexity. In this paper, to address this issue, we propose a novel SCMA decoder based on deep residual neural network (ResNet), wherein the decoder is trained to predict the transmit codewords. In our approach, residual blocks are employed to tackle the problems of accuracy saturation and vanishing gradients with deep learning based decoder, while batch normalization is utilized to enhance the stability and robustness of the decoder. The performance of the proposed ResNet decoder for SCMA is validated by means of simulations over AWGN and Rayleigh fading channels. The results show that besides a much reduced complexity, the proposed decoder leads to improvements in term of bit error rate (BER) over competing deep neural network (DNN) based decoders.

A deep residual neural network for semiconductor defect classification in imbalanced scanning electron microscope datasets

This paper makes a research for the speed control of wind turbine system under multiple operating regimes, which also studied the sleep residual recurrent neural network method in this work. We aim at designing deep residual recurrent neural network robust controllers, which guarantee the existence of the multiple regime system poles in some predefined zone and wind speed precise tracking. Moreover, the feedback gains which guarantee desired speed tracking performance are obtained by solving the Lyapunov stability functions. The results are applied to a directly driven wind energy conversion experiment systems and the numerical experiment, comparing with the existing results, shows the satisfactory performance of the proposed method.

Contact-map of a protein sequence dictates the global topology of structural fold. Accurate prediction of the contact-map is thus essential to protein 3D structure prediction, which is particularly useful for the protein sequences that do not have close homology templates in the Protein Data Bank. We developed a new method, ResPRE, to predict residue-level protein contacts using inverse covariance matrix (or precision matrix) of multiple sequence alignments (MSAs) through deep residual convolutional neural network training. The approach was tested on a set of 158 non-homologous proteins collected from the CASP experiments and achieved an average accuracy of 50.6% in the top-L long-range contact prediction with L being the sequence length, which is 11.7% higher than the best of other state-of-the-art approaches ranging from coevolution coupling analysis to deep neural network training. Detailed data analyses show that the major advantage of ResPRE lies at the utilization of precision matrix that helps rule out transitional noises of contact-maps compared with the previously used covariance matrix. Meanwhile, the residual network with parallel shortcut layer connections increases the learning ability of deep neural network training. It was also found that appropriate collection of MSAs can further improve the accuracy of final contact-map predictions. The standalone package and online server of ResPRE are made freely available, which should bring important impact on protein structure and function modeling studies in particular for the distant- and non-homology protein targets. https://zhanglab.ccmb.med.umich.edu/ResPRE and https://github.com/leeyang/ResPRE. Supplementary data are available at Bioinformatics online.

A key challenge in radar micro-Doppler classification is the difficulty in obtaining a large amount of training data due to costs in time and human resources. Small training datasets limit the depth of deep neural networks (DNNs), and, hence, attainable classification accuracy. In this work, a novel method for diversifying Kinect-based motion capture (MOCAP) simulations of human micro-Doppler to span a wider range of potential observations, e.g. speed, body size, and style, is proposed. By applying three transformations, a small set of MOCAP measurements is expanded to generate a large training dataset for network initialization of a 30-layer deep residual neural network. Results show that the proposed training methodology and residual DNN yield improved bottleneck feature performance and the highest overall classification accuracy among other DNN architectures, including transfer learning from the 1.5 million sample ImageNet database.

MotivationAntibody structure is largely conserved, except for a complementarity-determining region featuring six variable loops. Five of these loops adopt canonical folds which can typically be predicted with existing methods, while the remaining loop (CDR H3) remains a challenge due to its highly diverse set of observed conformations. In recent years, deep neural networks have proven to be effective at capturing the complex patterns of protein structure. This work proposes DeepH3, a deep residual neural network that learns to predict inter-residue distances and orientations from antibody heavy and light chain sequence. The output of DeepH3 is a set of probability distributions over distances and orientation angles between pairs of residues. These distributions are converted to geometric potentials and used to discriminate between decoy structures produced by RosettaAntibody and predict new CDR H3 loop structures de novo.ResultsWhen evaluated on the Rosetta antibody benchmark dataset of 49 targets, DeepH3-predicted potentials identified better, same and worse structures [measured by root-mean-squared distance (RMSD) from the experimental CDR H3 loop structure] than the standard Rosetta energy function for 33, 6 and 10 targets, respectively, and improved the average RMSD of predictions by 32.1% (1.4 Å). Analysis of individual geometric potentials revealed that inter-residue orientations were more effective than inter-residue distances for discriminating near-native CDR H3 loops. When applied to de novo prediction of CDR H3 loop structures, DeepH3 achieves an average RMSD of 2.2 ± 1.1 Å on the Rosetta antibody benchmark.Availability and ImplementationDeepH3 source code and pre-trained model parameters are freely available at https://github.com/Graylab/deepH3-distances-orientations.Supplementary information Supplementary data are available at Bioinformatics online.

Fourier ptychographic microscopy (FPM) is a newly developed microscopic technique for large field of view, high-resolution and quantitative phase imaging by combining the techniques from ptychographic imaging, aperture synthesizing and phase retrieval. In FPM, an LED array is utilized to illuminate the specimen from different angles and the corresponding intensity images are synthesized to reconstruct a high-resolution complex field. As a flexible and low-cost approach to achieve high-resolution, wide-field and quantitative phase imaging, FPM is of enormous potential in biomedical applications such as hematology and pathology. Conventionally, the FPM reconstruction problem is solved by using a phase retrieval method, termed Alternate Projection. By iteratively updating the Fourier spectrum with low-resolution-intensity images, the result converges to a high-resolution complex field. Here we propose a new FPM reconstruction framework with deep learning methods and design a multiscale, deep residual neural network for FPM reconstruction. We employ the widely used open-source deep learning library PyTorch to train and test our model and carefully choose the hyperparameters of our model. To train and analyze our model, we build a large-scale simulation dataset with an FPM imaging model and an actual dataset captured with an FPM system. The simulation dataset and actual dataset are separated as training datasets and test datasets, respectively. Our model is trained with the simulation training dataset and fine tuned with the fine-tune dataset, which contains actual training data. Both our model and the conventional method are tested on the simulation test dataset and the actual test dataset to evaluate the performances. We also show the reconstruction result of another neural network-based method for comparison. The experiments demonstrate that our model achieves better reconstruction results and consumes much less time than conventional methods. The results also point out that our model is more robust under system aberrations such as noise and blurring (fewer intensity images) compared with conventional methods.

Simple SummaryThe integration of advanced magnetic resonance imaging (MRI) has the potential to enable the improved prediction of the molecular diagnosis of adult-type gliomas. In this context, this study investigated whether deep learning-based predictive models can benefit from adding multi-shell diffusion MRI to conventional MRI. We evaluated the performance of an exemplar deep learning model for differentiating (1) isocitrate dehydrogenase (IDH)-mutation versus IDH-wildtype; (2) 1p/19q codeletion versus 1p/19q non-codeletion; and (3) IDH-mutation with or without 1p/19q codeletion, and IDH-wildtype. The model achieved the best prediction performance in our cohort of 146 patients in all three tasks when multi-shell diffusion MRI and conventional MRI are combined. These results demonstrate the specific added value provided by advanced diffusion MRI, extending the current literature on building deep learning models based on multiple MRI modalities.The WHO classification since 2016 confirms the importance of integrating molecular diagnosis for prognosis and treatment decisions of adult-type diffuse gliomas. This motivates the development of non-invasive diagnostic methods, in particular MRI, to predict molecular subtypes of gliomas before surgery. At present, this development has been focused on deep-learning (DL)-based predictive models, mainly with conventional MRI (cMRI), despite recent studies suggesting multi-shell diffusion MRI (dMRI) offers complementary information to cMRI for molecular subtyping. The aim of this work is to evaluate the potential benefit of combining cMRI and multi-shell dMRI in DL-based models. A model implemented with deep residual neural networks was chosen as an illustrative example. Using a dataset of 146 patients with gliomas (from grade 2 to 4), the model was trained and evaluated, with nested cross-validation, on pre-operative cMRI, multi-shell dMRI, and a combination of the two for the following classification tasks: (i) IDH-mutation; (ii) 1p/19q-codeletion; and (iii) three molecular subtypes according to WHO 2021. The results from a subset of 100 patients with lower grades gliomas (2 and 3 according to WHO 2016) demonstrated that combining cMRI and multi-shell dMRI enabled the best performance in predicting IDH mutation and 1p/19q codeletion, achieving an accuracy of 75 ± 9% in predicting the IDH-mutation status, higher than using cMRI and multi-shell dMRI separately (both 70 ± 7%). Similar findings were observed for predicting the 1p/19q-codeletion status, with the accuracy from combining cMRI and multi-shell dMRI (72 ± 4%) higher than from each modality used alone (cMRI: 65 ± 6%; multi-shell dMRI: 66 ± 9%). These findings remain when we considered all 146 patients for predicting the IDH status (combined: 81 ± 5% accuracy; cMRI: 74 ± 5%; multi-shell dMRI: 73 ± 6%) and for the diagnosis of the three molecular subtypes according to WHO 2021 (combined: 60 ± 5%; cMRI: 57 ± 8%; multi-shell dMRI: 56 ± 7%). Together, these findings suggest that combining cMRI and multi-shell dMRI can offer higher accuracy than using each modality alone for predicting the IDH and 1p/19q status and in diagnosing the three molecular subtypes with DL-based models.

Some industrial processes take place in confined settings only observable by sensors, e.g. infrared (IR) cameras. Drying processes take place while a material is transported by means of a conveyor through a black box equipped with internal IR cameras. While such sensors deliver data at high rates, this is beyond what human operators can analyze and calls for automation. Inspired by numerous implementations monitoring techniques that analyse IR images using deep learning, this paper shows how they can be applied to the confined microwave drying of porous foams, with benchmarking their effectiveness at condition monitoring to conduct fault detection. Convolutional neural networks, derived transfer learning, and deep residual neural network methods are already regarded as cutting-edge and are studied here, using a set of conventional approaches for comparative evaluation. Our comparison shows that state-of-the-art deep learning techniques significantly benefit condition monitoring, providing an increase in fault finding accuracy of up to 48% over conventional methods. Nevertheless, we also found that derived transfer learning and deep residual network techniques do not in our case yield increased performance over normal convolutional neural networks.

RIFATA: Remora improved invasive feedback artificial tree algorithm-enabled hybrid deep learning approach for root disease classification