High-performance decoding using deep neural networks

As an enabling technology for emerging and future generations of wireless networks, sparse code multiple access (SCMA) offers major improvements in terms of spectral efficiency and massive connectivity. Although the message passing algorithm (MPA) for SCMA decoding at the receiver side can achieve near optimum performance, it entails high computational complexity. In this paper, to address this issue, we propose a novel SCMA decoder based on deep residual neural network (ResNet), wherein the decoder is trained to predict the transmit codewords. In our approach, residual blocks are employed to tackle the problems of accuracy saturation and vanishing gradients with deep learning based decoder, while batch normalization is utilized to enhance the stability and robustness of the decoder. The performance of the proposed ResNet decoder for SCMA is validated by means of simulations over AWGN and Rayleigh fading channels. The results show that besides a much reduced complexity, the proposed decoder leads to improvements in term of bit error rate (BER) over competing deep neural network (DNN) based decoders.

With the rapid development of electronic computers and information technology, face recognition is widely used in fields such as enterprises, entertainment, information security, and daily life. However, the current face recognition technology is still relatively poor in distinguishing facial features, resulting in a low accuracy of face recognition, which cannot meet increasing application requirements. For this reason, it is necessary to develop more accurate face recognition technology. Residual neural network (ResNet) neural network is a deep residual learning convolutional neural network, which can be used to solve the degeneration problem (i.e., after adding more layers to the neural network, the performance drops rapidly) of deep-learning neural networks. We aim to study the specific principles, calculation methods, and characteristics of the ResNet neural network, analyze the complexity of the classroom environment, propose the use of ResNet network for face feature extraction, and use additive angular margin loss for deep face recognition (ArcFace) as the loss function of the ResNet network to improve the ResNet network. Because the ArcFace function has the advantages of high performance, easy programming, low complexity and high training efficiency. We use this scheme to conduct field tests of face recognition on many students in the classroom. The test results show that this scheme enhances the discrimination of facial features and improves the accuracy of face recognition. In the case of different numbers of people, facial defects, and strong light exposure, the system can still detect and recognize faces stably. In the case of facial defects, the recognition accuracy rate is still 63.3%, and the recognition accuracy rate is still more than 60% under the illumination of strong light, and the recognition accuracy rate under correct conditions is more than 70%.

Automatic recognition of surface defects for hot-rolled steel strip based on deep attention residual convolutional neural network

Lattice decoders constructed with neural networks are presented. Firstly, we show how the fundamental parallelotope is used as a compact set for the approximation by a neural lattice decoder. Secondly, we introduce the notion of Voronoi-reduced lattice basis. As a consequence, a first optimal neural lattice decoder is built from Boolean equations and the facets of the Voronoi cell. This decoder needs no learning. Finally, we present two neural decoders with learning. It is shown that L1 regularization and a priori information about the lattice structure lead to a simplification of the model.

Face recognition has made great progress due to the development of convolutional neural network (CNN), and face alignment is an important part of recognition, it is easily affected by gestures and occlusion. In this paper, we propose Residual Neural Network and Wing Loss for Face Alignment Network (RWAN), which consists of a plurality of stages, each stage ameliorates the position of facial landmarks estimated from the previous stage. Unlike the traditional cascading model, the network model uses the Residual Neural Network, which is easily to optimize and can improve accuracy by adding considerable depth, the internal residual block uses shortcut to alleviate the gradient explosion problem caused by increasing depth in deep neural network. Enter the face images and extract features from the entire image by introducing landmark heatmaps to obtain more accurate positioning. Using wing loss in the loss function part not only focuses on the landmark of large error points, but also on the small and medium errors of landmarks, it aims to improve the training ability of deep neural network with small and medium range error. The problem of unbalanced data not only confuses classification tasks, but also affects the accuracy of the model when face samples of different attitudes are not balanced in face key detection tasks. a simple but effective data enhancement method is proposed to deal with the problem, which solves the problem of unbalanced data processing by randomly rotating the training samples, amplifying and the like. The experimental results obtained by our method on the 300W dataset indicate the advantages of this method.

Sea ice is an important component of the polar circle and influences atmospheric change. Global navigation satellite system reflectometry (GNSS-R) not only realizes time-continuous and wide-area sea ice detection, but also greatly reduces the cost of sea ice remote sensing research, which has been a hot topic in recent years. To tackle the challenges of noise interference and the reduced accuracy of sea ice detection during the melting period, this paper proposes a sea ice detection method based on a residual neural network (ResNet). ResNet addresses the issue of vanishing gradients in deep neural networks and introduces residual connections, which allows the network to reuse learned features from previous layers. Delay-Doppler maps (DDMs) collected from TechDemoSat-1 (TDS-1) are used as input, and National Oceanic and Atmospheric Administration (NOAA) surface-type data above 60°N are selected as the true values. Based on ResNet, the sea ice detection achieved an accuracy of 98.61%, demonstrating high robustness to noise and strong stability during the sea ice melting period (June to September). In comparison to other sea ice detection algorithms, it stands out with its advantages of high accuracy, stability, and insensitivity to noise.

Training very deep neural networks: Rethinking the role of skip connections

The study presents the results of the development and testing of deep learning neural network architectures, which demonstrate high accuracy rates in classifying neurophysiological data, in particular hemodynamic brain activation patterns obtained by functional near-infrared spectroscopy, during solving mathematical problems on spatial-numerical associations. The analyzed signal represents a multidimensional time series of oxyhemoglobin and deoxyhemoglobin dynamics. Taking the specificity of the fNIRS signal into account, a comparative analysis of 2 types of neural network architectures was carried out: (1) architectures based on recurrent neural networks: recurrent neural network with long short-term memory, recurrent neural network with long short-term memory with fully connected layers, bidirectional recurrent neural network with long short-term memory, convolutional recurrent neural network with long short-term memory; (2) architectures based on convolutional neural networks with 1D convolutions: convolutional neural network, fully convolutional neural network, residual neural network. Trained long short-term memory recurrent neural network architectures showed worse results in accuracy in comparison with 1D convolutional neural network architectures. Residual neural network (model_Resnet) demonstrated the highest accuracy rates in three experimental conditions more than 88% in detecting age-related differences in brain activation during spatial-numerical association tasks considering the individual characteristics of the respondents’ signal.

Electron ionization (EI) mass spectrometry (MS) is a widely used technique for the compound identification and production of spectra. However, incomplete coverage of reference spectral libraries limits reliable analysis of newly characterized molecules. This study presents a hybrid deep learning model for predicting EI-MS spectra directly from molecular structure. The approach combines a graph neural network encoder with a residual neural network decoder, followed by refinement using cross-attention, bidirectional prediction, and probabilistic, chemistry-informed masks. Trained on the NIST14 EI-MS database (≤500 Da), the model achieves strong library matching performance (Recall@10 ≈ 80.8%) and high spectral similarity. The proposed hybrid GNN (Graph Neural Network)-ResNet (Residual Neural Network) model can generate high-quality synthetic EI-MS spectra to supplement existing libraries, potentially reducing the cost and effort of experimental spectrum acquisition. The obtained results demonstrate the potential of data-driven models to augment EI-MS libraries, while highlighting remaining challenges in generalization and spectral uniqueness.

Current deep learning based image enhancement algorithms attempt to learn the mapping relationship between degraded images and clear images directly. These algorithms often ignore the fidelity constraint of the observational model. In order to improve the image enhancement performance, an improved deep residual neural network based image enhancement algorithm (DRNN-IE) for low dose CT images is proposed in this paper. DRNN-IE embeds the image enhancement task into a deep neural network, and achieves data consistency using multiple enhancement modules and back-projection modules. The enhancement modules in DRNN-IE produce new features through fusing low-level and high-level features. In order to improve the algorithm’s generalization ability, a dual-parameter loss function is adopted to train and optimize the neural network. Experiments on real CT images show that the proposed algorithm has excellent enhancement performance and retains detailed information of low-dose CT images.

ResNet and its application to medical image processing: Research progress and challenges

Objective. Motor decoding is crucial to translate the neural activity for brain-computer interfaces (BCIs) and provides information on how motor states are encoded in the brain. Deep neural networks (DNNs) are emerging as promising neural decoders. Nevertheless, it is still unclear how different DNNs perform in different motor decoding problems and scenarios, and which network could be a good candidate for invasive BCIs. Approach. Fully-connected, convolutional, and recurrent neural networks (FCNNs, CNNs, RNNs) were designed and applied to decode motor states from neurons recorded from V6A area in the posterior parietal cortex (PPC) of macaques. Three motor tasks were considered, involving reaching and reach-to-grasping (the latter under two illumination conditions). DNNs decoded nine reaching endpoints in 3D space or five grip types using a sliding window approach within the trial course. To evaluate decoders simulating a broad variety of scenarios, the performance was also analyzed while artificially reducing the number of recorded neurons and trials, and while performing transfer learning from one task to another. Finally, the accuracy time course was used to analyze V6A motor encoding. Main results. DNNs outperformed a classic Naïve Bayes classifier, and CNNs additionally outperformed XGBoost and Support Vector Machine classifiers across the motor decoding problems. CNNs resulted the top-performing DNNs when using less neurons and trials, and task-to-task transfer learning improved performance especially in the low data regime. Lastly, V6A neurons encoded reaching and reach-to-grasping properties even from action planning, with the encoding of grip properties occurring later, closer to movement execution, and appearing weaker in darkness. Significance. Results suggest that CNNs are effective candidates to realize neural decoders for invasive BCIs in humans from PPC recordings also reducing BCI calibration times (transfer learning), and that a CNN-based data-driven analysis may provide insights about the encoding properties and the functional roles of brain regions.

Lightweight single image super-resolution with attentive residual refinement network

Image acquisition is an essential step in image processing. When the image acquisition is done the image that is generated is subjected to Impulse noise, Gaussian noise etc. We have performed the image denoising on images inflicted with impulse noise. Image denoising is an essential step in all types of image processing. Traditional techniques reduce the noise in the image but it also reduces the quality of the image. Traditional filters like gaussian filter, median filter is analyzed which work in the spatial domain and filters working in the frequency domain are also considered like Butterworth filters, Weiner filter. A Deep residual Neural Network filter is proposed which is compared with the Fuzzy Neural Network denoiser. Their performance is compared on the metrics PSNR and SSIM. The Fuzzy Neural Network system improves the SSIM significantly compared to a deep residual neural network and a comparison is made with traditional image denoising methods. We also compare the performance of the deep residual neural network, Fuzzy Neural Network system and Median denoising algorithm on impulse noise has been compared. The performance of deep neural networks depends on the total number of examples used and the performance can be improved if we have more image pairs.

Automatic modulation classification (AMC) is a signal processing technology used to identify the modulation type of unknown signals without prior information such as modulation parameters for drone communications. In recent years, deep learning (DL) has been widely used in AMC methods due to its powerful feature extraction ability. The significant performance of DL-based AMC methods is highly dependent on large amount of data. However, with the increasingly complex signal environment and the emergence of new signals, several recognition tasks have difficulty obtaining sufficient high-quality signals. To address this problem, we propose an AMC method based on a deep residual neural network with masked modeling (DRMM). Specifically, masked modeling is adopted to improve the performance of a deep neural network with limited signal samples. Both complex-valued and real-valued residual neural networks (ResNet) play an important role in extracting signal features for identification. Several typical experiments are conducted to evaluate our proposed DRMM-based AMC method on the RadioML 2016.10A dataset and a simulated dataset, and comparison experiments with existing AMC methods are also conducted. The simulation results illustrate that our proposed DRMM-based AMC method achieves better performance in the case of limited signal samples with low signal-to-noise ratio (SNR) than other existing methods.