Human behavior recognition based on residual generative adversarial networks

Hyperspectral remote sensing image classification based on residual generative Adversarial Neural Networks

ResNet and its application to medical image processing: Research progress and challenges

Early detection of cervical cancer is key to detecting and treating cancer. Applying the computer to the detection of cervical cancer can get more accurate results. However, data acquisition has become our main challenge. In order to solve the problem of insufficient data, this paper proposes a method based on residual network and generative adversarial network (RCGAN) for data augmentation of cervical single-cell images. At the same time, our experimental results are verified by a classification model. Experiments show that the method proposed in this paper can effectively expand the data set and improve the classification effect (accuracy rate is 95.18%, accuracy is 96.10%, recall rate is 98.50%, and F1-Score is 97.28%). Therefore, the work of this article is of great significance for the discovery and prevention of cervical cancer.

Automatic defect detection is of great significance to the production process of modern industries, which will affect the product appearance and quality. And a great number of economic loss may be caused due to the defects in the industrial production process. Traditional manual defect inspection method is labor-intensive and time-consuming with subjective factors, and the detection performance is random and uncertain. Machine learning has shown a good detection ability on small-scale samples, but the detect detection task against with poor contrast, weak texture, etc., will affect the effective feature representation. To address the detection task of steel surface defects, combined with the strong context extraction ability of deep learning, an accurate deep defect classification network is proposed in the paper to provide an end-to-end detection scheme. Fused with the residual network (Resnet50) and spatial attention block, a residual attention network is proposed for effective feature representation, which could make the classification network better focus on the defect areas. Meanwhile, due to the scale information change among different defects, a multi-scale context fusion (MCF) block is proposed for effective multi-scale feature extraction, which is conducive for multi-scale object detection. Experimental results on public defect data set show that the proposed defect detection network could acquire a superior classification performance compared with some typical classification networks.

With the advancement of remote sensing technology, high-resolution images are widely used in the field of computer vision. However, image quality is often degraded due to hardware limitations and environmental interference. This paper proposes a Residual Separable Super-Resolution Reconstruction Generative Adversarial Network (RSSRGAN) for remote sensing image super-resolution. The model aims to enhance the resolution and edge information of low-resolution images without hardware improvements. The main contributions include (1) designing an optimized generator network by improving the residual dense network and introducing depthwise separable convolutions to remove BN layers, thereby increasing training efficiency—two PatchGAN discriminators are designed to enhance multi-scale detail capture—and (2) introducing content loss and joint perceptual loss on top of adversarial loss to improve global feature representation. Experimental results show that compared to the widely used SRGAN model in remote sensing (exemplified by the satellite-specific SRGAN in this study), this model improves PSNR by approximately 18.8%, SSIM by 8.0%, reduces MSE by 3.6%, and enhances the PI metric by 13.6%. It effectively enhances object information, color, and brightness in images, making it more suitable for remote sensing image super-resolution.

To take full advantage of the information of images captured by drones and given that most existing monocular depth estimation methods based on supervised learning require vast quantities of corresponding ground truth depth data for training, the model of unsupervised monocular depth estimation based on residual neural network of coarse–refined feature extractions for drone is therefore proposed. As a virtual camera is introduced through a deep residual convolution neural network based on coarse–refined feature extractions inspired by the principle of binocular depth estimation, the unsupervised monocular depth estimation has become an image reconstruction problem. To improve the performance of our model for monocular depth estimation, the following innovations are proposed. First, the pyramid processing for input image is proposed to build the topological relationship between the resolution of input image and the depth of input image, which can improve the sensitivity of depth information from a single image and reduce the impact of input image resolution on depth estimation. Second, the residual neural network of coarse–refined feature extractions for corresponding image reconstruction is designed to improve the accuracy of feature extraction and solve the contradiction between the calculation time and the numbers of network layers. In addition, to predict high detail output depth maps, the long skip connections between corresponding layers in the neural network of coarse feature extractions and deconvolution neural network of refined feature extractions are designed. Third, the loss of corresponding image reconstruction based on the structural similarity index (SSIM), the loss of approximate disparity smoothness and the loss of depth map are united as a novel training loss to better train our model. The experimental results show that our model has superior performance on the KITTI dataset composed by corresponding left view and right view and Make3D dataset composed by image and corresponding ground truth depth map compared to the state-of-the-art monocular depth estimation methods and basically meet the requirements for depth information of images captured by drones when our model is trained on KITTI.

Owing to the loss of effective information and incomplete feature extraction caused by the convolution and pooling operations in a convolution subsampling network, the accuracy and speed of current speech processing architectures based on the conformer model are influenced because the shallow features of speech signals are not completely extracted. To solve these problems, in this study, we researched a method that used a capsule network to improve the accuracy of feature extraction in a conformer-based model, and then, we proposed a new end-to-end model architecture for speech recognition. First, to improve the accuracy of speech feature extraction, a capsule network with a dynamic routing mechanism was introduced into the conformer model; thus, the structural information in speech was preserved, and it was input to the conformer blocks via sequestered vectors; the learning ability of the conformed-based model was significantly enhanced using dynamic weight updating. Second, a residual network was added to the capsule blocks, thus, the mapping ability of our model was improved and the training difficulty was reduced. Furthermore, the bi-transformer model was adopted in the decoding network to promote the consistency of the hypotheses in different directions through bidirectional modeling. Finally, the effectiveness and robustness of the proposed model were verified against different types of recognition models by performing multiple sets of experiments. The experimental results demonstrated that our speech recognition model achieved a lower word error rate without a language model because of the higher accuracy of speech feature extraction and learning using our model architecture with a capsule network. Furthermore, our model architecture benefited from the advantage of the capsule network and the conformer encoder, and also has potential for other speech-related applications.

Atrial fibrillation (AF) is an arrhythmia that can cause blood clot and may lead to stroke and heart failure. To detect AF, deep learning-based detection algorithms have recently been developed. However, deep learning models were often trained with limited datasets and were evaluated within the same datasets, which makes their performance generally drops on the external datasets, known as data dependency. For this study, three different databases from PhysioNet were used to investigate the data dependency of deep learning-based AF detection algorithm using the residual neural network (Resnet). Resnet 18, 34, 50 and 152 model were trained with raw electrocardiogram (ECG) signal extracted from independent database. The highest accuracy was about 98–99% which is evaluation results of test dataset from the own database. On the other hand, the lowest accuracy was about 53–92% which was evaluation results of the external dataset extracted from different source. There are data dependency according to the train dataset and the test dataset. However, the data dependency decreased as a large amount of train data.

Sea ice is an important component of the polar circle and influences atmospheric change. Global navigation satellite system reflectometry (GNSS-R) not only realizes time-continuous and wide-area sea ice detection, but also greatly reduces the cost of sea ice remote sensing research, which has been a hot topic in recent years. To tackle the challenges of noise interference and the reduced accuracy of sea ice detection during the melting period, this paper proposes a sea ice detection method based on a residual neural network (ResNet). ResNet addresses the issue of vanishing gradients in deep neural networks and introduces residual connections, which allows the network to reuse learned features from previous layers. Delay-Doppler maps (DDMs) collected from TechDemoSat-1 (TDS-1) are used as input, and National Oceanic and Atmospheric Administration (NOAA) surface-type data above 60°N are selected as the true values. Based on ResNet, the sea ice detection achieved an accuracy of 98.61%, demonstrating high robustness to noise and strong stability during the sea ice melting period (June to September). In comparison to other sea ice detection algorithms, it stands out with its advantages of high accuracy, stability, and insensitivity to noise.

Accurate fault diagnosis is the key to ensure the safe and operation of rotating machines. Accuracy of bearing fault diagnosis will be reduced in complex noise environments and variable load conditions. Therefore, this paper proposes a fault diagnosis method based on attention mechanism and residual neural network (FaultCNN). Firstly, a residual attention module is designed to extract the original fault features of the noise signals. Secondly, the ability to extract features of the residual module is further enhanced by adding an average pooling layer to the residual structure shortcut. To validate the diagnostic performance of the model, CWRU datasets are used for experimental verification. In the noise environment of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{SNR}=-6 \text{dB}$</tex> , the accuracy of model fault diagnosis is 94.23%. Compared to existing bearing fault diagnostic methods, FaultCNN has higher anti-noise performance and variable load adaptability.

Mainland China has a poor distribution of meteorological stations. Existing models’ estimation accuracy for creating high-resolution surfaces of meteorological data is restricted for air temperature, and low for relative humidity and wind speed (few studies reported). This study compared the typical generalized additive model (GAM) and autoencoder-based residual neural network (hereafter, residual network for short) in terms of predicting three meteorological parameters, namely air temperature, relative humidity, and wind speed, using data from 824 monitoring stations across China’s mainland in 2015. The performance of the two models was assessed using a 10-fold cross-validation procedure. The air temperature models employ basic variables such as latitude, longitude, elevation, and the day of the year. The relative humidity models employ air temperature and ozone concentration as covariates, while the wind speed models use wind speed coarse-resolution reanalysis data as covariates, in addition to the fundamental variables. Spatial coordinates represent spatial variation, while the time index of the day captures time variation in our spatiotemporal models. In comparison to GAM, the residual network considerably improved prediction accuracy: on average, the coefficient of variation (CV) R2 of the three meteorological parameters rose by 0.21, CV root-mean square (RMSE) fell by 37%, and the relative humidity model improved the most. The accuracy of relative humidity models was considerably improved once the monthly index was included, demonstrating that varied amounts of temporal variables are crucial for relative humidity models. We also spoke about the benefits and drawbacks of using coarse resolution reanalysis data and closest neighbor values as variables. In comparison to classic GAMs, this study indicates that the residual network model may considerably increase the accuracy of national high spatial (1 km) and temporal (daily) resolution meteorological data. Our findings have implications for high-resolution and high-accuracy meteorological parameter mapping in China.

Competitive residual neural network for image classification

High-resolution digital elevation models (DEMs) are widely used in many fields such as mapping, hydrology, meteorology and geology, where they can improve the accuracy and reliability of many geographic analysis applications as an input. However, due to the high cost and difficulty of acquiring high-resolution DEMs, as well as the problems of edge smoothing, data distortion and fractures in reconstructed ground surfaces with traditional super-resolution DEM reconstruction techniques. Inspired by the excellence of generative adversarial neural networks in super-resolution image analysis, this paper investigates an approach for DEM super-resolution reconstruction with deep residual generative adversarial network. An advanced DEM Super-resolution Generative Adversarial Network (D-SRCAGAN) is proposed in this paper, which can reconstruct a quadruple higher resolution DEM by using low-resolution DEM. Compared with the bicubic and SRGAN methods, the D-SRCAGAN method reconstruction results can retain more topographic features and obtain higher RMSE values.

The technique of human action recognition can be applied in a number of fields, such as medical rehabilitation, posture in the domain of sports, and emotion perception in counselling. Deep learning action recognition models that consider the continuous changes in the human skeleton can efficiently identify action states, such as rehabilitation processes, posture, and changes in emotion. However, the keypoints of the human skeleton are susceptible to occlusion due to body movement, resulting in a lower confidence level in their positions, which in turn affects the training and execution of human action recognition models. This paper proposes a keypoint correction algorithm based on continuous-time images of temporal variation that can predict and correct the skeleton keypoints with low confidence, based on the previous and subsequent multiple images. The proposed temporal-variation skeleton keypoint correction algorithm can provide accurate feature association training for the locations of the skeleton keypoints. The proposed method is shown to improve the accuracy of human action recognition by at least 30% over three alternative models in the literature: STV-GCN, GCN-NAS, and 2S-AGCN. The proposed algorithm is also improved by at least 30% compared with the above-mentioned methods in the literature, based on an action recognition model with a Gaussian filter.

Reducing parameter number in residual networks by sharing weights