Learning multiscale and deep representations for classifying remotely sensed imagery

This paper proposes a novel intelligent fault diagnosis method to automatically identify different health conditions of wind turbine (WT) gearbox. Unlike traditional approaches, where feature extraction and classification are separately designed and performed, this paper aims to automatically learn effective fault features directly from raw vibration signals while classify the type of faults in a single framework, thus providing an end-to-end learning-based fault diagnosis system for WT gearbox without additional signal processing and diagnostic expertise. Considering the multiscale characteristics inherent in vibration signals of a gearbox, a new multiscale convolutional neural network (MSCNN) architecture is proposed to perform multiscale feature extraction and classification simultaneously. The proposed MSCNN incorporates multiscale learning into the traditional CNN architecture, which has two merits: 1) high-level fault features can be effectively learned by the hierarchical learning structure with multiple pairs of convolutional and pooling layers; and 2) multiscale learning scheme can capture complementary and rich diagnosis information at different scales. This greatly improves the feature learning ability and enables better diagnosis performance. The proposed MSCNN approach is evaluated through experiments on a WT gearbox test rig. Experimental results and comprehensive comparison analysis with respect to the traditional CNN and traditional multiscale feature extractors have demonstrated the superiority of the proposed method.

To address the issues of low diagnostic accuracy in traditional rolling bearing fault diagnosis models and the ineffective extraction of spatial and temporal features from vibration signals, this paper proposes a rolling bearing fault diagnosis method based on the improved sparrow search algorithm optimized VMD and multi-scale convolutional neural networks. First, the improved sparrow search algorithm is employed to adaptively optimize the penalty parameter and mode count in variational modal decomposition (VMD). This achieves finer frequency band segmentation and effectively suppresses energy leakage, thereby yielding high quality frequency domain representations. Second, a multi-scale convolutional neural networks (MSCNN) is constructed, with feature level fusion implemented. Subsequently, a bidirectional long short-term memory networks (BiLSTM) is introduced to model the temporal dependencies of the fused features, enabling fault mode learning. A softmax layer is employed to achieve multi-class classification. Finally experimental results and comparisons based on the CWRU bearing dataset demonstrate the effectiveness of the proposed method in the rolling bearing fault classification task, providing significant application value for achieving efficient and reliable bearing fault detection.

We propose Multi-scale convolutional recurrent neural networks (MCRNN) and data augmentation methods to detect polyphonic sound event with few training data. MCRNN consists of Multi-scale convolutional neural networks (MCNN) and recurrent neural networks (RNN). MCNN concatenates the higher level features extracted using multiple convolution kernels with different scales from the time domain and frequency domain at the same time. RNN is able to capture the longer term temporal context characteristics. A novel background spectrum random replacement (BSRR) data augmentation method is applied to expand training data, which uses standard normal distribution data with randomly selected position and length instead of the original time-domain, frequency-domain or time-frequency domain background spectrum features. Our method is tested on the datasets of DCASE 2019 Task3 (T3). The experimental results showed that the MCRNN and BSRR data augmentation method are efficient. We achieved better results than the first place and the single advanced on the T3 by applying BSRR and SpecAugment data augmentation method simultaneously. On the evaluation dataset (T3-eval), our best result shows 0.05 and 0.975 of error rate (ER) and F1 respectively. Our method got the best performance and relatively improved 17% and 1% than the corresponding values of the single advanced.

Bridge deflection values are significant for their health and safety, but current methods for predicting bridge deflection suffer from problems such as anomalous data and low prediction accuracy. To solve the problems of anomalous bias and loss of short-term trend in traditional smoothing methods, this paper proposes a preprocessing method for cascade residual smoothing. The method firstly uses Gaussian filtering to initially remove the high-frequency noise in the signal and retain the overall trend. Then, the residuals of the initial filtering and the original data are smoothed by quadratic exponential smoothing to extract the short-term trend in the deflection data, which is favorable for the data to have the advantages of both stabilization and retention of small fluctuations. In addition, to simultaneously acquire the temporal dependence and spatial features between long- and short-term temporal signals, this paper proposes a multiscale spatial attention network based on Multiscale Convolutional Neural Networks (MSCNNs), Gated Recurrent Units (GRUs), and self-attention (SA). The method obtains multi-level sensory field spatial information within each period through the MSCNN, focuses on the connection between different time steps using a GRU, and employs SA to automatically focus on the deflection features that have a significant impact and ignore unimportant perturbation variations, thus improving the prediction ability of the model. In this paper, compared with CNN-Attention-LSTM, the MAE is reduced by 25.79%, the RMSE is reduced by 24.69%, and the R2 is increased by 2.36%, which proves the superiority and advancement of the method.

In order to realize the quantitative assessment of muscle strength in hand function rehabilitation and then formulate scientific and effective rehabilitation training strategies, this paper constructs a multi-scale convolutional neural network (MSCNN) - convolutional block attention module (CBAM) - bidirectional long short-term memory network (BiLSTM) muscle strength prediction model to fully explore the spatial and temporal features of the data and simultaneously suppress useless features, and finally achieve the improvement of the accuracy of the muscle strength prediction model. To verify the effectiveness of the model proposed in this paper, the model in this paper is compared with traditional models such as support vector machine (SVM), random forest (RF), convolutional neural network (CNN), CNN - squeeze excitation network (SENet), MSCNN-CBAM and MSCNN-BiLSTM, and the effect of muscle strength prediction by each model is investigated when the hand force application changes from 40% of the maximum voluntary contraction force (MVC) to 60% of the MVC. The research results show that as the hand force application increases, the effect of the muscle strength prediction model becomes worse. Then the ablation experiment is used to analyze the influence degree of each module on the muscle strength prediction result, and it is found that the CBAM module plays a key role in the model. Therefore, by using the model in this article, the accuracy of muscle strength prediction can be effectively improved, and the characteristics and laws of hand muscle activities can be deeply understood, providing assistance for further exploring the mechanism of hand functions .

ABSTRACTObject detection in very-high-resolution (VHR) remote sensing images is one of the important technical means in many fields. In recent years, conventional object detection methods have been completely replaced by Convolutional Neural Network (CNN)-based methods, which are more accurate and efficient. However, most current CNN-based methods applied in VHR image sets have certain defects: (1) Scale preference is common in the framework designs, and the representation ability of feature maps for large and small objects is quite different, so accuracy promotion can hardly be made comprehensively in the detection of different objects. (2) The scale difference of the objects leads to training difficulties. (3) Some high-precision methods require high hardware costs, and the overall frameworks lack practicality. To address such problems, we propose a new object detection method in this paper, namely Balanced Multi-Scale Fusion-based CNN (BMF-CNN). It is a redesigned two-stage detection framework according to the region-based object detection methods, which enabled the detection accuracy of both large and small objects to reach a high level. Through the evaluation in the open VHR remote sensing image sets, we found that BMF-CNN showed a better integrative performance than the current mainstream detection methods.

We present an end-to-end multi-scale Convolutional Neural Network (CNN) framework for topic identification (topic ID). In this work, we examined multi -scale CNN for classification using raw text input. Topical word embeddings are learnt at multiple scales using parallel convolutional layers. A technique to integrate verification and identification objectives is examined to improve topic ID performance. With this approach, we achieved significant improvement in identification task. We evaluated our framework on two contrasting datasets: 20 newsgroups and Fisher. We obtained 92.93% accuracy on Fisher and 86.12% on 20 newsgroups, which to our know ledge are the best published results on these datasets at the moment.

Nowadays, the digital transformation of agriculture puts forward higher requirements for network security. In response to the complex network traffic data and chaotic attack data cycles in the current agricultural network environment, it is difficult for the network security situation awareness methods to effectively extract network security situation elements and perceive network security status. Therefore, this paper proposes a fusion model-based agricultural network security situation awareness method, namely, MSCNN-ResNeXt-Transformer. Firstly, ResNeXt is improved by fusion model, using Multi-Scale Convolutional Neural Network (MSCNN) instead of a single scale convolution structure. This enables the agricultural network security situation awareness model in digital economy to comprehensively extract network security situational elements from multiple scales. The Efficient Channel Attention (ECA) mechanism is then employed to further refine and characterize the data processed by the improved ResNeXt. Finally, the Transformer is used to optimize the proposed model and improve the accuracy of agricultural network security situation awareness in digital economy. The experimental results show that the accuracy, recall and F1 of MSCNN-ResNeXt-Transformer on MOORE, KDDCUP99 and WSN-DS are significantly better than traditional models, providing effective technical support for agricultural digital security protection.

In this paper, we address the semantic segmentation task with a deep network that combines contextual features and spatial information. The proposed Cross Attention Network is composed of two branches and a Feature Cross Attention (FCA) module. Specifically, a shallow branch is used to preserve low-level spatial information and a deep branch is employed to extract high-level contextual features. Then the FCA module is introduced to combine these two branches. Different from most existing attention mechanisms, the FCA module obtains spatial attention map and channel attention map from two branches separately, and then fuses them. The contextual features are used to provide global contextual guidance in fused feature maps, and spatial features are used to refine localizations. The proposed network outperforms other real-time methods with improved speed on the Cityscapes and CamVid datasets with lightweight backbones, and achieves state-of-the-art performance with a deep backbone.

. Batik is an Indonesian cultural heritage that has been recognized by UNESCO as an international cultural heritage on October 2, 2009. Patterns of batik produce geometric shapes unique, the number and name of the batik patterns make it difficult to recognize each motif. The objective classification of batik is split image into classes according to the pattern motif motive so easy to recognize in accordance with its feature. Batik can be classified based on the shape of the motive, namely geometric motifs, geometric motifs and motifs non specific. Spatial information is an important aspect of image processing such as computer vision and recognition structure / pattern in the context of modelling and resolution of the uncertainty caused by the ambiguity in the low-level features. Shortcomings inherent in combining two colours and spatial features are not adaptive pattern recognition process of the region across multiple images and histogram matching is not appropriate to capture the colours on the image content. This study discussed a model of spatial features and feature combinations topology with the aim to improve the validation batik image pattern recognition so that the level of the pattern recognition motif batik image could be better. Some of the features that have been used include colour features and spatial features. In addition, this paper discusses the possibility of combining the features in pattern recognition. This paper proposes a combination of features that will be able to improve the validation of image pattern recognition of batik.

Characterized by complicated backgrounds, various types, large size variations, and arbitrary orientations, the detection and recognition of arbitrary-oriented objects in remote sensing images are challenging. To address the aforementioned problem, an anchor-free arbitrary-oriented object detector using box boundary-aware vectors is proposed. With the idea of CenterNet to detect objects as points, oriented object detection is achieved by predicting the center, the box boundary-aware vectors, the size, and the type of the bounding box. In the feature extraction stage of the designed architecture, Res2Net, a multiscale convolutional neural network, is used to extract feature maps of different scales and adaptively spatial feature fusion is adopted to improve the detector's adaptability to objects of different sizes. In the detector, a context enhancement module with a multibranch network is designed to enhance the contextual information of the objects and improve the detector's robustness to the complicated backgrounds. Experiments are carried on three challenging benchmarks (i.e., HRSC2016, UCAS-AOD, and DOTA) and our method achieves state-of-the-art performance with 90.30%, 89.70%, and 77.18% mAP, respectively.

A hybrid approach for EEG motor imagery classification using adaptive margin disparity and knowledge transfer in convolutional neural networks.

Abstract. Developing a remote sensing-based monitoring system for detecting marine plastics requires a thorough investigation of their discrimination possibilities from other floating objects. To this end, this study aims to explore the ability to discriminate marine debris from other floating materials and sea features using high-resolution multispectral satellite data. To perform our analysis, we utilized the open-access Marine Debris Archive (MARIDA), which contains several marine classes on Sentinel-2 (S2) data and benchmarks machine learning frameworks. In particular, we investigated well-established spectral indices, GrayLevel Co-occurrence Matrix (GLCM), Local Binary Pattern (LBP) texture and other spatial features at multiple scales. A Random Forest (RF) classifier was also applied for the classification procedure, and the spectral and spatial features which contributed the most were underlined. The quantitative and qualitative assessment indicated that the spectral information alone is insufficient to distinguish marine plastic from other floating materials which exhibit similar spectral behavior, such as vessels. However, a strong potential even for challenging discrimination tasks is presented when combined with spatial information. By further evaluating our results qualitatively, significant insights are gained, and specific combinations are proposed for challenging floating materials discrimination.

Sentiment classification is an important topic in the field of natural language processing (NLP). The mainstream methods have also changed from the initial emotional dictionary to the later machine learning, and now most of them focus on deep learning. In sentiment classification tasks, comments in different fields have many similar expressions and relevant characteristics, which can effectively solve the problem of insufficient training data for current deep learning methods. In order to improve the classification effect of the model, we proposed a sentiment analysis model based on multi-task learning that Distillation Bidirectional Encoder Representations from Transformers combined with multi-scale convolution (MTL-DISTILBERT-MSCNN). It models comments in multiple fields together, and uses the advantages of the multi-task learning framework, the pre-training model DistilBert and the convolutional neural network (CNN) model to obtain the correlation between comments in multiple fields, the global features and local features of the text. We conduct experiments on 14 product review data sets and 2 movie review data sets. Using accuracy, F1-score and training speed as the evaluation criteria of the proposed model. By comparing single-task learning and other multi-task learning methods, the MTL-DISTILBERT-MSCNN model can be improved by 2%-5%, and the training efficiency is very high. In addition, we also adjusted the convolutional neural network of the model to further optimize the training effect of the model.

Hyperspectral Image (HSI) classification plays a key role remote sensing field. Recently, deep learning has demonstrated its effectiveness in HSI Classification field. This paper presents a spectral-spatial HSI classification technique established on the deep learning based deep belief network (DBN) for deep and abstract feature extraction and adaptive boundary adjustment based segmentation. Proposed approach focuses on integrating the deep learning based spectral features and segmentation based spatial features into a framework for improved performance. Specifically, first the deep DBN model is exploited as a spectral feature extraction based classifier to extract the deep spectral features. Second, spatial contextual features are obtained by utilizing effective adaptive boundary adjustment based segmentation technique. Finally, maximum voting based criteria is operated to integrate the results of extracted spectral and spatial information for improved HSI classification. In general, exploiting spectral features from DBN process and spatial features from segmentation and integration of spectral and spatial information by maximum voting based criteria, has a substantial effect on the performance of HSI classification. Experimental performance on real and widely used hyperspectral data sets with different contexts and resolutions demonstrates the accuracy of the proposed technique and performance is comparable to several recently proposed HSI classification techniques.