Seismic interpolation via multi-scale HU-Net

MSST-ResNet: Deep multi-scale spatiotemporal features for robust visual object tracking

Variable seismic waveforms representation: Weak-supervised learning based seismic horizon picking

We are developing a decision support system for treatment response assessment of bladder cancer in CT urography (CTU). Accurate segmentation of bladder cancer is a critical and challenging task. We previously developed a bladder cancer segmentation method using a deep learning convolutional neural network and level set (DL-CNN+LS) approach. In this study we investigated the application of a U-Net based deep learning (U-Net) model for bladder cancer segmentation. Our new U-Net method did not require the second-stage level set refinement, greatly simplifying the overall segmentation pipeline. The proposed U-Net model utilized a user-defined box to direct the attention of the U-Net to the lesion region by masking out the structured background outside the box. We trained and evaluated the performance of the U-Net segmentations by using hand-drawn 3D contours from a radiologist as reference standard. The segmentation accuracy was evaluated by the average volume intersection ratio (AVI), average percent volume error (AVE), average absolute volume error (AAVE), average minimum distance (AMD), and the Jaccard index (JI). On the validation set, the cropped U-Net achieved values of AVI = 65.5±20.2%, AVE = 3.7±44.4%, AAVE = 30.9±30.8%, AMD = 2.9±1.5 mm, and JI = 51.4±16.8%. The previous DL-CNN+LS approach achieved values of AVI = 36.2±28.4%, AVE = 51.1±39.3%, AAVE = 52.0±38.0%, AMD = 4.1±1.7 mm, and JI = 32.9±27.2%. On the independent test set, the cropped U-Net achieved values of AVI = 66.0±22.8%, AVE = -3.7±39.4%, AAVE = 28.1±27.3%, AMD = 4.5±3.3 mm, and JI = 49.2±20.0%. The DL-CNN+LS achieved values of AVI = 31.2±24.3%, AVE = 59.9±30.9%, AAVE = 60.8±29.0%, AMD = 5.5±2.2 mm, and JI = 27.9±20.8%, respectively. The results demonstrated that the U-Net model could achieve a higher accuracy than the previous DL-CNN+LS model while reducing the complexity of the segmentation pipeline.

As a core clean energy source in the global energy transition, natural gas price fluctuations directly affect the energy market supply demand balance, industrial chain cost control, etc. Thus, accurate natural gas price prediction is crucial for market participants’ decision making and policymakers’ regulation. To tackle the issue that traditional single models fail to capture data patterns of the New York Mercantile Exchange (NYMEX) natural gas futures daily prices—due to their nonlinearity, high volatility, and multi-scale features—this study proposes a hybrid model: VMD-CNN-BiLSTM-attention, integrating Variational Mode Decomposition (VMD), Convolutional Neural Network (CNN), Bidirectional Long Short-Term Memory (BiLSTM), and an attention mechanism. A one-step to four-step forecasting comparison was conducted using NYMEX natural gas futures daily closing prices, with the proposed model vs. CNN-BiLSTM-Attention and Autoregressive Integrated Moving Average (ARIMA) models. The empirical results show that the VMD-CNN-BiLSTM-attention model outperforms the comparison models in terms of Mean Absolute Percentage Error (MAPE), Mean Absolute Error (MAE), etc. Specifically, its four-step forecast MAPE stays ≤3.5% and R2 ≥ 98%, demonstrating a stronger ability to capture complex price fluctuations, better accuracy, and stability than traditional single models and deep learning models without VMD, and provides reliable technical support for short-to-medium-term natural gas price prediction.

Seismic data suffer from under-sampling along spatial dimensions due to physical and economic limitations during acquisition, which can adversely impact subsequent imaging processes. Various approaches have been proposed to address this issue, including rank-reduction, transform, and deep learning (DL) methods for reconstructing missing traces in observed seismic data. While many DL-based reconstruction methods employ convolutional neural networks (CNNs) as their core components in supervised or unsupervised settings, the accuracy of CNN-based DL reconstruction methods can still be enhanced due to the inherent localization of convolutional operators. In this study, we present an innovative method, termed Attentive CNN (ACNet), that incorporates a transformer model with a multi-head self-attention mechanism into the CNN framework for unsupervised seismic data reconstruction. ACNet consists of an encoder built with the transformer model and a CNN decoder, enabling an iterative reconstruction process within the proposed unsupervised DL-based approach. We compare the performance of our method with that of the damped rank-reduction (DRR) method using synthetic and field data examples. The results demonstrate that our proposed method exhibits superior signal reconstruction and random noise suppression capabilities compared to the DRR method.

Recent research on single image super-resolution (SISR) using convolutional neural networks (CNNs) with the utilization of residual structures and attention mechanisms to utilize image features has demonstrated excellent performance. However, previous SISR techniques mainly integrated extracted image features within a deep or wide network architecture, ignoring the interaction between multiscale features and the diversity of features. At the same time, SISR is also a typical ill-posed problem in that it allows for several predictions for a given LR image. These problems limit the great learning ability of CNNs. To solve these problems, we propose a closed-loop residual attention network (CLRAN) to extract and interact with all the available diversity of features features efficiently and limit the space of possible function solutions. Specifically, we design an enhanced residual attention block (ERA) to extract features, and it dynamically assigns weight to the internal attention branches. The ERA combines multi-scale block (MSB) and enhanced attention mechanism (EAM) base on the residual module. The MSB adaptively detects multiscale image features of different scales by using different 3 × 3 convolution kernels. The EAM combines multi-spectral channel attention (MSCA) and spatial attention (SA). Therefore, the EAM extracts different frequency component information and spatial information to utilize the diversity features. Furthermore, we apply the progressive network architecture and learn an additional map for model monitoring, which forms a closed-loop with the mapping already learned by the LR to HR function. Extensive experiments demonstrate that our CLRAN outperforms the state-of-the-art SISR methods on public datasets for both ×4 and ×8, proving its accuracy and visual perception.

Summary Wavefield separation is an important processing step to remove the interfering receiver ghost in seismic data. For ocean-bottom surveys, where typically both the pressure P (hydrophone) and particle velocity Vz (geophone) is measured, PZ-summation is a commonly used wavefield separation method. Although this method delivers excellent results, the required wavefield calibration makes it user-intensive to apply and QC. Supervised machine learning (ML) can offer a robust and user-friendly alternative to the industry-standard PZ-summation method. The convolutional neural network used in the proposed approach automatically calibrates the multi-component recordings and performs the wavefield separation without any user interaction required. A comparison study is conducted where the ML method is compared to the industry-standard PZ-summation method for a controlled shallow water data experiment and an OBN field data example with water depths ranging between 25m and 40m. After an easy to execute network training phase, the method delivers excellent wavefield separation results for both the complex synthetic and the shallow water field data example.

For understanding localized hydrological and climatological processes, downscaling gridded precipitation data to finer spatial resolutions is a crucial prerequisite. For a densely populated country like India, accurate downscaled data is crucial for building resilience to climate change impacts, supporting adaptation efforts, and enhancing disaster management. In recent years, deep learning (DL) has emerged as a powerful tool for advancing Earth system modelling and climate data downscaling. This study presents a comprehensive intercomparison of deep learning architectures, for downscaling precipitation across India. A few efficient DL architectures from recent studies are chosen for intercomparison such as simple dense, simple convolutional neural network, Fast Super Resolution Convolutional Neural Network (FSRCNN), Super Resolution Deep Residual Network (SRDRN), U-Net, and Nest-U-Net. The experiments are designed in synthetic style by using coarsened ECMWF Reanalysis version 5 (ERA5; 1ox1o) daily variables as the inputs and high-resolution Indian Monsoon Data Assimilation and Analysis reanalysis (IMDAA; 0.12ox0.12o) daily precipitation as training labels and benchmarks for the evaluation. Training and validation are conducted for the period 1980-2014, afterwards the trained models are evaluated on data from 2015-2020. To reduce the biases induced by the highly positive-skewed precipitation data and to enhance the model performance on extreme events, a weighted mean absolute error is implemented for training. The performance of the DL models is also compared with the Bias Correction and Spatial Disaggregation (BCSD), a renowned statistical downscaling method. The results indicate that all deep learning DL models outperformed the BCSD method. Among the DL models, U-Net and Nest-U-Net demonstrated superior performance in capturing fine-scale precipitation patterns and extreme precipitation events, owing to their encoder-decoder architecture, which effectively learns spatial features at different scales. In contrast, the FSRCNN and SRDRN produced results with slightly lower precision than the U-Net models, but at a significantly reduced inference time, making them more efficient for faster data generation. The findings underscore the potential of deep learning for improving regional precipitation downscaling across India, offering a promising alternative to traditional statistical methods like BCSD in handling complex, non-linear relationships inherent in climate data.

Lung field segmentation (LFS) in chest x-rays (CXR) is a key step for the computer-assisted diagnosis of respiratory diseases. However, achieving precise delineation remains challenging in images with poor contrast and superimposed anatomical structures, often resulting in imprecise lung outlines in deep learning (DL) models. Furthermore, existing models often lack generalizability, performing well on specific datasets but failing on unseen domains due to variations in imaging equipment, acquisition protocols, and patient demographics. To overcome these limitations, we propose a two-phase hybrid approach for robust and generalizable LFS in CXR. The first phase employs a heuristic approach to extract and refine multiscale edge features: The Laplacian of Gaussian (LoG) filter detects closed contours that effectively highlight the overall lung shape, whereas the Canny detector captures the finer, high-frequency edge details that preserve the lung outlines often missed by DL models due to overlapping structures. In the second phase, the enriched multiscale feature maps are fused along with the original contrast inputs and then used to train and evaluate various U-Net architectures. The proposed approach is evaluated on three benchmark datasets: Montgomery County (MC), Shenzhen Hospital (SH), and Japanese Society of Radiological Technology (JSRT). Experiments conducted on individual datasets using five-fold cross-validation and testing on a completely unseen separate test set to simulate real-world settings revealed that the proposed edge-assisted filters, especially the Canny filter, significantly improve segmentation performance metrics. Among all U-Net variants, deep attention U-Net achieved the highest performance using a consistent channel combination, attaining a dice coefficient of 0.9815, a Jaccard score of 0.9624 on the JSRT dataset. The proposed approach achieved a dice gain of +0.0011 to +0.0034 and an IoU gain of +0.009 to +0.0034 across the three datasets compared with the baseline configuration using original images. Furthermore, cross-dataset validation also depicted improvement gains in dice, IoU, and other metrics, demonstrating the generalizability of the proposed approach. Further cross-dataset validation confirmed the framework's strong generalization capabilities, demonstrating stable performance even on unseen domains. These findings validate the proposed method as a reliable and effective solution for LFS in diverse and clinically challenging environments, offering promising benefits for respiratory diagnosis and patient care.

Effective segmentation of cervical cancer tissue from magnetic resonance (MR) images is crucial for automatic detection, staging, and treatment planning of cervical cancer. This study develops an innovative deep learning model to enhance the automatic segmentation of cervical cancer lesions. We obtained 4063 T2WI small-field sagittal, coronal, and oblique axial images from 222 patients with pathologically confirmed cervical cancer. Using this dataset, we employed a convolutional neural network (CNN) along with TransUnet models for segmentation training and evaluation of cervical cancer tissues. In this approach, CNNs are leveraged to extract local information from MR images, whereas Transformers capture long-range dependencies related to shape and structural information, which are critical for precise segmentation. Furthermore, we developed three distinct segmentation models based on coronal, axial, and sagittal T2WI within a small field of view using multidirectional MRI techniques. The dice similarity coefficient (DSC) and mean Hausdorff distance (AHD) were used to assess the performance of the models in terms of segmentation accuracy. The average DSC and AHD values obtained using the TransUnet model were 0.7628 and 0.8687, respectively, surpassing those obtained using the U-Net model by margins of 0.0033 and 0.3479, respectively. The proposed TransUnet segmentation model significantly enhances the accuracy of cervical cancer tissue delineation compared to alternative models, demonstrating superior performance in overall segmentation efficacy. This methodology can improve clinical diagnostic efficiency as an automated image analysis tool tailored for cervical cancer diagnosis.

Aiming at solving the problems of limited training data, single input information, and limited diagnostic accuracy under the influence of strong background noise in fault diagnosis of rotating machinery, this paper proposes a fault diagnosis method based on the combination of discriminant correlation analysis (DCA) and convolutional neural network (CNN). Firstly, the original vibration signal is divided into several segments in the time domain, and the training data is directly processed by one CNN branch to extract multi-scale time domain features. Simultaneously, the divided data is subjected to discrete wavelet transform (DWT), and processed by another branch of CNN to extract multi-scale time-frequency features. Then, the DCA feature fusion mechanism is adopted to fuse the two-domain features extracted in the parallel branches to improve the model’ detection ability. Finally, the fused features are input into the deep CNN for training and learning to extract new features and output the classification results. Through the experimental analysis of two different types of data, the results show that the proposed method can be used for fault diagnosis of rotating machinery effectively. Compared with the single CNN network, the proposed method combines the multi-domain multi-scale feature extraction module with the DCA feature fusion module to enrich the feature information extraction ability. At the same time, the network performance is improved to get higher fault classification accuracy higher.

The classification of hyperspectral images (HSIs) is one of the most popular topics in the remote sensing community. Numerous feature extraction methods have been proposed to improve the classification accuracy of HSIs. Recently, deep features extracted by convolution neural network (CNN) have been introduced into the classification process of HSIs. Due to the nonlinear and invariant advantages of the features, CNN methods provide a powerful tool for representing geographic objects and classifying HSIs. However, traditional deep features only extracted at pixel-level and often neglect multiscale characteristics of geographic objects. In this study, a new deep feature extraction method is proposed, which takes advantage of multi-scale object analysis and the CNN model. Firstly, multiscale image objects are obtained by the multiscale segmentation algorithm and multiscale low-level features of objects are extracted. Secondly, the CNN is devoted to obtain deep features from low-level object features at each scale, respectively. Thirdly, the obtained deep features at all scales are stacked and fed to one fully connected layer to extract the multiscale deep learning features for classification. Finally, the logistic regression classifier is applied to hyperspectral image (HSI) classification based on object-oriented multiscale deep features. The proposed method was carried out on three widely used hyperspectral data sets: University of Pavia, Salinas, and Washington DC. The results reveal that the proposed method provides better results than other state-of-the-art methods.

We propose an automated platform for extra-coronary calcification detection on low dose CT scans. We utilize faster regional convolutional neural networks (R-CNN) to directly detect calcifications at the lesion-level without performing vessel extraction. To segment detected calcifications at the voxel-level, we employ holistically nested edge detection (HED). CT scans of 112 vasculitis patients and 3219 images with labeled calcifications were used to develop and evaluate our method. By employing a two-class faster R-CNN, the average precision (AP) increased from 49.2% to 84.4% for calcification detection. In addition, sensitivity of 85.0% at 1 false positive per image was observed. The Dice Similarity Coefficient (DSC) for calcification segmentation using HED (0.83±0.08) was significantly better (p≪0.01) than the traditional threshold-based method (0.59±0.26).

Wind turbine rolling bearings are crucial components for ensuring the reliability and stability of wind power systems. Their failure can lead to significant economic losses and equipment downtime. Therefore, the accurate diagnosis of bearing faults is of great importance. Although existing deep learning fault diagnosis methods have achieved certain results, they still face limitations such as inadequate feature extraction capabilities, insufficient generalization to complex working conditions, and ineffective multi-scale feature capture. To address these issues, this paper proposes an advanced fault diagnosis method named the two-stream feature fusion convolutional neural network (TSFFResNet-Net). Firstly, the proposed method combines the advantages of one-dimensional convolutional neural networks (1D-ResNet) and two-dimensional convolutional neural networks (2D-ResNet). It transforms one-dimensional vibration signals into two-dimensional images through the empirical wavelet transform (EWT) method. Then, parallel convolutional kernels in 1D-ResNet and 2D-ResNet are used to extract multi-scale features, respectively. Next, the Convolutional Block Attention Module (CBAM) is introduced to enhance the network’s ability to capture key features by focusing on important features in specific channels or spatial areas. After feature fusion, CBAM is introduced again to further enhance the effect of feature fusion, ensuring that the features extracted by different network branches can be effectively integrated, ultimately providing more accurate input features for the classification task of the fully connected layer. The experimental results demonstrate that the proposed method outperforms other traditional methods and advanced convolutional neural network models on different datasets. Compared with convolutional neural network models such as LeNet-5, AlexNet, and ResNet, the proposed method achieves a significantly higher accuracy on the test set, with a stable accuracy of over 99%. Compared with other models, it shows better generalization and stability, effectively improving the overall performance of rolling bearing vibration signal fault diagnosis. The method provides an effective solution for the intelligent fault diagnosis of wind turbine rolling bearings.

In general, most of the existing convolutional neural network (CNN)-based deep-learning models suffer from spatial-information loss and inadequate feature-representation issues. This is due to their inability to capture multiscale-context information and the exclusion of semantic information throughout the pooling operations. In the early layers of a CNN, the network encodes simple semantic representations, such as edges and corners, while, in the latter part of the CNN, the network encodes more complex semantic features, such as complex geometric shapes. Theoretically, it is better for a CNN to extract features from different levels of semantic representation because tasks such as classification and segmentation work better when both simple and complex feature maps are utilized. Hence, it is also crucial to embed multiscale capability throughout the network so that the various scales of the features can be optimally captured to represent the intended task. Multiscale representation enables the network to fuse low-level and high-level features from a restricted receptive field to enhance the deep-model performance. The main novelty of this review is the comprehensive novel taxonomy of multiscale-deep-learning methods, which includes details of several architectures and their strengths that have been implemented in the existing works. Predominantly, multiscale approaches in deep-learning networks can be classed into two categories: multiscale feature learning and multiscale feature fusion. Multiscale feature learning refers to the method of deriving feature maps by examining kernels over several sizes to collect a larger range of relevant features and predict the input images’ spatial mapping. Multiscale feature fusion uses features with different resolutions to find patterns over short and long distances, without a deep network. Additionally, several examples of the techniques are also discussed according to their applications in satellite imagery, medical imaging, agriculture, and industrial and manufacturing systems.