3D MEDICAL IMAGE SEGMENTATION WITH DEEP LEARNING METHODS

Corrosion is a natural phenomenon that causes significant material and economic losses. Timely inspection of corroded assets is vital for preventing such losses. Visual inspection is a non-destructive method that is commonly used for corrosion detection. The ability of deep learning methods to capture corrosion visual information allows for the automation of the manual inspection process. The literature includes several examples of deep learning models to detect metal corrosion. In developing such models, loss functions are crucial and directly influence the models’ performance. Although cross-entropy is a commonly used objective function for corrosion detection, systematic evaluation of alternative loss functions—and the performance gains they may unlock—has been largely neglected. This study uses eight loss functions, namely, cross-entropy loss, Focal loss, Unified Focal loss, Dice loss, Tversky loss, Focal Tversky loss, Combo loss, and Hybrid Focal loss, to train UNet-based models for corrosion segmentation and compares their performance. The results demonstrated that the model trained with the Dice loss achieved higher performance, with a mean Jaccard index value of 64.28%, while the Focal loss exhibited the lowest performance, with a mean Jaccard value of 57.20%. This study contributes to academia and practice by demonstrating the performance gains that can be achieved by tweaking the deep learning models using optimal loss functions. • Corrosion is a phenomenon that considerably impacts the environment and economy. • Deep learning models can be trained for corrosion detection, and loss functions are essential to developing accurate models. • This study evaluates the use of different loss functions, including the Focal loss, Unified Focal loss, Dice loss, Tversky loss, Focal Tversky loss, Combo loss, and Hybrid Focal loss, to train UNet-based models for corrosion segmentation. • On a standard dataset, the models developed with Combo loss, Tversky loss, Unified Focal loss, and Dice loss had higher performance than those developed with cross-entropy loss. • With the hyperparameter tuning, higher performance gains can be achieved.

Three-dimensional (3D) medical image segmentation plays a crucial role in medical care applications. Although various two-dimensional (2D) and 3D neural network models have been applied to 3D medical image segmentation and achieved impressive results, a trade-off remains between efficiency and accuracy. To address this issue, a novel mixture convolutional network (MixConvNet) is proposed, in which traditional 2D/3D convolutional blocks are replaced with novel MixConv blocks. In the MixConv block, 3D convolution is decomposed into a mixture of 2D convolutions from different views. Therefore, the MixConv block fully utilizes the advantages of 2D convolution and maintains the learning ability of 3D convolution. It acts as 3D convolutions and thus can process volumetric input directly and learn intra-slice features, which are absent in the traditional 2D convolutional block. By contrast, the proposed MixConv block only contains 2D convolutions; hence, it has significantly fewer trainable parameters and less computation budget than a block containing 3D convolutions. Furthermore, the proposed MixConvNet is pre-trained with small input patches and fine-tuned with large input patches to improve segmentation performance further. In experiments on the Decathlon Heart dataset and Sliver07 dataset, the proposed MixConvNet outperformed the state-of-the-art methods such as UNet3D, VNet, and nnUnet.

Lesion detection or segmentation on 3D medical image (e.g., MRI, CT-PET or hybrid modal) is a challenging task which aims to automatically mark the abnormal region on given input image. The methods to tackle the problem often construct a U-shape autoencoders to map the input to a compressed vector in latent space and predict the final probability map. However, approaches for 3D multimodal image segmentation are still not fully explored and localization accuracy can be improved by designing proper structure based on prior. In this paper, we propose a supervised contrastive approach for 3D multi-modality medical image segmentation. A hybrid 3D-2D multi-modality fusion model is designed based on the input image prior. The 3D segmentation problem is converted to a sequence prediction task by 2D decoder with contrastive modeling. The combined framework (HDC, Hybrid Dimension Contrastive) achieves comparable or even greater performance to 3D counterpart while memory efficient. We conduct extensive experiments on two multimodal datasets of brain image BraTS and our epilepsy MRI-PET. The validation results show the effectiveness of our framework.

Three-dimensional (3D) medical image segmentation is used to segment the target (a lesion or an organ) in 3D medical images. Through this process, 3D target information is obtained; hence, this technology is an important auxiliary tool for medical diagnosis. Although some methods have proved to be successful for two-dimensional (2D) image segmentation, their direct use in the 3D case has been unsatisfactory. To obtain more precise tumor segmentation results from 3D MR images, in this paper, we propose a method known as the 3D shape-weighted level set method (3D-SLSM). The proposed method first converts the LSM, which is superior with respect to 2D image segmentation, into a 3D algorithm that is suitable for overall calculations in 3D image models, and which improves the efficiency and accuracy of calculations. A 3D shape-weighted value is then added for each 3D-SLSM iterative process according to the changes in volume. Besides increasing the convergence rate and eliminating background noise, this shape-weighted value also brings the segmented contour closer to the actual tumor margins. To perform a quantitative analysis of 3D-SLSM and to examine its feasibility in clinical applications, we have divided our experiments into computer-simulated sequence images and actual breast MRI cases. Subsequently, we simultaneously compared various existing 3D segmentation methods. The experimental results demonstrated that 3D-SLSM exhibited precise segmentation results for both types of experimental images. In addition, 3D-SLSM showed better results for quantitative data compared with existing 3D segmentation methods.

3D convolution neural networks (CNN) have been proved very successful in parsing organs or tumours in 3D medical images, but it remains sophisticated and time-consuming to choose or design proper 3D networks given different task contexts. Recently, Neural Architecture Search (NAS) is proposed to solve this problem by searching for the best network architecture automatically. However, the inconsistency between search stage and deployment stage often exists in NAS algorithms due to memory constraints and large search space, which could become more serious when applying NAS to some memory and time-consuming tasks, such as 3D medical image segmentation. In this paper, we propose a coarse-to-fine neural architecture search (C2FNAS) to automatically search a 3D segmentation network from scratch without inconsistency on network size or input size. Specifically, we divide the search procedure into two stages: 1) the coarse stage, where we search the macro-level topology of the network, i.e. how each convolution module is connected to other modules; 2) the fine stage, where we search at micro-level for operations in each cell based on previous searched macro-level topology. The coarse-to-fine manner divides the search procedure into two consecutive stages and meanwhile resolves the inconsistency. We evaluate our method on 10 public datasets from Medical Segmentation Decalthon (MSD) challenge, and achieve state-of-the-art performance with the network searched using one dataset, which demonstrates the effectiveness and generalization of our searched models.

The understanding of narrative stories by computer is an important task for their automatic generation. To date, high-performance neural-network technologies such as BERT have been applied to tasks such as the Story Cloze Test and Story Completion. In this study, we focus on the text segmentation of novels into paragraphs, which is an important writing technique for readers to deepen their understanding of the texts. This type of segmentation, which we call “paragraph boundary recognition”, can be considered to be a binary classification problem in terms of the presence or absence of a boundary, such as a paragraph between target sentences. However, in this case, the data imbalance becomes a bottleneck because the number of paragraphs is generally smaller than the number of sentences. To deal with this problem, we introduced several cost-sensitive loss functions, namely. focal loss, dice loss, and anchor loss, which were robust for imbalanced classification in BERT. In addition, introducing the threshold-moving technique into the model was effective in estimating paragraph boundaries. As a result of the experiment on three newly created datasets, BERT with dice loss and threshold moving obtained a higher F1 than the original BERT had using cross-entropy loss as its loss function (76% to 80%, 50% to 54%, 59% to 63%).

Introduction: Precise segmentation of the liver from computed tomography (CT) scans is an essential component in the diagnosis and treatment of liver diseases such as hepatocellular carcinoma. Manual segmentation can be time-consuming and subject to variability, spurring the application of automated systems based on deep learning. We introduce a 3D U-Net-based convolutional neural network for end-to-end segmentation of the liver from volumetric CT images. The model is trained and tested on the LiTS dataset using a standard preprocessing pipeline and an in-house composite loss function—AdvancedComboLoss—combining Dice Loss, Binary Cross-Entropy, and Focal Tversky Loss to balance class imbalance and boundary accuracy. The network handles entire 3D volumes with spatial context retained between anatomical planes and is trained on a GPU-enabled setup with the PyTorch environment. Objectives: To develop a highly accurate and computationally efficient 3D U-Net-based deep learning model for automated liver segmentation from volumetric CT scans, addressing challenges like class imbalance, anatomical variability, and low tissue contrast, using a custom loss function (AdvancedComboLoss) and a standardized preprocessing pipeline. Methods: The methodology adopted is volumetric liver segmentation from a specially designed 3D U-Net architecture specific to medical CT scans. The process entails preprocessing of raw CT scans into a common shape and intensity range and training a fully convolutional 3D neural network that learns the spatial relationships among all three anatomical planes. The model uses an encoder-decoder architecture with skip connections to maintain fine-grained structural information, allowing for accurate delineation of liver regions. A hybrid loss function—integrating Dice, Binary Cross-Entropy, and Focal Tversky losses—is used to direct the learning process to optimize overlap of regions, accuracy of classification, and class imbalance. Results: Experimental outcomes demonstrate that the model has a Dice score of 0.9604 and voxel-wise accuracy of 0.9979, indicating excellent performance in segmentation as well as strong agreement with expert annotations. The model presented in this paper presents a solid, scalable, and clinically valuable approach to automated liver segmentation with future potential to integrate into multi-stage liver and tumor analysis workflows. Conclusions: Herein, we built and tested a 3D U-Net deep learning architecture for unsupervised liver segmentation from abdominal CT volumes. Through utilization of volumetric convolutions as well as through a well-engineered composite loss function, the model was successful in capturing complicated liver anatomy as well as being able to hold high performance even on varied CT volumes. The power of the method is not just in its quantitative performance but also in its capability to generate anatomically consistent and visually plausible segmentations, making it extremely relevant to actual clinical workflows. Although there are some limitations—constrained batch size and absence of data augmentation—there is room for improvement, the findings affirm that deep learning can be an effective tool for automating key tasks in medical image analysis. As a whole, this paper adds a robust and effective segmentation model that would be potentially embedded in wider diagnostic workflows, ending up improving the accuracy, speed, and scalability of liver evaluation in clinical practice. In brief, this paper adds a high-fidelity, scalable, and computationally effective solution for computer-aided liver segmentation, pushing the application of artificial intelligence in clinical decision-making and medical image analysis.

Recently, 3D medical image reconstruction (MIR) and segmentation (MIS) based on deep neural networks have been developed with promising results, and attention mechanism has been further designed for performance enhancement. However, the large size of 3D volume images poses a great computational challenge to traditional attention methods. In this paper, we propose a folded attention (FA) approach to improve the computational efficiency of traditional attention methods on 3D medical images. The main idea is that we apply tensor folding and unfolding operations to construct four small sub-affinity matrices to approximate the original affinity matrix. Through four consecutive sub-attention modules of FA, each element in the feature tensor can aggregate spatial-channel information from all other elements. Compared to traditional attention methods, with the moderate improvement of accuracy, FA can substantially reduce the computational complexity and GPU memory consumption. We demonstrate the superiority of our method on two challenging tasks for 3D MIR and MIS, which are quantitative susceptibility mapping and multiple sclerosis lesion segmentation.

Deep-learning has proved in recent years to be a powerful tool for image analysis and is now widely used to segment both 2D and 3D medical images. Deep-learning segmentation frameworks rely not only on the choice of network architecture but also on the choice of loss function. When the segmentation process targets rare observations, a severe class imbalance is likely to occur between candidate labels, thus resulting in sub-optimal performance. In order to mitigate this issue, strategies such as the weighted cross-entropy function, the sensitivity function or the Dice loss function, have been proposed. In this work, we investigate the behavior of these loss functions and their sensitivity to learning rate tuning in the presence of different rates of label imbalance across 2D and 3D segmentation tasks. We also propose to use the class re-balancing properties of the Generalized Dice overlap, a known metric for segmentation assessment, as a robust and accurate deep-learning loss function for unbalanced tasks.

Probability calibration and decision threshold selection are fundamental aspects of risk prediction and classification, respectively. A strictly proper loss function is used in clinical risk prediction applications to encourage a model to predict calibrated class-posterior probabilities or risks. Recent studies have shown that training with focal loss can improve the discriminatory power of gradient-boosted decision trees (GBDT) for classification tasks with an imbalanced or skewed class distribution. However, the focal loss function is not a strictly proper loss function. Therefore, the output of GBDT trained using focal loss is not an accurate estimate of the true class-posterior probability. This study aims to address the issue of poor calibration of GBDT trained using focal loss in the context of clinical risk prediction applications. The methodology utilizes a closed-form transformation of the confidence scores of GBDT trained with focal loss to estimate calibrated risks. The closed-form transformation relates the focal loss minimizer and the true-class posterior probability. Algorithms based on Bayesian hyperparameter optimization are provided to choose the focal loss parameter that optimizes discriminatory power and calibration, as measured by the Brier score metric. We assess how the calibration of the confidence scores affects the selection of a decision threshold to optimize the balanced accuracy, defined as the arithmetic mean of sensitivity and specificity. The effectiveness of the proposed strategy was evaluated using lung transplant data extracted from the Scientific Registry of Transplant Recipients (SRTR) for predicting post-transplant cancer. The proposed strategy was also evaluated using data from the Behavioral Risk Factor Surveillance System (BRFSS) for predicting diabetes status. Probability calibration plots, calibration slope and intercept, and the Brier score show that the approach improves calibration while maintaining the same discriminatory power according to the area under the receiver operating characteristics curve (AUROC) and the H-measure. The calibrated focal-aware XGBoost achieved an AUROC, Brier score, and calibration slope of 0.700, 0.128, and 0.968 for predicting the 10-year cancer risk, respectively. The miscalibrated focal-aware XGBoost achieved equal AUROC but a worse Brier score and calibration slope (0.140 and 1.579). The proposed method compared favorably to the standard XGBoost trained using cross-entropy loss (AUROC of 0.755 versus 0.736 in predicting the 1-year risk of cancer). Comparable performance was observed with other risk prediction models in the diabetes prediction task.

Gliomas, the most aggressive primary tumors in the central nervous system, are characterized by high morphological heterogeneity and diffusely infiltrating boundaries. Such complexity poses significant challenges for accurate segmentation in clinical practice. Although deep learning methods have shown promising results, they often struggle to achieve a satisfactory trade-off among precise boundary delineation, robust multi-scale feature representation, and computational efficiency, particularly when processing high-resolution three-dimensional (3D) magnetic resonance imaging (MRI) data. Therefore, the aim of this study is to develop a novel 3D segmentation framework that specifically addresses these challenges, thereby improving clinical utility in brain tumor analysis. To accomplish this, we propose a multi-level channel-spatial attention and light-weight scale-fusion network (MCSLF-Net), which integrates a multi-level channel-spatial attention mechanism (MCSAM) and a light-weight scale-fusion module. By strategically enhancing subtle boundary features while maintaining a compact network design, our approach seeks to achieve high accuracy in delineating complex glioma morphologies, reduce computational burden, and provide a more clinically feasible segmentation solution. We propose MCSLF-Net, a network integrating two key components: (I) MCSAM: by strategically inserting a 3D channel-spatial attention module at critical semantic layers, the network progressively emphasizes subtle, infiltrative edges and small, easily overlooked contours. This avoids reliance on an additional edge detection branch while enabling fine-grained localization in ambiguous transitional regions. (II) Light-weight scale fusion unit (LSFU): leveraging depth-wise separable convolutions combined with multi-scale atrous (dilated) convolutions, LSFU enhances computational efficiency and adapts to varying feature requirements at different network depths. In doing so, it effectively captures small infiltrative lesions as well as extensive tumor areas. By coupling these two modules, MCSLF-Net balances global contextual information with local fine-grained features, simultaneously reducing the computational burden typically associated with 3D medical image segmentation. Extensive experiments on the BraTS 2019, BraTS 2020, and BraTS 2021 datasets validated the effectiveness of our approach. On BraTS 2021, MCSLF-Net achieved a mean Dice similarity coefficient (DSC) of 0.8974 and a mean 95th percentile Hausdorff distance (HD95) of 2.52 mm. Notably, it excels in segmenting intricate transitional areas, including the enhancing tumor (ET) region and the tumor core (TC), thereby demonstrating superior boundary delineation and multi-scale feature fusion capabilities relative to existing methods. These findings underscore the clinical potential of deploying multi-level channel-spatial attention and light-weight multi-scale fusion strategies in high-precision 3D glioma segmentation. By striking an optimal balance among boundary accuracy, multi-scale feature capture, and computational efficiency, the proposed MCSLF-Net offers a practical framework for further advancements in automated brain tumor analysis and can be extended to a range of 3D medical image segmentation tasks.

We propose a generalized focal loss function based on the Tversky index to address the issue of data imbalance in medical image segmentation. Compared to the commonly used Dice loss, our loss function achieves a better trade off between precision and recall when training on small structures such as lesions. To evaluate our loss function, we improve the attention U-Net model by incorporating an image pyramid to preserve contextual features. We experiment on the BUS 2017 dataset and ISIC 2018 dataset where lesions occupy 4.84% and 21.4% of the images area and improve segmentation accuracy when compared to the standard U-Net by 25.7% and 3.6%, respectively.

Hyperspectral image (HSI) classification has been extensively applied for analyzing remotely sensed images. HSI data consist of multiple bands that provide abundant spatial information. Convolutional neural networks (CNNs) have emerged as powerful deep learning methods for processing visual data. In recent work, CNNs have shown impressive results in HSI classification. In this paper, we propose a hierarchical neural network architecture called feature extraction with hybrid spectral CNN (FE-HybridSN) to extract superior spectral–spatial features. FE-HybridSN effectively captures more spectral–spatial information while reducing computational complexity. Considering the prevalent issue of class imbalance in experimental datasets (IP, UP, SV) and real-world hyperspectral datasets, we apply the focal loss to mitigate these problems. The focal loss reconstructs the loss function and facilitates effective achievement of the aforementioned goals. We propose a framework (FEHN-FL) that combines FE-HybridSN and the focal loss for HSI classification and then conduct extensive HSI classification experiments using three remote sensing datasets: Indian Pines (IP), University of Pavia (UP), and Salinas Scene (SV). Using cross-entropy loss as a baseline, we assess the hyperspectral classification performance of various backbone networks and examine the influence of different spatial sizes on classification accuracy. After incorporating focal loss as our loss function, we not only compare the classification performance of the FE-HybridSN backbone network under different loss functions but also evaluate their convergence rates during training. The proposed classification framework demonstrates satisfactory performance compared to state-of-the-art end-to-end deep-learning-based methods, such as 2D-CNN, 3D-CNN, etc.

To train an automatic brain tumor segmentation model, a large amount of data is required. In this paper, we proposed a strategy to overcome the limited amount of clinically collected magnetic resonance image (MRI) data regarding meningiomas by pre-training a model using a larger public dataset of MRIs of gliomas and augmenting our meningioma training set with normal brain MRIs. Pre-operative MRIs of 91 meningioma patients (171 MRIs) and 10 non-meningioma patients (normal brains) were collected between 2016 and 2019. Three-dimensional (3D) U-Net was used as the base architecture. The model was pre-trained with BraTS 2019 data, then fine-tuned with our datasets consisting of 154 meningioma MRIs and 10 normal brain MRIs. To increase the utility of the normal brain MRIs, a novel balanced Dice loss (BDL) function was used instead of the conventional soft Dice loss function. The model performance was evaluated using the Dice scores across the remaining 17 meningioma MRIs. The segmentation performance of the model was sequentially improved via the pre-training and inclusion of normal brain images. The Dice scores improved from 0.72 to 0.76 when the model was pre-trained. The inclusion of normal brain MRIs to fine-tune the model improved the Dice score; it increased to 0.79. When employing BDL as the loss function, the Dice score reached 0.84. The proposed learning strategy for U-net showed potential for use in segmenting meningioma lesions.

Three-dimensional (3D) medical images are prone to overlap, and there are some problems, such as low detection efficiency and inconsistent with the actual situation. Therefore, a 3D medical image surface reconstruction method based on data mining and machine learning is proposed. The 3D medical images were classified according to different ways, the information frame of 3D medical images was established and the surface overlapping information model of 3D images was given. Based on this information framework, the nonlinear function of overlapping area information of 3D medical images was constructed. The weight of the nonlinear function was used to calculate the input and output results of overlapping area information. Combined with the input mode of 3D medical image information, the error between the information output and the expected output was set. The nonlinear function weight of the overlapping area information of 3D medical images was modified by using the learning rate and the use time of the overlapping area information, and the influence factors of the overlapping information detection were obtained by increasing the situation terms, so as to complete the detection of the surface reconstruction information of 3D medical images. The experimental results show that the information detection results of the proposed method fit well with the actual situation, and the information detection efficiency is high.