Image-free single-pixel semantic segmentation for complex scene based on multi-scale U-Net

This article addresses the semantic segmentation of laparoscopic surgery images, placing special emphasis on the segmentation of structures with a smaller number of observations. As a result of this study, adjustment parameters are proposed for deep neural network architectures, enabling a robust segmentation of all structures in the surgical scene. The U-Net architecture with five encoder-decoders (U-Net5ed), SegNet-VGG19, and DeepLabv3+ employing different backbones are implemented. Three main experiments are conducted, working with Rectified Linear Unit (ReLU), Gaussian Error Linear Unit (GELU), and Swish activation functions. The applied loss functions include Cross Entropy (CE), Focal Loss (FL), Tversky Loss (TL), Dice Loss (DiL), Cross Entropy Dice Loss (CEDL), and Cross Entropy Tversky Loss (CETL). The performance of Stochastic Gradient Descent with momentum (SGDM) and Adaptive Moment Estimation (Adam) optimizers is compared. It is qualitatively and quantitatively confirmed that DeepLabv3+ and U-Net5ed architectures yield the best results. The DeepLabv3+ architecture with the ResNet-50 backbone, Swish activation function, and CETL loss function reports a Mean Accuracy (MAcc) of 0.976 and Mean Intersection over Union (MIoU) of 0.977. The semantic segmentation of structures with a smaller number of observations, such as the hepatic vein, cystic duct, Liver Ligament, and blood, verifies that the obtained results are very competitive and promising compared to the consulted literature. The proposed selected parameters were validated in the YOLOv9 architecture, which showed an improvement in semantic segmentation compared to the results obtained with the original architecture.

Semantic segmentation of weed and crop images is a key component and prerequisite for automated weed management. For weeds in unmanned aerial vehicle (UAV) images, which are usually characterized by small size and easily confused with crops at early growth stages, existing semantic segmentation models have difficulties to extract sufficiently fine features. This leads to their limited performance in weed and crop segmentation of UAV images. We proposed a fine-grained feature-guided UNet, named FG-UNet, for weed and crop segmentation in UAV images. Specifically, there are two branches in FG-UNet, namely the fine-grained feature branch and the UNet branch. In the fine-grained feature branch, a fine feature-aware (FFA) module was designed to mine fine features in order to enhance the model's ability to segment small objects. In the UNet branch, we used an encoder-decoder structure to realize high-level semantic feature extraction in images. In addition, a contextual feature fusion (CFF) module was designed for the fusion of the fine features and high-level semantic features, thus enhancing the feature discrimination capability of the model. The experimental results showed that our proposed FG-UNet, achieved state-of-the-art performance compared to other semantic segmentation models, with mean intersection over union (MIOU) and mean pixel accuracy (MPA) of 88.06% and 92.37%, respectively. The proposed method in this study lays a solid foundation for accurate detection and intelligent management of weeds. It will have a positive impact on the development of smart agriculture. © 2024 Society of Chemical Industry.

Study of bubble behavior in a gas–solid dense-phase fluidized bed based on deep learning

FRET-SAM: SAM_Med2D-based automatic FRET two-hybrid analysis.

Ultrasound imaging can distinctly display the morphology and structure of internal organs within the human body, enabling the examination of organs like the breast, liver, and thyroid. It can identify the locations of tumors, nodules, and other lesions, thereby serving as an efficacious tool for treatment detection and rehabilitation evaluation. Typically, the attending physician is required to manually demarcate the boundaries of lesion locations, such as tumors, in ultrasound images. Nevertheless, several issues exist. The high noise level in ultrasound images, the degradation of image quality due to the impact of surrounding tissues, and the influence of the operator’s experience and proficiency on the determination of lesion locations can all contribute to a reduction in the accuracy of delineating the boundaries of lesion sites. In the wake of the advancement of deep learning, its application in medical image segmentation is becoming increasingly prevalent. For instance, while the U-Net model has demonstrated a favorable performance in medical image segmentation, the convolution layers of the traditional U-Net model are relatively simplistic, leading to suboptimal extraction of global information. Moreover, due to the significant noise present in ultrasound images, the model is prone to interference. In this research, we propose an Attention Residual Network model (ARU-Net). By incorporating residual connections within the encoder section, the learning capacity of the model is enhanced. Additionally, a spatial hybrid convolution module is integrated to augment the model’s ability to extract global information and deepen the vertical architecture of the network. During the feature fusion stage of the skip connections, a channel attention mechanism and a multi-convolutional self-attention mechanism are respectively introduced to suppress noisy points within the fused feature maps, enabling the model to acquire more information regarding the target region. Finally, the predictive efficacy of the model was evaluated using publicly accessible breast ultrasound and thyroid ultrasound data. The ARU-Net achieved mean Intersection over Union (mIoU) values of 82.59% and 84.88%, accuracy values of 97.53% and 96.09%, and F1-score values of 90.06% and 89.7% for breast and thyroid ultrasound, respectively.

Deep-sea mining vehicles rely on forward-looking sonar systems to collect critical environmental data for mapping, navigation, and operational safety. Real-time semantic segmentation of seabed terrain from sonar images is essential for accurately interpreting the underwater environment. However, these images present several challenges, including low resolution, substantial noise, and indistinct textures and shapes. Additionally, seabed terrain features vary across multiple spatial scales, which complicates the extraction of meaningful information using conventional image processing techniques. Such limitations hinder the precision and reliability of mapping efforts necessary for effective deep-sea operations. To address these challenges, a multi-scale semantic segmentation model has been developed specifically for sonar images. The model incorporates a downsampling module based on Discrete Wavelet Transform (WT-DM), which suppresses noise while preserving critical terrain features, ensuring clearer input for subsequent analysis. Moreover, a TriScale Attention Module (TAM) is introduced to capture features at varying spatial resolutions. This module enhances segmentation performance by allowing the model to focus on relevant patterns across different scales. In the absence of publicly available datasets, experiments were conducted using a custom-built seabed terrain dataset. The proposed model achieved a precision of 82.4% and a mean intersection over union (MIoU) of 79.6%, demonstrating its effectiveness. Comparative evaluations with other mainstream models further confirmed the superior performance and competitiveness of the proposed approach.

Accurate extraction of the therapeutic target region in the monitoring ultrasonic image is necessary for high intensity focused ultrasound (HIFU) therapy. However, due to the interference of complex background and large amount of noise in the ultrasonic image, traditional image segmentation algorithms have limitations on the target region segmentation of ultrasonic image. In order to extract the target region in ultrasonic image, a semantic segmentation method combining improved DeepLabv3+ network and improved loss function is proposed. On the basis of DeepLabv3+ network, dense atrous spatial pyramid pooling (DenseASPP) enhanced by attention mechanism through series connection of DenseASPP structure and channel attention mechanism is added into the DeepLabv3+ encoder, and the feature map weighting module is added into the DeepLabv3+ decoder to improve the network ability of feature extraction and the segmentation accuracy of the target region edge. In terms of the loss function, the improved loss obtained by combining the Huber loss and the noise-robust Dice loss is used to solve the proportional imbalance problem of multi-category pixel number, while improving the robustness against noise. The ablation experimental results show that the mean intersection over union (MIoU) of segmentation results with improved DeepLabv3+ network and with improved DeepLabv3+ network trained by the improved loss function increase by 0.46 percent and 0.99 percent respectively compared with the original DeepLabv3+ network. Proposed method effectively improves the segmentation accuracy of the target region in the monitoring ultrasonic image for HIFU therapy and has strong robustness against noise.

To accurately evaluate residual plastic film pollution in pre-sowing cotton fields, a method based on modified U-Net model was proposed in this research. Images of pre-sowing cotton fields were collected using UAV imaging from different heights under different weather conditions. Residual films were manually labelled, and the degree of residual film pollution was defined based on the residual film coverage rate. The modified U-Net model for evaluating residual film pollution was built by simplifying the U-Net model framework and introducing the inception module, and the evaluation results were compared to those of the U-Net, SegNet, and FCN models. The segmentation results showed that the modified U-Net model had the best performance, with a mean intersection over union (MIOU) of 87.53%. The segmentation results on images of cloudy days were better than those on images of sunny days, with accuracy gradually decreasing with increasing image-acquiring height. The evaluation results of residual film pollution showed that the modified U-Net model outperformed the other models. The coefficient of determination(R2), root mean square error (RMSE), mean relative error (MRE) and average evaluation time per image of the modified U-Net model on the CPU were 0.9849, 0.0563, 5.33% and 4.85 s, respectively. The results indicate that UAV imaging combined with the modified U-Net model can accurately evaluate residual film pollution. This study provides technical support for the rapid and accurate evaluation of residual plastic film pollution in pre-sowing cotton fields.

In interpreting a scene for numerous applications, including autonomous driving and robotic navigation, semantic segmentation is crucial. Compared to single-modal data, multi-modal data allow us to extract a richer set of features, which is the benefit of improving segmentation accuracy and effect. We propose a point cloud semantic segmentation method, and a fusion graph convolutional network (FGCN) which extracts the semantic information of each point involved in the two-modal data of images and point clouds. The two-channel k-nearest neighbors (KNN) module of the FGCN was created to address the issue of the feature extraction's poor efficiency by utilizing picture data. Notably, the FGCN utilizes the spatial attention mechanism to better distinguish more important features and fuses multi-scale features to enhance the generalization capability of the network and increase the accuracy of the semantic segmentation. In the experiment, a self-made semantic segmentation KITTI (SSKIT) dataset was made for the fusion effect. The mean intersection over union (MIoU) of the SSKIT can reach 88.06%. As well as the public datasets, the S3DIS showed that our method can enhance data features and outperform other methods: the MIoU of the S3DIS can reach up to 78.55%. The segmentation accuracy is significantly improved compared with the existing methods, which verifies the effectiveness of the improved algorithms.

In recent years, the level of science and technology in China has been significantly improved. Digital image processing technology is the product of the times with the rapid development of science and technology, and has been applied in many fields. Digital image processing technology includes many practical technologies, and image segmentation technology is one of them. At present, the widely used pattern recognition technology is realized by image segmentation technology. Based on this, this paper mainly studies the application of image segmentation technology in digital image processing. This paper first introduces the methods of image segmentation, including threshold segmentation, clustering segmentation and edge detection segmentation. Secondly, according to a need to extract a road image for image segmentation, and achieved very good results. Finally, this paper mainly analyzes the application of image segmentation in digital image processing. It is found that image segmentation has good application in automatic license plate recognition, biomedical engineering, remote sensing engineering and fire prevention and detection.

As an important link between human and nature, land use/land cover change (LUCC) has become a research hotspot since the first day of the issue initiated. Remote sensing observation supplies important data source for LUCC research. With the constant improvement of the remote sensed image spatial resolution and the continuous development of computer scientific algorithms and models, the study of semantic segmentation in the field of automatic interpretation of remote sensed images has promoted quickly. There have been a number of achievements focusing on the one-class extraction of LUCC classification system, but limited progress in multi-class classifying mission when facing high resolution remote sensing data by using semantic segmentation model, because of its sophisticated structure, the phenomenon of synonyms spectrum and foreign body with spectrum in images. Additionally, the limitation of the semantic segmentation model is a crucial issue which needs to be solv for precision of classification in class recognition and the perfection of semantic segmentation on pixel level as well. The High-Resolution network (HRnet) model was used and improved in the study for semantic segmentation because it has the advantage of low resolution loss. In the LUCC classification work, the distribution of classes (such as cultivated land, woodland, grassland, other land, water, etc.) are usually extremely imbalanced, that leads to the limited classification recognition results of land types when using HRnet. So a combined Loss function of Lovasz Softmax Loss+Cross Entropy Loss was used in HRnet to classify the land cover of remote sensing images, to meet the requirements of high-resolution image classification not only in terms of low resolution loss but also in terms of class recognition. The combined loss function provided the direction of the model, and the appropriate loss function improved the performance of the model when faced with the imbalanced distribution of land classes. The results are as follows: 1) Compared with Deeplab and Unet, HRnet model performed better in segmentation results; 2) HRnet model with combined Loss function Lovasz Softmax Loss+Cross Entropy Loss (HRnet+) had better performance, and its test accuracy PA(Pixel Accuracy) value was 0.9577, MIOU (Mean Intersection over Union) value was 0.8801, and Kappa value was 0.9455. This indicated that the HRnet model with low loss of resolution could achieve better results in the automatic segmentation of high-resolution remote sensing images, and the combined loss function constructed in this study could effectively solve the problem of imbalanced distribution of classes in remote sensing images, and the edge optimization ability was also more significant.

Need for medical imaging has immensely increased to diagnose functionality of organs and tissues. Application of magnetic resonance imaging (MRI) or computerized tomography scan (CT scan) has automated the process of disease prediction which otherwise would need manual intervention of experienced physicians. Critical transplants and tumor diagnosis of kidney, heart and liver could be very challenging without the application of high-resolution medical imaging. Semantic image segmentation helps to identify features within the image and label them to demarcate the background from organs and tumors formed on them. Current proposed model is built by applying image segmentation using deep learning models by training 3D image information obtained from KITS-19 using CNN U-Net architecture to segregate essential tumor details from the kidney background images. Considering the evaluation of medical condition based on imagery is very challenging due to lack of pre-processing methods to handle unbalanced nature of class distribution which is common in medical imaging. In the current research, we apply state-of-the-art convolution neural network and deep learning models to predict the tumor in MRI imagery. We performed evaluation of a subset of 60 CT scans with slice thickness values obtained from KiTS19 dataset. We also compared the image segmentation model with BRATS’2013 challenging dataset also to validate the prediction of tumor in brain MRI scan images. Performance evaluation of kidney and tumor features obtained through threefold cross validation comparing with ground truth gave us dice coefficient score of 0.96 while tumor segmentation prediction was close to 0.80. We believe the performance can be increased through higher GPU memory requirements, which was a limitation to our existing GPU hardware. Nevertheless, the possibility of applying improved deep learning model based on existing results and can be a promising step forward in medical imaging segmentation.

Robust Prediction of Isodose Distribution with a Fully Convolutional Networks (FCN)-Based Deep Learning Model

To solve the problems of rough edge and poor segmentation accuracy of traditional neural networks in small nucleus image segmentation, a nucleus image segmentation technology based on U-Net network is proposed. First, the U-Net network is used to segment the nucleus image, which stitches the feature images in the channel dimension to achieve feature fusion, and the skip structure is used to combine the low- and high-level features. Then, the subregional average pooling is proposed to improve the global average pooling in the attention module, and an attention channel expansion module is designed to improve the accuracy of image segmentation. Finally, the improved attention module is integrated into the U-Net network to achieve accurate segmentation of the nuclear image. Based on the Python platform, the experimental results show that the proposed segmentation technology can achieve fast convergence, and the mean intersection over union (MIoU) is 85.02%, which is better than other comparison technologies and has a good application prospect.

Obtaining animal regions and the relative position relationship of animals in the scene is conducive to further studying animal habits, which is of great significance for smart animal farming. However, the complex breeding environment still makes detection difficult. To address the problems of poor target segmentation effects and the weak generalization ability of existing semantic segmentation models in complex scenes, a semantic segmentation model based on an improved DeepLabV3+ network (Imp-DeepLabV3+) was proposed. Firstly, the backbone network of the DeepLabV3+ model was replaced by MobileNetV2 to enhance the feature extraction capability of the model. Then, the layer-by-layer feature fusion method was adopted in the Decoder stage to integrate high-level semantic feature information with low-level high-resolution feature information at multi-scale to achieve more precise up-sampling operation. Finally, the SENet module was further introduced into the network to enhance information interaction after feature fusion and improve the segmentation precision of the model under complex datasets. The experimental results demonstrate that the Imp-DeepLabV3+ model achieved a high pixel accuracy (PA) of 99.4%, a mean pixel accuracy (MPA) of 98.1%, and a mean intersection over union (MIoU) of 96.8%. Compared to the original DeepLabV3+ model, the segmentation performance of the improved model significantly improved. Moreover, the overall segmentation performance of the Imp-DeepLabV3+ model surpassed that of other commonly used semantic segmentation models, such as Fully Convolutional Networks (FCNs), Lite Reduced Atrous Spatial Pyramid Pooling (LR-ASPP), and U-Net. Therefore, this study can be applied to the field of scene segmentation and is conducive to further analyzing individual information and promoting the development of intelligent animal farming.