Automated Crack Detection via Semantic Segmentation Approaches Using Advanced U-Net Architecture

FRET-SAM: SAM_Med2D-based automatic FRET two-hybrid analysis.

Obtaining animal regions and the relative position relationship of animals in the scene is conducive to further studying animal habits, which is of great significance for smart animal farming. However, the complex breeding environment still makes detection difficult. To address the problems of poor target segmentation effects and the weak generalization ability of existing semantic segmentation models in complex scenes, a semantic segmentation model based on an improved DeepLabV3+ network (Imp-DeepLabV3+) was proposed. Firstly, the backbone network of the DeepLabV3+ model was replaced by MobileNetV2 to enhance the feature extraction capability of the model. Then, the layer-by-layer feature fusion method was adopted in the Decoder stage to integrate high-level semantic feature information with low-level high-resolution feature information at multi-scale to achieve more precise up-sampling operation. Finally, the SENet module was further introduced into the network to enhance information interaction after feature fusion and improve the segmentation precision of the model under complex datasets. The experimental results demonstrate that the Imp-DeepLabV3+ model achieved a high pixel accuracy (PA) of 99.4%, a mean pixel accuracy (MPA) of 98.1%, and a mean intersection over union (MIoU) of 96.8%. Compared to the original DeepLabV3+ model, the segmentation performance of the improved model significantly improved. Moreover, the overall segmentation performance of the Imp-DeepLabV3+ model surpassed that of other commonly used semantic segmentation models, such as Fully Convolutional Networks (FCNs), Lite Reduced Atrous Spatial Pyramid Pooling (LR-ASPP), and U-Net. Therefore, this study can be applied to the field of scene segmentation and is conducive to further analyzing individual information and promoting the development of intelligent animal farming.

The plant leaf veins coupling feature representation and measurement method based on DeepLabV3+ is proposed to solve problems of slow segmentation, partial occlusion of leaf veins, and low measurement accuracy of leaf veins parameters. Firstly, to solve the problem of slow segmentation, the lightweight MobileNetV2 is selected as the extraction network for DeepLabV3+. On this basis, the Convex Hull-Scan method is applied to repair leaf veins. Subsequently, a refinement algorithm, Floodfill MorphologyEx Medianblur Morphological Skeleton (F-3MS), is proposed, reducing the burr phenomenon of leaf veins’ skeleton lines. Finally, leaf veins’ related parameters are measured. In this study, mean intersection over union (MIoU) and mean pixel accuracy (mPA) reach 81.50% and 92.89%, respectively, and the average segmentation speed reaches 9.81 frames per second. Furthermore, the network model parameters are compressed by 89.375%, down to 5.813M. Meanwhile, leaf veins’ length and width are measured, yielding an accuracy of 96.3642% and 96.1358%, respectively.

Semantic segmentation of weed and crop images is a key component and prerequisite for automated weed management. For weeds in unmanned aerial vehicle (UAV) images, which are usually characterized by small size and easily confused with crops at early growth stages, existing semantic segmentation models have difficulties to extract sufficiently fine features. This leads to their limited performance in weed and crop segmentation of UAV images. We proposed a fine-grained feature-guided UNet, named FG-UNet, for weed and crop segmentation in UAV images. Specifically, there are two branches in FG-UNet, namely the fine-grained feature branch and the UNet branch. In the fine-grained feature branch, a fine feature-aware (FFA) module was designed to mine fine features in order to enhance the model's ability to segment small objects. In the UNet branch, we used an encoder-decoder structure to realize high-level semantic feature extraction in images. In addition, a contextual feature fusion (CFF) module was designed for the fusion of the fine features and high-level semantic features, thus enhancing the feature discrimination capability of the model. The experimental results showed that our proposed FG-UNet, achieved state-of-the-art performance compared to other semantic segmentation models, with mean intersection over union (MIOU) and mean pixel accuracy (MPA) of 88.06% and 92.37%, respectively. The proposed method in this study lays a solid foundation for accurate detection and intelligent management of weeds. It will have a positive impact on the development of smart agriculture. © 2024 Society of Chemical Industry.

An on-machine tool wear area identification method based on image augmentation and advanced segmentation

This study highlights the importance of unmanned aerial vehicle (UAV) multispectral (MS) imagery for the accurate delineation and analysis of wetland ecosystems, which is crucial for their conservation and management. We present an enhanced semantic segmentation algorithm designed for UAV MS imagery, which incorporates thermal infrared (TIR) data to improve segmentation outcomes. Our approach, involving meticulous image preprocessing, customized network architecture, and iterative training procedures, aims to refine wetland boundary delineation. The algorithm demonstrates strong segmentation results, including a mean pixel accuracy (MPA) of 90.35% and a mean intersection over union (MIOU) of 73.87% across different classes, with a pixel accuracy (PA) of 95.42% and an intersection over union (IOU) of 90.46% for the wetland class. The integration of TIR data with MS imagery not only enriches the feature set for segmentation but also, to some extent, helps address data imbalance issues, contributing to a more refined ecological analysis. This approach, along with the development of a comprehensive dataset that reflects the diversity of wetland environments and advances the utility of remote sensing technologies in ecological monitoring. This research lays the groundwork for more detailed and informative UAV-based evaluations of wetland health and integrity.

Underwater fish image segmentation is a crucial technique in marine fish monitoring. However, typical underwater fish images often suffer from issues such as color distortion, low contrast, and blurriness, primarily due to the complex and dynamic nature of the marine environment. To enhance the accuracy of underwater fish image segmentation, this paper introduces an innovative neural network model that combines the attention mechanism with a feature pyramid module. After the backbone network processes the input image through convolution, the data pass through the enhanced feature pyramid module, where it is iteratively processed by multiple weighted branches. Unlike conventional methods, the multi-scale feature extraction module that we designed not only improves the extraction of high-level semantic features but also optimizes the distribution of low-level shape feature weights through the synergistic interactions of the branches, all while preserving the inherent properties of the image. This novel architecture significantly boosts segmentation accuracy, offering a new solution for fish image segmentation tasks. To further enhance the model’s robustness, the Mix-up and CutMix data augmentation techniques were employed. The model was validated using the Fish4Knowledge dataset, and the experimental results demonstrate that the model achieves a Mean Intersection over Union (MIoU) of 95.1%, with improvements of 1.3%, 1.5%, and 1.7% in the MIoU, Mean Pixel Accuracy (PA), and F1 score, respectively, compared to traditional segmentation methods. Additionally, a real fish image dataset captured in deep-sea environments was constructed to verify the practical applicability of the proposed algorithm.

Objectives Measuring anatomical parameters in fetal heart ultrasound images is crucial for the diagnosis of congenital heart disease (CHD), which is highly dependent on the clinical experience of the sonographer. To address this challenge, we propose an automated segmentation method using the channel-wise knowledge distillation technique. Methods We design a teacher-student architecture to conduct channel-wise knowledge distillation. ROI-based cropped images and full-size images are used for the teacher and student models, respectively. It allows the student model to have both the fine-grained segmentation capability inherited from the teacher model and the ability to handle full-size test images. A total of 1,300 fetal heart ultrasound images of three-vessel view were collected and annotated by experienced doctors for training, validation, and testing. Results We use three evaluation protocols to quantitatively evaluate the segmentation accuracy: Intersection over Union (IoU), Pixel Accuracy (PA), and Dice coefficient (Dice). We achieved better results than related methods on all evaluation metrics. In comparison with DeepLabv3+, the proposed method gets more accurate segmentation boundaries and has performance gains of 1.8% on mean IoU (66.8% to 68.6%), 2.2% on mean PA (79.2% to 81.4%), and 1.2% on mean Dice (80.1% to 81.3%). Conclusions Our segmentation method could identify the anatomical structure in three-vessel view of fetal heart ultrasound images. Both quantitative and visual analyses show that the proposed method significantly outperforms the related methods in terms of segmentation results.

Aiming at the problems of low detection accuracy, small proportion of crack pixels in images and sensitivity to noise in highway pavement crack detection, an improved UNet model based on multi-scale and attention mechanism (UNet-MSAM) was proposed. In the encoding stage, in order to sense the crack information of different scales in pavement images, the UNet-MSAM added multi-scale feature fusion mechanism. In addition, UNet-MSAM introduced an attention mechanism to model the relationship between the channel and position on the low-level features of the crack, instead of simply integrating the low-level and high-level features of the crack like UNet. The F1 score of UNet-MSAM on DeepCrack dataset was 84.13%, and the Mean Intersection over Union (MIOU) was 85.61%, which was better than the other three methods such as UNet. The above experimental results show that the proposed network model can accurately detect cracks.

Robust Prediction of Isodose Distribution with a Fully Convolutional Networks (FCN)-Based Deep Learning Model

Study of bubble behavior in a gas–solid dense-phase fluidized bed based on deep learning

Dental caries is a bacterial infectious disease that destroys the structure of teeth. It is one of the main diseases that endanger human health [R. H. Selwitz, A. I. Ismail, and N. B. Pitts, Lancet 369(9555), 51–59 (2007)]. At present, dentists use both visual exams and radiographs for the detection of caries. Affected by the patient's dental health and the degree of caries demineralization, it is sometimes difficult to accurately identify some dental caries in x-ray images with the naked eye. Therefore, dentists need an intelligent and accurate dental caries recognition system to assist diagnosis, reduce the influence of doctors' subjective factors, and improve the efficiency of dental caries diagnosis. Therefore, this paper combines the U-Net model verified in the field of biomedical image segmentation with the convolution block attention module, designs an Attention U-Net model for caries image segmentation, and discusses the feasibility of deep learning technology in caries image recognition so as to prepare for the next clinical verification. After testing, the Dice similarity coefficient, mean pixel accuracy, mean intersection over union, and frequency-weighted intersection over the union of teeth segmentation with Attention U-Net are 95.30%, 94.46%, 93.10%, and 93.54%, respectively. The Dice similarity coefficient, mean pixel accuracy, mean intersection over union, and frequency-weighted intersection over the union of dental caries segmentation with Attention U-Net are 85.36%, 91.84%, 82.22%, and 97.08%, respectively. As a proof of concept study, this study was an initial evaluation of technology to assist dentists in the detection of caries. There is still more work needed before this can be used clinically.

Semantic segmentation is a fundamental task in image processing and computer vision domains. Therefore, it is a core component of most emerging industry applications such as medical imaging, autonomous driving, and agriculture. The last five years have witnessed a huge growth in the computer vision domain with the introduction of Generative Adversarial Networks (GANs) due to their ability to learn deep representations by a competitive process between two neural networks called the Generator the Discriminator. Neural Architecture Search (NAS) is undergoing a revolution aimed at automating the architectural engineering process. This paper discusses implementing a novel framework for semantic image segmentation called iSegmentor using NAS and GANs. The proposed approach is the first framework that utilizes NAS in GANs for semantic image segmentation to the best of our knowledge. The iSegmentor framework was evaluated using mean Intersection over Union (mIoU) and Mean Pixel Accuracy (MPA) metrics. The iSegmentor was mainly tested and evaluated for the autonomous driving domain. With the analysis of the past five-year literature, Cityscapes and CamVid datasets were chosen. The iSegmentor achieved 81.73 MPA and 71.90 mIoU for the Cityscapes dataset, whereas CamVid achieved 80.41 MPA and 70.63 mIoU. Further, we present future avenues that we identified with the conducted experiments to assist other researchers to extend the research area of NAS in GANs in semantic segmentation.

The fusion of spectral–polarimetric information can improve the autonomous reconnaissance capability of unmanned aerial vehicles (UAVs) in detecting artificial targets. However, the current spectral and polarization imaging systems typically suffer from low image sampling resolution, which can lead to the loss of target information. Most existing segmentation algorithms neglect the similarities and differences between multimodal features, resulting in reduced accuracy and robustness of the algorithms. To address these challenges, a real-time spectral–polarimetric segmentation algorithm for artificial targets based on an efficient attention fusion network, called ESPFNet (efficient spectral–polarimetric fusion network) is proposed. The network employs a coordination attention bimodal fusion (CABF) module and a complex atrous spatial pyramid pooling (CASPP) module to fuse and enhance low-level and high-level features at different scales from the spectral feature images and the polarization encoded images, effectively achieving the segmentation of artificial targets. Additionally, the introduction of the residual dense block (RDB) module refines feature extraction, further enhancing the network’s ability to classify pixels. In order to test the algorithm’s performance, a spectral–polarimetric image dataset of artificial targets, named SPIAO (spectral–polarimetric image of artificial objects) is constructed, which contains various camouflaged nets and camouflaged plates with different properties. The experimental results on the SPIAO dataset demonstrate that the proposed method accurately detects the artificial targets, achieving a mean intersection-over-union (MIoU) of 80.4%, a mean pixel accuracy (MPA) of 88.1%, and a detection rate of 27.5 frames per second, meeting the real-time requirement. The research has the potential to provide a new multimodal detection technique for enabling autonomous reconnaissance by UAVs in complex scenes.

As an important link between human and nature, land use/land cover change (LUCC) has become a research hotspot since the first day of the issue initiated. Remote sensing observation supplies important data source for LUCC research. With the constant improvement of the remote sensed image spatial resolution and the continuous development of computer scientific algorithms and models, the study of semantic segmentation in the field of automatic interpretation of remote sensed images has promoted quickly. There have been a number of achievements focusing on the one-class extraction of LUCC classification system, but limited progress in multi-class classifying mission when facing high resolution remote sensing data by using semantic segmentation model, because of its sophisticated structure, the phenomenon of synonyms spectrum and foreign body with spectrum in images. Additionally, the limitation of the semantic segmentation model is a crucial issue which needs to be solv for precision of classification in class recognition and the perfection of semantic segmentation on pixel level as well. The High-Resolution network (HRnet) model was used and improved in the study for semantic segmentation because it has the advantage of low resolution loss. In the LUCC classification work, the distribution of classes (such as cultivated land, woodland, grassland, other land, water, etc.) are usually extremely imbalanced, that leads to the limited classification recognition results of land types when using HRnet. So a combined Loss function of Lovasz Softmax Loss+Cross Entropy Loss was used in HRnet to classify the land cover of remote sensing images, to meet the requirements of high-resolution image classification not only in terms of low resolution loss but also in terms of class recognition. The combined loss function provided the direction of the model, and the appropriate loss function improved the performance of the model when faced with the imbalanced distribution of land classes. The results are as follows: 1) Compared with Deeplab and Unet, HRnet model performed better in segmentation results; 2) HRnet model with combined Loss function Lovasz Softmax Loss+Cross Entropy Loss (HRnet+) had better performance, and its test accuracy PA(Pixel Accuracy) value was 0.9577, MIOU (Mean Intersection over Union) value was 0.8801, and Kappa value was 0.9455. This indicated that the HRnet model with low loss of resolution could achieve better results in the automatic segmentation of high-resolution remote sensing images, and the combined loss function constructed in this study could effectively solve the problem of imbalanced distribution of classes in remote sensing images, and the edge optimization ability was also more significant.