Concrete bridge surface damage detection using a single-stage detector

This study was aimed to explore the adoption value of magnetic resonance imaging (MRI) under convolutional neural networks (CNN) in the therapeutic effect of nasopharyngeal carcinoma (NPC) radiotherapy. A total of 54 NPC patients were recruited. CNN was employed to perform 3D visualization processing on magnetic resonance (MR) images of NPC patients. MRI changes were analyzed before and after the patient received radiotherapy. The image segmentation and radiotherapy effects of CNN were evaluated by the Recall, intersection over union (IOU), postoperative apparent diffusion coefficient (ADC), and diagnostic coincidence rate. Moreover, gradient vector flow (GVF) algorithm, fuzzy c-means (FCM), and SegNet were adopted for comparative evaluation. Recall of CNN was 94.89% and the IOU was 84.16%, which was remarkably different from other algorithms (P < 0.05). After analysis of the MRI images of patients receiving radiotherapy, ADC of local residual patients was 1.108 ± 0.097 measured by CNN, the ADC was 1.826 ± 0.115, and the missed diagnosis rate was only 7.14%. In summary, CNN had a good effect on the localization and segmentation of NPC patients, and can accurately evaluate the effect of patients receiving radiotherapy, which can assist clinical diagnosis and treatment of NPC.

Monitoring damage in concrete structures is crucial for maintaining the health of structural systems. The implementation of computer vision has been the key for providing accurate and quantitative monitoring. Recent development uses the robustness of deep-learning-aided computer vision, especially the convolutional neural network model. The convolutional neural network is not only accurate but also flexible in various scenarios. The convolutional neural network has been constructed to classify image in terms of individual pixel, namely pixel-level detection, which is especially useful in detecting and classifying damage in fine-grained detail. Moreover, in the real-world scenario, the scenes are mostly very complex with varying foreign objects other than concrete. Therefore, this study will focus on implementing a pixel-level convolutional neural network for concrete surface damage detection with complicated surrounding image settings. Since there are multiple types of damage on concrete surfaces, the convolutional neural network model will be trained to detect three types of damages, namely cracks, spallings, and voids. The training architecture will adopt U-Net and DeepLabV3+. Both models are compared using the evaluation metrics and the predicted results. The dataset used for the neural network training is self-built and contains multiple concrete damages and complex foregrounds on every image. To deal with overfitting, the dataset is augmented, and the models are regularized using L1 and Spatial dropout. U-Net slightly outperforms DeepLabV3+ with U-Net scores 0.7199 and 0.5993 on F1 and mIoU, respectively, while DeepLabV3+ scores 0.6478 and 0.5174 on F1 and mIoU, respectively. Given the complexity of the dataset and extensive image labeling, the neural network models achieved satisfactory results.

BackgroundThe study of plant photosynthesis is essential for productivity and yield. Thanks to the development of high-throughput phenotyping (HTP) facilities, based on chlorophyll fluorescence imaging, photosynthetic traits can be measured in a reliable, reproducible and efficient manner. In most state-of-the-art HTP platforms, these traits are automatedly analyzed at individual plant level, but information at leaf level is often restricted by the use of manual annotation. Automated leaf tracking over time is therefore highly desired. Methods for tracking individual leaves are still uncommon, convoluted, or require large datasets. Hence, applications and libraries with different techniques are required. New phenotyping platforms are initiated now more frequently than ever; however, the application of advanced computer vision techniques, such as convolutional neural networks, is still growing at a slow pace. Here, we provide a method for leaf segmentation and tracking through the fine-tuning of Mask R-CNN and intersection over union as a solution for leaf tracking on top-down images of plants. We also provide datasets and code for training and testing on both detection and tracking of individual leaves, aiming to stimulate the community to expand the current methodologies on this topic.ResultsWe tested the results for detection and segmentation on 523 Arabidopsis thaliana leaves at three different stages of development from which we obtained a mean F-score of 0.956 on detection and 0.844 on segmentation overlap through the intersection over union (IoU). On the tracking side, we tested nine different plants with 191 leaves. A total of 161 leaves were tracked without issues, accounting to a total of 84.29% correct tracking, and a Higher Order Tracking Accuracy (HOTA) of 0.846. In our case study, leaf age and leaf order influenced photosynthetic capacity and photosynthetic response to light treatments. Leaf-dependent photosynthesis varies according to the genetic background.ConclusionThe method provided is robust for leaf tracking on top-down images. Although one of the strong components of the method is the low requirement in training data to achieve a good base result (based on fine-tuning), most of the tracking issues found could be solved by expanding the training dataset for the Mask R-CNN model.

Model optimization is a problem of great concern and challenge for developing an image classification model. In image classification, selecting the appropriate hyperparameters can substantially boost the model’s ability to learn intricate patterns and features from complex image data. Hyperparameter optimization helps to prevent overfitting by finding the right balance between complexity and generalization of a model. The ensemble genetic algorithm and convolutional neural network (EGACNN) are proposed to enhance image classification by fine-tuning hyperparameters. The convolutional neural network (CNN) model is combined with a genetic algorithm GA) using stacking based on the Modified National Institute of Standards and Technology (MNIST) dataset to enhance efficiency and prediction rate on image classification. The GA optimizes the number of layers, kernel size, learning rates, dropout rates, and batch sizes of the CNN model to improve the accuracy and performance of the model. The objective of this research is to improve the CNN-based image classification system by utilizing the advantages of ensemble learning and GA. The highest accuracy is obtained using the proposed EGACNN model which is 99.91% and the ensemble CNN and spiking neural network (CSNN) model shows an accuracy of 99.68%. The ensemble approaches like EGACNN and CSNN tends to be more effective as compared to CNN, RNN, AlexNet, ResNet, and VGG models. The hyperparameter optimization of deep learning classification models reduces human efforts and produces better prediction results. Performance comparison with existing approaches also shows the superior performance of the proposed model.

Chronic wounds cause a significant burden to affected patients and to society. Effective and objective diagnostic and monitoring methods are needed in wound care, and artificial intelligence offers one promising alternative. In this study, real-world wound images were used to train a convolutional neural network to automatically segment wound area and wound tissues on an image. The study included altogether 362 images of venous, arterial, vasculitis and pyoderma gangrenosum wounds. The model was based on a convolutional neural network architecture U-Net, and fully supervised learning was utilised during the training phase. Wound area reached a Dice Similarity Coefficient (DSC) of 0.927 and Intersection over Union (IoU) of 0.868 using an augmented dataset with pretraining. Fibrinous exudate and granulation performed fairly well with DSC 0.750 and 0.696, and with IoU 0.659 and 0.601, respectively. Necrosis present in only 56 images achieved lower performance with DSC 0.503 and IoU 0.502. In conclusion, this study suggested that it is possible to train a neural network to perform well with images taken for purely clinical purposes. Besides wound area, several wound structures can be identified, but wound structure identification performance is dependent on the number of images featuring the structure.

Funding Acknowledgements Type of funding sources: Private grant(s) and/or Sponsorship. Main funding source(s): Japan Organization of Occupational Health and Safety onbehalf J3D Introduction Improvements in cardiovascular imaging have contributed to the accurate diagnosis and treatment of cardiovascular diseases. In recent years, we can find a new demand from surgical departments, i.e., 3D heart models of patients for preoperative simulations. However, a 3D surgical model needs a detailed segmentation of heart structures from clinical images because these details are essential for achieving accurate polygon data of an exact 3D model. Thus, in most cases, manual segmentation is required for complicated heart shapes such as the chamber wall, valves, and papillary muscles, which causes a prolonged duration time. Purpose We aim to achieve an automated heart segmentation using a convolutional neural network (CNN) trained using deep learning techniques for the rapid creation of 3D surgical heart models of individual patients. Methods We constructed our original CNN program based on the latest artificial intelligence techniques and trained it to extract shapes of the heart from cardiac computed tomography (CT) images. The training data was 361 slices selected from CT scans of 10 patients. We used data augmentation to increase the amount and diversity of the data into 24,052 slices. The training result with the best Intersection over Union (IoU), one of the evaluation metrics used in deep learning, was saved. Finally, we used the best-trained CNN to construct 3D polygon data of two surgical cases of hypertrophic obstructive cardiomyopathy (HOCM) for preoperative assessments. Results The IoU attained 0.85 after deep learning. The time required to complete the 3D polygon data for the first HOCM case was 5 minutes for segmentation by the trained CNN and 3 hours for data correction by a human operator. Similarly, the time required for the second case was 5 minutes for segmentation without manual correction. We had to correct the segmentation for the first case because we needed an exact 3D model for the preoperative assessment (Fig. 1). According to our records of the other eight 3D heart models in the lab, the work for a 3D polygon shape from CT images needs a median 30 hours (quartiles 23-50 hours) when the procedure is fully manual and non-continuous with breaks in between. Conclusion The CNN-based segmentation aided the constructing heart shapes from cardiac CT images of preoperative patients. Although the performance, reaching IoU of 0.85, was insufficient for fully automatic segmentation, the methodology can shorten the process duration from several hours to several minutes for detailed segmentation of heart structures. We previously applied the CNN-based segmentation to the aorta, aortic stenosis valves, and atheromatous plaque in clinical images, demonstrating adequate segmentation performance. The proposed methodology can be applied as a fundamental technology of cardiovascular imaging for obtaining the actual structures of a target object as 3D coordinate values or a 3D model within a reasonable duration time. Abstract Figure 1

Concrete cracks significantly impact structural durability by allowing harmful elements such as carbon dioxide and chloride ions to penetrate. Thus, the cracks should be prevented and treated to maintain the lifespan of concrete structures. Convolutional neural network (CNN) has been widely applied in crack detection. This study investigated the use of a Bayesian convolutional neural network (BCNN) for concrete crack classification and compared the performance of BCNN with UNet, a deep CNN-based model, and CNN. BCNN weight uncertainty through a Bayesian method, improved prediction reliability and prevented overfitting. BCNN demonstrated superior expected calibration error (0.0030) and Brier score (0.0025) compared to CNN, highlighting its robustness in prediction reliability. Furthermore, Bayesian entropy analysis confirmed the ability of BCNN to quantify uncertainty, leading to more precise crack classification. These results imply that BCNN provides a balance between computational efficiency and performance, offering strong potential for smart construction applications. Validation results indicated that BCNN achieved a validation accuracy of 92.22%, higher than CNN (85.78%) and slightly lower than UNet (94.34%). However, BCNN demonstrated a superior intersection over union (IoU) of 0.8049, outperforming CNN (0.6685) and approaching UNet (0.9135). These results implied that BCNN provides a balance between computational efficiency and performance, offering strong potential for smart construction applications.

Understanding the spatial distribution of irrigated croplands is crucial for food security and water use. To map land cover classes with high-spatial-resolution images, it is necessary to analyze the semantic information of target objects in addition to the spectral or spatial–spectral information of local pixels. Deep convolutional neural networks (DCNNs) can characterize the semantic features of objects adaptively. This study uses DCNNs to extract irrigated croplands from Sentinel-2 images in the states of Washington and California in the United States. We integrated the DCNNs of 101 layers, discarded pooling layers, and employed dilation convolution to preserve location information; these are models which were used based on fully convolutional network (FCN) architectures. The findings indicated that irrigated croplands may be effectively detected at various phases of crop growth in the fields. A quantitative analysis of the trained models revealed that the three models in the two states had the lowest values of Intersection over Union (IoU) and Kappa, i.e., 0.88 and 0.91, respectively. The deep models’ temporal portability across different years was acceptable. The lowest values of recall and OA (overall accuracy) from 2018 to 2021 were 0.91 and 0.87, respectively. In Washington, the lowest OA value from 10 to 300 m resolution was 0.76. This study demonstrates the potential of FCNs + DCNNs approaches for mapping irrigated croplands across large regions, providing a solution for irrigation mapping. The spatial resolution portability of deep models could be improved further by designing model architectures.

Surface damages of reinforced concrete and steel bridges, for example, crack and corrosion, are usually regarded as indicators of internal structural defects, hence can be used to assess the structural health condition. Quantitative segmentation of these surface damages via computer vision is important yet challenging due to the limited accuracy of traditional semantic segmentation methods. To overcome this challenge, this study proposes a modified semantic segmentation method that can distinguish multiple surface damages, based on you only look once version 7 (YOLOv7) and global attention mechanism (GAM), namely, YOLOv7-SEG-GAM. Initially, the extended efficient layer aggregation network in the backbone network of YOLOv7 was substituted with GAM, followed by the integration of a segmentation head utilizing the three-scale feature map, thus establishing a segmentation network. Subsequently, graphical examples depicting five types of reinforced concrete and steel bridge surface damages, that is, concrete cracks, steel corrosion, exposed rebar, spalling, and efflorescence, are gathered and meticulously labeled to create a semantic segmentation dataset tailored for training the network. Afterwards, a comparative study is undertaken to analyze the effectiveness of GAM, squeeze-and-excitation networks, and convolutional block attention module in enhancing the network’s performance. Ultimately, a calibration device was developed utilizing a laser rangefinder and a smartphone to enable quantitative assessment of bridge damages in real size. Based on the identical dataset, the evaluated accuracy of YOLOv7-SEG-GAM was compared with DeepLabV3+, BiSeNet, and improved semantic segmentation networks. The results indicate that the mean pixel accuracy and mean intersection over union values achieved by YOLOv7-SEG-GAM were 0.881 and 0.782, respectively, surpassing those of DeepLabV3+ and BiSeNet. This study successfully enables pixel-level segmentation of bridge damages and offers valuable insights for quantitative segmentation.

IntroductionNarrow Band Imaging (NBI) is an endoscopic visualization technique useful for upper aero-digestive tract (UADT) cancer detection and margins evaluation. However, NBI analysis is strongly operator-dependent and requires high expertise, thus limiting its wider implementation. Recently, artificial intelligence (AI) has demonstrated potential for applications in UADT videoendoscopy. Among AI methods, deep learning algorithms, and especially convolutional neural networks (CNNs), are particularly suitable for delineating cancers on videoendoscopy. This study is aimed to develop a CNN for automatic semantic segmentation of UADT cancer on endoscopic images.Materials and MethodsA dataset of white light and NBI videoframes of laryngeal squamous cell carcinoma (LSCC) was collected and manually annotated. A novel DL segmentation model (SegMENT) was designed. SegMENT relies on DeepLabV3+ CNN architecture, modified using Xception as a backbone and incorporating ensemble features from other CNNs. The performance of SegMENT was compared to state-of-the-art CNNs (UNet, ResUNet, and DeepLabv3). SegMENT was then validated on two external datasets of NBI images of oropharyngeal (OPSCC) and oral cavity SCC (OSCC) obtained from a previously published study. The impact of in-domain transfer learning through an ensemble technique was evaluated on the external datasets.Results219 LSCC patients were retrospectively included in the study. A total of 683 videoframes composed the LSCC dataset, while the external validation cohorts of OPSCC and OCSCC contained 116 and 102 images. On the LSCC dataset, SegMENT outperformed the other DL models, obtaining the following median values: 0.68 intersection over union (IoU), 0.81 dice similarity coefficient (DSC), 0.95 recall, 0.78 precision, 0.97 accuracy. For the OCSCC and OPSCC datasets, results were superior compared to previously published data: the median performance metrics were, respectively, improved as follows: DSC=10.3% and 11.9%, recall=15.0% and 5.1%, precision=17.0% and 14.7%, accuracy=4.1% and 10.3%.ConclusionSegMENT achieved promising performances, showing that automatic tumor segmentation in endoscopic images is feasible even within the highly heterogeneous and complex UADT environment. SegMENT outperformed the previously published results on the external validation cohorts. The model demonstrated potential for improved detection of early tumors, more precise biopsies, and better selection of resection margins.

Hyperspectral image (HSI) classification has been extensively applied for analyzing remotely sensed images. HSI data consist of multiple bands that provide abundant spatial information. Convolutional neural networks (CNNs) have emerged as powerful deep learning methods for processing visual data. In recent work, CNNs have shown impressive results in HSI classification. In this paper, we propose a hierarchical neural network architecture called feature extraction with hybrid spectral CNN (FE-HybridSN) to extract superior spectral–spatial features. FE-HybridSN effectively captures more spectral–spatial information while reducing computational complexity. Considering the prevalent issue of class imbalance in experimental datasets (IP, UP, SV) and real-world hyperspectral datasets, we apply the focal loss to mitigate these problems. The focal loss reconstructs the loss function and facilitates effective achievement of the aforementioned goals. We propose a framework (FEHN-FL) that combines FE-HybridSN and the focal loss for HSI classification and then conduct extensive HSI classification experiments using three remote sensing datasets: Indian Pines (IP), University of Pavia (UP), and Salinas Scene (SV). Using cross-entropy loss as a baseline, we assess the hyperspectral classification performance of various backbone networks and examine the influence of different spatial sizes on classification accuracy. After incorporating focal loss as our loss function, we not only compare the classification performance of the FE-HybridSN backbone network under different loss functions but also evaluate their convergence rates during training. The proposed classification framework demonstrates satisfactory performance compared to state-of-the-art end-to-end deep-learning-based methods, such as 2D-CNN, 3D-CNN, etc.

Accurate lesion segmentation is critical for cancer diagnosis and treatment. Convolutional neural networks (CNNs) are widely used for medical image segmentation but struggle to capture long-range dependencies. Transformers mitigate this limitation but come with high computational costs. Mamba, a state-space model (SSM), efficiently models long-range dependencies but lacks precision in fine details. To address these challenges, this study aimed to develop a novel segmentation approach that combines the strengths of CNNs, Transformers, and Mamba, enhancing both global context understanding and local feature extraction in medical image segmentation. We propose AttmNet, a U-shaped network designed for medical image segmentation, which incorporates a novel structure called MAM (Multiscale-Convolution, Self-Attention, and Mamba). The MAM block integrates multi-layer convolution for multi-scale feature learning with an Att-Mamba component that combines self-attention and Mamba to effectively capture global context while preserving fine details. We evaluated AttmNet on four public datasets for breast, skin, and lung lesion segmentation. AttmNet outperformed state-of-the-art methods in terms of intersection over union (IoU) and Dice similarity coefficients. On the breast ultrasound (BUS) dataset, AttmNet achieved a 3.38% improvement in IoU and a 4.54% increase in Dice over the next best method. On the breast ultrasound images (BUSI) dataset, AttmNet's IoU and Dice coefficients were 1.17% and 3.21% higher than the closest competitor, respectively. In the PH2 Dermoscopy Image dataset, AttmNet surpassed the next best model by 0.25% in both IoU and Dice. On the larger coronavirus disease 2019 (COVID-19) Lung dataset, AttmNet maintained strong performance, achieving higher IoU and Dice scores than the next best models, SegMamba and TransUNet. AttmNet is a powerful and efficient tool for medical image segmentation, addressing the limitations of existing methods through its advanced design. The MAM block significantly enhances segmentation accuracy while maintaining computational efficiency, making AttmNet highly suitable for clinical applications. The code is available at https://github.com/hyb2840/AttmNet.

Most semantic segmentation models based on deep convolutional neural network (CNN) typically require a large number of weight parameters, high hardware resources for storage and computation. Moreover, redesigning a compact network suffers from some training problems, such as under-fitting. A human segmentation algorithm is proposed based on compressed deep CNN to optimize the convolution layers and filters. PSPNet-50 is fine-tuned on the human segmentation dataset to obtain the human segmentation model with higher accuracy. Then the convolutional-layer level pruning and corresponding structure optimization are performed so that the parameters of the model are substantially reduced. Finally, the two-stage global filter-level pruning strategy is used. Compared with the method of layer by layer pruning and retraining, our strategy not only reduces parameters of the model and saves the time of retraining, but also keeps the high IoU (Intersection over Union) accuracy. In addition, by adding auxiliary losses in the network during training CNN, the supervised training of the network is improved, and IoU is further increased. Compared to the model before compression, the sufficient experiments show that the parameter number, computation cost, memory consumption, and parameter storage are decreased by 1/7.5, 5.6/6.6, 0.7/1, 6.5/7.5, respectively, while the segmentation speed is accelerated by 2.4 times, and IoU on test set reaches 93.2%.

Since the emergence of deep learning in object detection in 2014, there has been rapid development and widespread application of detectors across various domains such as healthcare, road traffic, and industrial inspection. The evolution of convolutional neural network (CNN)-based detectors has reached a mature stage, with single-stage detectors playing a crucial role within CNN-based frameworks. This paper provides an analysis of the developmental process and distinctive features of classic single-stage detectors categorized into two types: Anchor-free and Anchor-based. Furthermore, the paper also provides a comparison of the accuracy trends for these two types of single-stage detectors over different years. Through this analysis and comparison, potential reasons behind the advancements in single-stage detectors during different time periods can be explored, such as network architecture optimization, increase in datasets, and algorithm improvements. Additionally, this paper offers insights into current research focal points and future prospects.

Unmanned aerial vehicles (UAVs) are increasingly being integrated into the domain of precision agriculture, revolutionizing the agricultural landscape. Specifically, UAVs are being used in conjunction with machine learning techniques to solve a variety of complex agricultural problems. This paper provides a careful survey of more than 70 studies that have applied machine learning techniques utilizing UAV imagery to solve agricultural problems. The survey examines the models employed, their applications, and their performance, spanning a wide range of agricultural tasks, including crop classification, crop and weed detection, cropland mapping, and field segmentation. Comparisons are made among supervised, semi-supervised, and unsupervised machine learning approaches, including traditional machine learning classifiers, convolutional neural networks (CNNs), single-stage detectors, two-stage detectors, and transformers. Lastly, future advancements and prospects for UAV utilization in precision agriculture are highlighted and discussed. The general findings of the paper demonstrate that, for simple classification problems, traditional machine learning techniques, CNNs, and transformers can be used, with CNNs being the optimal choice. For segmentation tasks, UNETs are by far the preferred approach. For detection tasks, two-stage detectors delivered the best performance. On the other hand, for dataset augmentation and enhancement, generative adversarial networks (GANs) were the most popular choice.