Trusted microelectronics: reverse engineering chip die using U-Net convolutional network

This study was aimed to explore the adoption value of magnetic resonance imaging (MRI) under convolutional neural networks (CNN) in the therapeutic effect of nasopharyngeal carcinoma (NPC) radiotherapy. A total of 54 NPC patients were recruited. CNN was employed to perform 3D visualization processing on magnetic resonance (MR) images of NPC patients. MRI changes were analyzed before and after the patient received radiotherapy. The image segmentation and radiotherapy effects of CNN were evaluated by the Recall, intersection over union (IOU), postoperative apparent diffusion coefficient (ADC), and diagnostic coincidence rate. Moreover, gradient vector flow (GVF) algorithm, fuzzy c-means (FCM), and SegNet were adopted for comparative evaluation. Recall of CNN was 94.89% and the IOU was 84.16%, which was remarkably different from other algorithms (P < 0.05). After analysis of the MRI images of patients receiving radiotherapy, ADC of local residual patients was 1.108 ± 0.097 measured by CNN, the ADC was 1.826 ± 0.115, and the missed diagnosis rate was only 7.14%. In summary, CNN had a good effect on the localization and segmentation of NPC patients, and can accurately evaluate the effect of patients receiving radiotherapy, which can assist clinical diagnosis and treatment of NPC.

Noise and aliasing artifacts arise in various accelerated cardiac magnetic resonance (CMR) imaging applications. In accelerated myocardial T1-mapping, the traditional three-parameter based nonlinear regression may not provide accurate estimates due to sensitivity to noise. A deep neural network-based framework is proposed to address this issue. The DeepT1 framework consists of recurrent and U-net convolution networks to produce a single output map from the noisy and incomplete measurements. The results show that DeepT1 provides noise-robust estimates compared to the traditional pixel-wise three parameter fitting.

Chronic wounds cause a significant burden to affected patients and to society. Effective and objective diagnostic and monitoring methods are needed in wound care, and artificial intelligence offers one promising alternative. In this study, real-world wound images were used to train a convolutional neural network to automatically segment wound area and wound tissues on an image. The study included altogether 362 images of venous, arterial, vasculitis and pyoderma gangrenosum wounds. The model was based on a convolutional neural network architecture U-Net, and fully supervised learning was utilised during the training phase. Wound area reached a Dice Similarity Coefficient (DSC) of 0.927 and Intersection over Union (IoU) of 0.868 using an augmented dataset with pretraining. Fibrinous exudate and granulation performed fairly well with DSC 0.750 and 0.696, and with IoU 0.659 and 0.601, respectively. Necrosis present in only 56 images achieved lower performance with DSC 0.503 and IoU 0.502. In conclusion, this study suggested that it is possible to train a neural network to perform well with images taken for purely clinical purposes. Besides wound area, several wound structures can be identified, but wound structure identification performance is dependent on the number of images featuring the structure.

Forest fires pose severe risks, including habitat loss and air pollution. Accurate forest flame segmentation is vital for effective fire management and protection of ecosystems. It improves detection, response, and understanding of fire behavior. Due to the easy accessibility and rich information content of forest remote sensing images, remote sensing techniques are frequently applied in forest flame segmentation. With the advancement of deep learning, convolutional neural network (CNN) techniques have been widely adopted for forest flame segmentation and have achieved remarkable results. However, forest remote sensing images often have high resolutions, and relative to the entire image, forest flame regions are relatively small, resulting in class imbalance issues. Additionally, mainstream semantic segmentation methods are limited by the receptive field of CNNs, making it challenging to effectively extract global features from the images and leading to poor segmentation performance when relying solely on labeled datasets. To address these issues, we propose a method based on the deeplabV3+ model, incorporating the following design strategies: (1) an adaptive Copy-Paste data augmentation method is introduced to learn from challenging samples (Images that cannot be adequately learned due to class imbalance and other factors) effectively, (2) transformer modules are concatenated and parallelly integrated into the encoder, while a CBAM attention mechanism is added to the decoder to fully extract image features, and (3) a dice loss is introduced to mitigate the class imbalance problem. By conducting validation on our self-constructed dataset, our approach has demonstrated superior performance across multiple metrics compared to current state-of-the-art semantic segmentation methods. Specifically, in terms of IoU (Intersection over Union), Precision, and Recall metrics for the flame category, our method has exhibited notable enhancements of 4.09%, 3.48%, and 1.49%, respectively, when compared to the best-performing UNet model. Moreover, our approach has achieved advancements of 11.03%, 9.10%, and 4.77% in the same aforementioned metrics as compared to the baseline model.

U-net-based segmentation of foreign bodies and ghost images in panoramic radiographs.

IntroductionNarrow Band Imaging (NBI) is an endoscopic visualization technique useful for upper aero-digestive tract (UADT) cancer detection and margins evaluation. However, NBI analysis is strongly operator-dependent and requires high expertise, thus limiting its wider implementation. Recently, artificial intelligence (AI) has demonstrated potential for applications in UADT videoendoscopy. Among AI methods, deep learning algorithms, and especially convolutional neural networks (CNNs), are particularly suitable for delineating cancers on videoendoscopy. This study is aimed to develop a CNN for automatic semantic segmentation of UADT cancer on endoscopic images.Materials and MethodsA dataset of white light and NBI videoframes of laryngeal squamous cell carcinoma (LSCC) was collected and manually annotated. A novel DL segmentation model (SegMENT) was designed. SegMENT relies on DeepLabV3+ CNN architecture, modified using Xception as a backbone and incorporating ensemble features from other CNNs. The performance of SegMENT was compared to state-of-the-art CNNs (UNet, ResUNet, and DeepLabv3). SegMENT was then validated on two external datasets of NBI images of oropharyngeal (OPSCC) and oral cavity SCC (OSCC) obtained from a previously published study. The impact of in-domain transfer learning through an ensemble technique was evaluated on the external datasets.Results219 LSCC patients were retrospectively included in the study. A total of 683 videoframes composed the LSCC dataset, while the external validation cohorts of OPSCC and OCSCC contained 116 and 102 images. On the LSCC dataset, SegMENT outperformed the other DL models, obtaining the following median values: 0.68 intersection over union (IoU), 0.81 dice similarity coefficient (DSC), 0.95 recall, 0.78 precision, 0.97 accuracy. For the OCSCC and OPSCC datasets, results were superior compared to previously published data: the median performance metrics were, respectively, improved as follows: DSC=10.3% and 11.9%, recall=15.0% and 5.1%, precision=17.0% and 14.7%, accuracy=4.1% and 10.3%.ConclusionSegMENT achieved promising performances, showing that automatic tumor segmentation in endoscopic images is feasible even within the highly heterogeneous and complex UADT environment. SegMENT outperformed the previously published results on the external validation cohorts. The model demonstrated potential for improved detection of early tumors, more precise biopsies, and better selection of resection margins.

Accurate lesion segmentation is critical for cancer diagnosis and treatment. Convolutional neural networks (CNNs) are widely used for medical image segmentation but struggle to capture long-range dependencies. Transformers mitigate this limitation but come with high computational costs. Mamba, a state-space model (SSM), efficiently models long-range dependencies but lacks precision in fine details. To address these challenges, this study aimed to develop a novel segmentation approach that combines the strengths of CNNs, Transformers, and Mamba, enhancing both global context understanding and local feature extraction in medical image segmentation. We propose AttmNet, a U-shaped network designed for medical image segmentation, which incorporates a novel structure called MAM (Multiscale-Convolution, Self-Attention, and Mamba). The MAM block integrates multi-layer convolution for multi-scale feature learning with an Att-Mamba component that combines self-attention and Mamba to effectively capture global context while preserving fine details. We evaluated AttmNet on four public datasets for breast, skin, and lung lesion segmentation. AttmNet outperformed state-of-the-art methods in terms of intersection over union (IoU) and Dice similarity coefficients. On the breast ultrasound (BUS) dataset, AttmNet achieved a 3.38% improvement in IoU and a 4.54% increase in Dice over the next best method. On the breast ultrasound images (BUSI) dataset, AttmNet's IoU and Dice coefficients were 1.17% and 3.21% higher than the closest competitor, respectively. In the PH2 Dermoscopy Image dataset, AttmNet surpassed the next best model by 0.25% in both IoU and Dice. On the larger coronavirus disease 2019 (COVID-19) Lung dataset, AttmNet maintained strong performance, achieving higher IoU and Dice scores than the next best models, SegMamba and TransUNet. AttmNet is a powerful and efficient tool for medical image segmentation, addressing the limitations of existing methods through its advanced design. The MAM block significantly enhances segmentation accuracy while maintaining computational efficiency, making AttmNet highly suitable for clinical applications. The code is available at https://github.com/hyb2840/AttmNet.

BackgroundThe study of plant photosynthesis is essential for productivity and yield. Thanks to the development of high-throughput phenotyping (HTP) facilities, based on chlorophyll fluorescence imaging, photosynthetic traits can be measured in a reliable, reproducible and efficient manner. In most state-of-the-art HTP platforms, these traits are automatedly analyzed at individual plant level, but information at leaf level is often restricted by the use of manual annotation. Automated leaf tracking over time is therefore highly desired. Methods for tracking individual leaves are still uncommon, convoluted, or require large datasets. Hence, applications and libraries with different techniques are required. New phenotyping platforms are initiated now more frequently than ever; however, the application of advanced computer vision techniques, such as convolutional neural networks, is still growing at a slow pace. Here, we provide a method for leaf segmentation and tracking through the fine-tuning of Mask R-CNN and intersection over union as a solution for leaf tracking on top-down images of plants. We also provide datasets and code for training and testing on both detection and tracking of individual leaves, aiming to stimulate the community to expand the current methodologies on this topic.ResultsWe tested the results for detection and segmentation on 523 Arabidopsis thaliana leaves at three different stages of development from which we obtained a mean F-score of 0.956 on detection and 0.844 on segmentation overlap through the intersection over union (IoU). On the tracking side, we tested nine different plants with 191 leaves. A total of 161 leaves were tracked without issues, accounting to a total of 84.29% correct tracking, and a Higher Order Tracking Accuracy (HOTA) of 0.846. In our case study, leaf age and leaf order influenced photosynthetic capacity and photosynthetic response to light treatments. Leaf-dependent photosynthesis varies according to the genetic background.ConclusionThe method provided is robust for leaf tracking on top-down images. Although one of the strong components of the method is the low requirement in training data to achieve a good base result (based on fine-tuning), most of the tracking issues found could be solved by expanding the training dataset for the Mask R-CNN model.

DeepDendro – A tree rings detector based on a deep convolutional neural network

This paper proposes to use Fast Fourier Transformation-based U-Net (a refined "fully convolutional networks") and perform image convolution in neural networks. Leveraging the Fast Fourier Transformation, it reduces the image convolution costs involved in the Convolutional Neural Networks (CNNs) and thus reduces the overall computational costs. The proposed model identifies the object information from the images. We apply the Fast Fourier transform algorithm on an image data set to obtain more accessible information about the image data, before segmenting them through the U-Net architecture. More specifically, we implement the FFT-based convolutional neural network to improve the training time of the network. The proposed approach was applied to publicly available Broad Bioimage Benchmark Collection (BBBC) dataset. Our model demonstrated improvement in training time during convolution from 600 - 700 ms/step to 400-500 ms/step. We evaluated the accuracy of our model using Intersection over Union (IoU) metric showing significant improvements.

In recent years, multilayer perceptron (MLP)-based networks have gained significant attention in the field of medical image segmentation. However, achieving high accuracy in breast ultrasound image segmentation remains challenging due to the difficulty in capturing sufficient target features. Shallow networks have proven insufficient for addressing the complexities involved in ultrasound image segmentation tasks. We propose a deep MLP-based network for ultrasound breast segmentation that incorporates boundary awareness. The architecture of the network comprises seven layers, with the initial three layers utilizing convolutional neural network (CNN), while the subsequent four layers are based on tokenized MLP(Tok-MLP). The CNN is utilized for extracting shallow features, while the Tok-MLP is employed to capture deeper features. Additionally, breast ultrasound images are often influenced by various factors that result in blurred tissue boundaries, which complicates the segmentation process. To address this issue, we introduce the Boundary Aware Module (BAM), designed to enhance boundary detection through two distinct strategies. This module is integrated with the original up-sampled feature maps to ensure that boundary features receive equal attention as internal features. Experimental results on two breast ultrasound datasets demonstrate that the proposed DCMNet significantly outperforms six state-of-the-art methods, as evidenced by superior Dice and Intersection over Union (IoU) scores. Specifically, the BUSI dataset achieved IoU and Dice scores of 67.11% and 80.01%, respectively.

Rheumatoid arthritis (RA) is a chronic autoimmune disease that can cause severe joint damage and functional impairment. Ultrasound imaging has shown promise in providing real-time assessment of synovium inflammation associated with the early stages of RA. Accurate segmentation of the synovium region and quantification of inflammation-specific imaging biomarkers are crucial for assessing and grading RA. However, automatic segmentation of the synovium in 3D ultrasound is challenging due to ambiguous boundaries, variability in synovium shape, and inhomogeneous intensity distribution. In this work, we introduce a novel network architecture, Swin Transformers with Deep Attentive Features for 3D segmentation (SwinDAF3D), which integrates Swin Transformers into a Deep Attentive Features framework. The developed architecture leverages the hierarchical structure and shifted windows of Swin Transformers to capture rich, multi-scale and attentive contextual information, improving the modeling of long-range dependencies and spatial hierarchies in 3D ultrasound images. In a six-fold cross-validation study with 3D ultrasound images of RA patients' finger joints (n = 72), our SwinDAF3D model achieved the highest performance with a Dice Score (DSC) of 0.838 ± 0.013, an Intersection over Union (IoU) of 0.719 ± 0.019, and Surface Dice Score (SDSC) of 0.852 ± 0.020, compared to 3D UNet (DSC: 0.742 ± 0.025; IoU: 0.589 ± 0.031; SDSC: 0.661 ± 0.029), DAF3D (DSC: 0.813 ± 0.017; IoU: 0.689 ± 0.022; SDSC: 0.817 ± 0.013), Swin UNETR (DSC: 0.808 ± 0.025; IoU: 0.678 ± 0.032; SDSC: 0.822 ± 0.039), UNETR++ (DSC: 0.810 ± 0.014; IoU: 0.684 ± 0.018; SDSC: 0.829 ± 0.027) and TransUNet (DSC: 0.818 ± 0.013; IoU: 0.692 ± 0.017; SDSC: 0.815 ± 0.016) models. This ablation study demonstrates the effectiveness of combining a Swin Transformers feature pyramid with a deep attention mechanism, improving the segmentation accuracy of the synovium in 3D ultrasound. This advancement shows great promise in enabling more efficient and standardized RA screening using ultrasound imaging.

Funding Acknowledgements Type of funding sources: Private grant(s) and/or Sponsorship. Main funding source(s): Japan Organization of Occupational Health and Safety onbehalf J3D Introduction Improvements in cardiovascular imaging have contributed to the accurate diagnosis and treatment of cardiovascular diseases. In recent years, we can find a new demand from surgical departments, i.e., 3D heart models of patients for preoperative simulations. However, a 3D surgical model needs a detailed segmentation of heart structures from clinical images because these details are essential for achieving accurate polygon data of an exact 3D model. Thus, in most cases, manual segmentation is required for complicated heart shapes such as the chamber wall, valves, and papillary muscles, which causes a prolonged duration time. Purpose We aim to achieve an automated heart segmentation using a convolutional neural network (CNN) trained using deep learning techniques for the rapid creation of 3D surgical heart models of individual patients. Methods We constructed our original CNN program based on the latest artificial intelligence techniques and trained it to extract shapes of the heart from cardiac computed tomography (CT) images. The training data was 361 slices selected from CT scans of 10 patients. We used data augmentation to increase the amount and diversity of the data into 24,052 slices. The training result with the best Intersection over Union (IoU), one of the evaluation metrics used in deep learning, was saved. Finally, we used the best-trained CNN to construct 3D polygon data of two surgical cases of hypertrophic obstructive cardiomyopathy (HOCM) for preoperative assessments. Results The IoU attained 0.85 after deep learning. The time required to complete the 3D polygon data for the first HOCM case was 5 minutes for segmentation by the trained CNN and 3 hours for data correction by a human operator. Similarly, the time required for the second case was 5 minutes for segmentation without manual correction. We had to correct the segmentation for the first case because we needed an exact 3D model for the preoperative assessment (Fig. 1). According to our records of the other eight 3D heart models in the lab, the work for a 3D polygon shape from CT images needs a median 30 hours (quartiles 23-50 hours) when the procedure is fully manual and non-continuous with breaks in between. Conclusion The CNN-based segmentation aided the constructing heart shapes from cardiac CT images of preoperative patients. Although the performance, reaching IoU of 0.85, was insufficient for fully automatic segmentation, the methodology can shorten the process duration from several hours to several minutes for detailed segmentation of heart structures. We previously applied the CNN-based segmentation to the aorta, aortic stenosis valves, and atheromatous plaque in clinical images, demonstrating adequate segmentation performance. The proposed methodology can be applied as a fundamental technology of cardiovascular imaging for obtaining the actual structures of a target object as 3D coordinate values or a 3D model within a reasonable duration time. Abstract Figure 1

X-ray imaging has been broadly adopted as a nondestructive testing method for product quality inspection. Deep learning has demonstrated powerful image scene understanding capabilities. In this article, U-Net with resnet101 is taken as the baseline for defect segmentation. First, there exist gray inhomogeneous and low-contrast regions in X-ray images, which can hardly be segmented. Contrast-limited adaptive histogram equalization (CLAHE) could be used to improve the contrast and consistency of the X-ray image. A two-stream convolutional neural network (CNN) is proposed that takes the original image and CLAHE processed image as inputs to address this issue. And then in CNN, low-level feature maps are lacking semantic information, which may lead to worse results. A gated multilayer fusion module is proposed to adaptively fuse the high-level features into low-level features. Furthermore, loss functions (such as cross entropy) in semantic segmentation are usually pixel level, ignoring the regional information. A weighted intersection over union (IOU) loss function is proposed to introduce IOU information to guide the model to focus on the objects that are easy to mine. The experimental results prove that the three proposed methods have better performance than the baseline for our dataset, achieving 42.2 in mIoU, 59.2 in Dice, and 54.5%, 74.9%, and 86.3% in small, middle, and large object recall rate, respectively.

Aim: Brain tumors are among the most fatal cancers worldwide. Diagnosing and manually segmenting tumors are time-consuming clinical tasks, and success strongly depends on the doctor's experience. Automatic quantitative analysis and accurate segmentation of brain tumors are greatly needed for cancer diagnosis.Methods:This paper presents an advanced three-dimensional multimodal segmentation algorithm called nested dilation networks (NDNs). It is inspired by the U-Net architecture, a convolutional neural network (CNN) developed for biomedical image segmentation and is modified to achieve better performance for brain tumor segmentation. Thus, we propose residual blocks nested with dilations (RnD) in the encoding part to enrich the low-level features and use squeeze-and-excitation (SE) blocks in both the encoding and decoding parts to boost significant features. To prove the reliability of the network structure, we compare our results with those of the standard U-Net and its transmutation networks. Different loss functions are considered to cope with class imbalance problems to maximize the brain tumor segmentation results. A cascade training strategy is employed to run NDNs for coarse-to-fine tumor segmentation. This strategy decomposes the multiclass segmentation problem into three binary segmentation problems and trains each task sequentially. Various augmentation techniques are utilized to increase the diversity of the data to avoid overfitting.Results: This approach achieves Dice similarity scores of 0.6652, 0.5880, and 0.6682 for edema, non-enhancing tumors, and enhancing tumors, respectively, in which the Dice loss is used for single-pass training. After cascade training, the Dice similarity scores rise to 0.7043, 0.5889, and 0.7206, respectively.Conclusion: Experiments show that the proposed deep learning algorithm outperforms other U-Net transmutation networks for brain tumor segmentation. Moreover, applying cascade training to NDNs facilitates better performance than other methods. The findings of this study provide considerable insight into the automatic and accurate segmentation of brain tumors.