WITHDRAWN: Instance Segmentation of Burn Areas Using Position-Sensitive Convolutional Networks

Panoptic Segmentation aims to provide an understanding of background (stuff) and instances of objects (things) at a pixel level. It combines the separate tasks of semantic segmentation (pixel level classification) and instance segmentation to build a single unified scene understanding task. Typically, panoptic segmentation is derived by combining semantic and instance segmentation tasks that are learned separately or jointly (multi-task networks). In general, instance segmentation networks are built by adding a foreground mask estimation layer on top of object detectors or using instance clustering methods that assign a pixel to an instance center. In this work, we present a fully convolution neural network that learns instance segmentation from semantic segmentation and instance contours (boundaries of things). Instance contours along with semantic segmentation yield a boundary aware semantic segmentation of things. Connected component labeling on these results produces instance segmentation. We merge semantic and instance segmentation results to output panoptic segmentation. We evaluate our proposed method on the CityScapes dataset to demonstrate qualitative and quantitative performances along with several ablation studies. Our overview video can be accessed from https://youtu.be/wBtcxRhG3e0.

Fatigue fracture is one of the most common metallic component failure cases in manufacturing industries. The observation on fractography can provide direct evidence for failure analysis. In this study, an image semantic segmentation method based on fully convolutional networks (FCNs) was proposed to figure out the boundary between fatigue crack propagation and fast fracture regions from optical microscope (OM) fractography images. Furthermore, a novel morphing‐based data augmentation method was adopted to enable few‐shot learning of sample images. The proposed framework can successfully segment two categories, namely, the crack propagation and fast fracture regions, thus differentiating the boundary of two regions in one image. This artificial intelligence (AI)‐assisted fatigue analysis architecture can complete the failure analysis procedure in 0.5 s and prove the feasibility of fatigue failure analysis. The segmentation accuracy of self‐developed network achieves 95.4% for the fatigue crack propagation region, as well as 97.2% for the fast fracture region. Not only for semantic segmentation DNN, we also prove that our novel data augmentation method can applied at the instance segmentation DNN, such as mask regional convolutional neural network (mask R‐CNN), one state‐of‐the‐art deep learning network for instance segmentation, to achieve similar accuracy.

Medical image analysis plays a pivotal role in modern healthcare, aiding clinicians in accurate diagnosis and treatment planning. However, the complexity and diversity of medical images pose significant challenges for traditional image processing methods. Existing methods often struggle to precisely delineate structures in medical images, leading to suboptimal diagnostic accuracy. The demand for automated and accurate segmentation tools in medical imaging has grown, highlighting the necessity for robust and efficient algorithms capable of handling diverse anatomical variations and pathologies. While CNNs have shown promise in image analysis, their application to medical images requires customization to accommodate unique challenges. The literature lacks comprehensive studies that bridge the gap between general-purpose CNNs and the specific demands of medical image segmentation, especially concerning the diverse and intricate structures present in medical imagery. This study addresses the need for advanced techniques by leveraging Convolutional Neural Networks (CNNs) for semantic segmentation in medical image analysis. Our approach involves the design and implementation of a specialized CNN architecture tailored to the nuances of medical image data. We employ state-of-the-art techniques for data preprocessing, model training, and validation. The model is trained on a diverse dataset encompassing various medical imaging modalities, ensuring its adaptability and generalizability. The proposed CNN-based semantic segmentation model demonstrates superior performance in accurately delineating anatomical structures compared to traditional methods. Evaluation metrics, including Dice coefficient and sensitivity, indicate the model efficacy in achieving precise segmentation. The results underscore the potential of CNNs in advancing medical image analysis for improved clinical outcomes.

Most current semantic segmentation approaches rely on deep convolutional neural networks (CNNs). However, their use of convolution operations with local receptive fields causes failures in modeling contextual spatial relations. Prior works have tried to address this issue by using graphical models or spatial propagation modules in networks. But such models often fail to capture long-range spatial relationships between entities, which leads to spatially fragmented predictions. In this work, we introduce a simple yet effective network unit, the spatial relation module, to learn and reason about global relationships between any two spatial positions, and then produce relation-enhanced feature representations. The spatial relation module is general and extensible, and can be used in a plug-and-play fashion with the existing fully convolutional network (FCN) framework. We evaluate spatial relation module-equipped networks on semantic segmentation tasks using two aerial image datasets. The networks achieve very competitive results, bringing significant improvements over baselines.

Carrier signal detection has been a problem for a long time, which is the first step for blind signal processing. In this paper, we propose a new method for carrier signal detection in the broadband power spectrum based on the fully convolutional network (FCN). FCN is a deep learning method and used in semantic image segmentation tasks. By regarding the broadband power spectrum sequence as a one-dimensional (1D) image and each subcarrier on the broadband as the target object, we can transform the carrier signal detection problem on the broadband into a semantic segmentation problem on a 1D image without prior knowledge. We design a 1D deep convolutional neural network (CNN) based on FCN to classify each point on broadband power spectrum array into two classes: subcarrier or noise, and then we can easily locate the subcarrier signals' position on the broadband power spectrum. We train the deep CNN on a simulation dataset and validate it on a real satellite broadband power spectrum dataset. The experimental results show that our method can effectively detect the subcarrier signal in the broadband power spectrum and achieve higher accuracy than the slope tracing method.

Unmanned Aerial Vehicles (UAVs) have revolutionized data acquisition across various domains, presenting immense potential for image processing and semantic segmentation. This literature review encompasses a thorough exploration of advancements, techniques, challenges, and datasets pertaining to UAV image semantic segmentation. It begins by introducing the fundamental concepts of UAVs, highlighting their pivotal role in capturing high-resolution imagery that serves diverse applications. The integration of deep learning algorithms with UAVs is emphasized, unlocking new horizons in autonomous flight, security, and environmental monitoring. Delving into the core principles of semantic segmentation, the review elucidates the critical task of classifying every pixel in an image. Convolutional Neural Networks (CNNs) are presented as the cornerstone technology, tracing their evolution from traditional CNNs to the highly adaptable Fully Convolutional Networks (FCNs). A substantial portion of the review is dedicated to FCNs, underscoring their ability to process images of varying dimensions while maintaining spatial coherence in the output. Their pivotal role in semantic segmentation, encompassing both classification and localization, is articulated. The subsequent sections delve into a comprehensive survey of state-of-the-art models, including SegNet, PSPNet, DeepLabNet, EfficientNet, DenseNet-C, and LinkNet. Each model's unique strengths and applications contribute to the evolving landscape of semantic segmentation tasks. The versatility of the U-Net architecture takes center stage in the latter parts of the review. Its fundamental structure is elucidated, followed by a comprehensive examination of its manifold adaptations—3D-U-Net, ResU-Net, U-Net++, Adversarial U-Net, Cascaded U-Net, and Improved U-Net 3+. These modifications address intrinsic challenges such as limited receptive fields and class imbalances, propelling U-Net to the forefront of image segmentation techniques. The subsequent sections pivot toward the application of U-Net in UAV image segmentation, illustrating its efficacy in diverse tasks, including land cover and crop classification. Nevertheless, persisting challenges, such as the scarcity of annotated datasets and the need for model generalization across varied environmental conditions, remain key areas of concern. The review culminates by underlining the significance of large, authentic datasets and data augmentation techniques. Furthermore, a brief exploration of publicly available UAV image datasets is presented, enhancing our understanding of the resources accessible for training and evaluating models. This comprehensive literature review encapsulates the dynamism of UAV image processing and semantic segmentation, illuminating recent developments and avenues for future research in this burgeoning field.

A Unified Architecture of Semantic Segmentation and Hierarchical Generative Adversarial Networks for Expression Manipulation

Accurate organ-level segmentation is essential for achieving high-throughput, non-destructive, and automated plant phenotyping. To address the challenge of intelligent acquisition of phenotypic parameters in tomato plants, we propose TSINet, an end-to-end dual-task segmentation network designed for effective and precise semantic labeling and instance recognition of tomato point clouds, based on the Pheno4D dataset. TSINet adopts an encoder–decoder architecture, where a shared encoder incorporates four Geometry-Aware Adaptive Feature Extraction Blocks (GAFEBs) to effectively capture local structures and geometric relationships in raw point clouds. Two parallel decoder branches are employed to independently decode shared high-level features for the respective segmentation tasks. Additionally, a Dual Attention-Based Feature Enhancement Module (DAFEM) is introduced to further enrich feature representations. The experimental results demonstrate that TSINet achieves superior performance in both semantic and instance segmentation, particularly excelling in challenging categories such as stems and large-scale instances. Specifically, TSINet achieves 97.00% mean precision, 96.17% recall, 96.57% F1-score, and 93.43% IoU in semantic segmentation and 81.54% mPrec, 81.69% mRec, 81.60% mCov, and 86.40% mWCov in instance segmentation. Compared with state-of-the-art methods, TSINet achieves balanced improvements across all metrics, significantly reducing false positives and false negatives while enhancing spatial completeness and segmentation accuracy. Furthermore, we conducted ablation studies and generalization tests to systematically validate the effectiveness of each TSINet component and the overall robustness of the model. This study provides an effective technological approach for high-throughput automated phenotyping of tomato plants, contributing to the advancement of intelligent agricultural management.

Environment monitoring, traffic management, and autonomous driving systems are numerous industries that use semantic segmentation of remote observation sensing data. High-resolution remote sensing satellite images have a large application area, include a lot of object information, and are challenging to extract information features due to the rapid growth of remote sensing technology. Motivated by the recent success of Convolutional Neural Network (CNN) for image semantic segmentation, this paper explores the power of CNN feature learning ability for remote sensing images, and serviceable image segmentation results are provided. A conventional Encoder-Decoder segmentation network is proposed for remotely sensed data. To further boost the performance of semantic segmentation, a variety of segmentation losses are explained and utilized. Evaluation metrics are introduced, and a series of experiments are conducted. The results demonstrate our proposed loss-modified CNN model's competitive performance against other methods.

Recent progress in semantic segmentation has been driven by improving the spatial resolution under Fully Convolutional Networks (FCNs). To address this problem, we propose a Stacked Deconvolutional Network (SDN) for semantic segmentation. In SDN, multiple shallow deconvolutional networks, which are called as SDN units, are stacked one by one to integrate contextual information and bring the fine recovery of localization information. Meanwhile, inter-unit and intra-unit connections are designed to assist network training and enhance feature fusion since the connections improve the flow of information and gradient propagation throughout the network. Besides, hierarchical supervision is applied during the upsampling process of each SDN unit, which enhances the discrimination of feature representations and benefits the network optimization. We carry out comprehensive experiments and achieve the new state-ofthe- art results on four datasets, including PASCAL VOC 2012, CamVid, GATECH, COCO Stuff. In particular, our best model without CRF post-processing achieves an intersection-over-union score of 86.6% in the test set.

Semantic segmentation with deep learning has achieved great progress in classifying the pixels in the image. However, the local location information is usually ignored in the high-level feature extraction by the deep learning, which is important for image semantic segmentation. To avoid this problem, we propose a graph model initialized by a fully convolutional network (FCN) named Graph-FCN for image semantic segmentation. Firstly, the image grid data is extended to graph structure data by a convolutional network, which transforms the semantic segmentation problem into a graph node classification problem. Then we apply graph convolutional network to solve this graph node classification problem. As far as we know, it is the first time that we apply the graph convolutional network in image semantic segmentation. Our method achieves competitive performance in mean intersection over union (mIOU) on the VOC dataset(about 1.34% improvement), compared to the original FCN model.

Medical image analysis is critical yet challenged by the need of jointly segmenting organs or tissues, and numerous instances for anatomical structures and tumor microenvironment analysis. Existing studies typically formulated different segmentation tasks in isolation, which overlooks the fundamental interdependencies between these tasks, leading to suboptimal segmentation performance and insufficient medical image understanding. To address this issue, we propose a Co-Seg++ framework for versatile medical segmentation. Specifically, we introduce a novel co-segmentation paradigm, allowing semantic and instance segmentation tasks to mutually enhance each other. We first devise a spatio-sequential prompt encoder (SSP-Encoder) to capture long-range spatial and sequential relationships between segmentation regions and image embeddings as prior spatial constraints. Moreover, we devise a multi-task collaborative decoder (MTC-Decoder) that leverages cross-guidance to strengthen the contextual consistency of both tasks, jointly computing semantic and instance segmentation masks. Extensive experiments on diverse CT and histopathology datasets demonstrate that the proposed Co-Seg++ outperforms state-of-the-arts in the semantic, instance, and panoptic segmentation of dental anatomical structures, histopathology tissues, and nuclei instances. The source code is available at https://github.com/xq141839/Co-Seg-Plus.

In this study, we propose a segmentation model based on convolutional neural networks (CNNs) to address image segmentation challenges in computer vision. Prior to designing the model, the activation function and other modules of the convolutional neural network were optimized to meet specific requirements. The segmentation task was transformed into binary classification problem to simplify network calculations and improve efficiency. Additionally, the model utilized a mask map obtained from the semantic segmentation model to aid in instance segmentation. Class activation technology was introduced to extract feature mapping maps. The corresponding thermal maps were obtained to achieve target instance segmentation. To further validate the effectiveness of the segmentation model, simulation experiments were conducted on semantic segmentation and instance segmentation respectively. The results show that the accuracy of the basic semantic segmentation model reached 87.58%, while the average accuracy of the entire class of the optimized instance segmentation model reached 97.9%. Therefore, the research and design of image segmentation models demonstrate high accuracy and good robustness.

One of the most common tasks in medical imaging is semantic segmentation. Achieving this segmentation automatically has been an active area of research, but the task has been proven very challenging due to the large variation of anatomy across different patients. However, recent advances in deep learning have made it possible to significantly improve the performance of image recognition and semantic segmentation methods in the field of computer vision. Due to the data driven approaches of hierarchical feature learning in deep learning frameworks, these advances can be translated to medical images without much difficulty. Several variations of deep convolutional neural networks have been successfully applied to medical images. Especially fully convolutional architectures have been proven efficient for segmentation of 3D medical images. In this article, we describe how to build a 3D fully convolutional network (FCN) that can process 3D images in order to produce automatic semantic segmentations. The model is trained and evaluated on a clinical computed tomography (CT) dataset and shows state-of-the-art performance in multi-organ segmentation.

Due to the lack of quality annotation in medical imaging community, semi-supervised learning methods are highly valued in image semantic segmentation tasks. In this paper, an advanced consistency-aware pseudo-label-based self-ensembling approach is presented to fully utilize the power of Vision Transformer (ViT) and Convolutional Neural Network (CNN) in semi-supervised learning. Our proposed framework consists of a feature-learning module which is enhanced by ViT and CNN mutually, and a guidance module which is robust for consistency-aware purposes. The pseudo labels are inferred and utilized recurrently and separately by views of CNN and ViT in the feature-learning module to expand the data set and are beneficial to each other. Meanwhile, a perturbation scheme is designed for the feature-learning module, and averaging network weight is utilized to develop the guidance module. By doing so, the framework combines the feature-learning strength of CNN and ViT, strengthens the performance via dual-view co-training, and enables consistency-aware supervision in a semi-supervised manner. A topological exploration of all alternative supervision modes with CNN and ViT are detailed validated, demonstrating the most promising performance and specific setting of our method on semi-supervised medical image segmentation tasks. Experimental results show that the proposed method achieves state-of-the-art performance on a public benchmark data set with a variety of metrics. The code is publicly available (https://github.com/ziyangwang007/CV-SSL-MIS).