Context from within: Hierarchical context modeling for semantic segmentation

This paper introduces a deep architecture for segmenting 3D objects into their labeled semantic parts. Our architecture combines image-based Fully Convolutional Networks (FCNs) and surface-based Conditional Random Fields (CRFs) to yield coherent segmentations of 3D shapes. The image-based FCNs are used for efficient view-based reasoning about 3D object parts. Through a special projection layer, FCN outputs are effectively aggregated across multiple views and scales, then are projected onto the 3D object surfaces. Finally, a surface-based CRF combines the projected outputs with geometric consistency cues to yield coherent segmentations. The whole architecture (multi-view FCNs and CRF) is trained end-to-end. Our approach significantly outperforms the existing state-of-the-art methods in the currently largest segmentation benchmark (ShapeNet). Finally, we demonstrate promising segmentation results on noisy 3D shapes acquired from consumer-grade depth cameras.

In technologically underdeveloped areas, water pollution threatens the living environment of local residents, so remote sensing monitoring of the features around reservoirs is necessary. Fully convolutional networks (FCNs) offer great potential for extracting high-resolution features due to their unlimited input image size and higher accuracy compared to convolutional neural networks. Therefore, a proposal to classify WorldView-2 images is implemented with a sixty-eight thousand iterations of fine-tuning and fully trained combined training method based on a fully convolutional network (SEFCN). The chosen images depict the urban area of Yingde, which is located to the northeast of the Feilaixia Reservoir, Qingyuan, Guangdong Province, China. The SEFCN combines an FCN-32s and FCN-16s to better integrate the deep features and shallow features and effectively improve the classification accuracy. Additionally, the loss value fully converges with enough iterations, and the overfitting caused by the superposition of two full trainings is avoided. The SEFCN model achieves the highest accuracy among all compared classification models and obtains the best classified results on the WorldView-2 images, as demonstrated by attaining the highest F1 scores in each category. After the classified images are optimized using a conditional random field, the current status of the study area is analyzed, and several suggestions for land use in the urban area of Yingde are made. The experiments still have deficiencies in the application of high-resolution remote sensing image classification with different sensors, and the classification results can be optimized and improved in other aspects.

Mass segmentation provides effective morphological features which are important for mass diagnosis. In this work, we propose a novel end-to-end network for mammographic mass segmentation which employs a fully convolutional network (FCN) to model a potential function, followed by a conditional random field (CRF) to perform structured learning. Because the mass distribution varies greatly with pixel position, the FCN is combined with a position priori. Further, we employ adversarial training to eliminate over-fitting due to the small sizes of mammogram datasets. Multi-scale FCN is employed to improve the segmentation performance. Experimental results on two public datasets, IN breast and DDSM-BCRP, demonstrate that our end-to-end network achieves better performance than state-of-the-art approaches.

Recently, fully convolutional network (FCN) has been successfully used to locate spliced regions in synthesized images. However, all the existing FCN-based algorithms use real-valued FCN to process each channel separately. As a consequence, they fail to capture the inherent correlation between color channels and the integrity of three channels. So, in this paper, quaternion fully convolutional network (QFCN) is proposed to generalize FCN to quaternion domain by replacing real-valued conventional blocks in FCN with quaternion conventional blocks. In addition, a new color image splicing localization algorithm is proposed by combining QFCNs and superpixel (SP)-enhanced pairwise conditional random field (CRF). QFCNs consider three different versions (QFCN32, QFCN16, and QFCN8) with different up-sampling layers. The SP-enhanced pairwise CRF is used to refine the results of QFCNs. Experimental results on three publicly available datasets demonstrate that the proposed algorithm outperforms the existing algorithms including some conventional algorithms and some deep learning-based algorithms.

Brain tumors can happen in the brain tissue. Nearby locations include nerves, the pituitary gland, the pineal gland, and the membranes that cover the surface of the brain. Although MRI images are a popular imaging method for evaluating these tumors, the volume of data it generates makes it difficult to manually segment the images in a reasonable amount of time, which restricts the use of precise quantitative assessments in clinical settings. The enormous spatial and structural heterogeneity among brain tumors makes automatic segmentation a difficult task, hence dependable and automatic segmentation methods are needed. This project focuses on developing deep learning models based on convolutional neural network and watershed algorithms to perform the automated semantic image segmentation of the MRI images of the brain. We explore the current state of the CNNs architecture and evaluate them on the BraTS dataset. Different regularization methods and hyperparameters are tested and optimized through a series of experiments. Finally, a web application is created so that the developed models can be used easily by medical practitioners. Key Words: CNN, Deep Learning, Histogram of Oriented Gradients (HOG), Fully Convolutional Networks (FCNs), Conditional Random Fields (CRFs), Support Vector Machine (SVM).

Cardiac Magnetic Resonance (CMR) imaging is commonly used to assess cardiac structure and function. One disadvantage of CMR is that postprocessing of exams is tedious. Without automation, precise assessment of cardiac function via CMR typically requires an annotator to spend tens of minutes per case manually contouring ventricular structures. Automatic contouring can lower the required time per patient by generating contour suggestions that can be lightly modified by the annotator. Fully convolutional networks (FCNs), a variant of convolutional neural networks, have been used to rapidly advance the state-of-the-art in automated segmentation, which makes FCNs a natural choice for ventricular segmentation. However, FCNs are limited by their computational cost, which increases the monetary cost and degrades the user experience of production systems. To combat this shortcoming, we have developed the FastVentricle architecture, an FCN architecture for ventricular segmentation based on the recently developed ENet architecture. FastVentricle is 4\(\times \) faster and runs with 6\(\times \) less memory than the previous state-of-the-art ventricular segmentation architecture while still maintaining excellent clinical accuracy.

Hybrid CNN-transformer network for interactive learning of challenging musculoskeletal images

Mapping horizontal and vertical urban densification in Denmark with Landsat time-series from 1985 to 2018: A semantic segmentation solution

Deep learning models represent the state of the art in medical image segmentation. Most of these models are fully-convolutional networks (FCNs), namely each layer processes the output of the preceding layer with convolution operations. The convolution operation enjoys several important properties such as sparse interactions, parameter sharing, and translation equivariance. Because of these properties, FCNs possess a strong and useful inductive bias for image modeling and analysis. However, they also have certain important shortcomings, such as performing a fixed and pre-determined operation on a test image regardless of its content and difficulty in modeling long-range interactions. In this work we show that a different deep neural network architecture, based entirely on self-attention between neighboring image patches and without any convolution operations, can achieve more accurate segmentations than FCNs. Our proposed model is based directly on the transformer network architecture. Given a 3D image block, our network divides it into non-overlapping 3D patches and computes a 1D embedding for each patch. The network predicts the segmentation map for the block based on the self-attention between these patch embeddings. Furthermore, in order to address the common problem of scarcity of labeled medical images, we propose methods for pre-training this model on large corpora of unlabeled images. Our experiments show that the proposed model can achieve segmentation accuracies that are better than several state of the art FCN architectures on two datasets. Our proposed network can be trained using only tens of labeled images. Moreover, with the proposed pre-training strategies, our network outperforms FCNs when labeled training data is small.

The manual systems for road safety are inefficient, very time consuming and prone to error. Automated systems face many challenges in detecting all road safety objects accurately and accurately measuring the distance between these objects. Therefore, in this paper, a novel approach is proposed to analyze roadside video data in order to assess important attributes pertaining to road safety. The specific aim is to learn and analyse Australian Road Assessment Program (AusRAP) attributes obtained from digital video recordings (DVRs). In this work, a technique for segmentation and classification based on Fully Convolutional Network (FCN) is investigated. The proposed approach takes video frames as input, extracts features through convolutional layer and learns them to classify in fully connected layer. The novelty in our approach is learning road attributes and its application in road safety. The proposed approach is optimized using multiple image sizes and training epochs. The 411 images obtained from roadside video data provided by the Department of Transport and Main Roads (DTMR), Queensland, Australia have been used to analyze the proposed technique. The evaluation results are discussed, and a comparative analysis is presented.

We propose a novel, multi-task, fully convolutional network (FCN) architecture for automatic segmentation of brain tumor. This network extracts multi-level contextual information by concatenating hierarchical feature representations extracted from multimodal MR images along with their symmetric-difference images. It achieves improved segmentation performance by incorporating boundary information directly into the loss function. The proposed method was evaluated on the BRATS13 and BRATS15 datasets and compared with competing methods on the BRATS13 testing set. Segmented tumor boundaries obtained were better than those obtained by single-task FCN and by FCN with CRF. The method is among the most accurate available and has relatively low computational cost at test time.

BackgroundAccurate preoperative differentiation of breast masses is critical for guiding individualized treatment strategies in Mammotome-assisted minimally invasive resection. While radiomics shows promise, existing methods rely on manual delineation, which is time-consuming and subjective. This study developed an ultrasound-based semi-automatic segmentation ensemble model to improve preoperative assessment.MethodsWe retrospectively analyzed preoperative ultrasound images from 773 patients (543 tumors, 230 non-tumors). Semi-automatic segmentation was performed using DeepLabv3_ResNet50 and fully convolutional network (FCN)_ResNet50. Radiomic and deep transfer learning (DTL) features were extracted to construct radiomic, deep learning, and combined models. An ensemble strategy integrated these with clinical models. Performance was evaluated via receiver operating characteristic (ROC) curves and decision curve analysis (DCA).ResultsThe cohort included 543 tumor patients and 230 non-tumor patients (95 adenosis, 135 other benign lesions). The semi-automatic segmentation model, DeepLabv3_ResNet50, achieved a peak global accuracy of 99.4% and an average Dice coefficient of 92.0% at its best epoch. On the other hand, the FCN_ResNet50 model exhibited a peak global accuracy of 99.5% and an average Dice coefficient of 93.7% at its best epoch. In the task of predicting tumor and non-tumor patients, age, maximum diameter, and BI-RADS (Breast Imaging Reporting and Data System) classification were ultimately identified as key indicators, and the stacking model ultimately demonstrated an area under the curve (AUC) of 0.890 in the training cohort (with a sensitivity of 0.844 and a specificity of 0.815) and an AUC of 0.780 in the testing cohort (with a sensitivity of 0.713 and a specificity of 0.739). In the task of predicting adenosis and other lesion types, focus emerged as a crucial factor, and the stacking model achieved an AUC of 0.813 in the training cohort (with a sensitivity of 0.613 and a specificity of 0.859) and an AUC of 0.771 in the testing cohort (with a sensitivity of 0.759 and a specificity of 0.765).ConclusionsOur study has established an ensemble learning model grounded in semi-automatic segmentation techniques. This model accurately distinguishes between tumor and non-tumor patients preoperatively, as well as discriminating adenosis from other lesion types among the non-tumor cohort, thus providing valuable insights for individualized treatment planning. The proposed stacking model demonstrates significant clinical utility by reducing unnecessary biopsies and saving diagnostic time compared to manual review. These improvements directly address the challenges of overtreatment and diagnostic delays in breast lesion management. By enhancing preoperative accuracy, our model supports tailored surgical planning and alleviates patient anxiety associated with indeterminate diagnoses.

A surgical simulator with elaborate artificial eyeball models has been developed for ophthalmic surgeries, in which sophisticated skills are required. To create the elaborate eyeball models with microstructures included in an eyeball, a database of eyeball models should be compiled by segmenting eye structures based on high-resolution medical images. Therefore, this paper presents an automated segmentation of eye structures from micro-CT images by using Fully Convolutional Networks (FCNs). In particular, we aim to construct a method for accurately segmenting eye structures from sparse annotation data. This method performs end-to-end segmentation of eye structures, including a workflow from training the FCN based on sparse annotation to obtaining the segmentation of the entire eyeball. We use the FCN trained on the slices sparsely annotated in a micro-CT volume to segment the remaining slices in the same volume. To achieve accurate segmentation from less annotated images, the multi-class segmentation is performed by using the network trained on the preprocessed and augmented micro-CT images; in the preprocessing, we apply filters for removing ring artifacts and random noises to the images, while in the data augmentation process, rotation and elastic deformation operations are performed on the sparsely-annotated training data. From the results of experiments for evaluating segmentation performances based on sparse annotation, we found that the FCN trained with data augmentation could achieve high segmentation accuracy of more than 90% even from a sparse training subset of only 2.5% of all slices.

In this paper, we propose a You Only Look Once version 11 (YOLOv11)-based road boundary detection method that uses visual images for autonomous vehicle navigation and maintenance of road condition. The main objective of this paper is to create the YOLOv11 model and then study the effectiveness of the YOLOv11 model under different image noises, such as the Salt-and-Pepper noise and the Gaussian noise, and image noise filtering techniques (Gaussian filter, Median filter, and moving average filter) in terms of benchmark metrics such as mean absolute error (MAE) and intersection over union (IoU) metrics. The evaluation results demonstrate that the performance of deep learning models, such as the fully convolutional networks (FCN) model, the PyTorch model, the DeepLabv3+ model and the YOLOv11 model, is degraded depending on the type of image noise and the level of noise (or noise density). The results further showed that the performance of the deep models can be improved with suitable image noise filtering techniques. Although data augmentation involves adding noise to training data, it helps models learn to be more robust to real-world imperfections and variations in data. The proposed YOLOv11-based road boundary detection model outperforms the other three models (FCN, Pytorch model and DeepLabv3+) for different image qualities and filtering techniques.

Hyper-fusion network for semi-automatic segmentation of skin lesions.