Deep supervision and atrous inception-based U-Net combining CRF for automatic liver segmentation from CT

Automatic segmentation of the liver and its lesion is an important step towards deriving quantitative biomarkers for accurate clinical diagnosis and computer-aided decision support systems. This paper presents a method to automatically segment liver and lesions in CT abdomen images using cascaded fully convolutional neural networks (CFCNs) and dense 3D conditional random fields (CRFs). We train and cascade two FCNs for a combined segmentation of the liver and its lesions. In the first step, we train a FCN to segment the liver as ROI input for a second FCN. The second FCN solely segments lesions from the predicted liver ROIs of step 1. We refine the segmentations of the CFCN using a dense 3D CRF that accounts for both spatial coherence and appearance. CFCN models were trained in a 2-fold cross-validation on the abdominal CT dataset 3DIRCAD comprising 15 hepatic tumor volumes. Our results show that CFCN-based semantic liver and lesion segmentation achieves Dice scores over 94% for liver with computation times below 100s per volume. We experimentally demonstrate the robustness of the proposed method as a decision support system with a high accuracy and speed for usage in daily clinical routine.

We aim at developing a foreground extraction method for automatic leaf identification in a monocotyledon image. In order to identify accurately all leaves in a two-dimensional plant image, it is critical to extract an exact plant region because accidental holes or breaks in the extracted foreground image may lead to wrong structural results in subsequent analysis steps. However, because monocotyledonous plants such as rice, wheat, and barley have many thin leaves and complex morphology, it is highly possible that their plant images have a lot of holes by self-occlusion between the leaves or by color change and image blurring. In addition, plant images usually have extremely thin regions caused by its thin shape. We propose a foreground extraction algorithm based on a fully convolutional neural network (CNN) and a dense conditional random field (CRF) to retain holes and breaks made by morphological characteristics of monocotyledonous plants and to eliminate the accidental holes and breaks. In our algorithm, a CNN plays a role in labeling pixels as foreground or not and a CRF strengthens connection between foreground pixels. By synergistic integration of both models, our proposed algorithm achieve a better foreground extraction accuracy for plant images. Experiments show that our proposed method effectively extracts foreground regions from a single 2-dimensional monocotyledonous plant image and is fast enough for high-throughput phenotyping.

Segmentation of livers and liver tumors is one of the most important steps in radiation therapy of hepatocellular carcinoma. The segmentation task is often done manually, making it tedious, labor intensive, and subject to intra-/inter- operator variations. While various algorithms for delineating organ-at-risks (OARs) and tumor targets have been proposed, automatic segmentation of livers and liver tumors remains intractable due to their low tissue contrast with respect to the surrounding organs and their deformable shape in CT images. The U-Net has gained increasing popularity recently for image analysis tasks and has shown promising results. Conventional U-Net architectures, however, suffer from three major drawbacks. First, skip connections allow for the duplicated transfer of low resolution information in feature maps to improve efficiency in learning, but this often leads to blurring of extracted image features. Secondly, high level features extracted by the network often do not contain enough high resolution edge information of the input, leading to greater uncertainty where high resolution edge dominantly affects the network's decisions such as liver and liver-tumor segmentation. Thirdly, it is generally difficult to optimize the number of pooling operations in order to extract high level global features, since the number of pooling operations used depends on the object size. To cope with these problems, we added a residual path with deconvolution and activation operations to the skip connection of the U-Net to avoid duplication of low resolution information of features. In the case of small object inputs, features in the skip connection are not incorporated with features in the residual path. Furthermore, the proposed architecture has additional convolution layers in the skip connection in order to extract high level global features of small object inputs as well as high level features of high resolution edge information of large object inputs. Efficacy of the modified U-Net (mU-Net) was demonstrated using the public dataset of Liver tumor segmentation (LiTS) challenge 2017. For liver-tumor segmentation, Dice similarity coefficient (DSC) of 89.72 %, volume of error (VOE) of 21.93 %, and relative volume difference (RVD) of - 0.49 % were obtained. For liver segmentation, DSC of 98.51 %, VOE of 3.07 %, and RVD of 0.26 % were calculated. For the public 3D Image Reconstruction for Comparison of Algorithm Database (3Dircadb), DSCs were 96.01 % for the liver and 68.14 % for liver-tumor segmentations, respectively. The proposed mU-Net outperformed existing state-of-art networks.

Dense conditional random fields (CRFs) have become a popular framework for modeling several problems in computer vision such as stereo correspondence and multiclass semantic segmentation. By modeli...

Selections are central to image editing, e.g., they are the starting point of common operations such as copy‐pasting and local edits. Creating them by hand is particularly tedious and scribble‐based techniques have been introduced to assist the process. By interpolating a few strokes specified by users, these methods generate precise selections. However, most of the algorithms assume a100% accurate input, and even small inaccuracies in the scribbles often degrade the selection quality, which imposes an additional burden on users. In this paper, we propose a selection technique tolerant to input inaccuracies. We use a dense conditional random field (CRF) to robustly infer a selection from possibly inaccurate input. Further, we show that patch‐based pixel similarity functions yield more precise selection than simple point‐wise metrics. However, efficiently solving a dense CRF is only possible in low‐dimensional Euclidean spaces, and the metrics that we use are high‐dimensional and often non‐Euclidean. We address this challenge by embedding pixels in a low‐dimensional Euclidean space with a metric that approximates the desired similarity function. The results show that our approach performs better than previous techniques and that two options are sufficient to cover a variety of images depending on whether the objects are textured.

In recent years, with the rapid development of artificial intelligence, simultaneous localization and mapping technology to solve robot perception problems has been widely used in automatic driving, robot navigation, augmented reality and other fields. In this paper, a method of reconstructing large outdoor 3D dense semantic map based on monocular vision is proposed. Firstly, the motion trajectory of the camera is estimated by visual odometer, and the depth data is obtained by monocular depth estimation algorithm. Meanwhile, a dense conditional random field (CRF) image segmentation system based on deep learning and super-pixel distribution is used for semantic segmentation. Then 2D image semantics are gradually transferred to 3D point clouds by Bayesian progressive label migration strategy and further optimize 3D labels through a novel CRF model. Super pixels are utilized to enforce smoothness and form robust PN high order potential. Finally, dense 3D semantic maps of urban environment is generated by arbitrary length image sequence. The algorithm is tested on KITTI dataset, and the constructed semantic map shows that the proposed algorithm can reconstruct globally consistent semantic map in large-scale outdoor scenes.

Intrinsic image decomposition aims to factorize an image into albedo (reflectance) and shading (illumination) sub-components. Being ill posed and under-constrained, it is a very challenging computer vision problem. There are infinite pairs of reflectance and shading images that can reconstruct the same input. To address the problem, Intrinsic Images in the Wild by Bell et al. provides an optimization framework based on a dense conditional random field (CRF) formulation that considers long-range material relations. We improve upon their model by introducing illumination invariant image descriptors: color ratios. The color ratios and the intrinsic reflectance are both invariant to illumination and thus are highly correlated. Through detailed experiments, we provide ways to inject the color ratios into the dense CRF optimization. Our approach is physics based and learning free and leads to more accurate and robust reflectance decompositions.

This paper presents an improved automatic liver segmentation method using a left partial histogram threshold (LPHT) algorithm. The LPHT algorithm removes other neighboring abdominal organs regardless of pixel variation of contrast enhanced computed tomography (CE-CT) images. After histogram transformation, adaptive multi-modal threshold is used to find the range of gray-level values of the liver structure. The LPHT algorithm is performed to removing other neighboring organs. Then, binary morphological filtering is processed to remove unnecessary objects and smooth the boundary. 48 CE-CT slices of twelve patients were selected to test the proposed automatic liver segmentation. As evaluation methods, normalized average area and area error rate were used. The results of experiments show similar performance between the proposed algorithm and the manual method by a medical doctor.KeywordsLiver RegionContrast Enhanced Compute TomographyAutomatic SegmentationLiver StructureLiver SegmentationThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This paper proposes a glioma segmentation method based on neural networks. The base of the network is a UNet, expanded by residual blocks. Several preprocessing steps were applied before training, such as intensity normalization, high intensity cutting, cropping, and random flips. 2D and 3D solutions are implemented and tested, and results show that the 3D network outperforms 2D directions, therefore we stayed with 3D directions.The novelty of the method is the energy-based post-processing. Snakes [10], and conditional random fields (CRF) [11] were applied to the neural network’s predictions. Snake or active contour needs an initial outline around the object – e.g. the network’s prediction outline - and it can correct the contours of the tumor based on calculating the energy minimum, based on the intensity values at a given area. CRF is a specific type of graphical model, it uses the network’s prediction and the raw image features to estimate the posterior distribution (the tumor contour) using energy function minimization.The proposed methods are evaluated within the framework of the BRATS 2020 challenge. Measured on the test dataset the mean dice scores of the whole tumor (WT), tumor core (TC) and enhancing tumor (ET) are 86.9%, 83.2% and 81.8% respectively. The results show high performance and promising future work in tumor segmentation, even outside of the brain.KeywordsTumor segmentationMedical imagingNeural networksSupervised segmentationUNetResNetRadiation therapy

In Cassini ISS (Imaging Science Subsystem) images, contour detection is often performed on disk-resolved objects to accurately locate their center. Thus, contour detection is a key problem. Traditional edge detection methods, such as Canny and Roberts, often extract the contour with too much interior details and noise. Although the deep convolutional neural network has been applied successfully in many image tasks, such as classification and object detection, it needs more time and computer resources. In this paper, a contour detection algorithm based on H-ELM (Hierarchical Extreme Learning Machine) and DenseCRF (Dense Conditional Random Field) is proposed for Cassini ISS images. The experimental results show that this algorithm’s performance is better than both traditional machine learning methods, such as Support Vector Machine, Extreme Learning Machine and even deep Convolutional Neural Network. The extracted contour is closer to the actual contour. Moreover, it can be trained and tested quickly on the general configuration of PC, and thus can be applied to contour detection for Cassini ISS images.

Accurate segmentation of medical images is critical for disease diagnosis and treatment planning. However, challenges such as fuzzy boundaries, low contrast, and complex anatomical structures often hinder performance. We propose EGCF‐Net, a novel U‐Net‐based architecture that integrates a hybrid encoder with an edge‐aware graph‐based attention network to address these limitations. The hybrid encoder integrates the Swin transformer and Kronecker convolution to capture global and local contextual dependencies. Additionally, skip connections are enhanced using an edge‐aware graph‐based attention module, which combines graph spatial attention and graph channel attention to dynamically model spatial correlations and edge‐aware contextual affinities. This design leads to edge‐enhanced boundary delineation and improved regional consistency. We evaluate EGCF‐Net on four benchmark datasets (Synapse, Kidney Stone, ISIC 2016, and ISIC 2018), achieving Dice scores of 84.70%, 93.07%, 91.07%, and 88.62%, respectively, surpassing existing state‐of‐the‐art methods. Quantitative and qualitative results further validate the efficacy and robustness of the proposed approach, highlighting its potential for advancing medical image segmentation.

Context from within: Hierarchical context modeling for semantic segmentation

Automatic 3D model-based method for liver segmentation in MRI based on active contours and total variation minimization

IntroductionThe segmentation of uterine fibroids is very important for the treatment of patients. However, uterine fibroids are small and have low contrast with surrounding tissue, making this task very challenging. To solve these problems, this paper proposes a 3D DA- VNet automatic segmentation method based on deep supervision and attention gate.MethodsThis method can accurately segment uterine fibroids in MRI images by convolutional information. We used 3DVnet as the underlying network structure and added a deep monitoring mechanism in the hidden layer. We introduce attention gates during the upsampling process to enhance focus on areas of interest. The network structure is composed of VNet, deep supervision module and attention gate module. The dataset contained 245 cases of uterine fibroids and was divided into a training set, a validation set, and a test set in a ratio of 6:2:2. A total of 147 patients' T2-weighted magnetic resonance (T2WI) images were used for training, 49 for validation, and 49 patients' MR Images were used for algorithm testing.ResultsExperimental results show that the proposed method achieves satisfactory segmentation results. Dice similarity coefficient (DSC), intersection ratio (IOU), sensitivity, precision and Hausdorff distance (HD) were 0.878, 0.784, 0.879, 0.885 and 11.180 mm, respectively.DiscussionThis shows that our proposed method can improve the automatic segmentation accuracy of magnetic resonance image (MRI) data of uterine fibroids to a certain extent

An abdominal aortic aneurysm (AAA) is a dangerous cardiovascular disease that can cause serious health complications and death. Methods that can provide automatic and accurate segmentation of the AAA can significantly help in preoperative planning and postoperative follow-ups. Therefore, in this work, we present an automatic method for AAA segmentation from CT images using a modified 3D U-Net network with deep supervision. We compare obtained results for AAA segmentation using original 3D U-Net, and modified 3D U-Net with deep supervision. The trained network is evaluated on 19 volumetric CT images from the publicly available dataset provided by the University Hospitals Leuven, Belgium, using four-fold cross-validation. We obtained DSC of 91.03% using modified 3D U-Net with deep supervision. Additionally, we provide a discussion of the effects of using up-sampling versus deconvolution layers and its influence on the performance of both networks for this specific clinical application.