Convolutional Neural Networks Based Level Set Framework for Pancreas Segmentation from CT Images

Image segmentation plays an important role in computer vision and image processing. Level set model is a classical image segmentation method. During level set evolution, almost all the level set energy minimization of image segmentation are based on the gradient descent method and the finite difference scheme. The speed of evolution is slow and easy to fall into local minima. In this paper, we propose a fast sweeping optimization algorithm to minimize global cosine fitting (GCF) model. When moving a pixel from the one side region to the another side region of evolving contour, the sweeping algorithm calculates the energy change directly and checks whether the cosine fitting energy is decreased. With this, we can avoid solving the Euler-Lagrange equation and the partial differential equation, which usually take a lot of time. Moreover, our proposal is robust to initial level set contour and it automatically handles the topological variety, algorithm automatic termination and no longer requires the reinitialization step, parameter adjustment and the distance regularization term. The experiments on real noise and synthetic images show the effectiveness of the sweeping algorithm.

X-ray computed tomography (CT) imaging plays an essential role in disease diagnosis due to its noninvasive, painless mode and superior penetration depth. However, the resolution of the soft tissue ...

Concurrent two-scale topological design of multiple unit cells and structure using combined velocity field level set and density model

Segmentation of hippocampus in the brain is one of a major challenge in medical image segmentation due to its’ imaging characteristics, with almost similar intensity between another adjacent gray matter structure, such as amygdala. The intensity similarity has causes the hippocampus to have weak or fuzzy boundaries. With this main challenge being demonstrated by hippocampus, a segmentation method that relies on image information alone may not produce accurate segmentation results. Therefore, it is needed an assimilation of prior information such as shape and spatial information into existing segmentation method to produce the expected segmentation. Previous studies has widely integrated prior information into segmentation methods. However, the prior information has been utilized through a global manner integration, and this does not reflect the real scenario during clinical delineation. Therefore, in this paper, a locally integrated prior information into a level set model is presented. This work utilizes a mean shape model to provide automatic initialization for level set evolution, and has been integrated as prior information into the level set model. The local integration of edge based information and prior information has been implemented through an edge weighting map that decides at voxel level which information need to be observed during a level set evolution. The edge weighting map shows which corresponding voxels having sufficient edge information. Experiments shows that the proposed integration of prior information locally into a conventional edge-based level set model, known as geodesic active contour has shown improvement of 9% in averaged Dice coefficient.

Detection of metastasis of mediastinal lymph nodes in lung cancer patients with an artificial intelligence model.

Automatic identification of consistently defined body regions in medical images is vital in many applications. In this paper, we describe a method to automatically demarcate the superior and inferior boundaries for neck, thorax, abdomen, and pelvis body regions in computed tomography (CT) images. For any three-dimensional (3D) CT image I, following precise anatomic definitions, we denote the superior and inferior axial boundary slices of the neck, thorax, abdomen, and pelvis body regions by NS(I), NI(I), TS(I), TI(I), AS(I), AI(I), PS(I), and PI(I), respectively. Of these, by definition, AI(I)=PS(I), and so the problem reduces to demarcating seven body region boundaries. Our method consists of a two-step approach. In the first step, a convolutional neural network (CNN) is trained to classify each axial slice in I into one of nine categories: the seven body region boundaries, plus legs (defined as all axial slices inferior to PI(I)), and the none-of-the-above category. This CNN uses a multichannel approach to exploit the interslice contrast, providing the neural network with additional visual context at the body region boundaries. In the second step, to improve the predictions for body region boundaries that are very subtle and that exhibit low contrast, a recurrent neural network (RNN) is trained on features extracted by CNN, limited to a flexible window about the predictions from the CNN. The method is evaluated on low-dose CT images from 442 patient scans, divided into training and testing sets with a ratio of 70:30. Using only the CNN, overall absolute localization error for NS(I), NI(I), TS(I), TI(I), AS(I), AI(I), and PI(I) expressed in terms of number of slices (nS) is (mean±SD): 0.61±0.58, 1.05±1.13, 0.31±0.46, 1.85±1.96, 0.57±2.44, 3.42±3.16, and 0.50±0.50, respectively. Using the RNN to refine the CNN's predictions for select classes improved the accuracy of TI(I) and AI(I) to: 1.35±1.71 and 2.83±2.75, respectively. This model outperforms the results achieved in our previous work by 2.4, 1.7, 3.1, 1.1, and 2 slices, respectively for TS(I), TI(I), AS(I), AI(I)=PS(I), and PI(I) classes with statistical significance. The model trained on low-dose CT images was also tested on diagnostic CT images for NS(I), NI(I), and TS(I) classes; the resulting errors were: 1.48±1.33, 2.56±2.05, and 0.58±0.71, respectively. Standardized body region definitions are a prerequisite for effective implementation of quantitative radiology, but the literature is severely lacking in the precise identification of body regions. The method presented in this paper significantly outperforms earlier works by a large margin, and the deviations of our results from ground truth are comparable to variations observed in manual labeling by experts. The solution presented in this work is critical to the adoption and employment of the idea of standardized body regions, and clears the path for development of applications requiring accurate demarcations of body regions. The work is indispensable for automatic anatomy recognition, delineation, and contouring for radiation therapy planning, as it not only automates an essential part of the process, but also removes the dependency on experts for accurately demarcating body regions in a study.

Computer algorithms that simulate lower-doses computed tomography (CT) images from clinical-dose images are widely available. However, most operate in the projection domain and assume access to the reconstruction method. Access to commercial reconstruction methods may often not be available in medical research, making image-domain noise simulation methods useful. However, the introduction of non-linear reconstruction methods, such as iterative and deep learning-based reconstruction, makes noise insertion in the image domain intractable, as it is not possible to determine the noise textures analytically. To develop a deep learning-based image-domain method to generate low-dose CT images from clinical-dose CT (CDCT) images for non-linear reconstruction methods. We propose a fully image domain-based method, utilizing a series of three convolutional neural networks (CNNs), which, respectively, denoise CDCT images, predict the standard deviation map of the low-dose image, and generate the noise power spectra (NPS) of local patches throughout the low-dose image. All three models have U-net-based architectures and are partly or fully three-dimensional. As a use case for this study and with no loss of generality, we use paired low-dose and clinical-dose brain CT scans. A dataset of paired scans was retrospectively obtained. All images were acquired with a wide-area detector clinical system and reconstructed using its standard clinical iterative algorithm. Each pair was registered using rigid registration to correct for motion between acquisitions. The data was randomly partitioned into training ( samples), validation ( samples), and test ( samples) sets. The performance of each of these three CNNs was validated separately. For the denoising CNN, the local standard deviation decrease, and bias were determined. For the standard deviation map CNN, the real and estimated standard deviations were compared locally. Finally, for the NPS CNN, the NPS of the synthetic and real low-dose noise were compared inside and outside the skull. Two proof-of-concept denoising studies were performed to determine if the performance of a CNN- or a gradient-based denoising filter on the synthetic low-dose data versus real data differed. The denoising network had a median decrease in noise in the cerebrospinal fluid by a factor of and introduced a median bias of HU. The network for standard deviation map estimation had a median error of HU. The noise power spectrum estimation network was able to capture the anisotropic and shift-variant nature of the noise structure by showing good agreement between the synthetic and real low-dose noise and their corresponding power spectra. The two proof of concept denoising studies showed only minimal difference in standard deviation improvement ratio between the synthetic and real low-dose CT images with the median difference between the two being 0.0 and +0.05 for the CNN- and gradient-based filter, respectively. The proposed method demonstrated good performance in generating synthetic low-dose brain CT scans without access to the projection data or to the reconstruction method. This method can generate multiple low-dose image realizations from one clinical-dose image, so it is useful for validation, optimization, and repeatability studies of image-processing algorithms.

In this paper, we propose a coupled level set (LS) framework for segmentation of bladder wall using T(1)-weighted magnetic resonance (MR) images with clinical applications to virtual cystoscopy (i.e., MR cystography). The framework uses two collaborative LS functions and a regional adaptive clustering algorithm to delineate the bladder wall for the wall thickness measurement on a voxel-by-voxel basis. It is significantly different from most of the pre-existing bladder segmentation work in four aspects. First of all, while most previous work only segments the inner border of the wall or at most manually segments the outer border, our framework extracts both the inner and outer borders automatically except that the initial seed point is given by manual selection. Secondly, it is adaptive to T(1)-weighted images with decreased intensities in urine, as opposed to enhanced intensities in T(2)-weighted scenario and computed tomography. Thirdly, by considering the image global intensity distribution and local intensity contrast, the defined image energy function in the framework is more immune to inhomogeneity effect, motion artifacts and image noise. Finally, the bladder wall thickness is measured by the length of integral path between the two borders which mimic the electric field line between two iso-potential surfaces. The framework was tested on six datasets with comparison to the well-known Chan-Vese (C-V) LS model. Five experts blindly scored the segmented inner and outer borders of the presented framework and the C-V model. The scores demonstrated statistically the improvement in detecting the inner and outer borders.

Achieving high feature reproducibility while preserving biological information is one of the main challenges for the generalizability of current radiomics studies. Non-clinical imaging variables, such as reconstruction kernels, have shown to significantly impact radiomics features. In this study, we retrain an open-source convolutional neural network (CNN) to harmonize computerized tomography (CT) images with various reconstruction kernels to improve feature reproducibility and radiomic model performance using epidermal growth factor receptor (EGFR) mutation prediction in lung cancer as a paradigm. In the training phase, the CNN was retrained and tested on 32 lung cancer patients’ CT images between two different groups of reconstruction kernels (smooth and sharp). In the validation phase, the retrained CNN was validated on an external cohort of 223 lung cancer patients’ CT images acquired using different CT scanners and kernels. The results showed that the retrained CNN could be successfully applied to external datasets with different CT scanner parameters, and harmonization of reconstruction kernels from sharp to smooth could significantly improve the performance of radiomics model in predicting EGFR mutation status in lung cancer. In conclusion, the CNN based method showed great potential in improving feature reproducibility and generalizability by harmonizing medical images with heterogeneous reconstruction kernels.

This paper presents a new local edge-based level set model that does not use initial contours. Unlike traditional edge-based active contours that use gradient to detect edges, our model derives the neighborhood distribution and edge information with two different localized region-based operators: a Gaussian mixture model-based intensity distribution estimator and the Hueckel operator. We incorporate the operator outcomes into the recently proposed local binary fitting (LBF) model as local distribution fitting (LDF) model, which enables a model without the initial contour selection, i.e., the level set function can be initialized with a random constant instead of a distance map. Thus our model overcomes the initialization sensitivity problem of most active contours. In addition, with region-based edge detection, the proposed LDF model provides more accurate and robust segmentation. Experiments on both synthetic and real images show the improved performance of our proposed model over the LBF model.

A deep learning based active contour framework is proposed for pancreas segmentation. Data extension and fractional differential operation are firstly applied for pre-processing. Second, deep learning method is designed to acquire the initial contour of pancreas. Subsequently, an intensity constrained term is designed to stop the contours at the edges. The intensity constrained term is integrated into a variational active contour model with three terms. The accurate pancreas segmentation is obtained by the evolution of the active contour model. Our approach reaches high detection dice similarity coefficient (DSC) of 83% and sensitivity of 85% in a dataset containing 40 abdominal CT scans. Comparisons with other level set models provide evidence that the proposed method offers desirable performances.

Based on Global Cancer 2015 statistics, the lung cancer of all types constitutes 27% of overall cancers while 19.5% of cancer deaths are due to lung cancer. In lieu of this, an effective lung cancer screening test using Computed Tomography (CT) scan is crucial to detect cancer at the early stage. The interpretation of the CT images requires an advanced CAD system of high accuracy for instance, in classifying the lung nodules. Recently, Deep Learning method that is Convolution Neural Network (CNN) shows an outstanding success in lung nodules classification. However, the training of CNN requires a great number of images. Such a requirement is an issue in the case of medical images. Generative adversarial network (GAN) has been introduced to generate new image datasets for CNN training. Thus, the main objective of this study is to compare the performance of CNN architectures with and without the implementation of GAN for lung nodules classification in CT images. Here, the study used Conditional GAN (cGAN) to generate benign nodules images. The classification accuracy of the combined cGAN-CNN architecture was compared among CNN pretraining networks namely GoogleNet, ShuffleNet, DenseNet, and MobileNet based on classification accuracy, specificity, sensitivity, and AUC-ROC values. The experiment was tested on LIDC-IDRI database. The results showed cGAN-CNN architecture improves the overall classification accuracy as compared to CNN alone with the cGAN-ShuffleNet architecture performed the best, achieving 98.38% accuracy, 98.13% specificity, 100% sensitivity and AUC-ROC at 99.90%. Overall, the classification performance of CNN can be improved by integrating GAN architecture to mitigate the constraint of having a large medical image dataset, in this case, CT lung nodules images.

Automatic segmentation of organs-at-risk from head-and-neck CT using separable convolutional neural network with hard-region-weighted loss

Segmentation of the target object(s) from images that have multiple complicated regions, mixture intensity distributions or are corrupted by noise poses a challenge for the level set models. In addition, the conventional piecewise smooth level set models normally require prior knowledge about the number of image segments. To address these problems, we propose a novel segmentation energy function with two distribution descriptors to model the background and the target. The single background descriptor models the heterogeneous background with multiple regions. Then, the target descriptor takes into account the intensity distribution and incorporates local spatial constraint. Our descriptors, which have more complete distribution information, construct the unique energy function to differentiate the target from the background and are more tolerant of image noise. We compare our approach to three other level set models: 1) the Chan-Vese; 2) the multiphase level set; and 3) the geodesic level set. This comparison using 260 synthetic images with varying levels and types of image noise and medical images with more complicated backgrounds showed that our method outperforms these models for accuracy and immunity to noise. On an additional set of 300 synthetic images, our model is also less sensitive to the contour initialization as well as to different types and levels of noise.

Interactive free-form level-set surface-editing operators