Vertebrae Identification and Localization Utilizing Fully Convolutional Networks and a Hidden Markov Model.

With the success of convolutional neural network (CNN) in the field of image processing and image recognition, an improved fully convolutional neural network (FCNN) based on CNN is proposed. Although FCNN has better performance than CNN, the number of parameters of FCNN has also increased significantly compared to CNN. Consequently, this paper proposes a multi‐feature fusion decision convolutional neural network (MFFD‐CNN), which use two kinds down‐sample methods in the pooling layer, and chooses the concatenate or add operation as the way that consolidates the feature of the convolutional layer with a stride of 2 and the max‐pooling layer to execute fusion operations. Meanwhile replaces the traditional softmax loss with an additive margin Softmax (AM‐Softmax) loss. The structure, which is based on the multi‐target classification experiment results of the MSTAR data set, shows can not only obtain an average correct recognition rate 1.2% higher than FCNN, but also better stability without augmenting the training samples of MSTAR data.

Abdominal image segmentation is a challenging, yet important clinical problem. Variations in body size, position, and relative organ positions greatly complicate the segmentation process. Historically, multi-atlas methods have achieved leading results across imaging modalities and anatomical targets. However, deep learning is rapidly overtaking classical approaches for image segmentation. Recently, Zhou et al. showed that fully convolutional networks produce excellent results in abdominal organ segmentation of computed tomography (CT) scans. Yet, deep learning approaches have not been applied to whole abdomen magnetic resonance imaging (MRI) segmentation. Herein, we evaluate the applicability of an existing fully convolutional neural network (FCNN) designed for CT imaging to segment abdominal organs on T2 weighted (T2w) MRI's with two examples. In the primary example, we compare a classical multi-atlas approach with FCNN on forty-five T2w MRI's acquired from splenomegaly patients with five organs labeled (liver, spleen, left kidney, right kidney, and stomach). Thirty-six images were used for training while nine were used for testing. The FCNN resulted in a Dice similarity coefficient (DSC) of 0.930 in spleens, 0.730 in left kidneys, 0.780 in right kidneys, 0.913 in livers, and 0.556 in stomachs. The performance measures for livers, spleens, right kidneys, and stomachs were significantly better than multi-atlas (p < 0.05, Wilcoxon rank-sum test). In a secondary example, we compare the multi-atlas approach with FCNN on 138 distinct T2w MRI's with manually labeled pancreases (one label). On the pancreas dataset, the FCNN resulted in a median DSC of 0.691 in pancreases versus 0.287 for multi-atlas. The results are highly promising given relatively limited training data and without specific training of the FCNN model and illustrate the potential of deep learning approaches to transcend imaging modalities.

This research study is to detect the Epileptic seizures using Novel Multi-Layered Convolutional Neural Networks (NMLCNN) and to compare the performance with Fully Convolutional Neural Networks (FCNN). By using Novel Multi-Layered Convolution Neural Networks algorithm works with the data collected from the observations of nervous system cells. The changes in the Epileptic seizures from the resource’s constraint, when the criteria met with randomizing, this will arrange in random ways for n times. The most typical beginning of a seizure is detectable through analysis of the Electroencephalogram (EEG). To examine this, the study utilized 20 samples in each group, with a Gpower value of 80%. The study used a predefined significance value and a Confidence Interval (CI) of 95%.This experiment found that NMLCNN has 92.25% and FCNN has 87.55% of accuracy for epileptic seizure. The Novel Multi-Layered Convolutional Neural Network achieved 0.012 significance using group analysis and provided significance. This probing concludes based on accuracy and significance results, detection of Epileptic seizures using Novel Multi-layered Convolutional Neural Networks has performed better compared to Fully Convolutional Neural Network.

With the leaps and bounds of computer performance and the advent of the era of big data, deep learning has drawn more and more attention from all walks of life. It can combine low-level features to form more abstract high-level features and describe the data more essentially. Therefore, it is widely used in various fields such as computer vision. Image segmentation is one of the most basic research topics in the field of computer vision. The main purpose is to extract regions of interest from images for later image processing research. In 3D reconstruction based on sequence images, the segmentation accuracy and speed of sequence images determine the quality and efficiency of target reconstruction. Therefore, when facing large-scale sequence images, the biggest problem is how to improve the segmentation speed while ensuring accuracy. Based on the above background, the research content of this article is an indoor scene segmentation algorithm based on full convolutional neural network. According to the characteristics of indoor application scenes, this paper proposes a fast convolutional neural network image segmentation method to segment the indoor scene image and construct the fast fully convolutional networks (FFCN) for indoor scene image segmentation uses inter-layer fusion to reduce the amount of network calculation parameters and avoid the loss of picture feature information by continuous convolution. In order to verify the effectiveness of the network, in this paper, a basic living object data set (XAUT data set) in an indoor environment is created. The XAUT data set is used to train the FFCN network under the Caffe framework to obtain an indoor scene segmentation model. In order to compare the effectiveness of the model, the structure of the worn FCN8s, FCN16s, and FCN32s models was fine-tuned, and the corresponding algorithm model for indoor scene segmentation was obtained by training with the XAUT data set. The experimental results show that the pixel recognition accuracy of all types of networks has reached 86%, and the mean IU ratio has reached more than 63%. The mean IU of the FCN8s network is the highest at 70.38%, but its segmentation speed is only 1/5 of FFCN. On the premise that other types of indicators are not much different, the average segmentation speed on FFCN fast segmentation convolutional neural network reaches 40 fps. It can be seen that the scale fusion technology can well avoid the loss of image feature information in the network convolution and reddening process. Compared with other FCN networks, it has a faster speed and is conducive to real-time image preprocessing.

This paper presents a new method for automatic localization and identification of vertebrae in arbitrary field-of-view CT scans. No assumptions are made about which section of the spine is visible or to which extent. Thus, our approach is more general than previous work while being computationally efficient. Our algorithm is based on regression forests and probabilistic graphical models. The discriminative, regression part aims at roughly detecting the visible part of the spine. Accurate localization and identification of individual vertebrae is achieved through a generative model capturing spinal shape and appearance. The system is evaluated quantitatively on 200 CT scans, the largest dataset reported for this purpose. We obtain an overall median localization error of less than 6mm, with an identification rate of 81%.

Unlike standard Convolutional Neural Networks (CNNs) with fully-connected layers, Fully Convolutional Neural Networks (FCN) are prevalent in computer vision applications such as object detection, semantic/image segmentation, and the most popular generative tasks based on Generative Adversarial Networks (GAN). In an FCN, traditional convolutional layers and deconvolutional layers contribute to the majority of the computation complexity. However, prior deep learning accelerator designs mostly focus on CNN optimization. They either use independent compute-resources to handle deconvolution or convert deconvolutional layers (Deconv) into general convolution operations, which arouses considerable overhead. To address this problem, we propose a unified fully convolutional accelerator aiming to handle both the deconvolutional and convolutional layers with a single processing element (PE) array. We re-optimize the conventional CNN accelerator architecture of regular 2D processing elements array, to enable it more efficiently support the data flow of deconvolutional layer inference. By exploiting the locality in deconvolutional filters, this architecture reduces the consumption of on-chip memory communication from 24.79 GB to 6.56 GB and improves the power efficiency significantly. Compared to prior baseline deconvolution acceleration scheme, the proposed accelerator achieves 1.3X -- 44.9X speedup and reduces the energy consumption by 14.6%-97.6% on a set of representative benchmark applications. Meanwhile, it keeps similar CNN inference performance to that of an optimized CNN-only accelerator with negligible power consumption and chip area overhead.

In the field of 3D Human Pose Estimation and Reconstruction based on body joints extracted from a 2D image; Exists challenges like self-occlusion and depth perception. These problems hinder approximate estimations. This article proposes a hybrid method that consists of semantic segmentation, sparse representation and 2D pose estimation for 3D estimation. Specifically, we train two fully Convolution neural networks (FCNs) to estimate 2D pose and semantic segmentation. Then, we narrow the estimation down using basic human body structure and results from the FCNs. Subsequently, utilizing the estimated 2D pose of the earlier step and a sparse representation model the 3D pose will estimate. Using a Convolution Neural Networks learning method for 2D human pose estimation and then by sparse representation will estimate. Results of the proposed method, show improvement relative to the previous works on 3D Human pose estimation. 3D Human Pose Estimation by this method demonstrates a significant mean error reduction compared to earlier studies.

Land cover and its change are crucial for many environmental applications. This study focuses on the land cover classification and change detection with multitemporal and multispectral Sentinel-2 satellite data. To address the challenging land cover change detection task, we rely on two different deep learning architectures and selected pre-processing steps. For example, we define an excluded class and deal with temporal water shoreline changes in the pre-processing. We employ a fully convolutional neural network (FCN), and we combine the FCN with long short-term memory (LSTM) networks. The FCN can only handle monotemporal input data, while the FCN combined with LSTM can use sequential information (multitemporal). Besides, we provided fixed and variable sequences as training sequences for the combined FCN and LSTM approach. The former refers to using six defined satellite images, while the latter consists of image sequences from an extended training pool of ten images. Further, we propose measures for the robustness concerning the selection of Sentinel-2 image data as evaluation metrics. We can distinguish between actual land cover changes and misclassifications of the deep learning approaches with these metrics. According to the provided metrics, both multitemporal LSTM approaches outperform the monotemporal FCN approach, about 3 to 5 percentage points (p.p.). The LSTM approach trained on the variable sequences detects 3 p.p. more land cover changes than the LSTM approach trained on the fixed sequences. Besides, applying our selected pre-processing improves the water classification and avoids reducing the dataset effectively by 17.6%. The presented LSTM approaches can be modified to provide applicability for a variable number of image sequences since we published the code of the deep learning models. The Sentinel-2 data and the ground truth are also freely available.

In recent years, convolutional neural networks (CNNs) have been successfully applied to inverse synthetic aperture radar (ISAR) sparse imaging because of their powerful ability in feature extraction. However, these CNNs only adopt single path feed-forward architectures and lack paths for directly transmitting original feature representations (OFRs) in shallow layers to reconstruction layers, which limits the complete reconstruction of target shape due to the underutilization of the OFRs that are efficient for recovering target details. Later, fully CNN (FCNN) introduces several skip connections (SKs) to establish the additional ways for directly passing the OFRs to the reconstruction layers. Nevertheless, the transmitted OFRs inevitably include the feature information of artifacts, which usually results the appearance of artifacts in final reconstructed target image. To address this issue, we introduce the gate units to FCNN, and refer to the improved FCNN as G-FCNN. Furthermore, the learnable gate units weight the OFRs transmitted by SKs and autonomously decide how many OFRs are transmitted further. To circumvent the shortage of the real data available for network training, we utilize the transfer learning strategy to guarantee a good performance of the G-FCNN. The imaging results of real data show that the G-FCNN-based imaging method is superior to the existing CNN-based imaging methods.

This paper introduces a new approach to automatically quantify the severity of knee OA using X-ray images. Automatically quantifying knee OA severity involves two steps: first, automatically localizing the knee joints; next, classifying the localized knee joint images. We introduce a new approach to automatically detect the knee joints using a fully convolutional neural network (FCN). We train convolutional neural networks (CNN) from scratch to automatically quantify the knee OA severity optimizing a weighted ratio of two loss functions: categorical cross-entropy and mean-squared loss. This joint training further improves the overall quantification of knee OA severity, with the added benefit of naturally producing simultaneous multi-class classification and regression outputs. Two public datasets are used to evaluate our approach, the Osteoarthritis Initiative (OAI) and the Multicenter Osteoarthritis Study (MOST), with extremely promising results that outperform existing approaches.

The visual inspection of massive civil infrastructure is a common trend for maintaining its reliability and structural health. However, this procedure, which uses human inspectors, requires long inspection times and relies on the subjective and empirical knowledge of the inspectors. To address these limitations, a machine vision-based autonomous crack detection method is proposed using a deep convolutional neural network (DCNN) technique. It consists of a fully convolutional neural network (FCN) with an encoder and decoder framework for semantic segmentation, which performs pixel-wise classification to accurately detect cracks. The main idea is to capture the global context of a scene and determine whether cracks are in the image while also providing a reduced and essential picture of the crack locations. The visual geometry group network (VGGNet), a variant of the DCCN, is employed as a backbone in the proposed FCN for end-to-end training. The efficacy of the proposed FCN method is tested on a publicly available benchmark dataset of concrete crack images. The experimental results indicate that the proposed method is highly effective for concrete crack classification, obtaining scores of approximately 92% for both the recall and F1 average.

We propose a learning method to generate synthetic CT (sCT) image for MRI-only radiation treatment planning. The proposed method integrated a dense-block concept into a cycle-generative adversarial network (cycle-GAN) framework, which is named as dense-cycle-GAN in this study. Compared with GAN, the cycle-GAN includes an inverse transformation between CT (ground truth) and sCT, which could further constrain the learning model. A 2.5D fully convolution neural network (FCN) with dense-block was introduced in generator to enable end-to-end transformation. A FCN is used in discriminator to urge the generator’s sCT to be similar with the ground-truth CT images. The well-trained model was used to generate the sCT of a new MRI. This proposed algorithm was evaluated using 14 patients’ data with both MRI and CT images. The mean absolute error (MAE), peak signal-to-noise ratio (PSNR) and normalized cross correlation (NCC) indexes were used to quantify the correction accuracy of the prediction algorithm. Overall, the MAE, PSNR and NCC were 60.9&minus;11.7 HU, 24.6&plusmn;0.9 dB, and 0.96&plusmn;0.01. We have developed a novel deep learning-based method to generate sCT with a high accuracy. The proposed method makes the sCT comparable to that of the planning CT. With further evaluation and clinical implementation, this method could be a useful tool for MRI-based radiation treatment planning and attenuation correction in a PET/MRI scanner.

Remote sensing data is an important way to reflect the comprehensive information of surface. In this paper, based on the semantic segmentation of high-resolution remote sensing images, a segmentation method based on full convolutional neural network (FCN) is proposed. The method improves the traditional convolutional neural network (CNN) and replaces the final fully connected layer of the CNN network with a convolutional layer. And then optimize the convolution operation by using the matrix expansion technique. The experimental results show that the FCN network with sufficient training and fine-tuning can effectively perform automatic semantic segmentation of high-resolution remote sensing images. The correct segmentation accuracy is higher than 85%, which improves the efficiency of convolution operations.

Instrument segmentation plays a vital role in 3D ultrasound (US) guided cardiac intervention. Efficient and accurate segmentation during the operation is highly desired since it can facilitate the operation, reduce the operational complexity, and therefore improve the outcome. Nevertheless, current image-based instrument segmentation methods are not efficient nor accurate enough for clinical usage. Lately, fully convolutional neural networks (FCNs), including 2D and 3D FCNs, have been used in different volumetric segmentation tasks. However, 2D FCN cannot exploit the 3D contextual information in the volumetric data, while 3D FCN requires high computation cost and a large amount of training data. Moreover, with limited computation resources, 3D FCN is commonly applied with a patch-based strategy, which is therefore not efficient for clinical applications. To address these, we propose a POI-FuseNet, which consists of a patch-of-interest (POI) selector and a FuseNet. The POI selector can efficiently select the interested regions containing the instrument, while FuseNet can make use of 2D and 3D FCN features to hierarchically exploit contextual information. Furthermore, we propose a hybrid loss function, which consists of a contextual loss and a class-balanced focal loss, to improve the segmentation performance of the network. With the collected challenging ex-vivo dataset on RF-ablation catheter, our method achieved a Dice score of 70.5%, superior to the state-of-the-art methods. In addition, based on the pre-trained model from ex-vivo dataset, our method can be adapted to the in-vivo dataset on guidewire and achieves a Dice score of 66.5% for a different cardiac operation. More crucially, with POI-based strategy, segmentation efficiency is reduced to around 1.3 seconds per volume, which shows the proposed method is promising for clinical use.

Diabetic retinopathy data augmentation and vessel segmentation through deep learning based three fully convolution neural networks