Detonator coded character spotting based on convolutional neural networks

Hybrid CNN-transformer network for interactive learning of challenging musculoskeletal images

Optical pressure sensors based plantar image segmenting using an improved fully convolutional network

Land cover is an important variable of the terrestrial ecosystem that provides information for natural resources management, urban sprawl detection, and environment research. To classify land cover with high-spatial-resolution multispectral remote sensing imagery is a difficult problem due to heterogeneous spectral values of the same object on the ground. Fully convolutional networks (FCNs) are a state-of-the-art method that has been increasingly used in image segmentation and classification. However, a systematic quantitative comparison of FCNs on high-spatial-multispectral remote imagery was not yet performed. In this paper, we adopted the three FCNs (FCN-8s, Segnet, and Unet) for Gaofen-2 (GF2) satellite imagery classification. Two scenes of GF2 with a total of 3329 polygon samples were used in the study area and a systematic quantitative comparison of FCNs was conducted with red, green, blue (RGB) and RGB+near infrared (NIR) inputs for GF2 satellite imagery. The results showed that: (1) The FCN methods perform well in land cover classification with GF2 imagery, and yet, different FCNs architectures exhibited different results in mapping accuracy. The FCN-8s model performed best among the Segnet and Unet architectures due to the multiscale feature channels in the upsampling stage. Averaged across the models, the overall accuracy (OA) and Kappa coefficient (Kappa) were 5% and 0.06 higher, respectively, in FCN-8s when compared with the other two models. (2) High-spatial-resolution remote sensing imagery with RGB+NIR bands performed better than RGB input at mapping land cover, and yet the advantage was limited; the OA and Kappa only increased an average of 0.4% and 0.01 in the RGB+NIR bands. (3) The GF2 imagery provided an encouraging result in estimating land cover based on the FCN-8s method, which can be exploited for large-scale land cover mapping in the future.

Early detection and treatment are crucial for recovery from cancer. Due to rapid advancement in medical imaging techniques, today doctors can scan various body parts from different perspectives to diagnose cancer or see the progress during a course of treatment. Such techniques are less painful and require less preparation and recovery. The recent evolution of AI, particularly deep learning algorithms have shown potential in computer-aided diagnosis of cancer. These include analysis of cancer images for detection of tumor region, prognosis, skin tissue inspection, etc. Convolutional Neural Network (CNN) based image processing techniques are showing promising results in the diagnosis of breast cancer, skin cancer, prostate cancer, lung cancer, etc. Literature shows many other CNN-based models such as feed-forward neural network, Deep CNN, Multi-Crop CNN, fully convolutional networks (FCNs), Deep Fully Convolutional Network (DFCNet), Recurrent neural networks, etc. The CNN focus solely on the image-specific features and hence require a lesser number of input parameters. A CNN model reduces the size of the input image vector without losing the features critical for making an accurate prediction. This makes it easier to process and reduces computational complexity. CNNs are also not sensitive to the object positioning in an image since the approximate position of image motifs relative to others is known. CNN deals with the variations in images through augmentation and other pre-processing techniques. This makes them robust to various straining conditions in images. This chapter presents an overview of a generic CNN architecture followed by a discussion on major advances in CNN models and their applications for cancer image analysis.

Objective: Accurate and fast automatic segmentation of organs is a key step for efficient planning for radiation therapy. In this paper, we propose the Multi-mode CNNs, which is for accurate and fast automatic segmentation of mandible, parotid glands, brainstem, optic nerves, cerebellum, eyes, lens, pituitary, thyroid, temporal lobes, brain and head respectively in 3D CT image of head and neck.Methods: The proposed Multi-mode CNNs consist of three convolutional neural networks. The first CNN is a simple classification network to distinguish slices of head and neck. It accelerates the next step by reducing the search space. The second network is a 3DCNN for locating the region of interest (ROI) of organs. It can enhance the robustness of the algorithm to CT images from different hospitals. The third network is a full convolution network (FCN) based on U-Net. This is a pixel-wise detailed segmented network to classify voxels in a region of interest and generate segmentation results. We used public and clinical datasets to evaluate our algorithm. The public datasets are from the MICCAI 2015 Head and Neck Auto Segmentation Grand Challenge. For the private datasets, we collected 3D CT images of 88 patients from A and B Hospital.Results: For the public datasets, our segmentation results of the nine organs in the challenge surpassed the first rank. For the private datasets, clinical trials have proved that it is effective for real image data in different hospitals. The average total segmentation time for one patient with above whole organs is less than one minute, which indicates proposed method having clinical value.Conclusion: The proposed Multi-mode CNNs can be used to accurate and fast automatic segmentation of organs in radiation therapy. It can assist doctors in outlining the organs during the planning of radiation therapy in the clinic.

In this paper, a new convolution neural network method is proposed for the inspection and classification of galvanized stamping parts. Firstly, all workpieces are divided into normal and defective by image processing, and then the defective workpieces extracted from the region of interest (ROI) area are input to the trained fully convolutional networks (FCN). The network utilizes an end-to-end and pixel-to-pixel training convolution network that is currently the most advanced technology in semantic segmentation, predicts result of each pixel. Secondly, we mark the different pixel values of the workpiece, defect and background for the training image, and use the pixel value and the number of pixels to realize the recognition of the defects of the output picture. Finally, the defect area's threshold depended on the needs of the project is set to achieve the specific classification of the workpiece. The experiment results show that the proposed method can successfully achieve defect detection and classification of galvanized stamping parts under ordinary camera and illumination conditions, and its accuracy can reach 99.6%. Moreover, it overcomes the problem of complex image preprocessing and difficult feature extraction and performs better adaptability.

Farmland is the cornerstone of agriculture and is important for food security and social production. Farmland assessment is essential but traditional methods are usually expensive and slow. Deep learning methods have been developed and widely applied recently in image recognition, semantic understanding, and many other application domains. In this research, we used fully convolutional networks (FCN) as the deep learning model to evaluate farmland grades. Normalized difference vegetation index (NDVI) derived from Landsat images was used as the input data, and the China National Cultivated Land Grade Database within Jiangsu Province was used to train the model on cloud computing. We also applied an image segmentation method to improve the original results from the FCN and compared the results with classical machine learning (ML) methods. Our research found that the FCN can predict farmland grades with an overall F1 score (the harmonic mean of precision and recall) of 0.719 and F1 score of 0.909, 0.590, 0.740, 0.642, and 0.023 for non-farmland, level I, II, III, and IV farmland, respectively. Combining the FCN and image segmentation method can further improve prediction accuracy with results of fewer noise pixels and more realistic edges. Compared with conventional ML, at least in farmland evaluation, FCN provides better results with higher precision, recall, and F1 score. Our research indicates that by using remote sensing NDVI data, the deep learning method can provide acceptable farmland assessment without fieldwork and can be used as a novel supplement to traditional methods. The method used in this research will save a lot of time and cost compared with traditional means.

Computed tomography (CT) is commonly used as a diagnostic and treatment planning imaging modality in craniomaxillofacial (CMF) surgery to correct patient's bony defects. A major disadvantage of CT is that it emits harmful ionizing radiation to patients during the exam. Magnetic resonance imaging (MRI) is considered to be much safer and noninvasive, and often used to study CMF soft tissues (e.g., temporomandibular joint and brain). However, it is extremely difficult to accurately segment CMF bony structures from MRI since both bone and air appear to be black in MRI, along with low signal-to-noise ratio and partial volume effect. To this end, we proposed a 3D deep-learning based cascade framework to solve these issues. Specifically, a 3D fully convolutional network (FCN) architecture is first adopted to coarsely segment the bony structures. As the coarsely segmented bony structures by FCN tend to be thicker, convolutional neural network (CNN) is further utilized for fine-grained segmentation. To enhance the discriminative ability of the CNN, we particularly concatenate the predicted probability maps from FCN and the original MRI, and feed them together into the CNN to provide more context information for segmentation. Experimental results demonstrate a good performance and also the clinical feasibility of our proposed 3D deep-learning based cascade framework.

Bearded seal vocalizations were recorded by four spatially separated receivers on the Chukchi Continental Slope in Alaska in 2016–2017. Bearded seals vocalizations are often analyzed manually or by using automatic detections that are manually validated. An automatic detection and classification system (DCS) based on convolutional neural networks (CNNs) is proposed. The DCS is divided into two sections. First, regions of interest (ROI) containing potential bearded seal calls are detected through a 2D normalized cross-correlation of the measured spectrogram and a representative template of the two calls of interest considered in this work. Second, CNNs are used to classify the ROIs to determine if they are noise or a specified vocalization. The CNNs are trained on 80% of the ROIs manually labeled from one of the recorders and validated on the remaining 20% with a classification accuracy above 95.5%. When assessing the generalization performance of the DCS, we tested on the remaining three recorders located at different positions, obtaining a precision above 89.2% for the main class of the two types of calls. This study provides evidence that CNNs are suitable for classifying bearded seal vocalizations from ROIs found by classic detection techniques. [Work supported by ONR via Grant No. N00014-18-1-2140.]

Lung segmentation of chest CT scan is utilised to identify lung cancer and this step is also critical in other diagnostic pathways. Therefore, powerful algorithms to accomplish this accurate segmentation task are highly needed in the medical imaging domain, where the tumours are required to be segmented with the lung parenchyma. Also, the lung parenchyma needs to be detached from the tumour regions that are often confused with the lung tissue. Recently, lung semantic segmentation is more suitable to allocate each pixel in the image to a predefined class based on fully convolutional networks (FCNs). In this paper, CT cancer scans from the Task06_Lung database were applied to FCN that was inspired by V.Net architecture for efficiently selecting a region of interest (ROI) using the 3D segmentation. This lung database is segregated into 64 training images and 32 testing images. The proposed system is generalised by three steps including data preprocessing, data augmentation and neural network based on the V-Net model. Then, it was evaluated by dice score coefficient (DSC) to calculate the ratio of the segmented image and the ground truth image. This proposed system outperformed other previous schemes for 3D lung segmentation with an average DCS of 80% for ROI and 98% for surrounding lung tissues. Moreover, this system demonstrated that 3D views of lung tumours in CT images precisely carried tumour estimation and robust lung segmentation.

Automatic fetal brain extraction from 2D in utero fetal MRI slices using deep neural network

Fully Convolutional Network (FCN), which can adopt various Convolutional Neural Networks (CNN), are now increasingly being used in remote sensing communities. CNN are improved constantly either in accuracy or by reducing parameters for a given equivalent accuracy. This paper investigates five widely used CNNs (AlexNet, VGG16, ResNet, SqueezeNet, and a pruned VGG16) in the context of FCN for coastal beach classification of imagery acquired by Unmanned Aerial Vehicles (UAV). Our experiments show that (1) not every CNN is suitable to FCN for semantic segmentations of images though each CNN approximately achieved an equivalent accuracy for image labeling; (2) band reduced pruning of existing CNN has the least impact on implementation and accuracy. To examine the capability of convolutional layers capturing semantic features, this paper also carries out beach classification experiments using hypercolumn methods with VGG16.

Nuclei segmentation is a fundamental step to measure cell motility and morphology in High-throughput microscopy images. One prerequisite of automatic nuclei segmentation is to manually label a huge amount of cell images, which is a labor-intensive and time-consuming process. This paper develops a semi-supervised learning approach to reduce the dependence on the amount of labeled images, and it consists of three main steps. First, cell regions are automatically extracted from the background through a novel convolutional neural network (CNN). Comparing with the state-of-art segmentation network structure, our designed network structure has fewer parameters and combines layers of the feature hierarchy that refines the spatial precision of the output probability map. Second, discriminator networks are explored to produce confidence maps for unlabeled microscopy images by applying a fully convolution network (FCN) as the discriminator network. It also can be used to improve the performance of the segmentation network by coupling the adversarial loss with standard cross-entropy loss. Finally, combining both labeled images and unlabeled images with confidence maps to train the segmentation network can identify individual nuclei on the extracted cell regions. Experimental results on only 84 images showed that SEG score could achieve 77.9% and F1 score could achieve 76.0%, which outperformed existing using 670 images segmentation methods. Such promising result suggested that the proposed semi-supervised nuclei segmentation approach could achieve high segmentation accuracy with a small set of labeled images, which is a desirable property to analyze microscopy images in the real clinical practice.

As a function of the spatial position of the optical coherence tomography (OCT) image, retinal layer thickness is an important diagnostic indicator for many retinal diseases. Reliable segmentation of the retinal layer is necessary for extracting useful clinical information. However, manual segmentation of these layers is time-consuming and prone to bias. Furthermore, due to speckle noise, low image contrast, retinal detachment, and also irregular morphological features make the automatic segmentation task challenging. To alleviate these challenges, in this paper, we propose a new coarse-fine framework combining the full convolutional network (FCN) with a multiphase level set (named FCN-MLS) for automatic segmentation of nine boundaries in retinal spectral OCT images. In the coarse stage, FCN is used to learn the characteristics of specific retinal layer boundaries and achieve classification of four retinal layers. The boundaries are then extracted and the remaining boundaries are initialized based on a priori information about the thickness of the retinal layer. In order to prevent the overlapping of the segmentation interfaces, a regional restriction technique is used in the multi-phase level to evolve the boundaries to achieve fine nine retinal layers segmentation. Experimental results on 1280 B-scans show that the proposed method can segment nine retinal boundaries accurately. Compared with the manual delineation, the overall mean absolute boundary location difference and the overall mean absolute thickness difference were 5.88 ± 2.38μm and 5.81 ± 2.19μm, which showed a good consistency with manual segmentation by the physicians. Our experimental results also outperform state-of-the-art methods.

This study proposes an automatic building footprint extraction framework that consists of a convolutional neural network (CNN)-based segmentation and an empirical polygon regularization that transforms segmentation maps into structured individual building polygons. The framework attempts to replace part of the manual delineation of building footprints that are involved in surveying and mapping field with algorithms. First, we develop a scale robust fully convolutional network (FCN) by introducing multiple scale aggregation of feature pyramids from convolutional layers. Two postprocessing strategies are introduced to refine the segmentation maps from the FCN. The refined segmentation maps are vectorized and polygonized. Then, we propose a polygon regularization algorithm consisting of a coarse and fine adjustment, to translate the initial polygons into structured footprints. Experiments on a large open building data set including 181 000 buildings showed that our algorithm reached a high automation level where at least 50% of individual buildings in the test area could be delineated to replace manual work. Experiments on different data sets demonstrated that our FCN-based segmentation method outperformed several most recent segmentation methods, and our polygon regularization algorithm is robust in challenging situations with different building styles, image resolutions, and even low-quality segmentation.