Automatic anatomical classification of esophagogastroduodenoscopy images using deep convolutional neural networks

Freezing of gait (FOG) is one of the most debilitating motor symptoms experienced by patients with Parkinson's disease (PD). FOG detection is possible using acceleration data from wearable sensors, and a convolutional neural network (CNN) is often used to determine the presence of FOG epochs. We compared the performance of a standard CNN for the detection of FOG with two more complex networks, which are well suited for time series data, the MiniRocket and the InceptionTime. We combined acceleration data of people with PD across four studies. The final data set was split into a training (80%) and hold-out test (20%) set. A fifth study was included as an unseen test set. The data were windowed (2 s) and five-fold cross-validation was applied. The CNN, MiniRocket, and InceptionTime models were evaluated using a receiver operating characteristic (ROC) curve and its area under the curve (AUC). Multiple sensor configurations were evaluated for the best model. The geometric mean was subsequently calculated to select the optimal threshold. The selected model and threshold were evaluated on the hold-out and unseen test set. A total of 70 participants (23.7 h, 9% FOG) were included in this study for training and testing, and in addition, 10 participants provided an unseen test set (2.4 h, 11% FOG). The CNN performed best (AUC = 0.86) in comparison to the InceptionTime (AUC = 0.82) and MiniRocket (AUC = 0.76) models. For the CNN, we found a similar performance for a seven-sensor configuration (lumbar, upper and lower legs and feet; AUC = 0.86), six-sensor configuration (upper and lower legs and feet; AUC = 0.87), and two-sensor configuration (lower legs; AUC = 0.86). The optimal threshold of 0.45 resulted in a sensitivity of 77% and a specificity of 58% for the hold-out set (AUC = 0.72), and a sensitivity of 85% and a specificity of 68% for the unseen test set (AUC = 0.90). We confirmed that deep learning can be used to detect FOG in a large, heterogeneous dataset. The CNN model outperformed more complex networks. This model could be employed in future personalized interventions, with the ultimate goal of using automated FOG detection to trigger real-time cues to alleviate FOG in daily life.

The Part 1 prequel to this review evaluated the evolution of modelling techniques used in evaluation of fruit quality over the past three decades and noted a progression towards the use of artificial neural networks (ANNs) and convolutional neural networks (CNNs). In this review, Part 2, the use of CNNs for NIR fruit quality evaluation is explored, given the success of CNNs in various other fields, such as image, video, speech, and audio processing, and the availability of large (open source) datasets of fruit spectra and reference quality attribute, which is required for the training of CNN models. The review provides an overview of deep learning and the CNN architectures and techniques used in NIR spectroscopy for regression modelling, with advantages and disadvantages identified. Studies using CNN for NIR based fruit quality evaluation are then critically examined. Eight publications have presented on models using the same open-source mango dry matter calibration and test set, enabling inter-method comparisons. CNN models have been demonstrated to be accurate, precise and robust. Techniques of transfer learning for CNN models offer an alternative solution to model updating and calibration transfer methods applied in traditional chemometrics. The review has highlighted crucial areas that require resolution and exploration in this application through future research, including, (i) data requirements for training a CNN (ii) optimal spectral pre-processing for CNN (iii) CNN architecture and hyper-parameter selection and tuning for fruit quality evaluation (iv) CNN model interpretability and explainability. Future studies must conduct clearer comparison to partial least squares (PLS) regression and shallow ANNs to better assess the prospective benefit of using CNN, a more complex model. The potential for visualisation of spectra relevance to the CNN model using techniques such as GradCam, currently employed in visualising 2D-CNN models, remains to be explored.

BackgroundIn recent months, multiple publications have demonstrated the use of convolutional neural networks (CNN) to classify images of skin cancer as precisely as dermatologists. However, these CNNs failed to outperform the International Symposium on Biomedical Imaging (ISBI) 2016 challenge which ranked the average precision for classification of dermoscopic melanoma images. Accordingly, the technical progress represented by these studies is limited. In addition, the available reports are impossible to reproduce, due to incomplete descriptions of training procedures and the use of proprietary image databases or non-disclosure of used images. These factors prevent the comparison of various CNN classifiers in equal terms.ObjectiveTo demonstrate the training of an image-classifier CNN that outperforms the winner of the ISBI 2016 CNNs challenge by using open source images exclusively.MethodsA detailed description of the training procedure is reported while the used images and test sets are disclosed fully, to insure the reproducibility of our work.ResultsOur CNN classifier outperforms all recent attempts to classify the original ISBI 2016 challenge test data (full set of 379 test images), with an average precision of 0.709 (vs. 0.637 of the ISBI winner) and with an area under the receiver operating curve of 0.85.ConclusionThis work illustrates the potential for improving skin cancer classification with enhanced training procedures for CNNs, while avoiding the use of costly equipment or proprietary image data.

In this study, a method of predicting the seismic responses of building structures based on a convolutional neural network (CNN) is proposed. In the method, the time histories of acceleration responses previously measured in a building during earthquakes are used in the CNN input layer, with the corresponding time histories of the displacement responses being used in the CNN output layer. The correlations between the features automatically extracted from the acceleration responses by the convolution and pooling operations in the various CNN layers and the displacement responses in the CNN output layer are established through CNN training. The trained CNN predicts the displacement responses for a future earthquake event using only the measured acceleration responses. To verify the validity of the proposed method, a numerical study on the ASCE benchmark model and an experimental study on a reinforced concrete frame structure are conducted. In the numerical study, the structural responses of the ASCE model subject to artificial earthquakes are utilized for CNN training, and the performance of the trained CNN is verified through seismic response prediction. In the experimental study, shaking table tests are performed for various earthquakes to measure the seismic responses of the reinforced concrete test structure. The seismic response prediction performance of the proposed method is examined using CNN trained with the measured seismic responses. Furthermore, the performances of CNNs trained with different numbers of datasets generated by the proposed data generation method based on data overlapping with the same data pool are discussed with related to the number of CNN training iteration number.

Tuberculosis (TB) remains a leading global health challenge, especially in resource-constrained regions due to limited access to expert radiology. To address this diagnostic gap, this study introduces a deep learning-based framework for automated TB detection using chest X-rays. The proposed model makes use of Convolutional Neural Networks (CNNs) to find and sort features that are important for the detection of TB wherein Ant Colony Optimization (ACO) is used as a metaheuristic method to improve hyperparameters. This approach enables the system to overcome local minima, a common limitation in traditional gradient-based algorithms by simulating the foraging behavior of ants, the ACO efficiently explores the search space for optimal learning rate, batch size, and network architecture. The pipeline achieves an accuracy of around 98% through extensive preprocessing steps-including adaptive histogram equalization, normalization, and data augmentation, CNN training, and careful evaluation to improve model generalization and address class imbalance on publicly available TB chest X-ray datasets from Kaggle, significantly outperforming traditional optimization methods. Local Interpretable Model-agnostic Explanations (LIME) are used to ensure transparency and build trust by letting the doctors view the most important areas that affect TB predictions, which boosts their confidence in the diagnosis. This research shows that scalable, explainable AI-powered TB screening is possible.

Background Deep learning may help to improve computer-aided detection of volume (CADv) measurement of pulmonary nodules at chest CT. Purpose To determine the efficacy of a deep learning method for improving CADv for measuring the solid and ground-glass opacity (GGO) volumes of a nodule, doubling time (DT), and the change in volume at chest CT. Materials and Methods From January 2014 to December 2016, patients with pulmonary nodules at CT were retrospectively reviewed. CADv without and with a convolutional neural network (CNN) automatically determined total nodule volume change per day and DT. Area under the curves (AUCs) on a per-nodule basis and diagnostic accuracy on a per-patient basis were compared among all indexes from CADv with and without CNN for differentiating benign from malignant nodules. Results The CNN training set was 294 nodules in 217 patients, the validation set was 41 nodules in 32 validation patients, and the test set was 290 nodules in 188 patients. A total of 170 patients had 290 nodules (mean size ± standard deviation, 11 mm ± 5; range, 4-29 mm) diagnosed as 132 malignant nodules and 158 benign nodules. There were 132 solid nodules (46%), 106 part-solid nodules (36%), and 52 ground-glass nodules (18%). The test set results showed that the diagnostic performance of the CNN with CADv for total nodule volume change per day was larger than DT of CADv with CNN (AUC, 0.94 [95% confidence interval {CI}: 0.90, 0.96] vs 0.67 [95% CI: 0.60, 0.74]; P < .001) and CADv without CNN (total nodule volume change per day: AUC, 0.69 [95% CI: 0.62, 0.75]; P < .001; DT: AUC, 0.58 [95% CI: 0.51, 0.65]; P < .001). The accuracy of total nodule volume change per day of CADv with CNN was significantly higher than that of CADv without CNN (P < .001) and DT of both methods (P < .001). Conclusion Convolutional neural network is useful for improving accuracy of computer-aided detection of volume measurement and nodule differentiation capability at CT for patients with pulmonary nodules. © RSNA, 2020 Online supplemental material is available for this article.

Man against machine reloaded: performance of a market-approved convolutional neural network in classifying a broad spectrum of skin lesions in comparison with 96 dermatologists working under less artificial conditions

A convolutional neural network trained with dermoscopic images performed on par with 145 dermatologists in a clinical melanoma image classification task

Using Convolutional Neural Networks for Enhanced Capture of Breast Parenchymal Complexity Patterns Associated with Breast Cancer Risk

Assess the efficacy of deep convolutional neural networks (DCNNs) in detection of critical enteric feeding tube malpositions on radiographs. 5475 de-identified HIPAA compliant frontal view chest and abdominal radiographs were obtained, consisting of 174 x-rays of bronchial insertions and 5301 non-critical radiographs, including normal course, normal chest, and normal abdominal x-rays. The ground-truth classification for enteric feeding tube placement was performed by two board-certified radiologists. Untrained and pretrained deep convolutional neural network models for Inception V3, ResNet50, and DenseNet 121 were each employed. The radiographs were fed into each deep convolutional neural network, which included untrained and pretrained models. The Tensorflow framework was used for Inception V3, ResNet50, and DenseNet. Images were split into training (4745), validation (630), and test (100). Both real-time and preprocessing image augmentation strategies were performed. Receiver operating characteristic (ROC) and area under the curve (AUC) on the test data were used to assess the models. Statistical differences among the AUCs were obtained. p < 0.05 was considered statistically significant. The pretrained Inception V3, which had an AUC of 0.87 (95 CI; 0.80–0.94), performed statistically significantly better (p < .001) than the untrained Inception V3, with an AUC of 0.60 (95 CI; 0.52–0.68). The pretrained Inception V3 also had the highest AUC overall, as compared with ResNet50 and DenseNet121, with AUC values ranging from 0.82 to 0.85. Each pretrained network outperformed its untrained counterpart. (p < 0.05). Deep learning demonstrates promise in differentiating critical vs. non-critical placement with an AUC of 0.87. Pretrained networks outperformed untrained ones in all cases. DCNNs may allow for more rapid identification and communication of critical feeding tube malpositions.

Aims: This study aimed to explore the effect of training set diversity on the performance of deep learning models for predicting early gastric cancer (EGC) under endoscopy. Methods: Images of EGC and non-cancerous lesions under narrow-band imaging (ME-NBI) and magnifying blue laser imaging (ME-BLI) were retrospectively collected. Training set 1 was composed of 150 non-cancerous and 309 EGC ME-NBI images, training set 2 was composed of 1505 non-cancerous and 309 EGC ME-BLI images, and training set 3 was the combination of training set 1 and 2. Test set 1 was composed of 376 non-cancerous and 1052 EGC ME-NBI images, test set 2 consisted of 529 non-cancerous and 71 EGC ME-BLI images, and test set 3 was the combination of test set 1 and test set 2. Three deep learning models, convolutional neural network (CNN) 1, CNN 2 and CNN 3 (CNN 1, CNN 2 and CNN 3 were independently trained using training set 1, training set 2 and training set 3, respectively), were constructed, and their performances on each test set were respectively evaluated. One hundred and thirty-eight ME-NBI videos and 17 ME-BLI videos were further collected to evaluate and compare the performance of each model in real time. Results: On the whole, the performance of CNN 3 was the best. The accuracy (Acc), sensitivity (Sn), specificity (Sp) and area under the curve (AUC) of test set 1 in CNN 3 were 87.89% (1255/1428), 90.96% (342/376), 86.79% (913/1052) and 94.60%, respectively. The Acc, Sn, Sp and AUC of test set 2 in CNN 3 were 95% (570/600), 97.92% (518/529), 73.24% (52/71) and 90.93% respectively. The Acc, Sn, Sp and AUC of test set 3 in CNN 3 were 89.99% (1825/2028), 95.03% (860/905), 85.93% (965/1123) and 94.89%, respectively. The performance of CNN 3 was also the best in videos test set. The Acc, Sn and Sp of videos test set in CNN 3 were 91.03% (142/156), 90.58% (125/138) and 94.44% (17/18), respectively. Conclusions: The deep learning model with the most diverse training data has the best diagnostic effect.

n the last decade, there has been an active digitalization of industrial production based on rapidly developing information technologies, including artificial intelligence technologies. This is largely due to the development of deep learning methods and their applications in computer vision. Since the mid 2010s convolutional neural networks demonstrate exceptional efficiency in solving problems such as the detection, classification and segmentation of various objects. As a result, computer vision methods are beginning to be actively used in the problems of quality control of raw materials and finished products. All this applies to the mining industry. However, in the Russian scientific literature there are practically no systematic reviews of computer vision applications in this area. The present study aims to fill this gap. The paper provides a systematic review of the history of development and the current state of the methods and technologies of machine vision used in the mining industry for the analysis of solid materials, demonstrates the latest achievements in this area and examples of their application in the mining industry. The authors have analyzed 29 research papers in the field of application of computer vision in the mining industry and classified the stages of technology development from the mid-1980s, when computer vision was used without the use of machine learning, and ending with modern research based on the use of deep convolutional neural networks for solving problems of classification and segmentation. The effectiveness of the methods used is compared, their advantages and disadvantages are discussed, and forecasts are made for the development of computer vision methods in the mining industry in the near future. Examples are given showing that the use of convolutional neural networks made it possible to move to a qualitatively higher level of quality in solving problems of classification and segmentation as applied to the analysis of output volume, particle size distribution, including flakiness, angularity and roughness, dust and clay content, bulk density and emptiness, etc.

This study presents a convolutional neural network (CNN)‐based response estimation model for structural health monitoring (SHM) of tall buildings subject to wind loads. In this model, the wind‐induced responses are estimated by CNN trained with previously measured sensor signals; this enables the SHM system to operate stably even when a sensor fault or data loss occurs. In the presented model, top‐level wind‐induced displacement in the time and frequency domains, and wind data in the frequency domain are configured into the input map of the CNN to reflect the resisting capacity of a tall building, the change in the dynamic characteristics of the building due to wind loads, and the relationship between wind load and the building. To evaluate stress, which is used as a safety indicator for structural members in the building, the maximum and minimum strains of columns are set as the output layer of the CNN. The CNN is trained using measured wind and wind response data to predict the column strains during a future wind load. The presented model is validated using data from a wind tunnel test of a building model. The performance of the presented model is verified through strain estimation with data that were not used in the CNN training. To assess the validity of the presented input map configuration, the estimation performance is compared with a CNN that considered only the time domain responses as input. Furthermore, the effects of the variations in the configuration of the CNN on the wind response estimation performance are examined.

Road detection aims at detecting the (drivable) road surface ahead vehicle and plays a crucial role in driver assistance system. To improve the accuracy and robustness of road detection approaches in complex environments, a new road detection method based on CNN (convolutional neural network) and MRF (markov random field) is proposed. The original road image is segmented into super-pixels of uniform size using simple linear iterative clustering (SLIC) algorithm. On this basis, we train the CNN which can automatically learn the features that are most beneficial to classification. Then, the trained CNN is applied to classify road region and non-road region. Finally, based on the relationship between the super-pixels neighborhood, we utilize MRF to optimize the classification results of CNN. Quantitative and qualitative experiments on the publicly datasets demonstrate that the proposed method is robust in complex environments. Furthermore, compared with state-of-the-art algorithms, the approach provides the better performance.

Information extraction and coding of free-text pathology reports is an important activity for cancer registries to support national cancer surveillance. Cancer registrars must process high volumes of pathology reports on an annual basis. In this study, we investigated an automated approach using a coarse-to-fine training of convolutional neural networks (CNNs) for extracting the primary site, histological grade and laterality from unstructured cancer pathology text reports. Our proposed training scheme consists of two stages. In the first stage, the multi-task learning (MTL) with hard parameter sharing approach is used to train a multi-task MT-CNN model for all the tasks. Then, the TM-CNN model parameters are used to initialize a CNN model for each task to be fine trained individually using its corresponding dataset. The performance of our proposed approach was compared against a state-of-the-art CNN and the commonly used SVM classifier. We observed that the proposed model consistently outperformed the base line models, especially for the less prevalent classes. Specifically, the proposed training approach achieved a micro-F score of 0.7749 over 12 ICD-O-3 topography codes which is a significant improvement as compared with state-of-the-art CNN (0.7101) and the SVM (0.6019) classifiers. Also, the results demonstrate the potential of the proposed method for handling class imbalance within each task. It significantly improves macro-F score by 24% and 12% of the primary site and histology grade tasks, respectively.