Vision-Based Autonomous Crack Detection of Concrete Structures Using a Fully Convolutional Encoder–Decoder Network

Automated identification and localization of vertebrae in spinal computed tomography (CT) imaging is a complicated hybrid task. This task requires detecting and indexing a long sequence in a 3-D image, and both image feature extraction and sequence modeling are needed to address the problem. In this paper, the powerful fully convolutional neural network (FCN) technique performs both of these tasks simultaneously because FCNs directly encode and decode the spatial interdependence of different components in images. The key module of our proposed framework is a 3-D FCN trained in an end-to-end manner at the spine level to capture the long-range contextual information in CT volumes. The large increase in the calculation due to the full-size image inputs is alleviated by the scale-down of the inputs and the use of an auxiliary FCN to compensate for the loss of details. The composite network pipeline design enables the integration of local image details and global image patterns. Furthermore, explicit spatial and sequential constraints are imposed by the hidden Markov model (HMM) for a higher robustness and a clearer interpretation of network outputs. The proposed framework is quantitatively evaluated on the public dataset from the MICCAI 2014 Computational Challenge on Vertebrae Localization and Identification and demonstrates an identification rate (within 20 mm) of 94.67%, a mean identification rate of 87.97%, and a mean error distance of 2.56 mm on the test set, thus achieving the highest performance reported on this dataset.

Infrastructure, such as buildings, bridges, pavement, etc., needs to be examined periodically to maintain its reliability and structural health. Visual signs of cracks and depressions indicate stress and wear and tear over time, leading to failure/collapse if these cracks are located at critical locations, such as in load-bearing joints. Manual inspection is carried out by experienced inspectors who require long inspection times and rely on their empirical and subjective knowledge. This lengthy process results in delays that further compromise the infrastructure’s structural integrity. To address this limitation, this study proposes a deep learning (DL)-based autonomous crack detection method using the convolutional neural network (CNN) technique. To improve the CNN classification performance for enhanced pixel segmentation, 40,000 RGB images were processed before training a pretrained VGG16 architecture to create different CNN models. The chosen methods (grayscale, thresholding, and edge detection) have been used in image processing (IP) for crack detection, but not in DL. The study found that the grayscale models (F1 score for 10 epochs: 99.331%, 20 epochs: 99.549%) had a similar performance to the RGB models (F1 score for 10 epochs: 99.432%, 20 epochs: 99.533%), with the performance increasing at a greater rate with more training (grayscale: +2 TP, +11 TN images; RGB: +2 TP, +4 TN images). The thresholding and edge-detection models had reduced performance compared to the RGB models (20-epoch F1 score to RGB: thresholding −0.723%, edge detection −0.402%). This suggests that DL crack detection does not rely on colour. Hence, the model has implications for the automated crack detection of concrete infrastructures and the enhanced reliability of the gathered information.

With the success of convolutional neural network (CNN) in the field of image processing and image recognition, an improved fully convolutional neural network (FCNN) based on CNN is proposed. Although FCNN has better performance than CNN, the number of parameters of FCNN has also increased significantly compared to CNN. Consequently, this paper proposes a multi‐feature fusion decision convolutional neural network (MFFD‐CNN), which use two kinds down‐sample methods in the pooling layer, and chooses the concatenate or add operation as the way that consolidates the feature of the convolutional layer with a stride of 2 and the max‐pooling layer to execute fusion operations. Meanwhile replaces the traditional softmax loss with an additive margin Softmax (AM‐Softmax) loss. The structure, which is based on the multi‐target classification experiment results of the MSTAR data set, shows can not only obtain an average correct recognition rate 1.2% higher than FCNN, but also better stability without augmenting the training samples of MSTAR data.

Rationale and Objectives: Accurate diagnosis and staging of cervical precancers is essential for practical medicine in determining the extent of the lesion extension and determines the most correct and effective therapeutic approach. For accurate diagnosis and staging of cervical precancers, we aim to create a diagnostic method optimized by artificial intelligence (AI) algorithms and validated by achieving accurate and favorable results by conducting a clinical trial, during which we will use the diagnostic method optimized by artificial intelligence (AI) algorithms, to avoid errors, to increase the understanding on interpretation of colposcopy images and improve therapeutic planning. Materials and Methods: The optimization of the method will consist in the development and formation of artificial intelligence models, using complicated convolutional neural networks (CNN) to identify precancers and cancers on colposcopic images. We will use topologies that have performed well in similar image recognition projects, such as Visual Geometry Group Network (VGG16), Inception deep neural network with an architectural design that consists of repeating components referred to as Inception modules (Inception), deeply separable convolutions that significantly reduce the number of parameters (MobileNet) that is a class of Convolutional Neural Network (CNN), Return of investment for machine Learning (ROI), Fully Convolutional Network (U-Net) and Overcomplete Convolutional Network Kite-Net (KiU-Net). Validation of the diagnostic method, optimized by algorithm of artificial intelligence will consist of achieving accurate results on diagnosis and staging of cervical precancers by conducting a randomized, controlled clinical trial, for a period of 17 months. Results: We will validate the computer assisted diagnostic (CAD) method through a clinical study and, secondly, we use various network topologies specified above, which have produced promising results in the tasks of image model recognition and by using this mixture. By using this method in medical practice, we aim to avoid errors, provide precision in diagnosing, staging and establishing the therapeutic plan in cervical precancers using AI. Conclusion: This diagnostic method, optimized by artificial intelligence algorithms and validated by the clinical trial, which we consider “second opinion”, improves the quality standard in diagnosing, staging and establishing therapeutic conduct in cervical precancer.

This research study is to detect the Epileptic seizures using Novel Multi-Layered Convolutional Neural Networks (NMLCNN) and to compare the performance with Fully Convolutional Neural Networks (FCNN). By using Novel Multi-Layered Convolution Neural Networks algorithm works with the data collected from the observations of nervous system cells. The changes in the Epileptic seizures from the resource’s constraint, when the criteria met with randomizing, this will arrange in random ways for n times. The most typical beginning of a seizure is detectable through analysis of the Electroencephalogram (EEG). To examine this, the study utilized 20 samples in each group, with a Gpower value of 80%. The study used a predefined significance value and a Confidence Interval (CI) of 95%.This experiment found that NMLCNN has 92.25% and FCNN has 87.55% of accuracy for epileptic seizure. The Novel Multi-Layered Convolutional Neural Network achieved 0.012 significance using group analysis and provided significance. This probing concludes based on accuracy and significance results, detection of Epileptic seizures using Novel Multi-layered Convolutional Neural Networks has performed better compared to Fully Convolutional Neural Network.

With the leaps and bounds of computer performance and the advent of the era of big data, deep learning has drawn more and more attention from all walks of life. It can combine low-level features to form more abstract high-level features and describe the data more essentially. Therefore, it is widely used in various fields such as computer vision. Image segmentation is one of the most basic research topics in the field of computer vision. The main purpose is to extract regions of interest from images for later image processing research. In 3D reconstruction based on sequence images, the segmentation accuracy and speed of sequence images determine the quality and efficiency of target reconstruction. Therefore, when facing large-scale sequence images, the biggest problem is how to improve the segmentation speed while ensuring accuracy. Based on the above background, the research content of this article is an indoor scene segmentation algorithm based on full convolutional neural network. According to the characteristics of indoor application scenes, this paper proposes a fast convolutional neural network image segmentation method to segment the indoor scene image and construct the fast fully convolutional networks (FFCN) for indoor scene image segmentation uses inter-layer fusion to reduce the amount of network calculation parameters and avoid the loss of picture feature information by continuous convolution. In order to verify the effectiveness of the network, in this paper, a basic living object data set (XAUT data set) in an indoor environment is created. The XAUT data set is used to train the FFCN network under the Caffe framework to obtain an indoor scene segmentation model. In order to compare the effectiveness of the model, the structure of the worn FCN8s, FCN16s, and FCN32s models was fine-tuned, and the corresponding algorithm model for indoor scene segmentation was obtained by training with the XAUT data set. The experimental results show that the pixel recognition accuracy of all types of networks has reached 86%, and the mean IU ratio has reached more than 63%. The mean IU of the FCN8s network is the highest at 70.38%, but its segmentation speed is only 1/5 of FFCN. On the premise that other types of indicators are not much different, the average segmentation speed on FFCN fast segmentation convolutional neural network reaches 40 fps. It can be seen that the scale fusion technology can well avoid the loss of image feature information in the network convolution and reddening process. Compared with other FCN networks, it has a faster speed and is conducive to real-time image preprocessing.

Abdominal image segmentation is a challenging, yet important clinical problem. Variations in body size, position, and relative organ positions greatly complicate the segmentation process. Historically, multi-atlas methods have achieved leading results across imaging modalities and anatomical targets. However, deep learning is rapidly overtaking classical approaches for image segmentation. Recently, Zhou et al. showed that fully convolutional networks produce excellent results in abdominal organ segmentation of computed tomography (CT) scans. Yet, deep learning approaches have not been applied to whole abdomen magnetic resonance imaging (MRI) segmentation. Herein, we evaluate the applicability of an existing fully convolutional neural network (FCNN) designed for CT imaging to segment abdominal organs on T2 weighted (T2w) MRI's with two examples. In the primary example, we compare a classical multi-atlas approach with FCNN on forty-five T2w MRI's acquired from splenomegaly patients with five organs labeled (liver, spleen, left kidney, right kidney, and stomach). Thirty-six images were used for training while nine were used for testing. The FCNN resulted in a Dice similarity coefficient (DSC) of 0.930 in spleens, 0.730 in left kidneys, 0.780 in right kidneys, 0.913 in livers, and 0.556 in stomachs. The performance measures for livers, spleens, right kidneys, and stomachs were significantly better than multi-atlas (p < 0.05, Wilcoxon rank-sum test). In a secondary example, we compare the multi-atlas approach with FCNN on 138 distinct T2w MRI's with manually labeled pancreases (one label). On the pancreas dataset, the FCNN resulted in a median DSC of 0.691 in pancreases versus 0.287 for multi-atlas. The results are highly promising given relatively limited training data and without specific training of the FCNN model and illustrate the potential of deep learning approaches to transcend imaging modalities.

Unlike standard Convolutional Neural Networks (CNNs) with fully-connected layers, Fully Convolutional Neural Networks (FCN) are prevalent in computer vision applications such as object detection, semantic/image segmentation, and the most popular generative tasks based on Generative Adversarial Networks (GAN). In an FCN, traditional convolutional layers and deconvolutional layers contribute to the majority of the computation complexity. However, prior deep learning accelerator designs mostly focus on CNN optimization. They either use independent compute-resources to handle deconvolution or convert deconvolutional layers (Deconv) into general convolution operations, which arouses considerable overhead. To address this problem, we propose a unified fully convolutional accelerator aiming to handle both the deconvolutional and convolutional layers with a single processing element (PE) array. We re-optimize the conventional CNN accelerator architecture of regular 2D processing elements array, to enable it more efficiently support the data flow of deconvolutional layer inference. By exploiting the locality in deconvolutional filters, this architecture reduces the consumption of on-chip memory communication from 24.79 GB to 6.56 GB and improves the power efficiency significantly. Compared to prior baseline deconvolution acceleration scheme, the proposed accelerator achieves 1.3X -- 44.9X speedup and reduces the energy consumption by 14.6%-97.6% on a set of representative benchmark applications. Meanwhile, it keeps similar CNN inference performance to that of an optimized CNN-only accelerator with negligible power consumption and chip area overhead.

Cracks are an indicator for a bridge’s structural health and functional failures. Crack detection is one of the major tasks needed to maintain the structural health and serviceability of a bridge. At present, the most commonly used crack detection technology is manual inspection, which has the disadvantages of being highly labor-intensive and time-consuming. In this paper, a crack detection method based on a convolutional neural network (CNN) is proposed. To automate quantitative measurements of an identified crack, hybrid image processing is proposed, as well. First, a dataset is compiled, including 12,000 cropped crack images and 19,500 cropped background images. Second, preprocessed images with the proposed method of Bilateral-Graying-Contrast (BGC) are fed into ResNet and a Visual Geometry Group Network (VGG) for training and testing. Finally, an automatic measurement system for bridge crack is developed which is not prone to weakened shooting conditions. The results demonstrate that ResNet achieves an accuracy of crack detection up to 97.44%, which is higher than VGG. Our crack measurement system significantly reduces the measurement error to 9.86% and can be assumed as a reliable method in the analysis of concrete bridge images.

Functional imaging has successfully been applied to capture functional changes in the pathological tissues of a body in recent years. Nuclear medicine functional imaging has been used to acquire information about areas of concerns (e.g. lesions and organs) in a non‐invasive manner, enabling semi‐automated or automated decision‐making for disease diagnosis, treatment, evaluation, and prediction. Focusing on functional nuclear medicine images, in this study, the authors review existing work on the classification of single‐photon emission computed tomography, positron emission tomography, and their hybrid modalities with computed tomography and magnetic resonance imaging images by using convolutional neural network (CNN) techniques. Specifically, they first present an overview of nuclear imaging and the CNN technique, such as nuclear imaging modalities, nuclear image data format, CNN architecture, and the main CNN classification models. According to the diseases of concern, they then classify the existing CNN‐based work on the classification of functional nuclear images into three different categories. For the typical work in each of these categories, they present details about their research objectives, adopted CNN models, and achieved main results. Finally, they discuss research challenges and directions for developing technological solutions to classify nuclear medicine images based on the CNN technique.

Millions of clinicians rely daily on automated preliminary electrocardiogram (ECG) interpretation. Critical comparisons of machine learning-based automated analysis against clinically accepted standards of care are lacking. To use readily available 12-lead ECG data to train and apply an explainability technique to a convolutional neural network (CNN) that achieves high performance against clinical standards of care. This cross-sectional study was conducted using data from January 1, 2003, to December 31, 2018. Data were obtained in a commonly available 12-lead ECG format from a single-center tertiary care institution. All patients aged 18 years or older who received ECGs at the University of California, San Francisco, were included, yielding a total of 365 009 patients. Data were analyzed from January 1, 2019, to March 2, 2021. A CNN was trained to predict the presence of 38 diagnostic classes in 5 categories from 12-lead ECG data. A CNN explainability technique called LIME (Linear Interpretable Model-Agnostic Explanations) was used to visualize ECG segments contributing to CNN diagnoses. Area under the receiver operating characteristic curve (AUC), sensitivity, and specificity were calculated for the CNN in the holdout test data set against cardiologist clinical diagnoses. For a second validation, 3 electrophysiologists provided consensus committee diagnoses against which the CNN, cardiologist clinical diagnosis, and MUSE (GE Healthcare) automated analysis performance was compared using the F1 score; AUC, sensitivity, and specificity were also calculated for the CNN against the consensus committee. A total of 992 748 ECGs from 365 009 adult patients (mean [SD] age, 56.2 [17.6] years; 183 600 women [50.3%]; and 175 277 White patients [48.0%]) were included in the analysis. In 91 440 test data set ECGs, the CNN demonstrated an AUC of at least 0.960 for 32 of 38 classes (84.2%). Against the consensus committee diagnoses, the CNN had higher frequency-weighted mean F1 scores than both cardiologists and MUSE in all 5 categories (CNN frequency-weighted F1 score for rhythm, 0.812; conduction, 0.729; chamber diagnosis, 0.598; infarct, 0.674; and other diagnosis, 0.875). For 32 of 38 classes (84.2%), the CNN had AUCs of at least 0.910 and demonstrated comparable F1 scores and higher sensitivity than cardiologists, except for atrial fibrillation (CNN F1 score, 0.847 vs cardiologist F1 score, 0.881), junctional rhythm (0.526 vs 0.727), premature ventricular complex (0.786 vs 0.800), and Wolff-Parkinson-White (0.800 vs 0.842). Compared with MUSE, the CNN had higher F1 scores for all classes except supraventricular tachycardia (CNN F1 score, 0.696 vs MUSE F1 score, 0.714). The LIME technique highlighted physiologically relevant ECG segments. The results of this cross-sectional study suggest that readily available ECG data can be used to train a CNN algorithm to achieve comparable performance to clinical cardiologists and exceed the performance of MUSE automated analysis for most diagnoses, with some exceptions. The LIME explainability technique applied to CNNs highlights physiologically relevant ECG segments that contribute to the CNN's diagnoses.

In the field of 3D Human Pose Estimation and Reconstruction based on body joints extracted from a 2D image; Exists challenges like self-occlusion and depth perception. These problems hinder approximate estimations. This article proposes a hybrid method that consists of semantic segmentation, sparse representation and 2D pose estimation for 3D estimation. Specifically, we train two fully Convolution neural networks (FCNs) to estimate 2D pose and semantic segmentation. Then, we narrow the estimation down using basic human body structure and results from the FCNs. Subsequently, utilizing the estimated 2D pose of the earlier step and a sparse representation model the 3D pose will estimate. Using a Convolution Neural Networks learning method for 2D human pose estimation and then by sparse representation will estimate. Results of the proposed method, show improvement relative to the previous works on 3D Human pose estimation. 3D Human Pose Estimation by this method demonstrates a significant mean error reduction compared to earlier studies.

This study aims to predict the risk of deep caries exposure in radiographic images based on the convolutional neural network model, compare the prediction results of the network model with those of senior dentists, evaluate the performance of the model for teaching and training stomatological students and young dentists, and assist dentists to clarify treatment plans and conduct good doctor-patient communication before surgery. A total of 206 cases of pulpitis caused by deep caries were selected from the Department of Stomatological Hospital of Tianjin Medical University from 2019 to 2022. According to the inclusion and exclusion criteria, 104 cases of pulpitis were exposed during the decaying preparation period and 102 cases of pulpitis were not exposed. The 206 radiographic images collected were randomly divided into three groups according to the proportion: 126 radiographic images in the training set, 40 radiographic images in the validation set, and 40 radiographic images in the test set. Three convolutional neural networks, visual geometry group network (VGG), residual network (ResNet), and dense convolutional network (DenseNet) were selected to analyze the rules of the radiographic images in the training set. The radiographic images of the validation set were used to adjust the super parameters of the network. Finally, 40 radiographic images of the test set were used to evaluate the performance of the three network models. A senior dentist specializing in dental pulp was selected to predict whether the deep caries of 40 radiographic images in the test set were exposed. The gold standard is whether the pulp is exposed after decaying the prepared hole during the clinical operation. The prediction effect of the three network models (VGG, ResNet, and DenseNet) and the senior dentist on the pulp exposure of 40 radiographic images in the test set were compared using receiver operating characteristic (ROC) curve, area under the ROC curve (AUC), accuracy, sensitivity, specificity, positive predictive value, negative predictive value, and F1 score to select the best network model. The best network model was DenseNet model, with AUC of 0.97. The AUC values of the ResNet model, VGG model, and the senior dentist were 0.89, 0.78, and 0.87, respectively. Accuracy was not statistically different between the senior dentist (0.850) and the DenseNet model (0.850)(P>0.05). Kappa consistency test showed moderate reliability (Kappa=0.6>0.4, P<0.05). Among the three convolutional neural network models, the DenseNet model has the best predictive effect on whether deep caries are exposed in imaging. The predictive effect of this model is equivalent to the level of senior dentists specializing in dental pulp.

In recent years, convolutional neural networks (CNNs) have been successfully applied to inverse synthetic aperture radar (ISAR) sparse imaging because of their powerful ability in feature extraction. However, these CNNs only adopt single path feed-forward architectures and lack paths for directly transmitting original feature representations (OFRs) in shallow layers to reconstruction layers, which limits the complete reconstruction of target shape due to the underutilization of the OFRs that are efficient for recovering target details. Later, fully CNN (FCNN) introduces several skip connections (SKs) to establish the additional ways for directly passing the OFRs to the reconstruction layers. Nevertheless, the transmitted OFRs inevitably include the feature information of artifacts, which usually results the appearance of artifacts in final reconstructed target image. To address this issue, we introduce the gate units to FCNN, and refer to the improved FCNN as G-FCNN. Furthermore, the learnable gate units weight the OFRs transmitted by SKs and autonomously decide how many OFRs are transmitted further. To circumvent the shortage of the real data available for network training, we utilize the transfer learning strategy to guarantee a good performance of the G-FCNN. The imaging results of real data show that the G-FCNN-based imaging method is superior to the existing CNN-based imaging methods.

This paper focuses on the design and comparison of different deep neural networks for the real-time prediction of locomotor intentions by using data from inertial measurement units. The deep neural network architectures are convolutional neural networks, recurrent neural networks, and convolutional recurrent neural networks. The input to the architectures are features in the time domain, which have been derived either from one inertial measurement unit placed on the upper right leg of ten healthy subjects, or two inertial measurement units placed on both the upper and lower right leg of ten healthy subjects. The study shows that a WaveNet, i.e., a full convolutional neural network, achieves a peak F1-score of 87.17% in the case of one IMU, and a peak of 97.88% in the case of two IMUs, with a 5-fold cross-validation.