A High-Level Modeling Framework for Estimating Hardware Metrics of CNN Accelerators

In this study, a method of predicting the seismic responses of building structures based on a convolutional neural network (CNN) is proposed. In the method, the time histories of acceleration responses previously measured in a building during earthquakes are used in the CNN input layer, with the corresponding time histories of the displacement responses being used in the CNN output layer. The correlations between the features automatically extracted from the acceleration responses by the convolution and pooling operations in the various CNN layers and the displacement responses in the CNN output layer are established through CNN training. The trained CNN predicts the displacement responses for a future earthquake event using only the measured acceleration responses. To verify the validity of the proposed method, a numerical study on the ASCE benchmark model and an experimental study on a reinforced concrete frame structure are conducted. In the numerical study, the structural responses of the ASCE model subject to artificial earthquakes are utilized for CNN training, and the performance of the trained CNN is verified through seismic response prediction. In the experimental study, shaking table tests are performed for various earthquakes to measure the seismic responses of the reinforced concrete test structure. The seismic response prediction performance of the proposed method is examined using CNN trained with the measured seismic responses. Furthermore, the performances of CNNs trained with different numbers of datasets generated by the proposed data generation method based on data overlapping with the same data pool are discussed with related to the number of CNN training iteration number.

Aims: This study aimed to explore the effect of training set diversity on the performance of deep learning models for predicting early gastric cancer (EGC) under endoscopy. Methods: Images of EGC and non-cancerous lesions under narrow-band imaging (ME-NBI) and magnifying blue laser imaging (ME-BLI) were retrospectively collected. Training set 1 was composed of 150 non-cancerous and 309 EGC ME-NBI images, training set 2 was composed of 1505 non-cancerous and 309 EGC ME-BLI images, and training set 3 was the combination of training set 1 and 2. Test set 1 was composed of 376 non-cancerous and 1052 EGC ME-NBI images, test set 2 consisted of 529 non-cancerous and 71 EGC ME-BLI images, and test set 3 was the combination of test set 1 and test set 2. Three deep learning models, convolutional neural network (CNN) 1, CNN 2 and CNN 3 (CNN 1, CNN 2 and CNN 3 were independently trained using training set 1, training set 2 and training set 3, respectively), were constructed, and their performances on each test set were respectively evaluated. One hundred and thirty-eight ME-NBI videos and 17 ME-BLI videos were further collected to evaluate and compare the performance of each model in real time. Results: On the whole, the performance of CNN 3 was the best. The accuracy (Acc), sensitivity (Sn), specificity (Sp) and area under the curve (AUC) of test set 1 in CNN 3 were 87.89% (1255/1428), 90.96% (342/376), 86.79% (913/1052) and 94.60%, respectively. The Acc, Sn, Sp and AUC of test set 2 in CNN 3 were 95% (570/600), 97.92% (518/529), 73.24% (52/71) and 90.93% respectively. The Acc, Sn, Sp and AUC of test set 3 in CNN 3 were 89.99% (1825/2028), 95.03% (860/905), 85.93% (965/1123) and 94.89%, respectively. The performance of CNN 3 was also the best in videos test set. The Acc, Sn and Sp of videos test set in CNN 3 were 91.03% (142/156), 90.58% (125/138) and 94.44% (17/18), respectively. Conclusions: The deep learning model with the most diverse training data has the best diagnostic effect.

Every year, new Convolutional Neural Network (CNN) architectures appear to deal with different problems in the activity of image and video recognition. These architectures usually work along the ImageNet dataset for looking for the best performance of the CNNs without taking into account the video task where they are used. This can represent a problem if the task is Human Action Recognition (HAR) in video, since the CNN architectures are pre-trained with an image dataset that can practically contain any object, while HAR problem requires consecutive frames of people doing actions. To prove the idea that using CNNs pre-trained on an image dataset does not always achieve the best performance on a video dataset and that, therefore, it is worth comparing the performance of different CNNs under similar circumstances for the HAR problem, this work proposes an analysis between eight different CNN architectures. Each one of the CNN was exclusively trained with RGB images, which were extracted from the frames of the different classes of videos of HMDB51 dataset. To make the classification of an activity in video, we average the predictions taking into account the successes. We also made some ensembles with the best performance CNNs to measure the improvement in accuracy. Our results suggest that Xception is a strong baseline model that could be used by the community to make their comparisons of their proposals more robust.

This study presents a convolutional neural network (CNN)‐based response estimation model for structural health monitoring (SHM) of tall buildings subject to wind loads. In this model, the wind‐induced responses are estimated by CNN trained with previously measured sensor signals; this enables the SHM system to operate stably even when a sensor fault or data loss occurs. In the presented model, top‐level wind‐induced displacement in the time and frequency domains, and wind data in the frequency domain are configured into the input map of the CNN to reflect the resisting capacity of a tall building, the change in the dynamic characteristics of the building due to wind loads, and the relationship between wind load and the building. To evaluate stress, which is used as a safety indicator for structural members in the building, the maximum and minimum strains of columns are set as the output layer of the CNN. The CNN is trained using measured wind and wind response data to predict the column strains during a future wind load. The presented model is validated using data from a wind tunnel test of a building model. The performance of the presented model is verified through strain estimation with data that were not used in the CNN training. To assess the validity of the presented input map configuration, the estimation performance is compared with a CNN that considered only the time domain responses as input. Furthermore, the effects of the variations in the configuration of the CNN on the wind response estimation performance are examined.

Information extraction and coding of free-text pathology reports is an important activity for cancer registries to support national cancer surveillance. Cancer registrars must process high volumes of pathology reports on an annual basis. In this study, we investigated an automated approach using a coarse-to-fine training of convolutional neural networks (CNNs) for extracting the primary site, histological grade and laterality from unstructured cancer pathology text reports. Our proposed training scheme consists of two stages. In the first stage, the multi-task learning (MTL) with hard parameter sharing approach is used to train a multi-task MT-CNN model for all the tasks. Then, the TM-CNN model parameters are used to initialize a CNN model for each task to be fine trained individually using its corresponding dataset. The performance of our proposed approach was compared against a state-of-the-art CNN and the commonly used SVM classifier. We observed that the proposed model consistently outperformed the base line models, especially for the less prevalent classes. Specifically, the proposed training approach achieved a micro-F score of 0.7749 over 12 ICD-O-3 topography codes which is a significant improvement as compared with state-of-the-art CNN (0.7101) and the SVM (0.6019) classifiers. Also, the results demonstrate the potential of the proposed method for handling class imbalance within each task. It significantly improves macro-F score by 24% and 12% of the primary site and histology grade tasks, respectively.

The convolution neural networks have revolutionized the computer vision domain. It has proven to be a dominant technology to carry out tasks such as image classification, semantic segmentation, and object detection. The convolution neural networks surpass the performance of the existing algorithms such as SIFT, HOG, etcetera. Where, instead of manually engineering the features, supervised learning help to learn the essential low-level and high-level features necessary for classifications. The convolution neural networks have become a popular tool to counter computer vision problems. However, it is computationally, and memory intensive to train and deploy the network because of the model size of a deep convolution neural networks. However, the research in the field of design space exploration (DSE) of neural networks and compression techniques to develop compact architectures, have made convolution neural networks memory and computationally efficient. These techniques have also improved the feasibility of convolution neural network for deployment on embedded targets. The paper explores the concept of compact convolution filters to reduce the number of parameters in a convolution neural network. The intuition behind the approach is that replacing convolution filters with a stack of compact convolution filters helps in developing a compact architecture with competitive accuracy. This paper explores the fire module a compact convolution filter and proposes a method of recreating a state-of-the-art architecture VGG-16 using the fire modules to develop a compact architecture, which is further trained on the CIFAR-10 dataset and deployed on a real-time embedded platform known as Bluebox 2.0 by NXP using RTMaps software framework.

Abstract. In recent years, there has been a significant improvement in the detection, identification and classification of objects and images using Convolutional Neural Networks. To study the potential of the Convolutional Neural Network, in this paper three approaches are investigated to train classifiers based on Convolutional Neural Networks. These approaches allow Convolutional Neural Networks to be trained on datasets containing only a few hundred training samples, which results in a successful classification. Two of these approaches are based on the concept of transfer learning. In the first approach features, created by a pretrained Convolutional Neural Network, are used for a classification using a support vector machine. In the second approach a pretrained Convolutional Neural Network gets fine-tuned on a different data set. The third approach includes the design and training for flat Convolutional Neural Networks from the scratch. The evaluation of the proposed approaches is based on a data set provided by the IEEE Geoscience and Remote Sensing Society (GRSS) which contains RGB and LiDAR data of an urban area. In this work it is shown that these Convolutional Neural Networks lead to classification results with high accuracy both on RGB and LiDAR data. Features which are derived by RGB data transferred into LiDAR data by transfer learning lead to better results in classification in contrast to RGB data. Using a neural network which contains fewer layers than common neural networks leads to the best classification results. In this framework, it can furthermore be shown that the practical application of LiDAR images results in a better data basis for classification of vehicles than the use of RGB images.

Glass forming ability prediction of bulk metallic glasses based on fused strategy

Background: Memristor-based implementations of Convolutional Neural Networks (CNNs) offer the potential to reduce power consumption and improve computational efficiency significantly. However, the non-ideal characteristics of memristive devices, such as variability and limited switching behavior, can introduce nonlinearity and degrade network performance. It is therefore essential to simulate memristor-based deep learning systems and evaluate the impact of these imperfections before proceeding with hardware-level deployment. Objective: This study aims to design a lightweight memristor-based MobileNetV2 model for smoking behavior detection and analyze the influence of memristor non-idealities on the network’s detection accuracy. Methods: A lightweight MobileNetV2 network was constructed and trained on a self-developed dataset for smoking behavior detection, achieving a peak accuracy of 92.25%. Based on the trained model, a memristor-based MobileNetV2 architecture was implemented using the Mem- Torch simulation framework. Two memristor models, VTEAM and Data-Driven Verilog-A RRAM, were employed to replace the convolutional and fully connected layers with memristor crossbar arrays. The impact of non-ideal memristor properties, including conductance variability and resistance state faults, was systematically evaluated. Results: Simulation results demonstrated that the memristor-based MobileNetV2 network achieved detection accuracies of 92.11% with the VTEAM model and 91.82% with the Verilog-A RRAM model. As device non-idealities increased, the detection accuracy declined significantly, indicating a strong correlation between memristor imperfections and the performance of CNN- based target detection tasks. Conclusion: Memristor non-idealities substantially affect the performance of neuromorphic CNNs. This study provides targeted analyses and proposes mitigation strategies, such as differential encoding and improved fabrication techniques, to support the practical deployment of memristor- based neural networks in energy-efficient AI applications.

Recently, the deep learning methods have achieved great success in pattern recognition tasks. Especially for character recognition, most of the state-of-the-art results belong to the deep learning models. Among those models, the convolutional neural network (CNN) becomes the most popular due to its outstanding performance. Therefore, many trials were made in order to make improvements on CNN. However, most of the trials only focused on the network structure or training skills, the inter-class information is usually ignored. In this paper, we have proposed a novel CNN model with two training feedbacks: the reconstruction feedback and the classification feedback. By using the reconstruction feedback, the inter-class information (for example, shape similarity) of the characters is taken into account. Consequently, without enlarging the network structure, our model can outperform those state-of-the-art improved CNN models, which is proved by the experimental results.

Man against machine reloaded: performance of a market-approved convolutional neural network in classifying a broad spectrum of skin lesions in comparison with 96 dermatologists working under less artificial conditions

A retrospective analysis of magnetic resonance imaging (MRI). The study aimed to evaluate the performance of a convolutional neural network (CNN) to differentiate spinal pyogenic spondylitis from Modic change on MRI. We compared the performance of CNN to that of four clinicians. Discrimination between pyogenic spondylitis and spinal Modic change is crucial in clinical practice. CNN deep-learning approaches for medical imaging are being increasingly utilized. We retrospectively reviewed MRIs from pyogenic spondylitis and spinal Modic change patients. There were 50 patients per group. Sagittal T1-weighted (T1WI), sagittal T2-weighted (T2WI), and short TI inversion recovery (STIR) MRIs were used for CNN training and validation. The deep learning framework Tensorflow was used to construct the CNN architecture. To evaluate CNN performance, we plotted the receiver operating characteristic curve and calculated the area under the curve. We compared the accuracy, sensitivity, and specificity of CNN diagnosis to that of a radiologist, spine surgeon, and two orthopedic surgeons. The CNN-based area under the curves of the receiver operating characteristic curve from the T1WI, T2WI, and STIR were 0.95, 0.94, and 0.95, respectively. The accuracy of the CNN was significantly greater than that of the four clinicians on T1WI and STIR (P<0.05), and better than a radiologist and one orthopedic surgeon on the T2WI (P<0.05). The sensitivity was significantly better than that of the four clincians on T1WI and STIR (P<0.05), and better than a radiologist and one orthopedic surgeon on the T2WI (P<0.05). The specificity was significantly better than one orthopedic surgeon on T1WI and T2WI (P<0.05) and better than both orthopedic surgeons on STIR (P<0.05). We differentiated between Modic changes and pyogenic spondylitis using a CNN that interprets MRI. The performance of the CNN was comparable to, or better than, that of the four clinicians.

In recent years, convolutional neural networks (CNNs) have made great achievements in object extraction from very high-resolution (VHR) images. However, most existing approaches require large quantities of clean and accurate training data to achieve impressive classification results. The presence of inaccurate labels in training datasets is known to deteriorate the performance of CNNs. In this paper, we introduce a novel efficient method for improving the robustness when training CNN on the dataset with relatively noisy labels. First, we propose a feature and label noise model (FLNM) to model the noisy label distribution in the training dataset. Then, we use a multitask deep learning framework (MDLF) to integrate the FLNM into the training process of CNN. Finally, a novel loss function concerning the high-level features is introduced to efficiently train the MDLF. We evaluate our method on datasets from Massachusetts and compare this method with other state-of-the-art methods. The experimental results demonstrate the effectiveness of the proposed method in improving the classification performance of CNNs trained with noisy training dataset.

Convolutional neural networks (CNNs) have been successfully applied to many tasks such as digit and object recognition. In this paper we study the size of the receptive field of deep convolutional neural networks, in particular, we check the idea of a redundant receptive field. We run a set of experiments on two common CNN models — VGG16 and ResNet18 — in order to explore the influence of receptive field size on CNN's training time, accuracy, and performance. We run experiments using the MakiFlow framework on the CALTECH256 dataset. The experiments' results show that the optimization of neural networks (NNs) by reducing the size of the receptive field allows to reduce the NN’s training time by 5-7% while maintaining the accuracy of the network.

Training neural networks with many processors can reduce time-to-solution; however, it is challenging to maintain convergence and efficiency at large scales. The Kroneckerfactored Approximate Curvature (K-FAC) was recently proposed as an approximation of the Fisher Information Matrix that can be used in natural gradient optimizers. We investigate here a scalable K-FAC design and its applicability in convolutional neural network (CNN) training at scale. We study optimization techniques such as layer-wise distribution strategies, inverse-free second-order gradient evaluation, and dynamic K-FAC update decoupling to reduce training time while preserving convergence. We use residual neural networks (ResNet) applied to the CIFAR10 and ImageNet-1k datasets to evaluate the correctness and scalability of our K-FAC gradient preconditioner. With ResNet-50 on the ImageNet-1k dataset, our distributed K-FAC implementation converges to the 75.9% MLPerf baseline in 18-25% less time than does the classic stochastic gradient descent (SGD) optimizer across scales on a GPU cluster.