On the Evaluation of CNN Models in Remote-Sensing Scene Classification Domain

Emotions expressed by humans can be identified from facial expressions, speech signals, or physiological signals. Among them, the use of physiological signals for emotion classification is a notable emerging area of research. In emotion recognition, a person’s electrocardiogram (ECG) and galvanic skin response (GSR) signals cannot be manipulated, unlike facial and voice signals. Moreover, wearables such as smartwatches and wristbands enable the detection of emotions in people’s naturalistic environment. During the COVID-19 pandemic, it was necessary to detect people’s emotions in order to ensure that appropriate actions were taken according to the prevailing situation and achieve societal balance. Experimentally, the duration of the emotion stimulus period and the social and non-social contexts of participants influence the emotion classification process. Hence, classification of emotions when participants are exposed to the elicitation process for a longer duration and taking into consideration the social context needs to be explored. This work explores the classification of emotions using five pretrained convolutional neural network (CNN) models: MobileNet, NASNetMobile, DenseNet 201, InceptionResnetV2, and EfficientNetB7. The continuous wavelet transform (CWT) coefficients were detected from ECG and GSR recordings from the AMIGOS database with suitable filtering. Scalograms of the sum of frequency coefficients versus time were obtained and converted into images. Emotions were classified using the pre-trained CNN models. The valence and arousal emotion classification accuracy obtained using ECG and GSR data were, respectively, 91.27% and 91.45% using the InceptionResnetV2 CNN classifier and 99.19% and 98.39% using the MobileNet CNN classifier. Other studies have not explored the use of scalograms to represent ECG and GSR CWT features for emotion classification using deep learning models. Additionally, this study provides a novel classification of emotions built on individual and group settings using ECG data. When the participants watched long-duration emotion elicitation videos individually and in groups, the accuracy was around 99.8%. MobileNet had the highest accuracy and shortest execution time. These subject-independent classification methods enable emotion classification independent of varying human behavior.

BackgroundSurgical process modeling automatically identifies surgical phases, and further improvement in recognition accuracy is expected with deep learning. Surgical tool or time series information has been used to improve the recognition accuracy of a model. However, it is difficult to collect this information continuously intraoperatively. The present study aimed to develop a deep convolution neural network (CNN) model that correctly identifies the surgical phase during laparoscopic cholecystectomy (LC).MethodsWe divided LC into six surgical phases (P1–P6) and one redundant phase (P0). We prepared 115 LC videos and converted them to image frames at 3 fps. Three experienced doctors labeled the surgical phases in all image frames. Our deep CNN model was trained with 106 of the 115 annotation datasets and was evaluated with the remaining datasets. By depending on both the prediction probability and frequency for a certain period, we aimed for highly accurate surgical phase recognition in the operation room.ResultsNine full LC videos were converted into image frames and were fed to our deep CNN model. The average accuracy, precision, and recall were 0.970, 0.855, and 0.863, respectively.ConclusionThe deep learning CNN model in this study successfully identified both the six surgical phases and the redundant phase, P0, which may increase the versatility of the surgical process recognition model for clinical use. We believe that this model can be used in artificial intelligence for medical devices. The degree of recognition accuracy is expected to improve with developments in advanced deep learning algorithms.

Comparative analysis of deep convolution neural network models on small scale datasets

Background Incidence rate of thyroid cancer is steadily increasing due to overdiagnosis and overtreatment. Thyroid ultrasound is commonly used to diagnose thyroid cancer. The aim of this study is to examine the accuracy of using deep convolutional neural network (DCNN) models to improve diagnosis of thyroid cancer by analyzing sonographic imaging data from clinical thyroid ultrasound. Methods A total of 131,731 sonographic images from 17,627 thyroid cancer patients and 180,668 sonographic images from 25,325 controls used as training set were obtained from Tianjin Cancer Hospital. Images from anatomical sites that did not have cancer according to location sign on the image were not included. All thyroid cancer patients and 13·2% of controls (51,255 images) were confirmed by pathological reports. DCNN is a specific type of neural network optimized for image recognition. We trained two DCNN models on the training set and subsequently evaluated the performance on one independent internal (Tianjin, 1,118 individuals) and two external (Jilin,154 individuals; Weihai, 1,420 individuals) validation sets. Individuals in the validation sets all have pathological examinations. We compared the specificity/sensitivity of DCNN models with the performance of six thyroid ultrasound radiologists on these three validation sets. Findings DCNN model achieved high performance in identifying thyroid cancer patients versus six experience radiologists: for Tianjin validation set, sensitivity was 92·2% versus 96·9% (95% CI 89·7% - 94·3% vs. 93·9% - 98·6%; p = 0·003), and specificity was 85·6% versus 59·4% (95% CI 82·4% - 88·4% vs. 53% - 65·6%; p < 0·0001); for Jilin validation set, sensitivity was 84·3% versus 92·9% (95% CI 73·6% - 91·9% vs. 84·1% - 97·6%; p = 0·05), and specificity was 86·9% versus 57·1% (95% CI 77·8% - 93·3% vs. 45·9% - 67·9%; p < 0·0001); for Weihai validation set, sensitivity was 84·5% versus 89% (95% CI 81·2% - 87·4% vs. 81·9% - 94%; p = 0·2), and specificity was 87·5% versus 68·6% (95% CI 85·1% - 89·6% vs. 60·7% - 75·8%; p < 0·0001). Interpretation DCNN models exhibited high accuracy, sensitivity, and specificity in identifying thyroid cancer patients at levels comparable to or higher than six experienced radiologists. Conferred by the high specificity of DCNN models, the rate of overdiagnosis and overtreatment of patients with thyroid cancer is expected to decrease. This supports future application of the deep learning models to clinical practice for thyroid cancer diagnosis. However, further validation of these DCNN models in prospective clinical trials is warranted. Funding The Program for Changjiang Scholars and Innovative Research Team in University in China (IRT_14R40), National Natural Science Foundation of China (31801117). Citation Format: Xiangchun Li, Sheng Zhang, Qiang Zhang, Xi Wei, Yi Pan, Jing Zhao, Xiaojie Xin, Xiaoqing Wang, Fan Yang, Jianxin Li, Meng Yang, Qinghua Wang, Xiangqian Zheng, Yanhui Zhao, Lun Zhang, Xudong Wang, Zhimin Zheng, Christopher T. Whitlow, Metin N. Gurcan, Boris Pasche, Ming Gao, Wei Zhang, Kexin Chen. Diagnosis of thyroid cancer using deep convolutional neural network models applied to sonographic images from clinical ultrasound exams [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2019; 2019 Mar 29-Apr 3; Atlanta, GA. Philadelphia (PA): AACR; Cancer Res 2019;79(13 Suppl):Abstract nr 1394.

In the realm of computer vision, image classification is a critical issue with many applications, including multimedia content analysis, security and surveillance, and medical imaging. The accuracy of image classification algorithms has considerably increased with the development of deep learning. The discipline of image classification has dramatically benefited from the usage of pre-trained Convolutional Neural Network (CNN) models. In this study, we perform image classification on the Wang dataset composed of 1000 images separated into ten categories, using six different pre-trained CNN models. The research made use of pre-trained versions of VGG16, Densenet, Mobilenet, Inception V3, Resnet50, and Xception models. We assessed the model performances in terms of training and testing accuracy. Testing accuracies for ten unique batches of data were calculated and averaged out. Densenet outperformed state-of-the-art models like Xception by a small margin, mainly due to low data availability. The findings of this study show how pre-trained models have advanced the field of image classification and shed light on how well these models perform in diverse image classification tasks. This study can serve as a valuable reference for researchers and practitioners in the field of computer vision and deep learning and help inform their choices when selecting pre-trained models for image classification, depending on their needs, while also considering data availability and computational constraints.

Fingerprint classification helps in reducing the number of comparisons during the matching stage in automatic fingerprint identification system. In this study, a convolutional neural network model is proposed for classification of plain, rolled and latent fingerprints. We first propose a new convolutional neural network model initialised with random weights and train the model on fingerprint images. Then we fine-tune two pre-trained convolutional neural network models on fingerprint images. Finally, we compare these three models: two pre-trained models and a custom convolutional neural network model initialised with random weights. We show that pre-trained models can achieve the state-of-the-art results on other similar tasks with no or little fine-tuning. We also show that training data size and depth of the network have a serious impact on the overall performance of deep networks. Dropout is used to enhance the performance of deep networks where the labelled training data is not of sufficient size. All the three models trained on NIST DB4 fingerprint and IIIT-D latent fingerprint databases report good accuracy. By only fine-tuning the pre-trained convolutional neural network model, we get the accuracy of 99%, easily out-performing the state-of-the-art.

Excessive withdrawal of groundwater for agricultural irrigation can cause seawater intrusion into coastal aquifers. Such a case will in turn results in deterioration of irrigation water quality. Determination of irrigation water quality with traditional methods is a time-consuming and costly process. However, machine learning algorithms can be useful tools for modeling and estimating groundwater quality used for irrigation water purposes. In this study, TDS, PS, SAR, and Cl parameters of groundwater were estimated with models based on EC and pH variables. For this purpose, prediction performances of two different deep learning methods (convolutional neural network (CNN) and deep neural network (DNN)) and two different classical machine learning (Random Forest (RF) and extreme gradient boosting (XGBoost)) methods were compared. In addition, predictive uncertainty of the models was determined by quantile regression (QR) analysis. Performance criteria and results of uncertainty analysis revealed that CNN (in testing phase, NSE = 0.95 for TDS, NSE = 0.96 for PS, NSE = 0.67 for SAR and NSE = 0.93 for CI) and DNN (in testing phase, NSE = 0.91 for TDS, NSE = 0.91 for PS, NSE = 0.57 for SAR and NSE = 0.94 for Cl) models had quite a close performance in estimation of TDS, PS, SAR, and Cl parameters and higher than the other two classical machine learning methods. As a result, the CNN model can be considered the best performing model in estimating all quality parameters due to the highest NSE and lowest RMSE values. In addition, the Taylor diagram showed that the values estimated using the CNN model had the highest correlation with the measured data. It was determined that the model with the lowest uncertainty based on the PICP statistics was DNN, followed by the CNN model. However, the CNN model has predicted outliers more accurately. Present findings proved that deep learning models could offer efficient tools for predicting irrigation water quality parameters.

A case study on computer-aided diagnosis of nonerosive reflux disease using deep learning techniques

For the purpose of applying information technology to the creation of ink style painting, the algorithm of ink painting rendering based on the deep learning framework and convolutional neural network model is designed and improved. Firstly, the ink style rendering program is written in Python. Secondly, VGG under Caffe architecture and Illustration 2Vec models are transplanted to TensorFlow architecture, and the image is rendered in ink style based on deep learning framework and convolutional neural network model. Finally, based on Node.js, the server-side program for image ink style rendering is built. Among them, Express is adopted as the Web-side framework, and the front-end page effect is completed. The results show that the ink rendering logic program is applicable, and the expected purpose is achieved.

Thousands of women worldwide are diagnosed with breast cancer yearly, which may be fatal if not treated. The diagnosis of the condition may take years, by which time the patient has little choice except to have the affected breast removed. Early diagnosis and treatments are the best ways to stop this disease's spread. In this study, the authors presented a Computer Aided Diagnosis (CAD) system to assist in breast cancer diagnosis. The study uses the Wisconsin breast cancer dataset to classify benign and malignant data. For the classification, three pre-trained Deep learning algorithms: Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), Multilayer Perceptron (MLP), were used. A novel CNN model that exceeds the performance efficiency of three pre-trained models and requires minimal compilation time is proposed. A number of evaluation matrices are used to analyze the models' classification abilities. Upon closer inspection, it has been established that the proposed CNN model outperforms CNN, LSTM, and MLP models with validation accuracy of 97.85%. CNN and LSTM performed with accuracies of 94.12% with the Adagrad optimizer and 93.5% with the Adam optimizer, respectively. Furthermore, MLP performance with 92.44% accuracy using the Adam optimizer. The proposed CNN model achieves the lowest Loss value and compilation time. In addition, the models' recall value, precision, and f1-score are computed to pick out the most effective model for diagnosing breast cancer on numeric data.

Visual attention mechanism has been widely used in computer vision and plays a vital role in the research of human action recognition. In this paper, we explore a novel moving target detection mechanism for human action recognition and propose a new 3D CNN (3D Convolutional Neural Network) model, dubbed LaM-2SRN (Local Features Enhanced and Moving target detected 2Stream-ResNet) for extracting and learning attention-enhanced spatiotemporal features. The contributions of this paper are as follow: First, the traditional CAM (Class Activation Map) based visual attention algorithm is used to obtain the optical flow information of the human region, thus eliminating the influence of irrelevant optical flow information (such as background clutter). Second, the ViBe algorithm is used to identify the moving target in the continuous frame, retain the optical flow information of the moving target, and make it complement with the optical flow information of the human region, to obtain a complete motion descriptor. After the motion information is marked in the video frame, we put the marked video frames and the original video frames into the 3D CNN and 2D CNN models respectively, to acquired mixed descriptor, which serves as the basis for video classification. Different from most of the previous target detection algorithms, this paper not only detects the human target, but also detects the moving foreground targets in the video, so that the pre-trained CNN model can obtain more complete motion information from videos. We use RGB-only video data for evaluation on two benchmarks: UCF101 and HMDB51, the experimental results demonstrate that our LaM-2SRN is comparable to the previous state-of-the-art algorithms.

This paper addresses the problems related to the mapless navigation control of wheeled mobile robots based on deep learning technology. The traditional navigation control framework is based on a global map of the environment, and its navigation performance depends on the quality of the global map. In this paper, we proposes a mapless Light Detection and Ranging (LiDAR) navigation control method for wheeled mobile robots based on deep imitation learning. The proposed method is a data-driven control method that directly uses LiDAR sensors and relative target position for mobile robot navigation control. A deep convolutional neural network (CNN) model is proposed to predict motion control commands of the mobile robot without the requirement of the global map to achieve navigation control of the mobile robot in unknown environments. While collecting the training dataset, we manipulated the mobile robot to avoid obstacles through manual control and recorded the raw data of the LiDAR sensor, the relative target position, and the corresponding motion control commands. Next, we applied a data augmentation method on the recorded samples to increase the number of training samples in the dataset. In the network model design, the proposed CNN model consists of a LiDAR CNN module to extract LiDAR features and a motion prediction module to predict the motion behavior of the robot. In the model training phase, the proposed CNN model learns the mapping between the input sensor data and the desired motion behavior through end-to-end imitation learning. Experimental results show that the proposed mapless LiDAR navigation control method can safely navigate the mobile robot in four unseen environments with an average success rate of 75%. Therefore, the proposed mapless LiDAR navigation control system is effective for robot navigation control in an unknown environment without the global map.

Electricity theft is harmful to power grids. Integrating information flows with energy flows, smart grids can help to solve the problem of electricity theft owning to the availability of massive data generated from smart grids. The data analysis on the data of smart grids is helpful in detecting electricity theft because of the abnormal electricity consumption pattern of energy thieves. However, the existing methods have poor detection accuracy of electricity theft since most of them were conducted on one-dimensional (1-D) electricity consumption data and failed to capture the periodicity of electricity consumption. In this paper, we originally propose a novel electricity-theft detection method based on wide and deep convolutional neural networks (CNN) model to address the above concerns. In particular, wide and deep CNN model consists of two components: the wide component and the deep CNN component. The deep CNN component can accurately identify the nonperiodicity of electricity theft and the periodicity of normal electricity usage based on 2-D electricity consumption data. Meanwhile, the wide component can capture the global features of 1-D electricity consumption data. As a result, wide and deep CNN model can achieve the excellent performance in electricity-theft detection. Extensive experiments based on realistic dataset show that wide and deep CNN model outperforms other existing methods.

Chest X-rays (CXR) images are a useful noninvasive diagnostic tool for assessing various lung diseases. In this paper, we propose transfer learning with a fine-tuning-based model to detect and classify COVID-19 and pneumonia using CXR images to assist the radiologist with diagnosis. One of the difficulties with the medical imaging classification is the limited number of available datasets, and hence training a deep Convolutional Neural Network (CNN) model for medical image classification on a small dataset is challenging. We address this issue by exploiting transfer learning via fine-tuning. In this paper, we use a pre-trained deep CNN model and then fine-tune the layers of the neural network to perform multi-class classification using CXR images. The model is trained to perform multi-class classification, such as two-class (COVID-19 vs normal), three-class (COVID-19 vs Bacterial Pneumonia vs normal), four-class (COVID-19 vs Bacterial Pneumonia vs lung opacity vs normal), and five-class (COVID-19 vs Bacterial Pneumonia vs Viral Pneumonia vs lung opacity vs normal) classification. The performance of the model is evaluated in terms of accuracy, precision, recall, and F1-score.

The aim of the present study was to explore the diagnostic value of a deep convolutional neural network (DCNN) model for the diagnosis of pulmonary nodules in adolescent and young adult patients with osteosarcoma. For the present study, 675 chest CT images were retrospectively collected from 109 patients with clinically confirmed osteosarcoma who underwent chest CT examination at Hangzhou Third People's Hospital (Hangzhou, China) from March 2011 to February 2022. CT images were then evaluated using the DCNN and manual models. Subsequently, pulmonary nodules of osteosarcoma were divided into calcified nodules, solid nodules, partially solid nodules and ground glass nodules using the DCNN model. Those patients with osteosarcoma who were diagnosed and treated were followed up to observe dynamic changes in the pulmonary nodules. A total of 3,087 nodules were detected, while 278 nodules were missed compared with those determined using the reference standard given by the consensus of three Experienced radiologists., which was analyzed by two diagnostic radiologists. In the manual model group, 2,442 nodules were detected, while 657 nodules were missed. The DCNN model showed significantly higher sensitivity and specificity compared with the manual model (sensitivity, 0.923 vs. 0.908; specificity, 0.552 vs. 0.351; P<0.05). In addition, the DCNN model yielded an area under the curve (AUC) value of 0.795 [95% confidence interval (CI), 0.743-0.846], outperforming that of the manual model (AUC, 0.687; 95% CI, 0.629-0.732; P<0.05). The film reading time of the DCNN model was also significantly shorter compared with that of the manual model [mean ± standard deviation (SD); 173.25±24.10 vs. 328.32±22.72 sec; P<0.05)]. The AUC of calcified nodules, solid nodules, partially solid nodules and ground glass nodules was calculated to be 0.766, 0.771, 0.761 and 0.796, respectively, using the DCNN model. Using this model, the majority of the pulmonary nodules were detected in patients with osteosarcoma at the initial diagnosis (69/109, 62.3%), and the majority of these were found with multiple pulmonary nodules instead of a single nodule (71/109, 65.1% vs. 38/109, 34.9%). These data suggest that, compared with the manual model, the DCNN model proved to be beneficial for the detection of pulmonary nodules in adolescent and young adult patients with osteosarcoma, which may reduce the time of artificial radiograph reading. In conclusion, the proposed DCNN model, developed using data from 675 chest CT images retrospectively collected from 109 patients with clinically confirmed osteosarcoma, may be used as an effective tool to evaluate pulmonary nodules in patients with osteosarcoma.