An efficient algorithm for recognition of emotions from speaker and language independent speech using deep learning

Deep learning-based computed tomography applied to the diagnosis of rib fractures

Background Incidence rate of thyroid cancer is steadily increasing due to overdiagnosis and overtreatment. Thyroid ultrasound is commonly used to diagnose thyroid cancer. The aim of this study is to examine the accuracy of using deep convolutional neural network (DCNN) models to improve diagnosis of thyroid cancer by analyzing sonographic imaging data from clinical thyroid ultrasound. Methods A total of 131,731 sonographic images from 17,627 thyroid cancer patients and 180,668 sonographic images from 25,325 controls used as training set were obtained from Tianjin Cancer Hospital. Images from anatomical sites that did not have cancer according to location sign on the image were not included. All thyroid cancer patients and 13·2% of controls (51,255 images) were confirmed by pathological reports. DCNN is a specific type of neural network optimized for image recognition. We trained two DCNN models on the training set and subsequently evaluated the performance on one independent internal (Tianjin, 1,118 individuals) and two external (Jilin,154 individuals; Weihai, 1,420 individuals) validation sets. Individuals in the validation sets all have pathological examinations. We compared the specificity/sensitivity of DCNN models with the performance of six thyroid ultrasound radiologists on these three validation sets. Findings DCNN model achieved high performance in identifying thyroid cancer patients versus six experience radiologists: for Tianjin validation set, sensitivity was 92·2% versus 96·9% (95% CI 89·7% - 94·3% vs. 93·9% - 98·6%; p = 0·003), and specificity was 85·6% versus 59·4% (95% CI 82·4% - 88·4% vs. 53% - 65·6%; p < 0·0001); for Jilin validation set, sensitivity was 84·3% versus 92·9% (95% CI 73·6% - 91·9% vs. 84·1% - 97·6%; p = 0·05), and specificity was 86·9% versus 57·1% (95% CI 77·8% - 93·3% vs. 45·9% - 67·9%; p < 0·0001); for Weihai validation set, sensitivity was 84·5% versus 89% (95% CI 81·2% - 87·4% vs. 81·9% - 94%; p = 0·2), and specificity was 87·5% versus 68·6% (95% CI 85·1% - 89·6% vs. 60·7% - 75·8%; p < 0·0001). Interpretation DCNN models exhibited high accuracy, sensitivity, and specificity in identifying thyroid cancer patients at levels comparable to or higher than six experienced radiologists. Conferred by the high specificity of DCNN models, the rate of overdiagnosis and overtreatment of patients with thyroid cancer is expected to decrease. This supports future application of the deep learning models to clinical practice for thyroid cancer diagnosis. However, further validation of these DCNN models in prospective clinical trials is warranted. Funding The Program for Changjiang Scholars and Innovative Research Team in University in China (IRT_14R40), National Natural Science Foundation of China (31801117). Citation Format: Xiangchun Li, Sheng Zhang, Qiang Zhang, Xi Wei, Yi Pan, Jing Zhao, Xiaojie Xin, Xiaoqing Wang, Fan Yang, Jianxin Li, Meng Yang, Qinghua Wang, Xiangqian Zheng, Yanhui Zhao, Lun Zhang, Xudong Wang, Zhimin Zheng, Christopher T. Whitlow, Metin N. Gurcan, Boris Pasche, Ming Gao, Wei Zhang, Kexin Chen. Diagnosis of thyroid cancer using deep convolutional neural network models applied to sonographic images from clinical ultrasound exams [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2019; 2019 Mar 29-Apr 3; Atlanta, GA. Philadelphia (PA): AACR; Cancer Res 2019;79(13 Suppl):Abstract nr 1394.

BackgroundSurgical process modeling automatically identifies surgical phases, and further improvement in recognition accuracy is expected with deep learning. Surgical tool or time series information has been used to improve the recognition accuracy of a model. However, it is difficult to collect this information continuously intraoperatively. The present study aimed to develop a deep convolution neural network (CNN) model that correctly identifies the surgical phase during laparoscopic cholecystectomy (LC).MethodsWe divided LC into six surgical phases (P1–P6) and one redundant phase (P0). We prepared 115 LC videos and converted them to image frames at 3 fps. Three experienced doctors labeled the surgical phases in all image frames. Our deep CNN model was trained with 106 of the 115 annotation datasets and was evaluated with the remaining datasets. By depending on both the prediction probability and frequency for a certain period, we aimed for highly accurate surgical phase recognition in the operation room.ResultsNine full LC videos were converted into image frames and were fed to our deep CNN model. The average accuracy, precision, and recall were 0.970, 0.855, and 0.863, respectively.ConclusionThe deep learning CNN model in this study successfully identified both the six surgical phases and the redundant phase, P0, which may increase the versatility of the surgical process recognition model for clinical use. We believe that this model can be used in artificial intelligence for medical devices. The degree of recognition accuracy is expected to improve with developments in advanced deep learning algorithms.

Automated emotion recognition (AEE) plays a crucial role in numerous industries that depend on understanding human emotional responses, such as advertising, technology, and human-robot interaction, particularly within the Information Technology (IT) field. However, current systems can often come up short in comprehensively understanding an individual's emotions, as prior research has mainly focused on assessing facial expressions and categorizing them into seven primary emotions, including neutrality. In this study, we present several Deep Convolutional Neural Network (CNN) models designed specifically for the task of facial emotion recognition, utilizing the FER2013 and RAF datasets. The baseline CNN model is established through a trial-and-error method, and its results are compared with more complex deep learning techniques, including ResNet18, VGGNet16, VGGNet19, and Efficient Net-B0 models. Among these models, the VGGNet19 model achieved the best results with a test accuracy of 71.02% on the FER2013 dataset. In comparison, the ResNet18 model outperformed all other models with an 86.02% test accuracy on the RAF-DB dataset. These results underscore the potential for advancing automated emotion recognition through complex deep-learning techniques.

The aim of the present study was to explore the diagnostic value of a deep convolutional neural network (DCNN) model for the diagnosis of pulmonary nodules in adolescent and young adult patients with osteosarcoma. For the present study, 675 chest CT images were retrospectively collected from 109 patients with clinically confirmed osteosarcoma who underwent chest CT examination at Hangzhou Third People's Hospital (Hangzhou, China) from March 2011 to February 2022. CT images were then evaluated using the DCNN and manual models. Subsequently, pulmonary nodules of osteosarcoma were divided into calcified nodules, solid nodules, partially solid nodules and ground glass nodules using the DCNN model. Those patients with osteosarcoma who were diagnosed and treated were followed up to observe dynamic changes in the pulmonary nodules. A total of 3,087 nodules were detected, while 278 nodules were missed compared with those determined using the reference standard given by the consensus of three Experienced radiologists., which was analyzed by two diagnostic radiologists. In the manual model group, 2,442 nodules were detected, while 657 nodules were missed. The DCNN model showed significantly higher sensitivity and specificity compared with the manual model (sensitivity, 0.923 vs. 0.908; specificity, 0.552 vs. 0.351; P<0.05). In addition, the DCNN model yielded an area under the curve (AUC) value of 0.795 [95% confidence interval (CI), 0.743-0.846], outperforming that of the manual model (AUC, 0.687; 95% CI, 0.629-0.732; P<0.05). The film reading time of the DCNN model was also significantly shorter compared with that of the manual model [mean ± standard deviation (SD); 173.25±24.10 vs. 328.32±22.72 sec; P<0.05)]. The AUC of calcified nodules, solid nodules, partially solid nodules and ground glass nodules was calculated to be 0.766, 0.771, 0.761 and 0.796, respectively, using the DCNN model. Using this model, the majority of the pulmonary nodules were detected in patients with osteosarcoma at the initial diagnosis (69/109, 62.3%), and the majority of these were found with multiple pulmonary nodules instead of a single nodule (71/109, 65.1% vs. 38/109, 34.9%). These data suggest that, compared with the manual model, the DCNN model proved to be beneficial for the detection of pulmonary nodules in adolescent and young adult patients with osteosarcoma, which may reduce the time of artificial radiograph reading. In conclusion, the proposed DCNN model, developed using data from 675 chest CT images retrospectively collected from 109 patients with clinically confirmed osteosarcoma, may be used as an effective tool to evaluate pulmonary nodules in patients with osteosarcoma.

Automated accurate emotion recognition system using rhythm-specific deep convolutional neural network technique with multi-channel EEG signals

Deep learning has been widely adopted in automatic emotion recognition and has lead to significant progress in the field. However, due to insufficient training data, pre-trained models are limited in their generalisation ability, leading to poor performance on novel test sets. To mitigate this challenge, transfer learning performed by fine-tuning pr-etrained models on novel domains has been applied. However, the fine-tuned knowledge may overwrite and/or discard important knowledge learnt in pre-trained models. In this paper, we address this issue by proposing a PathNet-based meta-transfer learning method that is able to (i) transfer emotional knowledge learnt from one visual/audio emotion domain to another domain and (ii) transfer emotional knowledge learnt from multiple audio emotion domains to one another to improve overall emotion recognition accuracy. To show the robustness of our proposed method, extensive experiments on facial expression-based emotion recognition and speech emotion recognition are carried out on three bench-marking data sets: SAVEE, EMODB, and eNTERFACE. Experimental results show that our proposed method achieves superior performance compared with existing transfer learning methods.

This work aims to construct and optimize the public emotion network communication model under the artificial intelligence (AI) environment and provide a new research direction for exploring public emotion network communication with intelligent human–computer interaction (HCI). The main challenge of this research is the lack of interactivity of intelligent human–computer expression design, and the accuracy of emotion recognition is not high. First, the theoretical analysis of network public opinion is made, and the network public opinion calculation method and emotion recognition method based on deep learning are proposed. On this basis, the public emotion network communication model is established, and the model’s communication mechanism and rules are described in detail. Next, the emotion calculation of human–computer expression interaction and the emotion expression mode of the humanoid robot are proposed, which provides an experimental basis for HCI and public emotion network communication analysis. According to the data of email network and microblog network in the real network, the network degree distribution map is established. The statistical results show that email and microblog networks belong to power law distribution, and the aggregation coefficient is relatively high. Microblog network degree changes slowly, and more points are scattered in the middle area. In a practical sense, when the positive and negative emotions in the public emotion network communication model are input into the bionic robot, the robot can show happy and lost expressions. The results show that compared with the traditional convolutional neural network (CNN) model or recurrent neural network (RNN) model, the RNN–CNN structure model of RNN combined with CNN reduces the waiting time by about 20% and improves the algorithm accuracy by at least 3.1%. The public emotion network communication model based on deep learning and intelligent HCI can accurately reflect the public emotion state in the network, which provides a practical basis for the application of AI technology in network public opinion judgment.

Humans have traditionally found it simple to identify emotions from facial expressions, but it is far more difficult for a computer system to do the same. The social signal processing subfield of emotion recognition from facial expression is used in a wide range of contexts, particularly for human-computer interaction. Automatic emotion recognition has been the subject of numerous studies, most of which use a machine learning methodology. The recognition of simple emotions like anger, happiness, contempt, fear, sadness, and surprise, however, continues to be a difficult topic in computer vision. Deep learning has recently drawn increased attention as a solution to a variety of practical issues, including emotion recognition. In this study, we improved the convolutional neural network technique to identify 7 fundamental emotions and evaluated several preprocessing techniques to demonstrate how they affected the CNN performance. This research focuses on improving facial features and expressions based on emotional recognition. By identifying or recognising facial expressions that elicit human responses, it is possible for computers to make more accurate predictions about a person's mental state and to provide more tailored responses. As a result, we examine how a deep learning technique that employs a convolutional neural network might improve the detection of emotions based on facial features (CNN). Multiple facial expressions are included in our dataset, which consists of about 32,298 photos for testing and training. The preprocessing system aids in removing noise from the input image, and the pretraining phase aids in revealing face detection after noise removal, including feature extraction. As a result, the existing paper generates the classification of multiple facial reactions like the seven emotions of the facial acting coding system (FACS) without using the optimization technique, but our proposed paper reveals the same seven emotions of the facial acting coding system.

Electricity theft is harmful to power grids. Integrating information flows with energy flows, smart grids can help to solve the problem of electricity theft owning to the availability of massive data generated from smart grids. The data analysis on the data of smart grids is helpful in detecting electricity theft because of the abnormal electricity consumption pattern of energy thieves. However, the existing methods have poor detection accuracy of electricity theft since most of them were conducted on one-dimensional (1-D) electricity consumption data and failed to capture the periodicity of electricity consumption. In this paper, we originally propose a novel electricity-theft detection method based on wide and deep convolutional neural networks (CNN) model to address the above concerns. In particular, wide and deep CNN model consists of two components: the wide component and the deep CNN component. The deep CNN component can accurately identify the nonperiodicity of electricity theft and the periodicity of normal electricity usage based on 2-D electricity consumption data. Meanwhile, the wide component can capture the global features of 1-D electricity consumption data. As a result, wide and deep CNN model can achieve the excellent performance in electricity-theft detection. Extensive experiments based on realistic dataset show that wide and deep CNN model outperforms other existing methods.

Developing a treatment strategy that effectively prolongs the lives of people with brain tumors requires an accurate diagnosis of the condition. Therefore, improving the preoperative classification of meningiomas is a priority. Machine learning (ML) has made great strides thanks to the development of convolutional neural networks (CNNs) and computer-aided tumor detection systems. The deep convolutional layers automatically extract important and dependable information from the input space, in contrast to more traditional neural network layers. One recent and promising advancement in this field is ML. Still, there is a dearth of studies being carried out in this area. Therefore, starting with the analysis of magnetic resonance images, we have suggested in this research work a tried-and-tested and methodical strategy for real-time meningioma diagnosis by image segmentation using a very deep transfer learning CNN model or DNN model (VGG-16) with CUDA. Since the VGGNet CNN model has a greater level of accuracy than other deep CNN models like AlexNet, GoogleNet, etc., we have chosen to employ it. The VGG network that we have constructed with very small convolutional filters consists of 13 convolutional layers and 3 fully connected layers. Our VGGNet model takes in an sMRI FLAIR image input. The VGG's convolutional layers leverage a minimal receptive field, i.e., 3 × 3, the smallest possible size that still captures up/down and left/right. Moreover, there are also 1 × 1 convolution filters acting as a linear transformation of the input. This is followed by a ReLU unit. The convolution stride is fixed at 1 pixel to keep the spatial resolution preserved after convolution. All the hidden layers in our VGG network also use ReLU. A dataset consisting of 264 3D FLAIR sMRI image segments from three different classes (meningioma, tuberculoma, and normal) was employed. The number of epochs in the Sequential Model was set to 10. The Keras layers that we used were Dense, Dropout, Flatten, Batch Normalization, and ReLU. According to the simulation findings, our suggested model successfully classified all of the data in the dataset used, with a 99.0% overall accuracy. The performance metrics of the implemented model and confusion matrix for tumor classification indicate the model's high accuracy in brain tumor classification. The good outcomes demonstrate the possibility of our suggested method as a useful diagnostic tool, promoting better understanding, a prognostic tool for clinical outcomes, and an efficient brain tumor treatment planning tool. It was demonstrated that several performance metrics we computed using the confusion matrix of the previously used model were very good. Consequently, we think that the approach we have suggested is an important way to identify brain tumors.

A dual framework for implicit and explicit emotion recognition: An ensemble of language models and computational linguistics

The accuracy in diagnosing prostate cancer (PCa) has increased with the development of multiparametric magnetic resonance imaging (mpMRI). Biparametric magnetic resonance imaging (bpMRI) was found to have a diagnostic accuracy comparable to mpMRI in detecting PCa. However, prostate MRI assessment relies on human experts and specialized training with considerable inter-reader variability. Deep learning may be a more robust approach for prostate MRI assessment. Here we present a method for autosegmenting the prostate zone and cancer region by using SegNet, a deep convolution neural network (DCNN) model. We used PROSTATEx dataset to train the model and combined different sequences into three channels of a single image. For each subject, all slices that contained the transition zone (TZ), peripheral zone (PZ), and PCa region were selected. The datasets were produced using different combinations of images, including T2-weighted (T2W) images, diffusion-weighted images (DWI) and apparent diffusion coefficient (ADC) images. Among these groups, the T2W + DWI + ADC images exhibited the best performance with a dice similarity coefficient of 90.45% for the TZ, 70.04% for the PZ, and 52.73% for the PCa region. Image sequence analysis with a DCNN model has the potential to assist PCa diagnosis.

In recent years, deep learning especially deep convolutional neural networks (DCNN) has made great progress. Many researchers take advantage of different DCNN models to do object detection in remote sensing. Different DCNN models have different advantages and disadvantages. But in the field of remote sensing, many scholars usually do comparison between DCNN models and traditional machine learning. In this paper, we compare different state-of-the-art DCNN models mainly over two publicly available remote sensing datasets—airplane dataset and car dataset. Such comparison can provide guidance for related researchers. Besides,we provide suggestions for fine-tuning different DCNN models. Moreover, for DCNN models including fully connected layers, we provide a method to save storage space.

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.