Classification using deep learning neural networks for brain tumors

Classification of Bisyllabic Lexical Stress Patterns Using Deep Neural Networks

Heartbeat classification is a crucial step for arrhythmia diagnosis during electrocardiographic (ECG) analysis. The new scenario of wireless body sensor network- (WBSN-) enabled ECG monitoring puts forward a higher-level demand for this traditional ECG analysis task. Previously reported methods mainly addressed this requirement with the applications of a shallow structured classifier and expert-designed features. In this study, modified frequency slice wavelet transform (MFSWT) was firstly employed to produce the time-frequency image for heartbeat signal. Then the deep learning (DL) method was performed for the heartbeat classification. Here, we proposed a novel model incorporating automatic feature abstraction and a deep neural network (DNN) classifier. Features were automatically abstracted by the stacked denoising auto-encoder (SDA) from the transferred time-frequency image. DNN classifier was constructed by an encoder layer of SDA and a softmax layer. In addition, a deterministic patient-specific heartbeat classifier was achieved by fine-tuning on heartbeat samples, which included a small subset of individual samples. The performance of the proposed model was evaluated on the MIT-BIH arrhythmia database. Results showed that an overall accuracy of 97.5% was achieved using the proposed model, confirming that the proposed DNN model is a powerful tool for heartbeat pattern recognition.

Brain tumour classification using conventional techniques involving human intervention is prone to errors. Errors in feature extraction and/or classification may turn out to be fatal. Hence focus has shifted on automated techniques for the feature extraction and classification of brain tumour images. In the present work, a mechanism for the same has been proposed involving Artificial Neural Networks (ANN). Three categories have been incorporated in the present work, viz. normal brain, brain with benign tumour and brain with malignant tumour. In the present case, MRI images of the brain have been used to train an Artificial Neural Network which subsequently predicts the category of some new MRI data. The key challenge in designing such a system is attaining high accuracy of classification. This can be achieved by accurate feature extraction mechanism and then designing an appropriate Neural Network for training and testing. At the outset, segmentation has been employed to separate the region of the tumour (if any). Later the Discrete Wavelet Transform (DWT) is used as a preprocessing tool. The DWT helps in removing the non linearities of the image signal and smoothen it out. Since images exhibit abrupt changes in pixel values, conventional Fourier methods like the Fourier Transform prove to be unfit for image analysis. Subsequently, features like contrast, correlation, energy, homogeneity, mean, standard deviation, entropy, RMS value, variance, smoothness, kurtosis and skewness have been evaluated. Prior to training the Neural Network using the feature values, Principal Component Analysis (PCA) has been employed to find trends in the feature values. A Probabilistic Neural Network (PNN) has been designed and it has been trained using the feature values. I has been found that brain tumour MRI image are classified into normal, benign and malignant category using PNN classifier with an accuracy of 97% for the used data set which can be attributed to the efficacy of the proposed method. Key Words: Brain Tumor Classification, Machine Learning, Feature Extraction, Neural Networks, Confusion Matrix, Classification Accuracy.

Learning ability is the basic characteristic of human intelligence. The July 1, 2005 issue of Science published a list of 125 important questions in science. Among them, the question 94 “What are the limits of learning by machines?”. The annotation “Computers can already beat the world’s best chess players, and they have a wealth of information on the Web to draw on. But abstract reasoning is still beyond any machine”. In recent artificial intelligence has made great progresses. In 1997, the rise of the man-machine war, IBM Supercomputer Deep Blue defeated the chess master Garry Kasparov. On February 14, 2011, IBM’s Watson supercomputer won a practice round against Jeopardy champions Ken Jennings and Brad Rutter. In March 2016, Google DeepMind’s AlphaGo sealed a 4-1 victory over a South Korean Go grandmaster Lee Se-dol. This paper focuses on the machine learning methods of AlphaGo, including reinforcement learning, deep learning, deep reinforcement learning, analysis of the existing problems and the latest research progress. Deep reinforcement learning is the combination of deep learning and reinforcement learning, which can realize the learning algorithm from the perception to action. Simply said, this is the same as human behavior, input sensing information such as vision, and then, direct output action through the deep neural network. Deep reinforcement learning has the potential to learn a variety of skills for the robot to achieve full autonomy. Even though reinforcement learning is practiced successfully, but feature states need to manually set, for complex scene is a difficult thing, especially easy to cause the dimension disaster, and expression is not good. In 2010, Sascha Lange and Martin Riedmiller proposed deep auto-encoder neural networks in reinforcement learning to extract feature, which is used to control the visual correlation. In 2013, DeepMind proposed deep Q-network (DQN) in NIPS 2013, using convolution neural network to extract features, and then applied in reinforcement learning. They continue to improve and published an improved version of DQN on Nature in 2015, which has aroused widespread concern. In order to break through the limits of learning by machines, cognitive machine learning is proposed, which is the combination of machine learning and brain cognition, so that the machine intelligence is constantly evolving, and gradually reaches the human level of artificial intelligence. A cognitive model entitled Consciousness And Memory (CAM) is proposed by author, which consists of memory, consciousness, high-level cognitive functions, perception and motor. High-level cognitive functions of the brain include learning, language, thinking, decision making, emotion, and so on. Learning is a course to accept the stimulus through the nervous system and obtain new behavior, habits and accumulation experience. According to the current research progress of brain science and cognitive science, cognitive machine learning may be interested in learning emergence, procedural memory knowledge learning, learning evolution and so on. For intelligence, so-called evolution is refers to the learning of learning and the structure also follows the change. It is important to record the learning result by structure changing and improve the learning method.

Automated image classification is an essential task of the computer vision field. The tagging of images into a set of predefined groups is referred to as image classification. The implementation of computer vision to automate image classification would be beneficial because manual image evaluation and identification can be time-consuming, particularly when there are many images of different classes. Deep learning approaches are proven to overperform existing machine learning techniques in many fields in recent years, and computer vision is one of the most notable examples. The very deep neural network (VDCNN) is a powerful deep learning model for image classification, and this paper examines it briefly using MNIST handwritten digit dataset. This dataset is used to prove the efficacy of very deep neural networks over other deep learning models. The proposed study aims to comprehend the very deep neural network architecture used to accomplish a handwritten digit recognition task. The feasibility of the proposed model is evaluated using mean accuracy, validation accuracy, and standard deviation. The study results of the very deep neural network model are compared to a convolutional neural network and convolutional neural network with batch normalization. According to the results of the comparison study, very deep neural networks achieve high accuracy of 99.1% for a handwritten dataset. The outcome of the proposed work is used to interpret how well a very deep neural network performs when compared to the other two models of deep neural networks. This proposed architecture may be used to automate the classification of handwritten digits dataset.

Due to the advent of modern embedded systems and mobile devices with constrained resources, there is a great demand for incredibly efficient deep neural networks for machine learning purposes. There is also a growing concern of privacy and confidentiality of user data within the general public when their data is processed and stored in an external server which has further fueled the need for developing such efficient neural networks for real-time inference on local embedded systems. The scope of our work presented in this paper is limited to image classification using a convolutional neural network. A Convolutional Neural Network (CNN) is a class of Deep Neural Network (DNN) widely used in the analysis of visual images captured by an image sensor, designed to extract information and convert it into meaningful representations for real-time inference of the input data. In this paper, we propose a neoteric variant of deep convolutional neural network architecture to ameliorate the performance of existing CNN architectures for real-time inference on embedded systems. We show that this architecture, dubbed CondenseNeXt, is remarkably efficient in comparison to the baseline neural network architecture, CondenseNet, by reducing trainable parameters and FLOPs required to train the network whilst maintaining a balance between the trained model size of less than 3.0 MB and accuracy trade-off resulting in an unprecedented computational efficiency.

In the past decade, Deep Neural Networks (DNNs) have achieved breakthrough collaborations in developing smart intelligent systems within the field of computer vision, natural language processing, autonomous systems, etc. Recent research has revealed that stability of such smart systems is at greater risk when they come across to adversarial perturbations. Although, these perturbations may not be perceivable in nature when seen from naked eye, yet, they are capable enough to fool state-of-the-art DNN classifiers. Till now, much of the previous work related to fool such classifiers focuses on generating adversaries that directly change pixel values of an image in spatial-domain. In this paper, we propose a novel transform-domain imperceptible attack methodology “TDIAM” to generate adversaries based on image steganography-approach using a “single carefully selected targeted watermark”. We use three different frequency-domain approaches, i.e., Discrete Wavelet Transform (DWT), Discrete Cosine Transform (DCT) and Fast Fourier Transform (FFT) to craft perturbations in selective frequency component which makes it robust and it requires less computational time as it is a non-gradient approach. We present our case study on MNIST handwritten digits dataset. Our results demonstrate that the generated perturbation vector successfully fool simple Convolutional Neural Network (CNN), LeNet-5 and AlexNet architectures by increasing probability of adversarial examples for the targeted class (to which the targeted watermark belongs) in both “black-box” and “white-box” adversarial attacks. The results have shown that among these three perturbation approaches, DWT based perturbation shown promising results by effectively fooling DNNs while ensuring the high imperceptibility as well.

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

Deep learning is a fast flattering highest development in the machine learning technique leading to augment performance in diverse medical applications. The tributary artificial intelligence is rapidly becoming metamorphic in the domain of Healthcare Technology rendering facility to analyze the medical data with high rapidity and exactness. Due to the overwhelming development of artificial intelligence (AI), the contemporary encroachments of deep learning employing advanced and futuristic deep learning methods has become a vivacious research domain for processing medical data in the health care industries. This paper presents a complete analysis about the deep structured learning based methods used in medical image analysis and also elucidate about the advancement of convolutional neural network which is a major class of deep neural networks used in medical image processing. It also gives insight about the fundamentals of deep learning methods and reviews in image interpretation. It also explains about the implementation of an integrated framework for medical image synthesis combining Speckle GAN and Di-Conv-AE-Net.

15 - Analysis on the prediction of central line-associated bloodstream infections (CLABSI) using deep neural network classification

Deep neural networks (DNNs) have achieved unprecedented success across many scientific and engineering fields in the last decades. Despite its empirical success, unfortunately, recent studies have shown that there are various failure modes and blindspots in DNN models which may result in unexpected serious failures and potential harms, e.g. the existence of adversarial examples and small perturbations. This is not acceptable especially for safety critical and high stakes applications in the real-world, including healthcare, self-driving cars, aircraft control systems, hiring and malware detection protocols. Moreover, it has been challenging to understand why and when DNNs will fail due to their complicated structures and black-box behaviors. Lacking interpretability is one critical issue that may seriously hinder the deployment of DNNs in high-stake applications, which need interpretability to trust the prediction, to understand potential failures, and to be able to mitigate harms and eliminate biases in the model. To make DNNs trustworthy and reliable for deployment, it is necessary and urgent to develop methods and tools that can (i) quantify and improve their robustness against adversarial and natural perturbations, and (ii) understand their underlying behaviors and further correct errors to prevent injuries and damages. These are the important first steps to enable Trustworthy AI and Trustworthy Machine Learning. In this talk, I will survey a series of research efforts in my lab contributed to tackling the grand challenges in (i) and (ii). In the first part of my talk, I will overview our research effort in Robust Machine Learning since 2017, where we have proposed the first attack-agnostic robustness evaluation metric, the first efficient robustness certification algorithms for various types of perturbations, and efficient robust learning algorithms across supervised learning to deep reinforcement learning. In the second part of my talk, I will survey a series of exciting results in my lab on accelerating interpretable machine learning and explainable AI. Specifically, I will show how we could bring interpretability into deep learning by leveraging recent advances in multi-modal models. I'll present recent works in our group on automatically dissecting neural networks with open vocabulary concepts, designing interpretable neural networks without concept labels, and briefly overview our recent efforts on demystifying black-box DNN training process, automated neuron explanations for Large Language Models and the first robustness evaluation of a family of neuron-level interpretation techniques.

This study presents a fault diagnosis system for vehicle heating, ventilation and air conditioning (HVAC) acoustic signal with various feature extractions in deep learning neural network. Traditionally, sound used for fault diagnosis or signal classification is observed the difference of energy in time or frequency domains. Unfortunately, the frequency smearing effect often arises in some critical conditions. In the present study, discrete wavelet transform (DWT) and wavelet packet transform (WPT) are proposed in fault diagnosis. Meanwhile, when using mechanical learning methods, the data are relatively large, in order to reduce the amount of data, DWT and WPT low-frequency decomposition could be used to improve the performance. Furthermore, the signal characteristics more comprehensive, this study attempts to use the feature extraction method of wavelet packet conversion to improve the signal characteristics. In the experiment process, the operation state of the blade blower in the vehicle air conditioner, four different faults were designed, test database was established through sound to classify, and identify the data using deep neural networks to achieve the purpose of blower fault diagnosis. In data analysis, the original signal is presented through wavelet packet decomposition and discrete packet conversion technology, compared with traditional time and frequency domain signals to explore the identification rate, identification speed and related issues. Experimental results show that using WPT combined with deep neural networks have good fault diagnosis and discrimination capabilities, training, and identification time is shorter than time-frequency domain signals.

Modern software systems are increasingly including machine learning (ML) as an integral component. However, we do not yet understand the difficulties faced by software developers when learning about ML libraries and using them within their systems. To fill that gap this thesis reports on a detailed (manual) examination of 3,243 highly-rated Q&A posts related to ten ML libraries, namely Tensorflow, Keras, scikitlearn, Weka, Caffe, Theano, MLlib, Torch, Mahout, and H2O, on Stack Overflow, a popular online technical Q&A forum. Our findings reveal the urgent need for software engineering (SE) research in this area. The second part of the thesis particularly focuses on understanding the Deep Neural Network (DNN) bug characteristics. We study 2,716 high-quality posts from Stack Overflow and 500 bug fix commits from Github about five popular deep learning libraries Caffe, Keras, Tensorflow, Theano, and Torch to understand the types of bugs, their root causes and impacts, bug-prone stage of deep learning pipeline as well as whether there are some common antipatterns found in this buggy software. While exploring the bug characteristics, our findings imply that repairing software that uses DNNs is one such unmistakable SE need where automated tools could be beneficial; however, we do not fully understand challenges to repairing and patterns that are utilized when manually repairing DNNs. So, the third part of this thesis presents a comprehensive study of bug fix patterns to address these questions. We have studied 415 repairs from Stack Overflow and 555 repairs from Github for five popular deep learning libraries Caffe, Keras, Tensorflow, Theano, and Torch to understand challenges in repairs and bug repair patterns. Our key findings reveal that DNN bug fix patterns are distinctive compared to traditional bug fix patterns and the most common bug fix patterns are fixing data dimension and neural network connectivity. Finally, we propose an automatic technique to detect ML Application Programming Interface (API) misuses. We started with an empirical study to understand ML API misuses. Our study shows that ML API misuse is prevalent and distinct compared to non-ML API misuses. Inspired by these findings, we contributed Amimla (Api Misuse In Machine Learning Apis) an approach and a tool for ML API misuse detection. Amimla relies on several technical innovations. First, we proposed an abstract representation of ML pipelines to use in misuse detection. Second, we proposed an abstract representation of neural networks for deep learning related APIs. Third, we have developed a representation strategy for constraints on ML APIs. Finally, we have developed a misuse detection strategy for both single and multi-APIs. Our experimental evaluation shows that Amimla achieves a high average accuracy of ∼80% on two benchmarks of misuses from Stack Overflow and Github.

Power system fault type classification and location prediction is critical in assessing the reliability of the power system, and later restoring it to a stable operating point followed by a fault. State of the art methods include sequence component, impedance measurement from the origin of the fault, and traveling wave based methods for fault type detection and classification problem. It is important to identify and classify the fault as quickly as possible for restoring power system stability to normal operation. Machine Learning and Deep Learning methods allow the analysis of large data of fault voltages and currents by using fast and efficient algorithms. These methods require large amount of data, however, with the recent advances in the field of power system, data acquisition using smart meters and Phasor Measurement Units (PMU), huge amount of system-wide data can be made available to analyze the problem of fault type classification and location prediction. This paper presents a comparative study of Stochastic Gradient Descent (SGD) based Deep Neural Network (DNN) and Machine Learning (ML) applied to power system fault type and location prediction problem. DNN architecture uses 10 hidden layers, each layer having 60 units with hyperbolic tangent as activation function, and a combination of Support Vector Machine (SVM) and Principal Component Analysis (PCA) method are considered. Comparative results in terms of time taken to run the algorithms, and accuracy of the results obtained are presented for a 3 machine 9 bus system. Results indicate that SVM method is an optimal choice for fault classification with high accuracy for location prediction and low computational requirements compared with DNN.

Deep neural networks for texture classification—A theoretical analysis