Demystifying Neural Networks: A Stage-Based Learning Approach for Novice Students

The non-caloric sweeteners market is catching up with the market of conventionally used sugars due to the benefits of preventing obesity, tooth decay and other health problems. Developing strategies for designing easier-to-produce novel molecules with a sweet taste and less toxicity are up-to-date motivations for the food industry. In this sense, Machine Learning (ML) approaches have been reported as cutting-edge technologies to guide the design of new molecules towards specific objectives, including sweet taste. The largest known dataset of sweet molecules is here provided. The dataset contains fully integrated 9541 sweeteners and 1141 bitterants from FooDB, FlavorDB and literature. This robust dataset allowed the development of standard Machine and Deep Learning pipelines towards conceiving Structure-Activity Relationships (SAR) between molecules and sweetness. In this work, we showcase that Textual Convolutional Neural Networks (TextCNN), Graph Convolutional Networks (GCN), and Deep Neural Networks (DNNs) outperformed most of traditional “shallow” learning approaches. These Deep Learning (DL) models produced platforms to guide the design of new sweeteners and repurposing existing compounds. Sixty million compounds from PubChem were evaluated using these models. Herein, we deliver a dataset of 67724 compounds that present high probabilities of being sweet. Quick searches in literature allowed us to find 13 molecules reported as potent sweetening agents, revealing that our approach is suitable for finding new sweeteners, valuable to expand food chemistry databases, repurposing existing chemicals and designing novel molecules with a sweet taste.

Neural network observer based on fuzzy auxiliary sliding-mode-control for nonlinear systems

The purpose of this study is to familiarise the reader with the foundations of neural networks. Artificial Neural Networks (ANNs) are algorithm-based systems that are modelled after Biological Neural Networks (BNNs). Neural networks are an effort to use the human brain's information processing skills to address challenging real-world AI issues. The evolution of neural networks and their significance are briefly explored. ANNs and BNNs are contrasted, and their qualities, benefits, and disadvantages are discussed. The drawbacks of the perceptron model and their improvement by the sigmoid neuron and ReLU neuron are briefly discussed. In addition, we give a bird's-eye view of the different Neural Network models. We study neural networks (NNs) and highlight the different learning approaches and algorithms used in Machine Learning and Deep Learning. We also discuss different types of NNs and their applications. A brief introduction to Neuro-Fuzzy and its applications with a comprehensive review of NN technological advances is provided.

Today breast cancer is the leading type of cancer among women undergoing cancer screening. A slight delay in detecting and diagnosing this disease may result in irreversible convolutions. Histopathological images from the biopsy examination present a large amount of structural information that can signi cantly improve the prognosis for breast cancer. The pathological analysis, which involves the microscopic examination of the histopathological slides, is a challenging task. An automated computer-aided detection (CAD) procedure is inevitable, as it may decrease the pathologist examination time and help detect the disease at an early stage. Lately, deep learning methods using arti cial neural networks are consistently in use to improve the performance of CAD methods. A common practice among recent studies is to use the transfer learning approach of training deep neural network architectures. Transfer Learning is an established learning approach that facilitates a deep neural network to train quickly on a speci c dataset and resolve an interdisciplinary problem. Deep Learning methods employing the transfer learning approach have provided highly competitive results on the datasets consisting of the whole slide images, which are captured generally at high resolutions. However, the performance is not remarkably appreciable on the small and low-resolution image datasets, in particular the datasets that include patch samples. In this direction, the present study proposes a novel domain-speci c learning strategy, Breast Histo-Fusion, which aims to detect breast cancer even from images of low resolution and small size. Further, four state-of-the-art deep CNNs (AlexNet, VGG19, ResNet, and DenseNet) are trained using both learning approaches: Transfer Learning and Histo-Fusion on the IDC dataset. The proposed Histo-Fusion learning approach has improved the discriminating abilities and performance of each deep CNN by providing better results of (AlexNet -95.75%, VGG19-95.96%, ResNet34-96.17%, ResNet50-96.67%, and DenseNet121-97.49%) compared to (AlexNet -90.41, VGG19-90.51%, ResNet34-90.83%, ResNet50-92.27%, and DenseNet121-93.05%) using the transfer learning strategy. As a result, the procedure can help expert pathologists to perform accurate diagnoses and reduce false-positive rates.

Intrusion detection is a major challenge for security experts in the cyber world. Traditional IDS failed to detect complex and unknown cyber-attacks. Machine learning has become a vibrant technology for cybersecurity. There exists several machine learning algorithms to detect intrusion. Most classifiers are well suited to detect the attacks. However, improving accuracy and detecting unknown attacks in existing IDSs is a great challenge. Therefore, the detailed comparative study of various machine learning approaches such as artificial neural networks, support vector machine, decision tree, and hybrid classifiers used by researchers for intrusion detection are done. Deep learning is an emerging approach which suits well for large data. Deep learning techniques find optimal feature set and classify low-frequency attacks better than other techniques. This study also summarizes literatures in deep learning approaches such as deep auto-encoder, Boltzmann machine, recurrent neural networks, convolutional neural networks, and deep neural networks. Moreover, the datasets used in various literatures and the analysis of deep learning approaches based on the performance metrics are also done. Future directions to detect intrusion are also provided. This study in fact will be helpful to develop IDS based on artificial intelligence approaches such as machine learning and deep learning.KeywordsMachine learningIntrusion detectionFeature selectionDeep learningCyber securityClassifier

Satellite low-Earth-orbiting (LEO) and geostationary (GEO) imager estimates of cloud-top pressure (CTP) have many applications in both operations and in studying long-term variations in cloud properties. Recently, machine learning (ML) approaches have shown improvement upon physically based algorithms. However, ML approaches, and especially neural networks, can suffer from a lack of interpretability, making it difficult to understand what information is most useful for accurate predictions of cloud properties. We trained several neural networks to estimate CTP from the infrared channels of the Visible Infrared Imaging Radiometer Suite (VIIRS) and the Advanced Baseline Imager (ABI). The main focus of this work is assessing the relative importance of each instrument’s infrared channels in neural networks trained to estimate CTP. We use several ML explainability methods to offer different perspectives on feature importance. These methods show many differences in the relative feature importance depending on the exact method used, but most agree on a few points. Overall, the 8.4- and 8.6-μm channels appear to be the most useful for CTP estimation on ABI and VIIRS, respectively, with other native infrared window channels and the 13.3-μm channel playing a moderate role. Furthermore, we find that the neural networks learn relationships that may account for properties of clouds such as opacity and cloud-top phase that otherwise complicate the estimation of CTP. Significance Statement Model interpretability is an important consideration for transitioning machine learning models to operations. This work applies several explainability methods in an attempt to understand what information is most important for estimating the pressure level at the top of a cloud from satellite imagers in a neural network model. We observe much disagreement between approaches, which motivates further work in this area but find agreement on the importance of channels in the infrared window region around 8.6 and 10–12 μm, informing future cloud property algorithm development. We also find some evidence suggesting that these neural networks are able to learn physically relevant variability in radiation measurements related to key cloud properties.

Human emotions play a key role in numerous decision-making processes. The ability to correctly identify likes and dislikes as well as excitement and boredom would facilitate novel applications in neuromarketing, affective entertainment, virtual rehabilitation and forensic neuroscience that leverage on sub-conscious human affective states. In this neuroinformatics investigation, we seek to recognize human preferences and excitement passively through the use of electroencephalography (EEG) when a subject is presented with some 3D visual stimuli. Our approach employs the use of machine learning in the form of deep neural networks to classify brain signals acquired using a brain-computer interface (BCI). In the first part of our study, we attempt to improve upon our previous work, which has shown that EEG preference classification is possible although accuracy rates remain relatively low at 61%-67% using conventional deep learning neural architectures, where the challenge mainly lies in the accurate classification of unseen data from a cohort-wide sample that introduces inter-subject variability on top of the existing intra-subject variability. Such an approach is significantly more challenging and is known as subject-independent EEG classification as opposed to the more commonly adopted but more time-consuming and less general approach of subject-dependent EEG classification. In this new study, we employ deep networks that allow dropouts to occur in the architecture of the neural network. The results obtained through this simple feature modification achieved a classification accuracy of up to 79%. Therefore, this study has shown that the use of a deep learning classifier was able to achieve an increase in emotion classification accuracy of between 13% and 18% through the simple adoption of the use of dropouts compared to a conventional deep learner for EEG preference classification. In the second part of our study, users are exposed to a roller-coaster experience as the emotional stimuli which are expected to evoke the emotion of excitement, while simultaneously wearing virtual reality goggles, which delivers the virtual reality experience of excitement, and an EEG headset, acquires the raw brain signals detected when exposed to this excitement stimuli. Here, a deep learning approach is used to improve the excitement detection rate to well above the 90% accuracy level. In a prior similar study, the use of conventional machine learning approaches involving k-Nearest Neighbour (kNN) classifiers and Support Vector Machines (SVM) only achieved prediction accuracy rates of between 65% and 89%. Using a deep learning approach here, rates of 78%-96% were achieved. This demonstrates the superiority of adopting a deep learning approach over other machine learning approaches for detecting human excitement when immersed in an immersive virtual reality environment.

Neural network models have demonstrated outstanding performance in a variety of applications, from image classification to natural language processing. However, deploying the models to hardware raises efficiency and reliability issues. From the efficiency perspective, the storage, computation, and communication cost of neural network processing is considerably large because the neural network models have a large number of parameters and operations. From the standpoint of robustness, the perturbation in hardware is unavoidable and thus the performance of neural networks can be degraded. As a result, this paper investigates effective learning and optimization approaches as well as advanced hardware designs in order to build efficient and robust neural network designs.

Every data and kind of data need a physical drive to store it. There has been an explosion in the volume of images, video, and other similar data types circulated over the internet. Users using the internet expect intelligible data, even under the pressure of multiple resource constraints such as bandwidth bottleneck and noisy channels. Therefore, data compression is becoming a fundamental problem in wider engineering communities. There has been some related work on data compression using neural networks. Various machine learning approaches are currently applied in data compression techniques and tested to obtain better lossy and lossless compression results. A very efficient and variety of research is already available for image compression. However, this is not the case for video compression. Because of the explosion of big data and the excess use of cameras in various places globally, around 82% of the data generated involve videos. Proposed approaches have used Deep Neural Networks (DNNs), Recurrent Neural Networks (RNNs), and Generative Adversarial Networks (GANs), and various variants of Autoencoders (AEs) are used in their approaches. All newly proposed methods aim to increase performance (reducing bitrate up to 50% at the same data quality and complexity). This paper presents a bibliometric analysis and literature survey of all Deep Learning (DL) methods used in video compression in recent years. Scopus and Web of Science are well-known research databases. The results retrieved from them are used for this analytical study. Two types of analysis are performed on the extracted documents. They include quantitative and qualitative results. In quantitative analysis, records are analyzed based on their citations, keywords, source of publication, and country of publication. The qualitative analysis provides information on DL-based approaches for video compression, as well as the advantages, disadvantages, and challenges of using them.

Bringing artificial intelligence to embedded devices has become a central research topic in many scientific domains (environment, agriculture, sociology, health…). For Human Activity Recognition, Artificial Neural Networks (ANNs) have shown their capability to provide better performance compared to other machine learning methods. However, ANNs suffer from two major limitations. First, ANNs are often trained using supervised learning requiring labelled databases, which are often difficult to build in real applications. Then, those algorithms are usually very expensive in terms of computing power. For that reason, their integration into low-power microcontrollers has been so far only evaluated to a limited extent. In this paper, we propose to evaluate quantitatively and qualitatively the embedded implementation of different neural networks for human activity recognition. First, supervised learning approaches are presented, followed by an exploratory study of unsupervised learning approaches using Self-Organizing Maps. Finally, some aspects of embedded unsupervised online learning are investigated to improve classification results using subject-specific data over a more general training. Each neural network is tested on a Human Activity Recognition dataset acquired from a smartphone using accelerometer and gyroscope sensing information (UCI HAR) and deployed on the SparkFun Edge board. This board hosts a low-power ARM Cortex-M4F-based microcontroller.

Evidence now suggests that precision psychiatry is becoming a cornerstone of medical practices by providing the patient of psychiatric disorders with the right medication at the right dose at the right time. In light of recent advances in neuroimaging and multi-omics, more and more biomarkers associated with psychiatric diseases and treatment responses are being discovered in precision psychiatry applications by leveraging machine learning and neural network approaches. In this article, we focus on the most recent developments for research in precision psychiatry using machine learning, deep learning, and neural network algorithms, together with neuroimaging and multi-omics data. First, we describe different machine learning approaches that are employed to assess prediction for diagnosis, prognosis, and treatment in various precision psychiatry studies. We also survey probable biomarkers that have been identified to be involved in psychiatric diseases and treatment responses. Furthermore, we summarize the limitations with respect to the mentioned precision psychiatry studies. Finally, we address a discussion of future directions and challenges.

Real-world knowledge is represented by a knowledge graph that provides assistance for various applications built on the basis of artificial intelligence. Awareness of the neighbourhood is obtained from the individuals and relationships of the Knowledge Graph. High-dimensional data analysis is a difficult task in many applications and this article discusses the dimensionality by specifying a limited collection of features that implies high-dimensional data without visible or substantial data loss. An unsupervised learning approach based on learning that uses the neural network principle and learns the features using the graph. The Positive Feature Selection approach using the Neural Network (PFSNN) approach in this paper defines features using a graph where the classification is carried out by the NN process and analyses the output of the proposed system. The efficiency of the PFSNN is evaluated by contrasting it with existing classification methods and using different datasets. Performance is measured using the classification performance metrics and it is defined from the observation that the proposed PFSNN algorithm has the best outcome.

Background/Aim: Deep learning is a class of machine learning algorithms. Convolutional neural network, a deep learning algorithm, has brought about breakthroughs in processing image, speech, audio and text. Inside it, a convolutional layer takes information from nearby pixels to create high-level abstraction to improve model performance. This abstraction capacity is what traditional air pollution modelling lacks: traditional air pollution modelling uses variable values at monitor stations (in-situ information) to establish relationship and make prediction. However, neighbouring information (e.g., nearby traffic volume, neighbouring land-use type) also impacts local PM2.5 measurements but is often ignored in previous modelling. Convolutional neural network can potentially take neighbouring information into account and improve model performance. Methods: We used a convolutional neural network with multiple predictors, including aerosol optical depth, chemical transport model outputs, land-use variables, meteorological variables, surface reflectance, absorbing aerosol index as predictors to model ground-level PM2.5 from monitoring stations. For each variable, we extract its values at monitor stations as well as neighbouring locations as input information of the neural network. We incorporated multiple convolutional layers, pooling layers and full connection layers inside the neural network to model complex relationship between variables. Results: Model performance on validation data set indicated a good performance with daily R2 = 0.84 and MSE = 2.94 µg/m3. Model performance also exhibited regional variations with higher model performance in the Eastern and Central U.S. than the Western U.S. The Model still performed well at low PM2.5 levels (<12 µg/m3). Prediction results indicated higher PM2.5 concentrations in the Eastern and Central U.S. Summer time had higher PM2.5 levels than other seasons. Conclusions: This study explored a deep learning approach to model air pollutions with high accuracy, which facilitates follow-up epidemiological studies. This study suggests wider application of deep learning techniques in the field of environmental epidemiology.

Increasing number of social networking sites made people more engaged in their virtual life more than ever and at the same time amount data people put online is enormous and also heaven for researchers for conducting their researches. People tend to put their thoughts online to share with the whole world which also includes suicidal thoughts. Suicide is a social problem and is a major concern of recent times. In this research paper we mainly focused on Twitter which is one of the well-known networking sites. We adopted an approach of machine learning and neural network for this research. Support Vector Machine (SVM) is one of the best machine leaning algorithm for text analysis and neural network is also well known for performance in complex cases. In case of neural network we used three types weight optimizers namely Limited-memory BFGS, Stochastic gradient descent and an extension of stochastic gradient descent which is Adam to attain maximum accuracy. At the end we were able to attain accuracy of 95.2% using SVM and 97.6% using neural network and also we performed 10 fold cross validation for further model performance evaluation.

Abstract. Satellite-based aerosol retrievals provide global spatially distributed estimates of atmospheric aerosol parameters that are commonly needed in applications such as estimation of atmospherically corrected satellite data products, climate modelling and air quality monitoring. However, a common feature of the conventional satellite aerosol retrievals is that they have reasonably low spatial resolution and poor accuracy caused by uncertainty in auxiliary model parameters, such as fixed aerosol model parameters, and the approximate forward radiative transfer models utilized to keep the computational complexity feasible. As a result, the improvement and reprocessing of the operational satellite data retrieval algorithms would become a tedious and computationally excessive problem. To overcome these problems, we have developed a machine-learning-based post-process correction approach to correct the existing operational satellite aerosol data products. Our approach combines the existing satellite retrieval data and a post-processing step where a machine learning algorithm is utilized to predict the approximation error in the conventional retrieval. With approximation error, we refer to the discrepancy between the true aerosol parameters and the ones retrieved using the satellite data. Our hypothesis is that the prediction of the approximation error with a finite training dataset is a less complex and easier task than the direct, fully learned machine-learning-based prediction in which the aerosol parameters are directly predicted given the satellite observations and measurement geometry. Our approach does not require reprocessing of the satellite retrieval products; it requires only a computationally fast machine-learning-based post-processing step of the existing retrieval product. Our approach is based on neural networks trained based on collocated satellite data and accurate ground-based Aerosol Robotic Network (AERONET) aerosol data. Based on our post-processing approach, we propose a post-process-corrected high-resolution Sentinel-3 Synergy aerosol product, which gives a spectral estimate of the aerosol optical depth at five different wavelengths with a high spatial resolution equivalent to the native resolution of the Sentinel-3 Level-1 data (300 m at nadir). With aerosol data from Sentinel-3A and 3B satellites, we demonstrate that our approach produces high-resolution aerosol data with clearly better accuracy than the operational Sentinel-3 Level-2 Synergy aerosol product, and it also results in slightly better accuracy than the conventional fully learned machine learning approach. We also demonstrate better generalization capabilities of the post-process correction approach over the fully learned approach.