Machine Learning in Neural Networks.

Chemical heuristics have been fundamental to the advancement of chemistry and materials science. These heuristics are typically established by scientists using knowledge and creativity to extract patterns from limited datasets. Machine learning offers opportunities to perfect this approach using computers and larger datasets. Here, we discuss the relationships between traditional heuristics and machine learning approaches. We show how traditional rules can be challenged by large-scale statistical assessment and how traditional concepts commonly used as features are feeding the machine learning techniques. We stress the waste involved in relearning chemical rules and the challenges in terms of data size requirements for purely data-driven approaches. Our view is that heuristic and machine learning approaches are at their best when they work together.

Classification and prediction tasks are common in health research. With the increasing availability of vast health data repositories (e.g. electronic medical record databases) and advances in computing power, traditional statistical approaches are being augmented or replaced with machine learning (ML) approaches to classify and predict health outcomes. ML describes the automated process of identifying ("learning") patterns in data to perform tasks. Developing an ML model includes selecting between many ML models (e.g. decision trees, support vector machines, neural networks); model specifications such as hyperparameter tuning; and evaluation of model performance. This process is conducted repeatedly to find the model and corresponding specifications that optimize some measure of model performance. ML models can make more accurate classifications and predictions than their statistical counterparts and confer greater flexibility when modelling unstructured data or interactions between covariates; however, many ML models require larger sample sizes to achieve good classification or predictive performance and have been criticized as "black box" for their poor transparency and interpretability. ML holds potential in family medicine for risk profiling of patients' disease risk and clinical decision support to present additional information at times of uncertainty or high demand. In the future, ML approaches are positioned to become commonplace in family medicine. As such, it is important to understand the objectives that can be addressed using ML approaches and the associated techniques and limitations. This article provides a brief introduction into the use of ML approaches for classification and prediction tasks in family medicine.

A point of interest (POI) is a particular location point that is useful or interesting for people, such as restaurants, museums, parks and hotels. POIs are mostly used on location based social media applications, especially for place recommendation. Social media users share the places they like and discovering such new POIs has importance for understanding the taste and preference of city citizens and for understanding the city. An important problem is, in a given social media message, detecting which word can be POI. This process of retrieving points of interest from a text is called POI extraction. In this work, we propose methods to extract POIs from microblogs. We explore both machine learning and artificial neural network based approaches. As machine learning approach, we use Conditional Random Fields (CRF) for sequential tagging. We investigate the effect of various additional features such as sentiment of tweets, POI density and population density of the location where the tweet was posted. We also use built-in features of CRF. As a hybrid approach, we generate word embeddings by Word2vec and apply K-Nearest Neighbors classification algorithm on the vectors constructed. Finally we construct a deep, feed-forward neural network to extract POIs from microblog text. These techniques are applied on a collection of tweets in Turkish posted by users from Ankara. Experimental results show that CRF constructed with POI density feature outperforms CRF with other feature sets along with other neural network approaches in terms of POI extraction accuracy.

Intelligent ground vehicles require automated steering control systems. The complexity of the automated steering systems has increased as the concurrent data receiving, processing, decision-making and monitoring all happen simultaneously in super-fast sampling rates. The research reported in this paper focuses on the neural network training and machine learning approach for an automated steering system. The essential information of the steering controller was trained using artificial neural network (ANN) and pattern recognition algorithm approaches. The objective is to investigate the design of ANN training and deep machine learning on collected data from sensors of steering wheel angular position, speed, steering column torque, and ego vehicle speed and generating acceptable steering commands for the vehicle. The results of the research showed that the proposed network control system had trained and validated more than 96.5% steering system behavior patterns and adapted large random disturbances of the steering controller commands. It is, therefore, necessary to develop artificial intelligence methodologies in automated steering systems of autonomous vehicles with neural network representing the main topology blocks of the control system architecture and utilize ANN abstraction in the control system of autonomous vehicle steering control system.

Optimized machine learning approaches for identifying vertical temperature gradient on ballastless track in natural environments

Combining machine learning in neural networks with multimodal fusion strategies offers an interesting potential for classification tasks but the optimum fusion strategies for many applications have yet to be determined. Here we address this issue in the context of human activity recognition, making use of a state-of-the-art convolutional network architecture (Inception I3D) and a huge dataset (NTU RGB+D). As modalities we consider RGB video, optical flow, and skeleton data. We determine whether the fusion of different modalities can provide an advantage as compared to uni-modal approaches, and whether a more complex early fusion strategy can outperform the simpler late-fusion strategy by making use of statistical correlations between the different modalities. Our results show a clear performance improvement by multi-modal fusion and a substantial advantage of an early fusion strategy,

Background: Adolescent depression is highly prevalent in low- and middle-income countries (LMICs). Identifying top key risk factors is necessary to inform effective prevention program design. Machine learning (ML) offers a powerful approach to analyze complex multidomain of data to identify the most relevant predictors and estimate risks for mental health problems. This paper applies ML to study risks for adolescent depression to enhance adolescent depression prevention efforts in LMICs. Methods: Six ML approaches (e.g., Explainable Boosting Machine, random forests, and XGBoost) were applied to study the risks of depression. Data were drawn from a digital health intervention study conducted in Kenya (year 2024–2025, n = 269). Multiple domains of childhood and adolescent adversity and stress experiences were used to predict adolescent depression (using PHQ9-A). Findings: We found that ML was a valuable approach in the early identification of adolescents at risk for depression. Among the six ML approaches applied, the random forest approach outperformed other ML approaches, especially when multiple domains of risks were included. We also found that childhood adversity or home adversity alone were not strong predictors for depression. Adding adolescent stress experiences and community school adversity experiences significantly improves the accuracy and predictability of depression. Using the top-15 and top-20 ranking factors, we achieved 74.8% and 75.1% accuracy in depression prediction, which was similar to the accuracy when all 49 adverse/stress factors were included in the predictive model (78.3%). Conclusions: Innovative ML and modern predictive modeling approaches have the potential to transform modern preventive mental health care by better utilizing multidomain data to identify individuals at risk for developing depression early and identify top risk factors (for targeted individuals and/or populations). Findings from ML can inform tailored intervention design to better mitigate risks in order to prevent depression problem development. They can also inform the better utilization of resources to target high-need cases and key determinants, which is particularly useful for LMICs and low-resource settings. This paper illustrates an example of how to move toward this direction. Future research is needed to validate the approach.

Performance optimization of wireless networks is typically complicated because of high computational complexity and dynamic channel conditions. Considering a specific case, the recent introduction of intelligent reflecting surface (IRS) can reshape the wireless channels by controlling the scattering elements' phase shifts, namely, passive beamforming. However, due to the large size of scattering elements, the IRS's beamforming optimization becomes intractable. In this article, we focus on machine learning (ML) approaches for complex optimization problems in wireless networks. ML approaches can provide flexibility and robustness against uncertain and dynamic systems. However, practical challenges still remain due to slow convergence in offline training or online learning. This motivated us to design a novel optimization-driven ML framework that exploits the efficiency of model-based optimization and the robustness of model-free ML approaches. Splitting the control variables into two parts allows one part to be updated by the outer loop ML approach while the other part is solved by the inner loop optimization. The case study in IRS-assisted wireless networks confirms that the optimization-driven ML framework can improve learning efficiency and the reward performance significantly compared to conventional model-free ML approaches.

Background: Insulin resistance is a common etiology of metabolic syndrome, but receiver operating characteristic (ROC) curve analysis shows a weak association in Koreans. Using a machine learning (ML) approach, we aimed to generate the best model for predicting insulin resistance in Korean adults aged > 40 of the Ansan/Ansung cohort using a machine learning (ML) approach. Methods: The demographic, anthropometric, biochemical, genetic, nutrient, and lifestyle variables of 8842 participants were included. The polygenetic risk scores (PRS) generated by a genome-wide association study were added to represent the genetic impact of insulin resistance. They were divided randomly into the training (n = 7037) and test (n = 1769) sets. Potentially important features were selected in the highest area under the curve (AUC) of the ROC curve from 99 features using seven different ML algorithms. The AUC target was ≥0.85 for the best prediction of insulin resistance with the lowest number of features. Results: The cutoff of insulin resistance defined with HOMA-IR was 2.31 using logistic regression before conducting ML. XGBoost and logistic regression algorithms generated the highest AUC (0.86) of the prediction models using 99 features, while the random forest algorithm generated a model with 0.82 AUC. These models showed high accuracy and k-fold values (>0.85). The prediction model containing 15 features had the highest AUC of the ROC curve in XGBoost and random forest algorithms. PRS was one of 15 features. The final prediction models for insulin resistance were generated with the same nine features in the XGBoost (AUC = 0.86), random forest (AUC = 0.84), and artificial neural network (AUC = 0.86) algorithms. The model included the fasting serum glucose, ALT, total bilirubin, HDL concentrations, waist circumference, body fat, pulse, season to enroll in the study, and gender. Conclusion: The liver function, regular pulse checking, and seasonal variation in addition to metabolic syndrome components should be considered to predict insulin resistance in Koreans aged over 40 years.

Intrusion detection is a major challenge for security experts in the cyber world. Traditional IDS failed to detect complex and unknown cyber-attacks. Machine learning has become a vibrant technology for cybersecurity. There exists several machine learning algorithms to detect intrusion. Most classifiers are well suited to detect the attacks. However, improving accuracy and detecting unknown attacks in existing IDSs is a great challenge. Therefore, the detailed comparative study of various machine learning approaches such as artificial neural networks, support vector machine, decision tree, and hybrid classifiers used by researchers for intrusion detection are done. Deep learning is an emerging approach which suits well for large data. Deep learning techniques find optimal feature set and classify low-frequency attacks better than other techniques. This study also summarizes literatures in deep learning approaches such as deep auto-encoder, Boltzmann machine, recurrent neural networks, convolutional neural networks, and deep neural networks. Moreover, the datasets used in various literatures and the analysis of deep learning approaches based on the performance metrics are also done. Future directions to detect intrusion are also provided. This study in fact will be helpful to develop IDS based on artificial intelligence approaches such as machine learning and deep learning.KeywordsMachine learningIntrusion detectionFeature selectionDeep learningCyber securityClassifier

Satellite low-Earth-orbiting (LEO) and geostationary (GEO) imager estimates of cloud-top pressure (CTP) have many applications in both operations and in studying long-term variations in cloud properties. Recently, machine learning (ML) approaches have shown improvement upon physically based algorithms. However, ML approaches, and especially neural networks, can suffer from a lack of interpretability, making it difficult to understand what information is most useful for accurate predictions of cloud properties. We trained several neural networks to estimate CTP from the infrared channels of the Visible Infrared Imaging Radiometer Suite (VIIRS) and the Advanced Baseline Imager (ABI). The main focus of this work is assessing the relative importance of each instrument’s infrared channels in neural networks trained to estimate CTP. We use several ML explainability methods to offer different perspectives on feature importance. These methods show many differences in the relative feature importance depending on the exact method used, but most agree on a few points. Overall, the 8.4- and 8.6-μm channels appear to be the most useful for CTP estimation on ABI and VIIRS, respectively, with other native infrared window channels and the 13.3-μm channel playing a moderate role. Furthermore, we find that the neural networks learn relationships that may account for properties of clouds such as opacity and cloud-top phase that otherwise complicate the estimation of CTP. Significance Statement Model interpretability is an important consideration for transitioning machine learning models to operations. This work applies several explainability methods in an attempt to understand what information is most important for estimating the pressure level at the top of a cloud from satellite imagers in a neural network model. We observe much disagreement between approaches, which motivates further work in this area but find agreement on the importance of channels in the infrared window region around 8.6 and 10–12 μm, informing future cloud property algorithm development. We also find some evidence suggesting that these neural networks are able to learn physically relevant variability in radiation measurements related to key cloud properties.

Commentary: Machine learning and cardiac surgery risk prediction

SUMMARYIn many seismological, environmental and engineering applications a detailed S-wave velocity model of the shallow subsurface is required. This is generally achieved by the inversion of surface wave dispersion curves using various inversion methods. The classical inversion approaches suffer from several shortcomings, such as inaccurate solutions due to local minima or large computation times in case of a wide parameter space. A number of machine learning (ML) approaches have been suggested to tackle these problems, which however do not provide probabilistic solutions and/or constrain layer number and layer thickness to a fixed value. In this study, we develop a novel neural network (NN) approach in order to characterize the shallow velocity structure from Love and Rayleigh wave dispersion curves. The novelty of our method lies in the simultaneous estimation of layer numbers, layer depth and a complete probability distribution of the S-wave velocity structure. This is achieved by a two-step ML approach, where (1) a regular NN classifies the number of layers within the upper 100 m of the subsurface and (2) a mixture density network outputs the depth estimates together with a fully probabilistic solution of the S-wave velocity structure. We show the advantages of our ML approach compared to a conventional neighbourhood inversion and a Markov chain Monte Carlo algorithm. Our ML approach is then applied to dispersion curves extracted from recorded noise data in Munich, Germany. The resulting velocity profile is in accordance with lithologic information at the site, which highlights the potential of our approach.

ABSTRACTIntroduction:This study aimed to evaluate the usefulness of machine learning (ML) and neural network (NN) models versus traditional statistical methods for estimating biochemical recurrence (BCR) in men following robot-assisted radical prostatectomy (RARP).Methods:Patients who underwent RARP from November 2011 to July 2022 were taken in the study. Patients with BCR were assigned to Group 2, whereas those without BCR were placed in Group 1. Preoperative and postoperative parameters, together with demographic data, were recorded in the database. This study used one NN, the radial basis function NN (RBFNN), and two ML approaches, the K-nearest neighbor and XGboost ML models, to predict BCR.Results:Following the application of exclusion criteria, 516 patients were deemed eligible for the study. Of those, 234 (45.3%) developed BCR, and 282 (54.7%) did not. The results showed that the median follow-up period was 24 (15–42) months, and the median BCR diagnosis was 12.23 ± 15.58 months. The area under the curve (AUC) for the Cox proportional hazard analysis was 0.77. The receiver-operating characteristic curves (AUCs) for the XGBoost and K closest neighbor models were 0.82 and 0.69, respectively. The RBFNN’s AUC was 0.82.Conclusions:The classical statistical model was outperformed by XGBoost and RBFNN models in predicting BCR.

BackgroundMalaria is still a major global health burden, with more than 3.2 billion people in 91 countries remaining at risk of the disease. Accurately distinguishing malaria from other diseases, especially uncomplicated malaria (UM) from non-malarial infections (nMI), remains a challenge. Furthermore, the success of rapid diagnostic tests (RDTs) is threatened by Pfhrp2/3 deletions and decreased sensitivity at low parasitaemia. Analysis of haematological indices can be used to support the identification of possible malaria cases for further diagnosis, especially in travellers returning from endemic areas. As a new application for precision medicine, we aimed to evaluate machine learning (ML) approaches that can accurately classify nMI, UM, and severe malaria (SM) using haematological parameters.MethodsWe obtained haematological data from 2,207 participants collected in Ghana: nMI (n = 978), SM (n = 526), and UM (n = 703). Six different ML approaches were tested, to select the best approach. An artificial neural network (ANN) with three hidden layers was used for multi-classification of UM, SM, and uMI. Binary classifiers were developed to further identify the parameters that can distinguish UM or SM from nMI. Local interpretable model-agnostic explanations (LIME) were used to explain the binary classifiers.ResultsThe multi-classification model had greater than 85% training and testing accuracy to distinguish clinical malaria from nMI. To distinguish UM from nMI, our approach identified platelet counts, red blood cell (RBC) counts, lymphocyte counts, and percentages as the top classifiers of UM with 0.801 test accuracy (AUC = 0.866 and F1 score = 0.747). To distinguish SM from nMI, the classifier had a test accuracy of 0.96 (AUC = 0.983 and F1 score = 0.944) with mean platelet volume and mean cell volume being the unique classifiers of SM. Random forest was used to confirm the classifications, and it showed that platelet and RBC counts were the major classifiers of UM, regardless of possible confounders such as patient age and sampling location.ConclusionThe study provides proof of concept methods that classify UM and SM from nMI, showing that the ML approach is a feasible tool for clinical decision support. In the future, ML approaches could be incorporated into clinical decision-support algorithms for the diagnosis of acute febrile illness and monitoring response to acute SM treatment particularly in endemic settings.