Multisided Quality Inspection in Smart Manufacturing Systems

Summary A meticulous interpretation of steady-state or unsteady-state relative permeability (Kr) experimental data is required to determine a complete set of Kr curves. In this work, different machine learning (ML) models were developed to assist in a faster estimation of these curves from steady-state drainage coreflooding experimental runs. These ML algorithms include gradient boosting (GB), random forest (RF), extreme gradient boosting (XGB), and deep neural network (DNN) with a main focus on and comparison of the two latter algorithms (XGB and DNN). Based on existing mathematical models, a leading-edge framework was developed where a large database of Kr and capillary pressure (Pc) curves were generated. This database was used to perform thousands of coreflood simulation runs representing oil-water drainage steady-state experiments. The results obtained from these simulation runs, mainly pressure drop along with other conventional core analysis data, were used to estimate analytical Kr curves based on Darcy’s law. These analytically estimated Kr curves along with the previously generated Pc curves were fed as features into the ML model. The entire data set was split into 80% for training and 20% for testing. The k-fold cross-validation technique was applied to increase the model’s accuracy by splitting 80% of the training data into 10 folds. In this manner, for each of the 10 experiments, nine folds were used for training and the remaining fold was used for model validation. Once the model was trained and validated, it was subjected to blind testing on the remaining 20% of the data set. The ML model learns to capture fluid flow behavior inside the core from the training data set. In terms of applicability of these ML models, two sets of experimental data were needed as input; the first was the analytically estimated Kr curves from the steady-state drainage coreflooding experiments, while the other was the Pc curves estimated from centrifuge or mercury injection capillary pressure (MICP) measurements. The trained/tested model was then able to estimate Kr curves based on the experimental results fed as input. Furthermore, to test the performance of the ML model when only one set of experimental data is available to an end user, a recurrent neural network (RNN) algorithm was trained/tested to predict Kr curves in the absence of Pc curves as an input. The performance of the three developed models (XGB, DNN, and RNN) was assessed using the values of the coefficient of determination (R2) along with the loss calculated during training/validation of the model. The respective crossplots along with comparisons of ground truth vs. artificial intelligence (AI)-predicted curves indicated that the model is capable of making accurate predictions with an error percentage between 0.2% and 0.6% on history-matching experimental data for all three tested ML techniques. This implies that the AI-based model exhibits better efficiency and reliability in determining Kr curves when compared to conventional methods. The developed ML models by no means replace the need to conduct drainage coreflooding or centrifuge experiments but act as an alternative to existing commercial platforms that are used to interpret experimental data to predict Kr curves. The two main advantages of the developed ML models are their capability of predicting Kr curves within a matter of a few minutes as well as with limited intervention from the end user. The results also include a comparison between classical ML approaches, shallow neural networks, and DNNs in terms of accuracy in predicting the final Kr curves. The research presented here is an extension of the state-of-the-art framework proposed by Mathew et al. (2021). However, the two main aspects of the current study are the application of deep learning for the prediction of Kr curves and the application of feature engineering. The latter not only reduces the training/testing time for the ML models but also enables the end user to obtain the final predictions with the least set of experimental data. The various models discussed in this research work currently focus on the prediction of Kr curves for drainage steady-state experiments; however, the work can be extended to capture the imbibition cycle as well.

Background: Early identification of probable post-traumatic stress disorder (PTSD) can lead to early intervention and treatment.Aims: This study aimed to evaluate supervised machine learning (ML) classifiers for the identification of probable PTSD in those who are serving, or have recently served in the United Kingdom (UK) Armed Forces.Methods: Supervised ML classification techniques were applied to a military cohort of 13,690 serving and ex-serving UK Armed Forces personnel to identify probable PTSD based on self-reported service exposures and a range of validated self-report measures. Data were collected between 2004 and 2009.Results: The predictive performance of supervised ML classifiers to detect cases of probable PTSD were encouraging when compared to a validated measure, demonstrating a capability of supervised ML to detect the cases of probable PTSD. It was possible to identify which variables contributed to the performance, including alcohol misuse, gender and deployment status. A satisfactory sensitivity was obtained across a range of supervised ML classifiers, but sensitivity was low, indicating a potential for false negative diagnoses.Conclusions: Detection of probable PTSD based on self-reported measurement data is feasible, may greatly reduce the burden on public health and improve operational efficiencies by enabling early intervention, before manifestation of symptoms.

Chapter 6 - Cardiovascular disease prediction using machine learning models and ensemble technique

To develop and internally validate a machine learning (ML) model that identifies older outpatients with MCI using routine electronic health record (EHR) data. We conducted a retrospective cross-sectional study of community outpatients aged ≥60 years in Zhejiang, China. Structured EHR predictors included demographics, comorbidities/medications, lifestyle, and visit patterns. The outcome was adjudicated MCI based on cognitive screening (MoCA plus supplemental tests). Supervised ML classifiers were compared using 10-fold cross-validation and an independent held-out test set; class imbalance was addressed with SMOTE. Performance was assessed by the area under the ROC curve (AUC) and by sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy, and F1 score. The test set included ~640 patients (≈20% MCI). Gradient Boosting performed best: cross-validation mean AUC 0.855 (SD 0.031) and accuracy 0.862 (SD 0.013); test AUC 0.850, accuracy 0.833, and F1 0.402. At the default threshold, sensitivity was 0.286 and specificity 0.967 (PPV 0.679; NPV 0.847). Prioritizing sensitivity (~0.82) lowered specificity (~0.64). At a high-sensitivity threshold of 0.159, the model achieved a sensitivity of 0.802 with a specificity of 0.751 (PPV 0.441; NPV 0.939). Important predictors included older age, female sex, lower education, smaller family size, and higher depression scores. An ML model using routine outpatient EHR can discriminate MCI in older adults (AUC ≈ 0.85), supporting potential for automated, low-cost screening in primary care. Using the predicted probabilities generated in this analysis, we assessed calibration and conducted a decision-curve analysis. While the model shows good discrimination and calibration, external validation is still required to confirm clinical utility and refine operating thresholds.

Classification and prediction tasks are common in health research. With the increasing availability of vast health data repositories (e.g. electronic medical record databases) and advances in computing power, traditional statistical approaches are being augmented or replaced with machine learning (ML) approaches to classify and predict health outcomes. ML describes the automated process of identifying ("learning") patterns in data to perform tasks. Developing an ML model includes selecting between many ML models (e.g. decision trees, support vector machines, neural networks); model specifications such as hyperparameter tuning; and evaluation of model performance. This process is conducted repeatedly to find the model and corresponding specifications that optimize some measure of model performance. ML models can make more accurate classifications and predictions than their statistical counterparts and confer greater flexibility when modelling unstructured data or interactions between covariates; however, many ML models require larger sample sizes to achieve good classification or predictive performance and have been criticized as "black box" for their poor transparency and interpretability. ML holds potential in family medicine for risk profiling of patients' disease risk and clinical decision support to present additional information at times of uncertainty or high demand. In the future, ML approaches are positioned to become commonplace in family medicine. As such, it is important to understand the objectives that can be addressed using ML approaches and the associated techniques and limitations. This article provides a brief introduction into the use of ML approaches for classification and prediction tasks in family medicine.

Modern manufacturing systems are complex and heterogeneous distributed information systems that consist of manufacturing hardware and networking software. Unlike traditional manufacturing environments, modern ones generate large amounts of data in real-time that take the form of sensor measurements from the shopfloor, images from the product inspection, technical text from maintenance records, and many more. The issue of collecting, processing, and analyzing this data is at the core of the Industry 4.0 paradigm's success. What is more important, challenging, and rewarding is that such valuable real-time streaming data can be used for diagnostics, prognostics, and health monitoring of systems and products. This dissertation explores the area of health monitoring of modern manufacturing environments to support the state-of-the-art Industry 4.0 paradigms. We deploy Machine Learning (ML), Deep Learning (DL), and Natural Language Processing (NLP) based approaches to assess the performance of systems and products by identifying anomalies and defects, estimating the Remaining Useful Life (RUL), and determining what sort of corrective maintenance actions may be necessary. In the context of the foregoing, advanced manufacturing environments are very intricate and have a vast array of complex problems that can be researched. This dissertation focuses on four important questions: (1) what are the requirements to achieve interoperability in smart manufacturing? (2) how can the next hardware failure, be it of a machine tool or a cutting tool, be predicted before it occurs? (3) how can we identify the products that fail inspection, and how can we improve the overall quality inspection process by considering the factors affecting it? and (4) how can we leverage the resource-rich maintenance logs from manufacturing systems to find the most frequently occurring problems, and more importantly, how can we predict corrective actions for future maintenance problems? This dissertation addresses the above problems by making the following research contributions. In the first work, we discuss interoperability in manufacturing by reviewing semantic, syntactic, factory, and cloud-based interoperability. We then identify key research challenges that impact the implementation of interoperable manufacturing, and review frameworks for modern manufacturing such as Reference Architecture Model for Industry 4.0 (RAMI 4.0), Industrial Internet Reference Architecture (IIRA), and several others. The second part of the research proposes a framework for Smart Prognostics and Health Management (SPHM) to support manufacturing system health. We apply it to a milling machine operation and assess the condition of the milling machine's tool tip by identifying whether it is safe or degraded and estimate its RUL using ML models. To address the third question, we introduce a methodology for Smart Quality Inspection (SQI) of manufacturing products by using a custom Convolutional Neural Network (CNN) architecture. A shopfloor computer application is developed that helps minimize the effects of the factors affecting the visual inspection process. The fourth component of this dissertation introduces a Technical Language Processing (TLP) framework to deal with the unstructured and complex nature of technical text data from maintenance work orders. We demonstrate its capabilities by applying it to maintenance logs of aircraft to predict corrective actions, and link maintenance problems to standardized maintenance codes as prescribed by the industry. The rapid growth of Artificial Intelligence (AI) suggests that its potential to transform health monitoring capabilities is going to improve in the coming years. Future research in health monitoring of manufacturing systems and products can be aimed towards the following areas: development of standardized communication protocols for interoperability, further automation of the inspection process by integrating localized fault detection, and incorporation of TLP with predictive maintenance approaches and quality control to form a prescriptive approach, to suggest a few.--Author's abstract

Over the years, drone usage have become an increasing part of the ever-connected society that we are currently living in. Its usages have proliferated beyond the military sector to various commercial and consumer activities such as package delivery, disaster relief, agriculture and filming. With the rise of Wi-Fi controlled drone. Wi-Fi controlled drone has increased its popularity for personal use due to its affordability, and the ease of operating the drone through smart-devices like mobile phone, tablets and computers. As such, this increases the likelihood of drone presence in various environments, especially in critical government infrastructure, leading to various privacy and security concern by the authorities and the public with malicious intent. Therefore, various signature-based methodology of drone detection has emerged such as the visual and Radio Frequency (RF) signature-based detection method. Visual signature-based detection relies on camera capture and image processing but this is an expensive approach. Whereas, RF signature- based detection relies on the identification of the emission of RF signal by the drone. However, since most commercial electronics devices were built based on Wi-Fi technology, the differentiation of the RF signals transmitted between a drone or a standard Wi-Fi device in a crowded Wi-Fi environment such as a school campus or city area is an challenging task. In this paper, we propose a novel machine learning approach that leverages on the identified unique signatures of Wi-Fi devices in terms of Radio Frequency (RF) and network packets measurement to differentiate the presence of Wi-Fi drone and standard Wi-Fi devices in an urban setting. Furthermore, we also carried out a meticulous pre-processing procedure and a better training scheme of using Stratified K-Fold Cross-Validation (SKFCV), to enhance the richness in the data signature and fully exploit the permutation of the data during training respectively for better performance of the ML models. Two supervised classification Machine Learning (ML) models, namely the Logistic Regression (LR), and Artificial Neural Network (ANN) were applied using the joint data measurements to identify the presence of drone in dense Wi-Fi environment. The experimental results have shown that the proposed novel ML approach of using both RF and network measurement signatures coupled with the pre-processing and training methodology on LR and ANN ML models have outperformed the traditional RF signature-based drone detection ML accuracy results by 15.1% and 21.63% respectively in a crowded Wi-Fi environment.

Freshwater ecosystems play a crucial role in maintaining ecological balance and supporting biodiversity. Assessing water quality and identifying pollution factors affected by seasonal changes is of utmost importance. The traditional approach for identifying pollution sources and seasonal water quality assessment is often labor-intensive, time-consuming, and complex. In response, our study introduces a hybrid receptor, multivariate statistical and machine learning (ML) based framework as an alternative. Positive matrix factorization (PMF) and receptor model were used to identify key pollutant sources, while Principal Component Analysis (PCA) was used to corroborate pollutant sources. Additionally, various supervised ML classification algorithms are used to predict and identify pollution sources on full physicochemical datasets and reduced in situ parameter scenarios. ML model stability and generalization capacity were evaluated using learning curves, cross-validation method. The findings indicate significant seasonal variability in water quality index, ranging from 38.5 (poor) to 78.6 (good). PMF and PCA revealed three dominant pollution factors, such as biogeochemical processes, sediment resuspension/nutrient influx, and anthropogenic pollution, affecting the seasonal water quality by 15-30%, 25-45%, and 40-70%, respectively. Model performance indicators using confusion matrices, ROC-AUC, and other evaluation metrics, for full dataset identified XGBoost as the best model, achieving the highest accuracy of 96.3%, followed by CatBoosting (93.6%), Random Forest (93.6%), Support Vector Classifier (91.8%), K-Nearest Neighbors (87.2%) and Decision Tree (84.3%). However, with the in situ data the ML models show stable and consistent classification accuracy in the range of approximately 70-81%. Overall, the modelling framework provides an effective approach for capturing seasonal variability and pollution source attribution, contributing to improved ecological understanding and predictive management of freshwater system.

Artificial intelligence (AI) is a tool thought to revolutionize the field of reproductive medicine in the years to come. Specifically, machine learning (ML), which is a subset of AI methods used to detect patterns and make predictions based on large datasets, has been used in ART to predict implantation outcome, embryo transfer strategies or adverse outcomes for IVF treatments (for a review see, Wang et al., 2019). However, ML methods have not yet been widely adopted in medically assisted reproduction (MAR). This slow uptake could be due to a lack of in-depth interdisciplinary communication between ML experts and clinicians/embryologists, as well as some uncertainty on how ML models could be generalized for different populations. The February edition of the ESHRE Journal Club discussed a paper from Yland et al. (2022) where the authors compared different ML algorithms to predict the chance of pregnancy in couples actively trying to conceive (without undergoing MAR). By using data from the Pregnancy Study Online (PRESTO), a web-based prospective cohort study of 4133 couples in North America (Wise et al., 2015), the authors analysed 163 potential variables and used supervised ML classification algorithms to identify the strength of each variable in predicting pregnancy outcomes of couples across the fertility spectrum (infertile: <12 menstrual cycles; subfertile: within 6 menstrual cycles; and fecundability: the average probability of pregnancy per menstrual cycle). The authors were able to predict pregnancy with a discrimination as high as 71.2% and identify the variables that most consistently predicted conception. These variables included lifestyle and reproductive characteristics such as age, BMI, history of infertility, daily use of vitamins or folic acid, intercourse timing, diet quality and reduced stress. ESHRE Journal Club discussion focused on promoting interdisciplinarity among ML and reproductive medicine specialists. It hosted 50 participants on Twitter together with experts Michelle Perugini, Vajira Thambawita and authors Jennifer Yland, Yannis Paschalidis and Lauren Wise. The discussion resulted in 902 tweets and around 1 million impressions over a 24-h period.

Machine learning has achieved remarkable success across diverse domains. Nevertheless, concerns about interpretability in black-box models, especially within Deep Neural Networks (DNNs), have become pronounced in safety-critical fields like healthcare and finance. Classical machine learning (ML) classifiers, known for their higher interpretability, are preferred in these domains. Similar to DNNs, classical ML classifiers can exhibit bugs that could lead to severe consequences in practice. Test input prioritization has emerged as a promising approach to ensure the quality of an ML system, which prioritizes potentially misclassified tests so that such tests can be identified earlier with limited manual labeling costs. However, when applying to classical ML classifiers, existing DNN test prioritization methods are constrained from three perspectives: 1) Coverage-based methods are inefficient and time-consuming; 2) Mutation-based methods cannot be adapted to classical ML models due to mismatched model mutation rules; 3) Confidence-based methods are restricted to a single dimension when applying to binary ML classifiers, solely depending on the model’s prediction probability for one class. To overcome the challenges, we propose MLPrior, a test prioritization approach specifically tailored for classical ML models. MLPrior leverages the characteristics of classical ML classifiers (i.e., interpretable models and carefully engineered attribute features) to prioritize test inputs. The foundational principles are: 1) tests more sensitive to mutations are more likely to be misclassified, and 2) tests closer to the model’s decision boundary are more likely to be misclassified. Building on the first concept, we design mutation rules to generate two types of mutation features (i.e., <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">model mutation features</b> and <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">input mutation features</b> ) for each test. Drawing from the second notion, MLPrior generates <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">attribute features</b> of each test based on its attribute values, which can indirectly reveal the proximity between the test and the decision boundary. For each test, MLPrior combines all three types of features of it into a final vector. Subsequently, MLPrior employs a pre-trained ranking model to predict the misclassification probability of each test based on its final vector and ranks tests accordingly. We conducted an extensive study to evaluate MLPrior based on 185 subjects, encompassing natural datasets, mixed noisy datasets, and fairness datasets. The results demonstrate that MLPrior outperforms all the compared test prioritization approaches, with an average improvement of 14.74%∼66.93% on natural datasets, 18.55%∼67.73% on mixed noisy datasets, and 15.34%∼62.72% on fairness datasets.

COVID-19 is highly infectious and causes acute respiratory disease. Machine learning (ML) and deep learning (DL) models are vital in detecting disease from computerized chest tomography (CT) scans. The DL models outperformed the ML models. For COVID-19 detection from CT scan images, DL models are used as end-to-end models. Thus, the performance of the model is evaluated for the quality of the extracted feature and classification accuracy. There are four contributions included in this work. First, this research is motivated by studying the quality of the extracted feature from the DL by feeding these extracted to an ML model. In other words, we proposed comparing the end-to-end DL model performance against the approach of using DL for feature extraction and ML for the classification of COVID-19 CT scan images. Second, we proposed studying the effect of fusing extracted features from image descriptors, e.g., Scale-Invariant Feature Transform (SIFT), with extracted features from DL models. Third, we proposed a new Convolutional Neural Network (CNN) to be trained from scratch and then compared to the deep transfer learning on the same classification problem. Finally, we studied the performance gap between classic ML models against ensemble learning models. The proposed framework is evaluated using a CT dataset, where the obtained results are evaluated using five different metrics The obtained results revealed that using the proposed CNN model is better than using the well-known DL model for the purpose of feature extraction. Moreover, using a DL model for feature extraction and an ML model for the classification task achieved better results in comparison to using an end-to-end DL model for detecting COVID-19 CT scan images. Of note, the accuracy rate of the former method improved by using ensemble learning models instead of the classic ML models. The proposed method achieved the best accuracy rate of 99.39%.

Rapid growth of network traffic causes the need for the development of new network technologies. Artificial intelligence provides suitable tools to improve currently used network optimization methods. In this paper, we propose a procedure for network traffic prediction. Based on optical networks’ (and other network technologies) characteristics, we focus on the prediction of fixed bitrate levels called traffic levels. We develop and evaluate two approaches based on different supervised machine learning (ML) methods—classification and regression. We examine four different ML models with various selected features. The tested datasets are based on real traffic patterns provided by the Seattle Internet Exchange Point (SIX). Obtained results are analyzed using a new quality metric, which allows researchers to find the best forecasting algorithm in terms of network resources usage and operational costs. Our research shows that regression provides better results than classification in case of all analyzed datasets. Additionally, the final choice of the most appropriate ML algorithm and model should depend on the network operator expectations.

Improving autocoding performance of rare categories in injury classification: Is more training data or filtering the solution?

Stream temperature (Ts) is an important water quality parameter that affects ecosystem health and human water use for beneficial purposes. Accurate Ts predictions at different spatial and temporal scales can inform water management decisions that account for the effects of changing climate and extreme events. In particular, widespread predictions of Ts in unmonitored stream reaches can enable decision makers to be responsive to changes caused by unforeseen disturbances. In this study, we demonstrate the use of classical machine learning (ML) models, support vector regression and gradient boosted trees (XGBoost), for monthly Ts predictions in 78 pristine and human-impacted catchments of the Mid-Atlantic and Pacific Northwest hydrologic regions spanning different geologies, climate, and land use. The ML models were trained using long-term monitoring data from 1980–2020 for three scenarios: (1) temporal predictions at a single site, (2) temporal predictions for multiple sites within a region, and (3) spatiotemporal predictions in unmonitored basins (PUB). In the first two scenarios, the ML models predicted Ts with median root mean squared errors (RMSE) of 0.69–0.84 °C and 0.92–1.02 °C across different model types for the temporal predictions at single and multiple sites respectively. For the PUB scenario, we used a bootstrap aggregation approach using models trained with different subsets of data, for which an ensemble XGBoost implementation outperformed all other modeling configurations (median RMSE 0.62 °C).The ML models improved median monthly Ts estimates compared to baseline statistical multi-linear regression models by 15–48% depending on the site and scenario. Air temperature was found to be the primary driver of monthly Ts for all sites, with secondary influence of month of the year (seasonality) and solar radiation, while discharge was a significant predictor at only 10 sites. The predictive performance of the ML models was robust to configuration changes in model setup and inputs, but was influenced by the distance to the nearest dam with RMSE &lt;1 °C at sites situated greater than 16 and 44 km from a dam for the temporal single site and regional scenarios, and over 1.4 km from a dam for the PUB scenario. Our results show that classical ML models with solely meteorological inputs can be used for spatial and temporal predictions of monthly Ts in pristine and managed basins with reasonable (&lt;1 °C) accuracy for most locations.

Regulating inflammatory microglia presents a promising strategy for treating neurodegenerative and autoimmune disorders, yet effective therapeutic agents delivery to these cells remains a challenge. This study investigates modified lipid nanoparticles (LNP) for mRNA delivery to hyperactivated microglia, particularly those with pro-inflammatory characteristics, utilizing supervised machine learning (ML) classifiers. We developed and screened a library of 216 LNP formulations with varying lipid compositions, N/P ratios, and hyaluronic acid (HA) modifications. The transfection efficiency of eGFP mRNA was assessed in the BV-2 murine microglia cell line under different immunological states, including resting and activated conditions (LPS-activated and IL4/IL13-activated). ML-guided morphometric analysis tracked the phenotypes of various microglia subtypes before and after transfection. Four supervised ML classifiers were investigated to predict transfection efficiency and phenotypic changes based on LNP design parameters. The Multi-Layer Perceptron (MLP) neural network emerged as the best-performing model, achieving weighted F1-scores ≥0.8. While it accurately predicted responses from LPS-activated and resting cells, it struggled with IL4/IL13-activated cells. The MLP model was validated by predicting the performance of four unseen LNP formulations delivering eGFP mRNA to LPS-activated BV2 cells. HA-LNP2 emerged as optimal formulation for delivering target IL10 mRNA, effectively suppressing inflammatory phenotypes, evidenced by shifts in cell morphology, increased IL10 expression, and reduced TNF-α levels. We also evaluated HA-LNP2 on LPS-activated human iPSC-derived microglia, confirming its efficacy in modulating inflammatory responses. This study highlights the potential of tailored LNP design and ML techniques to enhance mRNA therapy for neuroinflammatory disorders by leveraging carrier’s immunogenic properties to modulate microglial responses.