A Survey of Topological Machine Learning Methods

COVID-19 is highly infectious and causes acute respiratory disease. Machine learning (ML) and deep learning (DL) models are vital in detecting disease from computerized chest tomography (CT) scans. The DL models outperformed the ML models. For COVID-19 detection from CT scan images, DL models are used as end-to-end models. Thus, the performance of the model is evaluated for the quality of the extracted feature and classification accuracy. There are four contributions included in this work. First, this research is motivated by studying the quality of the extracted feature from the DL by feeding these extracted to an ML model. In other words, we proposed comparing the end-to-end DL model performance against the approach of using DL for feature extraction and ML for the classification of COVID-19 CT scan images. Second, we proposed studying the effect of fusing extracted features from image descriptors, e.g., Scale-Invariant Feature Transform (SIFT), with extracted features from DL models. Third, we proposed a new Convolutional Neural Network (CNN) to be trained from scratch and then compared to the deep transfer learning on the same classification problem. Finally, we studied the performance gap between classic ML models against ensemble learning models. The proposed framework is evaluated using a CT dataset, where the obtained results are evaluated using five different metrics The obtained results revealed that using the proposed CNN model is better than using the well-known DL model for the purpose of feature extraction. Moreover, using a DL model for feature extraction and an ML model for the classification task achieved better results in comparison to using an end-to-end DL model for detecting COVID-19 CT scan images. Of note, the accuracy rate of the former method improved by using ensemble learning models instead of the classic ML models. The proposed method achieved the best accuracy rate of 99.39%.

During the past decade, it has been quite successful to observe how computational topology has incorporated some of the fundamental concepts and ideas from algebraic and differential topology into applications. This fusion of theories gave birth to a new field named Topological Data Analysis (TDA), which has a significant value in various fields, ranging from computational biology to personalized medicine and dynamic data analysis. Going beyond its foundational applications, TDA has enriched and complemented classical machine and deep learning frameworks in establishing what is now known as “topological machine learning.” In this paper, we review the present landscape of this emerging field, emphasizing how it merges with machine learning algorithms such as deep neural networks. Each method confers special advantages, targeting areas like machine learning integration, network reconstruction, classification of network regimes, or reduction of noisy data. We described common methodologies, discussed current implementations, and anticipate future challenges in topological machine learning.

Machine learning has achieved remarkable success across diverse domains. Nevertheless, concerns about interpretability in black-box models, especially within Deep Neural Networks (DNNs), have become pronounced in safety-critical fields like healthcare and finance. Classical machine learning (ML) classifiers, known for their higher interpretability, are preferred in these domains. Similar to DNNs, classical ML classifiers can exhibit bugs that could lead to severe consequences in practice. Test input prioritization has emerged as a promising approach to ensure the quality of an ML system, which prioritizes potentially misclassified tests so that such tests can be identified earlier with limited manual labeling costs. However, when applying to classical ML classifiers, existing DNN test prioritization methods are constrained from three perspectives: 1) Coverage-based methods are inefficient and time-consuming; 2) Mutation-based methods cannot be adapted to classical ML models due to mismatched model mutation rules; 3) Confidence-based methods are restricted to a single dimension when applying to binary ML classifiers, solely depending on the model’s prediction probability for one class. To overcome the challenges, we propose MLPrior, a test prioritization approach specifically tailored for classical ML models. MLPrior leverages the characteristics of classical ML classifiers (i.e., interpretable models and carefully engineered attribute features) to prioritize test inputs. The foundational principles are: 1) tests more sensitive to mutations are more likely to be misclassified, and 2) tests closer to the model’s decision boundary are more likely to be misclassified. Building on the first concept, we design mutation rules to generate two types of mutation features (i.e., <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">model mutation features</b> and <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">input mutation features</b> ) for each test. Drawing from the second notion, MLPrior generates <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">attribute features</b> of each test based on its attribute values, which can indirectly reveal the proximity between the test and the decision boundary. For each test, MLPrior combines all three types of features of it into a final vector. Subsequently, MLPrior employs a pre-trained ranking model to predict the misclassification probability of each test based on its final vector and ranks tests accordingly. We conducted an extensive study to evaluate MLPrior based on 185 subjects, encompassing natural datasets, mixed noisy datasets, and fairness datasets. The results demonstrate that MLPrior outperforms all the compared test prioritization approaches, with an average improvement of 14.74%∼66.93% on natural datasets, 18.55%∼67.73% on mixed noisy datasets, and 15.34%∼62.72% on fairness datasets.

Summary A meticulous interpretation of steady-state or unsteady-state relative permeability (Kr) experimental data is required to determine a complete set of Kr curves. In this work, different machine learning (ML) models were developed to assist in a faster estimation of these curves from steady-state drainage coreflooding experimental runs. These ML algorithms include gradient boosting (GB), random forest (RF), extreme gradient boosting (XGB), and deep neural network (DNN) with a main focus on and comparison of the two latter algorithms (XGB and DNN). Based on existing mathematical models, a leading-edge framework was developed where a large database of Kr and capillary pressure (Pc) curves were generated. This database was used to perform thousands of coreflood simulation runs representing oil-water drainage steady-state experiments. The results obtained from these simulation runs, mainly pressure drop along with other conventional core analysis data, were used to estimate analytical Kr curves based on Darcy’s law. These analytically estimated Kr curves along with the previously generated Pc curves were fed as features into the ML model. The entire data set was split into 80% for training and 20% for testing. The k-fold cross-validation technique was applied to increase the model’s accuracy by splitting 80% of the training data into 10 folds. In this manner, for each of the 10 experiments, nine folds were used for training and the remaining fold was used for model validation. Once the model was trained and validated, it was subjected to blind testing on the remaining 20% of the data set. The ML model learns to capture fluid flow behavior inside the core from the training data set. In terms of applicability of these ML models, two sets of experimental data were needed as input; the first was the analytically estimated Kr curves from the steady-state drainage coreflooding experiments, while the other was the Pc curves estimated from centrifuge or mercury injection capillary pressure (MICP) measurements. The trained/tested model was then able to estimate Kr curves based on the experimental results fed as input. Furthermore, to test the performance of the ML model when only one set of experimental data is available to an end user, a recurrent neural network (RNN) algorithm was trained/tested to predict Kr curves in the absence of Pc curves as an input. The performance of the three developed models (XGB, DNN, and RNN) was assessed using the values of the coefficient of determination (R2) along with the loss calculated during training/validation of the model. The respective crossplots along with comparisons of ground truth vs. artificial intelligence (AI)-predicted curves indicated that the model is capable of making accurate predictions with an error percentage between 0.2% and 0.6% on history-matching experimental data for all three tested ML techniques. This implies that the AI-based model exhibits better efficiency and reliability in determining Kr curves when compared to conventional methods. The developed ML models by no means replace the need to conduct drainage coreflooding or centrifuge experiments but act as an alternative to existing commercial platforms that are used to interpret experimental data to predict Kr curves. The two main advantages of the developed ML models are their capability of predicting Kr curves within a matter of a few minutes as well as with limited intervention from the end user. The results also include a comparison between classical ML approaches, shallow neural networks, and DNNs in terms of accuracy in predicting the final Kr curves. The research presented here is an extension of the state-of-the-art framework proposed by Mathew et al. (2021). However, the two main aspects of the current study are the application of deep learning for the prediction of Kr curves and the application of feature engineering. The latter not only reduces the training/testing time for the ML models but also enables the end user to obtain the final predictions with the least set of experimental data. The various models discussed in this research work currently focus on the prediction of Kr curves for drainage steady-state experiments; however, the work can be extended to capture the imbibition cycle as well.

Despite the promise for performance and accuracy improvements of quantum inspired (QI) algorithms over classical machine learning (ML) algorithms, such gains have not been realized in practice. The quantum inspired algorithms can theoretically achieve significant speed up based on sampling assumptions and have thus far failed to outperform the existing classical ML models in practical applications. The speedup of quantum machine learning (QML) algorithms assume the access to data in quantum random access memory (QRAM) which is a strong assumption with current quantum architectures. QI algorithms assume sample and query (SQ) access to input vector and norms of matrices using a dynamic data structure. We explore the components of these models and the assumptions in this paper by surveying the recent works in QML and QI Machine learning (QIML) algorithms. We limit our study to QML and QIML models on achieving a speed up over classical ML techniques rather than individual proofs of these algorithms. This study highlights the assumptions being made that are currently not practical for QML and QIML algorithms in achieving performance advantage over classical ML algorithms.

BackgroundAttention-deficit/hyperactivity disorder (ADHD) is currently diagnosed by a diagnostic interview, mainly based on subjective reports from parents or teachers. It is necessary to develop methods that rely on objectively measureable neurobiological data to assess brain-behavior relationship in patients with ADHD. We investigated the application of a topological data analysis tool, Mapper, to analyze the brain functional connectivity data from ADHD patients.MethodsTo quantify the disease severity using the neuroimaging data, the decomposition of individual functional networks into normal and disease components by the healthy state model (HSM) was performed, and the magnitude of the disease component (MDC) was computed. Topological data analysis using Mapper was performed to distinguish children with ADHD (n = 196) from typically developing controls (TDC) (n = 214).ResultsIn the topological data analysis, the partial clustering results of patients with ADHD and normal subjects were shown in a chain-like graph. In the correlation analysis, the MDC showed a significant increase with lower intelligence scores in TDC. We also found that the rates of comorbidity in ADHD significantly increased when the deviation of the functional connectivity from HSM was large. In addition, a significant correlation between ADHD symptom severity and MDC was found in part of the dataset.ConclusionsThe application of HSM and topological data analysis methods in assessing the brain functional connectivity seem to be promising tools to quantify ADHD symptom severity and to reveal the hidden relationship between clinical phenotypic variables and brain connectivity.

Improving autocoding performance of rare categories in injury classification: Is more training data or filtering the solution?

Management of software development demands including bug or defect fixes and new feature or change requests is a crucial part of software maintenance. Failure to prioritize demands correctly might result in inefficient planning and use of resources as well as user or customer dissatisfaction. In order to overcome the difficulty and inefficiency of manual processing, many automated prioritization approaches were proposed in the literature. However, existing body of research generally focused on bug report repositories of open-source software, where textual bug descriptions are in English. Additionally, they proposed solutions to the problem using mostly classical text mining methods and machine learning (ML) algorithms. In this study, we first introduce a demand prioritization dataset in Turkish, which is composed of manually labeled demand records taken from the demand management system of a private insurance company in Turkey. Second, we propose several deep learning (DL) architectures to improve software development demand prioritization. Through an extensive experimentation, we compared the effectiveness of our DL architectures trained with several combinations of different optimizers and activation functions in order to reveal the best combination for demand prioritization in Turkish. We empirically show that DL models can achieve much higher accuracy than classical ML models even with a small amount of training data.

Stream temperature (Ts) is an important water quality parameter that affects ecosystem health and human water use for beneficial purposes. Accurate Ts predictions at different spatial and temporal scales can inform water management decisions that account for the effects of changing climate and extreme events. In particular, widespread predictions of Ts in unmonitored stream reaches can enable decision makers to be responsive to changes caused by unforeseen disturbances. In this study, we demonstrate the use of classical machine learning (ML) models, support vector regression and gradient boosted trees (XGBoost), for monthly Ts predictions in 78 pristine and human-impacted catchments of the Mid-Atlantic and Pacific Northwest hydrologic regions spanning different geologies, climate, and land use. The ML models were trained using long-term monitoring data from 1980–2020 for three scenarios: (1) temporal predictions at a single site, (2) temporal predictions for multiple sites within a region, and (3) spatiotemporal predictions in unmonitored basins (PUB). In the first two scenarios, the ML models predicted Ts with median root mean squared errors (RMSE) of 0.69–0.84 °C and 0.92–1.02 °C across different model types for the temporal predictions at single and multiple sites respectively. For the PUB scenario, we used a bootstrap aggregation approach using models trained with different subsets of data, for which an ensemble XGBoost implementation outperformed all other modeling configurations (median RMSE 0.62 °C).The ML models improved median monthly Ts estimates compared to baseline statistical multi-linear regression models by 15–48% depending on the site and scenario. Air temperature was found to be the primary driver of monthly Ts for all sites, with secondary influence of month of the year (seasonality) and solar radiation, while discharge was a significant predictor at only 10 sites. The predictive performance of the ML models was robust to configuration changes in model setup and inputs, but was influenced by the distance to the nearest dam with RMSE &lt;1 °C at sites situated greater than 16 and 44 km from a dam for the temporal single site and regional scenarios, and over 1.4 km from a dam for the PUB scenario. Our results show that classical ML models with solely meteorological inputs can be used for spatial and temporal predictions of monthly Ts in pristine and managed basins with reasonable (&lt;1 °C) accuracy for most locations.

Introduction: Arthritis is one of the most common chronic diseases. Early detection of arthritis and its progression can facilitate early intervention measures, lowering disease severity in patients. As electronic health records (EHR) become more accessible, this study assesses whether general health information and arthritis-related questionnaires can be used in arthritis diagnosis, without the involvement of costly imaging methods. Therefore, we created deep learning (DL) and machine learning (ML) models to explore the feasibility of combining EHR and modern computational tools to diagnose arthritis. Methods: A total of 782 arthritis patients and 4014 control patients were identified from the Osteoarthritis Initiative (OAI) – a ten-year-long observational study that included patient EHR in five time points. Six hundred variables were filtered by random forest classifier followed by manual filtering. Data were split properly to training, testing and validation set, and the training set was balanced. Sequential, nonsequential DL models, and five independent DL models for each time points were used. The accuracy, positive prevalence value (PPV), negative prevalence value (NPV), and area under curve (AUC), were assessed and compared with four classical ML models. SHAP (SHapley Additive exPlanations) summary analysis was also conducted. Results: Sequential and non-sequential deep learning models showed accuracies of ~ 0.97, and the four classical machine learning approaches showed accuracies of above 0.9. High positive and negative predicted values (&gt; 0.90) for all of the models suggested the potential clinical applicability of the model, while the SHAP analysis demonstrated its interpretability. Discussion: We tested various models and showed the ability to use machine learning methods for early diagnosis of arthritis with EHR. The models can be used as a screening tool to select susceptible patients for confirmatory tests such as X-ray and MRI. Identification of early disease states could facilitate protective measures that slow disease progression.

Evaluation of multispectral imaging for freeze damage assessment in strawberries using AI-based computer vision technology

Image classification is a highly significant field in machine learning (ML), especially when applied to address longstanding and challenging issues in the biological sciences, such as specie recognition and biodiversity conservation. In this study, we present the development of a hybrid machine learning-based tool suitable for deployment on mobile devices. This tool is aimed at processing and classifying three-dimensional samples of endemic lizard species from the Amazon rainforest. The dataset used in our experiment was collected at the Museu Paraense Emílio Goeldi (MPEG), Belém-PA, Brazil, and comprises three species: (a) Anolis fuscoauratus; (b) Hoplocercus spinosus; and (c) Polychrus marmoratus. We compared the effectiveness of four artificial neural networks (ANN) for feature extraction: (a) MobileNet; (b) MobileNetV2; (c) MobileNetV3-Small; and (d) MobileNetV3-Large. Additionally, we evaluated five classical ML models for classifying the extracted patterns: (a) Support Vector Machine (SVM); (b) GaussianNB (GNB); (c) AdaBoost (ADB); (d) K-Nearest Neighbors (KNN); and (e) Random Forest (RF). The performance metrics of all classifiers were very close, we used the McNemar’s test on each model’s confusion matrix to evaluate and compare their statistical significance. Our best model was a combination of a 2.9 million parameters MobileNetV3-Small as the feature extractor, with a linear kernel-based SVM as the classifier, which achieved accuracy of 0.955, precision of 0.948, recall of 0.948, and f1-score of 0.948. The results indicated that the use of a small deep learning (DL) model, in combination with a classical ML algorithm, emerges as a viable technique for classifying three-dimensional representations of lizard species samples. Such an approach facilitates taxonomic identification work for professionals in the field and provides a tool adaptable for integration into mobile data recording equipment, such as smartphones, and benefiting from more morphological features extracted from three-dimensional samples instead of two-dimensional images.

Asthma is a complex respiratory disease characterised by spatial ventilation heterogeneity (VH) in the lung. Multiple breath washout (MBW) and forced oscillation technique (FOT) testing has been shown to quantify VH at the mouth. Methods to provide fresh insight into the mechanisms driving VH are urgently needed to support treatment development. Computed tomography (CT) with parametric response map (PRM) analysis provides spatial and functional lung data. Integrated data sets of imaging features and VH markers present the problem of extracting meaningful information from high dimensional objects; topological data analysis (TDA) has demonstrated recent success in solving this problem. In this thesis topological methods are applied to extract information about lung anatomy and function from imaging data to study association with VH in innately high dimensional integrated data sets. It was sought to describe the spatial distributions of voxel based metrics, visualise the lungs using TDA methods, and combine PRM with simulated airway tree models to improve computational modelling of FOT. Data was obtained from a clinical trial involving 52 subjects, 41 asthmatic (11 healthy). An algorithm was developed to quantify directional Hounsfield unit change (ΔHU) gradients. TDA was applied to integrated data sets and voxel sets with measures of airway tree depth. FOT modelling was augmented with ΔHU about simulated acinar airways.Inferior-to-superior ΔHU gradient best differentiated VH markers. Visualisations of the lungs illustrated gradient reversal and increased heterogeneity in ΔHU in asthmatics. FOT modelling using ΔHU parameterisation showed increased correlation between clinically measured and simulated FOT small airway resistance (R5-R20). In this thesis additional insight is presented into mechanisms affecting VH markers derived from MBW and FOT. Topological methods to analyse imaging data were developed and implemented, including workflows to visualise the lungs and lung function. Linking simulated airway tree geometry with ΔHU showed improvement in FOT modelling.

This study examines the impact of the COVID‐19 epidemic on students in Bangladesh through text classification using various machine learning (ML) algorithms and deep learning (DL) models. The pandemic led to emergency crisis protocols in the country, including self‐quarantine and the closure of educational and governmental institutions, resulting in significant negative impacts on individuals’ physical and mental health, including anxiety, sadness, and terror. To better understand the psychological effects of the epidemic, the authors collected survey data from 400 students in various divisions of Bangladesh using self‐administered questionnaires through Google Forms. Preprocessing techniques such as tokenization, filtering, and n‐gram modeling were used in the analysis. The study deployed eight different ML algorithms and DL models, including LSTM, BiLSTM, and CNN, to classify the effects on students’ academic, mental, and social lives. The results show that the ML classifier algorithms were highly effective, achieving accuracies of 95.00%, 93.75%, and 95.00% for academic, mental, and social life impact, respectively. Furthermore, hybrid DL models, such as CNN‐LSTM and CNN‐BiLSTM, produced good scores in predicting the impacts on students’ lives. Overall, this study provides valuable insights into the impacts of the COVID‐19 epidemic on students’ academic, mental, and social well‐being in Bangladesh.

BackgroundKnee osteoarthritis (OA) is the most common form of OA and a leading cause of disability worldwide. Chronic pain and functional loss secondary to knee OA put patients at risk of developing depression, which can also impair their treatment response. However, no tools exist to assist clinicians in identifying patients at risk. Machine learning (ML) predictive models may offer a solution. We investigated whether ML models could predict the development of depression in patients with knee OA and examined which features are the most predictive.ObjectiveThe primary aim of this study was to develop and test an ML model to predict depression in patients with knee OA at 2 years and to validate the models using an external data set. The secondary aim was to identify the most important predictive features used by the ML algorithms.MethodsOsteoarthritis Initiative Study (OAI) data were used for model development and external validation was performed using Multicenter Osteoarthritis Study (MOST) data. Forty-two features were selected, which denoted routinely collected demographic and clinical data such as patient demographics, past medical history, knee OA history, baseline examination findings, and patient-reported outcome measures. Six different ML classification models were trained (logistic regression, least absolute shrinkage and selection operator [LASSO], ridge regression, decision tree, random forest, and gradient boosting machine). The primary outcome was to predict depression at 2 years following study enrollment. The presence of depression was defined using the Center for Epidemiological Studies Depression Scale. Model performance was evaluated using the area under the receiver operating characteristic curve (AUC) and F1 score. The most important features were extracted from the best-performing model on external validation.ResultsA total of 5947 patients were included in this study, with 2969 in the training set, 742 in the test set, and 2236 in the external validation set. For the test set, the AUC ranged from 0.673 (95% CI 0.604-0.742) to 0.869 (95% CI 0.824-0.913), with an F1 score of 0.435 to 0.490. On external validation, the AUC varied from 0.720 (95% CI 0.685-0.755) to 0.876 (95% CI 0.853-0.899), with an F1 score of 0.456 to 0.563. LASSO modeling offered the highest predictive performance. Blood pressure, baseline depression score, knee pain and stiffness, and quality of life were the most predictive features.ConclusionsTo our knowledge, this is the first study to apply ML classification models to predict depression in patients with knee OA. Our study showed that ML models can deliver a clinically acceptable level of performance (AUC>0.7) in predicting the development of depression using routinely available demographic and clinical data. Further work is required to address the class imbalance in the training data and to evaluate the clinical utility of the models in facilitating early intervention and improved outcomes.