Putting the Scientist in the Loop -- Accelerating Scientific Progress with Interactive Machine Learning

With the Internet of Things paradigm, the data generated by the rapidly increasing number of connected devices lead to new possibilities, such as using machine learning for activity recognition in smart environments. However, it also introduces several challenges. The sensors of different devices might be of different types, making the fusion of data non-trivial. Moreover, the devices are often mobile, resulting in that data from a particular sensor is not always available, i.e. there is a need to handle data from a dynamic set of sensors. From a machine learning perspective, the data from the sensors arrives in a streaming fashion, i.e., online learning, as compared to many learning problems where a static dataset is assumed. Machine learning is in many cases a good approach for classification problems, but the performance is often linked to the quality of the data. Having a good data set to train a model can be an issue in general, due to the often costly process of annotating the data. With dynamic and heterogeneous data, annotation can be even more problematic, because of the ever-changing environment. This means that there might not be any, or a very small amount of, annotated data to train the model on at the start of learning, often referred to as the cold start problem. To be able to handle these issues, adaptive systems are needed. With adaptive we mean that the model is not static over time, but is updated if there for instance is a change in the environment. By including human-in-the-loop during the learning process, which we refer to as interactive machine learning, the input from users can be utilized to build the model. The type of input used is typically annotations of the data, i.e. user input in the form of correctly labelled data points. Generally, it is assumed that the user always provides correct labels in accordance with the chosen interactive learning strategy. In many real-world applications these assumptions are not realistic however, as users might provide incorrect labels or not provide labels at all in line with the chosen strategy. In this thesis we explore which interactive learning strategies are possible in the given scenario and how they affect performance, as well as the effect of machine learning algorithms on performance. We also study how a user who is not always reliable, i.e. that does not always provide a correct label when expected to, can affect performance. We propose a taxonomy of interactive online machine learning strategies and test how the different strategies affect performance through experiments on multiple datasets. The findings show that the overall best performing interactive learning strategy is one where the user provides labels when previous estimations have been incorrect, but that the best performing machine learning algorithm depends on the problem scenario. The experiments also show that a decreased reliability of the user leads to decreased performance, especially when there is a limited amount of labelled data.

In this paper, we highlight three issues that limit performance of machine learning on biomedical images, and tackle them through 3 case studies: 1) Interactive Machine Learning (IML): we show how IML can drastically improve exploration time and quality of direct volume rendering. 2) transfer learning: we show how transfer learning along with intelligent pre-processing can result in better Alzheimer's diagnosis using a much smaller training set 3) data imbalance: we show how our novel focal Tversky loss function can provide better segmentation results taking into account the imbalanced nature of segmentation datasets. The case studies are accompanied by in-depth analytical discussion of results with possible future directions.

Chapter 15 - Clinical decision support systems to improve the diagnosis and management of respiratory diseases

Material characterization techniques are widely used to characterize the physical and chemical properties of materials at the nanoscale and, thus, play central roles in material scientific discoveries. However, the large and complex datasets generated by these techniques often require significant human effort to interpret and extract meaningful physicochemical insights. Artificial intelligence (AI) techniques such as machine learning (ML) have the potential to improve the efficiency and accuracy of surface analysis by automating data analysis and interpretation. In this perspective paper, we review the current role of AI in surface analysis and discuss its future potential to accelerate discoveries in surface science, materials science, and interface science. We highlight several applications where AI has already been used to analyze surface analysis data, including the identification of crystal structures from XRD data, analysis of XPS spectra for surface composition, and the interpretation of TEM and SEM images for particle morphology and size. We also discuss the challenges and opportunities associated with the integration of AI into surface analysis workflows. These include the need for large and diverse datasets for training ML models, the importance of feature selection and representation, and the potential for ML to enable new insights and discoveries by identifying patterns and relationships in complex datasets. Most importantly, AI analyzed data must not just find the best mathematical description of the data, but it must find the most physical and chemically meaningful results. In addition, the need for reproducibility in scientific research has become increasingly important in recent years. The advancement of AI, including both conventional and the increasing popular deep learning, is showing promise in addressing those challenges by enabling the execution and verification of scientific progress. By training models on large experimental datasets and providing automated analysis and data interpretation, AI can help to ensure that scientific results are reproducible and reliable. Although integration of knowledge and AI models must be considered for the transparency and interpretability of models, the incorporation of AI into the data collection and processing workflow will significantly enhance the efficiency and accuracy of various surface analysis techniques and deepen our understanding at an accelerated pace.

This review explores the use of machine learning (ML) techniques for detecting pests and diseases in crops, which is a significant challenge in agriculture, leading to substantial yield losses worldwide. This study focuses on the integration of ML models, particularly Convolutional Neural Networks (CNNs), which have shown promise in accurately identifying and classifying plant diseases from images. By analyzing studies published from 2019 to 2024, this work summarizes the common methodologies involving stages of data acquisition, preprocessing, segmentation, feature extraction, and prediction to develop robust ML models. The findings indicate that the incorporation of advanced image processing and ML algorithms significantly enhances disease detection capabilities, leading to the early and precise diagnosis of crop ailments. This can not only improve crop yield and quality but also reduce the dependency on chemical pesticides, contributing to more sustainable agricultural practices. Future research should focus on enhancing the robustness of these models to varying environmental conditions and expanding the datasets to include a wider variety of crops and diseases. CNN-based models, particularly specialized architectures like ResNet, are the most widely used in the studies reviewed, making up 42.36% of all models, with ResNet alone contributing 7.65%. This highlights ResNet’s appeal for tasks that demand deep architectures and sophisticated feature extraction. Additionally, SVM models account for 9.41% of the models examined. The prominence of both ResNet and MobileNet reflects a trend toward architectures with residual connections for deeper networks, alongside efficiency-focused designs like MobileNet, which are well-suited for mobile and edge applications.

Applying data science and machine learning to transportation engineering is essential for addressing modern urban mobility challenges. This thesis investigates whether these technologies improve decision-making in mobility systems, aiming to create more efficient, sustainable, and cohesive mobility solutions. By leveraging data science and machine learning, the research introduces innovative tools across various urban mobility applications to support organizations, stakeholders, and policymakers in navigating the complexities of urban mobility. The research follows a comprehensive methodology encompassing domain understanding, data collection, data cleaning, data exploration, feature engineering, predictive modeling, and stakeholder communication. At its core is a compendium of four scientific publications, each providing empirical evidence and insights aligned with the broader research objectives. The specific objectives focus on: (i) identifying effective data sources and collection techniques for urban mobility analysis; (ii) determining the role of feature engineering in enhancing data quality and relevance; (iii) evaluating the suitability and added value of machine learning techniques for urban mobility challenges; and (iv) assessing model performance while defining policy recommendations for mobility stakeholders. These objectives are tested through three practical research applications addressing real-world urban mobility challenges: (i) promoting micromobility for urban shopping; (ii) setting prices in ride-hailing services; and (iii) forecasting moped scooter-sharing demand. The first research application examines factors influencing the adoption of bicycles and scooters (primarily active modes) for urban shopping in cities with low micromobility maturity. Using machine learning models applied to survey data, the study identifies barriers and provides policy recommendations to improve infrastructure and safety for micromobility, emphasizing potential growth post-COVID-19. The second research application explores pricing dynamics in ride-hailing services. One study analyzes fare patterns before and during the COVID-19 pandemic in Boston and Atlanta, uncovering fare dynamics during crises. Another examines determinants of fares in Madrid, considering factors such as weather conditions and peak hours. Together, these studies provide insights into efficient pricing strategies and policy recommendations for better regulation and integration of ride-hailing services. The third research application focuses on forecasting demand for moped scooter-sharing services. Using machine learning techniques, the study predicts travel demand based on comprehensive datasets, including sociodemographic characteristics, transportation network attributes, and policy-related variables. The results demonstrate the effectiveness of the random forest model and underscore the significance of neighborhood characteristics and short-distance trip preferences in urban planning and transportation management. This thesis demonstrates that data science and machine learning significantly enhance decision-making in mobility systems by improving prediction accuracy, optimizing data utilization, and providing actionable recommendations for sustainable urban mobility. Furthermore, it acknowledges persistent challenges such as the need for high-quality data, robust algorithms, and enhanced model interpretability. Addressing these limitations is critical for ensuring successful real-world implementation and maximizing the transformative potential of these technologies. RESUMEN La aplicación de la ciencia de datos y el aprendizaje automático a la ingeniería del transporte es esencial para abordar los desafíos modernos de la movilidad urbana. Esta tesis investiga si estas tecnologías mejoran la toma de decisiones en los sistemas de movilidad, con el objetivo de crear soluciones más eficientes, sostenibles y cohesionadas. Al aprovechar estas herramientas, la investigación introduce innovaciones en diversas aplicaciones de movilidad urbana para apoyar a organizaciones, partes interesadas y responsables políticos en la gestión de la movilidad urbana. La metodología incluye comprensión del dominio, recopilación, limpieza y exploración de datos, ingeniería de características, modelado predictivo y comunicación con partes interesadas. En su núcleo está un compendio de cuatro publicaciones científicas, cada una aportando evidencia empírica y conocimientos alineados con los objetivos generales. Los objetivos específicos son: (i) identificar fuentes de datos y técnicas de recopilación efectivas para el análisis de movilidad urbana; (ii) determinar el papel de la ingeniería de características en la mejora de la calidad y relevancia de los datos; (iii) evaluar la idoneidad y el valor añadido de las técnicas de aprendizaje automático en desafíos de movilidad urbana; y (iv) analizar el rendimiento de los modelos y definir recomendaciones de políticas para las partes interesadas de la movilidad. Estos objetivos se prueban a través de tres aplicaciones prácticas: (i) promover la micromovilidad para compras urbanas; (ii) establecer precios en servicios de vehículo de transporte con conductor (VTC); y (iii) predecir la demanda de motos compartidas. La primera aplicación examina los factores que influyen en la adopción de bicicletas y patinetes (principalmente modos activos) para compras urbanas en ciudades con baja madurez en micromovilidad. Usando modelos de aprendizaje automático aplicados a datos de encuestas, se identifican barreras y se proponen recomendaciones para mejorar infraestructura y seguridad, destacando su potencial de crecimiento post-COVID-19. La segunda aplicación explora la dinámica de precios en servicios de VTC. Un estudio analiza patrones tarifarios antes y durante la pandemia en Boston y Atlanta, identificando la dinámica de tarifas en tiempos de crisis. Otro examina los determinantes de tarifas en Madrid, considerando clima y horarios pico. Estos estudios ofrecen estrategias de precios y recomendaciones de regulación para integrar mejor los servicios VTC. La tercera aplicación se centra en predecir la demanda de motos compartidas. Usando técnicas de aprendizaje automático, se predice la demanda basándose en datos sociodemográficos, atributos de la red y variables políticas. Los resultados destacan la efectividad del modelo de random forest y la importancia de las características de los barrios y las preferencias por trayectos de corta distancia en la planificación urbana y gestión del transporte. Esta tesis demuestra que la ciencia de datos y el aprendizaje automático mejoran la toma de decisiones en sistemas de movilidad al incrementar la precisión de predicciones, optimizar el uso de datos y ofrecer recomendaciones prácticas para la movilidad sostenible. También reconoce desafíos como la necesidad de datos de alta calidad, algoritmos robustos y modelos más interpretables. Abordar estas limitaciones es clave para garantizar implementaciones exitosas y maximizar el impacto de estas tecnologías.

Machine learning (ML) is the fastest growing field in computer science, and health informatics is among the greatest challenges. The goal of ML is to develop algorithms which can learn and improve over time and can be used for predictions. Most ML researchers concentrate on automatic machine learning (aML), where great advances have been made, for example, in speech recognition, recommender systems, or autonomous vehicles. Automatic approaches greatly benefit from big data with many training sets. However, in the health domain, sometimes we are confronted with a small number of data sets or rare events, where aML-approaches suffer of insufficient training samples. Here interactive machine learning (iML) may be of help, having its roots in reinforcement learning, preference learning, and active learning. The term iML is not yet well used, so we define it as “algorithms that can interact with agents and can optimize their learning behavior through these interactions, where the agents can also be human.” This “human-in-the-loop” can be beneficial in solving computationally hard problems, e.g., subspace clustering, protein folding, or k-anonymization of health data, where human expertise can help to reduce an exponential search space through heuristic selection of samples. Therefore, what would otherwise be an NP-hard problem, reduces greatly in complexity through the input and the assistance of a human agent involved in the learning phase.

Interactive Machine Learning (IML) can enable intelligent systems to interactively learn from their end-users, and is quickly becoming more and more relevant to many application domains. Although it places the human in the loop, interactions are mostly performed via mutual explanations that miss contextual information. Furthermore, current model-agnostic IML strategies such as CAIPI are limited to ’destructive’ feedback, meaning that they solely allow an expert to prevent a learner from using irrelevant features. In this work, we propose a novel interaction framework called Semantic Interactive Learning for the domain of document classification, located at the intersection between Natural Language Processing (NLP) and Machine Learning (ML). We frame the problem of incorporating constructive and contextual feedback into the learner as a task involving finding an architecture that enables more semantic alignment between humans and machines while at the same time helping to maintain the statistical characteristics of the input domain when generating user-defined counterexamples based on meaningful corrections. Therefore, we introduce a technique called SemanticPush that is effective for translating conceptual corrections of humans to non-extrapolating training examples such that the learner’s reasoning is pushed towards the desired behavior. Through several experiments we show how our method compares to CAIPI, a state of the art IML strategy, in terms of Predictive Performance and Local Explanation Quality in downstream multi-class classification tasks. Especially in the early stages of interactions, our proposed method clearly outperforms CAIPI while allowing for contextual interpretation and intervention. Overall, SemanticPush stands out with regard to data efficiency, as it requires fewer queries from the pool dataset to achieve high accuracy.

Network alarm triage refers to grouping and prioritizing a stream of low-level device health information to help operators find and fix problems. Today, this process tends to be largely manual because existing rule-based tools cannot easily evolve with the network. We present CueT, a system that uses interactive machine learning to constantly learn from the triaging decisions of operators. It then uses that learning in novel visualizations to help them quickly and accurately triage alarms. Unlike prior interactive machine learning systems, CueT handles a highly dynamic environment where the groups of interest are not known a priori and evolve constantly. Our evaluations with real operators and data from a large network show that CueT significantly improves the speed and accuracy of alarm triage.

PurposeDental implant abutments are defined as medical devices by their intended use. Surfaces of custom-made CAD/CAM two-piece abutments may become contaminated during the manufacturing process in the dental lab. Inadequate reprocessing prior to patient care may contribute to implant-associated complications. Risk-adapted hygiene management is required to meet the requirements for medical devices.MethodsA total of 49 CAD/CAM-manufactured zirconia copings were bonded to prefabricated titanium bases. One group was bonded, polished, and cleaned separately in dental laboratories throughout Germany (LA). Another group was left untreated (NC). Five groups received the following hygiene regimen: three-stage ultrasonic cleaning (CP and FP), steam (SC), argon–oxygen plasma (PL), and simple ultrasonic cleaning (UD). Contaminants were detected using scanning electron microscopy (SEM) and energy-dispersive X-ray spectroscopy (EDS) and segmented and quantified using interactive machine learning (ML) and thresholding (SW). The data were statistically analysed using non-parametric tests (Kruskal–Wallis test, Dunn’s test).ResultsSignificant differences in contamination levels with the different cleaning procedures were found (p ≤ 0.01). The FP–NC/LA groups showed the most significant difference in contamination levels for both measurement methods (ML, SW), followed by CP–LA/NC and UD–LA/NC for SW and CP–LA/NC and PL–LA/NC for ML (p ≤ 0.05). EDS revealed organic contamination in all specimens; traces of aluminum, silicon, and calcium were detected.ConclusionsChemothermal cleaning methods based on ultrasound and argon–oxygen plasma effectively removed process-related contamination from zirconia surfaces. Machine learning is a promising assessment tool for quantifying and monitoring external contamination on zirconia abutments.Graphical

Subfield of AI, is going to change the healthcare industry forever. However, it is not yet considered part of standard of care, especially when it comes to the treatment of specific patients. Whether or not data-driven methods are being used to support clinical making a Call (CDS). To date, there has been no comprehensive analysis of how research in machine learning and other data-driven techniques might effectively contribute to clinical care and what kinds of support they can bring to doctors. In this study, we investigate the potential contributions to clinical decision support systems of two data-driven fields: machine learning and data visualization. Here, we survey the research on three distinct CDS and how heuristic knowledge, machine learning, and visualization are now being used to analyse and improve them. Predictive modelling for alerts has been the subject of extensive study, although this technology is not yet integrated into CDS systems. Interactive visualizations and machine learning inferences are gaining popularity as a means to organize and review patient data, however these methods are still in the prototype stage and have not been implemented. We still lack CDS systems that could take use of prescriptive machine learning (e.g., individualized therapy suggestions). Possible explanations for the slow adoption of data-driven approaches in CDS are offered, along with suggestions for future study in this area. Clinical decision assistance; Visual analytics; Machine Learning.

This article proposes a data-centric strategy that emphasises data preprocessing, interpretation of machine learning models' performance, improving data quality and modifying models to deal with issues identified during the iterative loop of classification model development. The framework consists of three stages: stage-1 focuses on data collection and pre-processing, followed by data quality improvement and feature extraction in stage-2, and the final stage-3 with model hyper-parameter tuning. The concept of model interpretation is added within the framework that helps to understand the learning behaviour of machine learning (ML) models. This makes the models' performance more explainable and is known as Explainable Artificial Intelligence (XAI). For stage-1, the data is generated from a simulation of cyber-attacks in CIGRE low voltage microgrid network, which is then preprocessed. In stage-2, data is augmented using ensembled Synthetic Minority Over-sampling Technique (SMOTE) and Edited Nearest Neighbour (ENN) methods, followed by feature extraction using the Boruta python package. Finally, the hyper-parameters are tuned through a Tree-structured Parzen Estimator (TPE) algorithm. A time-series transformer model is also presented for cyber-attack detection. The findings from the proposed approach demonstrate that the model's predictive performance increases with subsequent stages.

Explanations have gained an increasing level of interest in the AI and Machine Learning (ML) communities in order to improve model transparency and allow users to form a mental model of a trained ML model. However, explanations can go beyond this one way communication as a mechanism to elicit user control, because once users understand, they can then provide feedback. The goal of this paper is to present an overview of research where explanations are combined with interactive capabilities as a mean to learn new models from scratch and to edit and debug existing ones. To this end, we draw a conceptual map of the state-of-the-art, grouping relevant approaches based on their intended purpose and on how they structure the interaction, highlighting similarities and differences between them. We also discuss open research issues and outline possible directions forward, with the hope of spurring further research on this blooming research topic.

As technology has advanced, the cyber threats are more advanced and common, thus it is difficult for the conventional methods to detect it. This paper explores the role of Machine Learning (ML) in the improvement of cybersecurity strategies in relation to threat detection and management. ML's pattern recognition ability is used to enhance threat recognition performance and decrease false positives. Study of past research indicates that several ML algorithms like decision tree, random forest, and convolution neural networks (CNN's), have been useful in interpreting the anomalies and distinguishing various sorts of attacks. In this paper the impact of ML algorithms on the improvement of cybersecurity especially in detecting, assessing and addressing innumerable threats present in the network traffic is reviewed. In consideration of these challenges, a system with the aim of threat classification and features extraction through use of some ML capabilities is presented. This would help in making the cyber security defences flexible and quick, encouraging timely and effective assessment and management of the threats. Some prominent areas of application of ML in the protection of cyber security are in-detecting intrusions in the networks, malware classification, preventing phishing attacks, as well as automating incident response.

Chemical heuristics have been fundamental to the advancement of chemistry and materials science. These heuristics are typically established by scientists using knowledge and creativity to extract patterns from limited datasets. Machine learning offers opportunities to perfect this approach using computers and larger datasets. Here, we discuss the relationships between traditional heuristics and machine learning approaches. We show how traditional rules can be challenged by large-scale statistical assessment and how traditional concepts commonly used as features are feeding the machine learning techniques. We stress the waste involved in relearning chemical rules and the challenges in terms of data size requirements for purely data-driven approaches. Our view is that heuristic and machine learning approaches are at their best when they work together.