Using Interactive Machine Learning to Sonify Visually Impaired Dancers' Movement

With the Internet of Things paradigm, the data generated by the rapidly increasing number of connected devices lead to new possibilities, such as using machine learning for activity recognition in smart environments. However, it also introduces several challenges. The sensors of different devices might be of different types, making the fusion of data non-trivial. Moreover, the devices are often mobile, resulting in that data from a particular sensor is not always available, i.e. there is a need to handle data from a dynamic set of sensors. From a machine learning perspective, the data from the sensors arrives in a streaming fashion, i.e., online learning, as compared to many learning problems where a static dataset is assumed. Machine learning is in many cases a good approach for classification problems, but the performance is often linked to the quality of the data. Having a good data set to train a model can be an issue in general, due to the often costly process of annotating the data. With dynamic and heterogeneous data, annotation can be even more problematic, because of the ever-changing environment. This means that there might not be any, or a very small amount of, annotated data to train the model on at the start of learning, often referred to as the cold start problem. To be able to handle these issues, adaptive systems are needed. With adaptive we mean that the model is not static over time, but is updated if there for instance is a change in the environment. By including human-in-the-loop during the learning process, which we refer to as interactive machine learning, the input from users can be utilized to build the model. The type of input used is typically annotations of the data, i.e. user input in the form of correctly labelled data points. Generally, it is assumed that the user always provides correct labels in accordance with the chosen interactive learning strategy. In many real-world applications these assumptions are not realistic however, as users might provide incorrect labels or not provide labels at all in line with the chosen strategy. In this thesis we explore which interactive learning strategies are possible in the given scenario and how they affect performance, as well as the effect of machine learning algorithms on performance. We also study how a user who is not always reliable, i.e. that does not always provide a correct label when expected to, can affect performance. We propose a taxonomy of interactive online machine learning strategies and test how the different strategies affect performance through experiments on multiple datasets. The findings show that the overall best performing interactive learning strategy is one where the user provides labels when previous estimations have been incorrect, but that the best performing machine learning algorithm depends on the problem scenario. The experiments also show that a decreased reliability of the user leads to decreased performance, especially when there is a limited amount of labelled data.

In this paper, we highlight three issues that limit performance of machine learning on biomedical images, and tackle them through 3 case studies: 1) Interactive Machine Learning (IML): we show how IML can drastically improve exploration time and quality of direct volume rendering. 2) transfer learning: we show how transfer learning along with intelligent pre-processing can result in better Alzheimer's diagnosis using a much smaller training set 3) data imbalance: we show how our novel focal Tversky loss function can provide better segmentation results taking into account the imbalanced nature of segmentation datasets. The case studies are accompanied by in-depth analytical discussion of results with possible future directions.

The goal of this paper is to develop an interactive web-based machine learning application to assist architects with multimodal inputs (sketches and textual information) for conceptual design. With different textual inputs, the application generates the architectural stylistic variations of a user’s initial sketch input as a design inspiration. A novel machine learning model for the multimodal input application is introduced and compared to others. The machine learning model is performed through procedural training with the content curation of training data (1) to control the fidelity of generated designs from the input and (2) to manage their diversity. The web-based interface is at its work in progress as a frontend of the proposed application for better user experience and future data collection. In this paper, the framework of the proposed interactive application is explained. Furthermore, the implementation of its prototype is demonstrated with various examples.

Chemical heuristics have been fundamental to the advancement of chemistry and materials science. These heuristics are typically established by scientists using knowledge and creativity to extract patterns from limited datasets. Machine learning offers opportunities to perfect this approach using computers and larger datasets. Here, we discuss the relationships between traditional heuristics and machine learning approaches. We show how traditional rules can be challenged by large-scale statistical assessment and how traditional concepts commonly used as features are feeding the machine learning techniques. We stress the waste involved in relearning chemical rules and the challenges in terms of data size requirements for purely data-driven approaches. Our view is that heuristic and machine learning approaches are at their best when they work together.

The study was conducted to design and develop an interactive alphabet learning application software and to find out its effectiveness in enhancing letter name and sound knowledge of kindergarten learners. To achieve this goal, this study utilized sequential developmental research design, adapting the ADDIE (Analysis, Design, Development, Implementation and Evaluation) instructional model to design and develop the interactive learning application. The development of the instructional material was anchored on Information Processing Theory, Bottom-up Theory, Constructivist Learning Theory and Cognitive Development Theory. Having five kindergarten learners as participants, this study also utilized quasi-experimental design to find out its effectiveness when used by kindergarten learners as a supplementary tool to develop and improve the alphabet knowledge. Based on the results, the least learned letters were Ii, Ee, Nn, Pp, Bb and Dd. These could be enhanced using technology tools such as the internet and smartphone which are both common among learners. Moreover, the Interactive Application Software was found out to be appropriate to address their needs. This study concluded that has significantly improved the alphabet knowledge of the kindergarten learners. Thus, it is recommended to be used as a supplementary and intervention tool in learning the letter names and sounds. Kindergarten teachers are also recommended to devise similar learning material which addresses least learned competencies brough about by distance learning. Finally, it is suggested that future studies be conducted to investigate the effectiveness of the same materials to another set of learners.

Interactive Machine Learning (IML) can enable intelligent systems to interactively learn from their end-users, and is quickly becoming more and more relevant to many application domains. Although it places the human in the loop, interactions are mostly performed via mutual explanations that miss contextual information. Furthermore, current model-agnostic IML strategies such as CAIPI are limited to ’destructive’ feedback, meaning that they solely allow an expert to prevent a learner from using irrelevant features. In this work, we propose a novel interaction framework called Semantic Interactive Learning for the domain of document classification, located at the intersection between Natural Language Processing (NLP) and Machine Learning (ML). We frame the problem of incorporating constructive and contextual feedback into the learner as a task involving finding an architecture that enables more semantic alignment between humans and machines while at the same time helping to maintain the statistical characteristics of the input domain when generating user-defined counterexamples based on meaningful corrections. Therefore, we introduce a technique called SemanticPush that is effective for translating conceptual corrections of humans to non-extrapolating training examples such that the learner’s reasoning is pushed towards the desired behavior. Through several experiments we show how our method compares to CAIPI, a state of the art IML strategy, in terms of Predictive Performance and Local Explanation Quality in downstream multi-class classification tasks. Especially in the early stages of interactions, our proposed method clearly outperforms CAIPI while allowing for contextual interpretation and intervention. Overall, SemanticPush stands out with regard to data efficiency, as it requires fewer queries from the pool dataset to achieve high accuracy.

Machine learning applications for anomaly detection in Smart Water Metering Networks: A systematic review

Interactive machine learning systems allow end-users, often non-experts, to build and apply statistical models for their own uses. Constructivism is the view that learning occurs when ideas and experiences interact. I argue that the objectives of interactive machine learning can be interpreted as constructivist. By so characterising them, I show how constructivist learning environments pose critical questions for the design of interactive machine learning systems.

Machine learning (ML) is the fastest growing field in computer science, and health informatics is among the greatest challenges. The goal of ML is to develop algorithms which can learn and improve over time and can be used for predictions. Most ML researchers concentrate on automatic machine learning (aML), where great advances have been made, for example, in speech recognition, recommender systems, or autonomous vehicles. Automatic approaches greatly benefit from big data with many training sets. However, in the health domain, sometimes we are confronted with a small number of data sets or rare events, where aML-approaches suffer of insufficient training samples. Here interactive machine learning (iML) may be of help, having its roots in reinforcement learning, preference learning, and active learning. The term iML is not yet well used, so we define it as “algorithms that can interact with agents and can optimize their learning behavior through these interactions, where the agents can also be human.” This “human-in-the-loop” can be beneficial in solving computationally hard problems, e.g., subspace clustering, protein folding, or k-anonymization of health data, where human expertise can help to reduce an exponential search space through heuristic selection of samples. Therefore, what would otherwise be an NP-hard problem, reduces greatly in complexity through the input and the assistance of a human agent involved in the learning phase.

The oil and gas industry faces constant pressure to enhance operational efficiency, reduce costs, and minimize equipment downtime. Machine learning (ML) applications in predictive maintenance have emerged as a transformative approach to achieving these goals. This review explores the role of machine learning in predictive maintenance, highlighting its potential to revolutionize maintenance strategies and improve asset management across the industry. Predictive maintenance leverages advanced algorithms to analyze historical and real-time data from equipment and sensors, enabling the identification of patterns and anomalies that precede equipment failures. By utilizing techniques such as supervised learning, unsupervised learning, and reinforcement learning, organizations can forecast equipment malfunctions and schedule maintenance activities proactively, thereby reducing unexpected downtimes and extending asset lifecycles. The implementation of ML in predictive maintenance provides several key benefits. Firstly, it enhances operational efficiency by optimizing maintenance schedules and minimizing unplanned outages, which are critical in the capital-intensive oil and gas sector. Secondly, it enables cost savings through more efficient resource allocation and reduced labor costs associated with reactive maintenance strategies. Thirdly, machine learning algorithms can continuously learn from new data, refining their predictive capabilities and improving accuracy over time. Several case studies illustrate the successful application of machine learning in predictive maintenance within the oil and gas industry. For example, ML models have been employed to predict pump failures, optimize drilling operations, and improve pipeline integrity monitoring. These applications not only lead to significant financial savings but also enhance safety by reducing the risk of catastrophic failures. In conclusion, machine learning applications in predictive maintenance represent a crucial advancement for the oil and gas industry. By harnessing the power of data-driven insights, organizations can enhance operational efficiency, reduce costs, and ultimately drive sustainable growth. This review emphasizes the transformative potential of machine learning in predictive maintenance, establishing it as a key strategy for success in the ever-evolving oil and gas landscape.

Chapter 15 - Clinical decision support systems to improve the diagnosis and management of respiratory diseases

A cost effective, easily scalable, and application independent FPGA cluster co-processing platform for machine learning (ML) applications is proposed. The work in this paper focuses on delivering an economical platform for the researchers and developers without the knowledge of FPGAs who seek budget friendly solutions to increase the performance of their ML applications without relying on the highly expensive solutions available in the today's FPGA market. The approach consists of two main parts, with a CPU based host machine learning application and the FPGA cluster sharing the heavy workload through an Ethernet protocol. An experiment was conducted using a perceptron layer implementation on the hardware and the test results show the execution time improvements due to the utilization of the proposed cluster approach. The solution proposed is easily scalable with any FPGA platform with an Ethernet support.

Technology drives advances in science. Giving scientists access to more powerful tools for collecting and understanding data enables them to both ask and answer new kinds questions that were previously beyond their reach. Of these new tools at their disposal, machine learning offers the opportunity to understand and analyze data at unprecedented scales and levels of detail. The standard machine learning pipeline consists of data labeling, feature extraction, training, and evaluation. However, without expert machine learning knowledge, it is difficult for scientists to optimally construct this pipeline to fully leverage machine learning in their work. Using ecology as a motivating example, we analyze a typical scientist's data collection and processing workflow and highlight many problems facing practitioners when attempting to capitalize on advances in machine learning and pattern recognition. Understanding these shortcomings allows us to outline several novel and underexplored research directions. We end with recommendations to motivate progress in future cross-disciplinary work.

Network alarm triage refers to grouping and prioritizing a stream of low-level device health information to help operators find and fix problems. Today, this process tends to be largely manual because existing rule-based tools cannot easily evolve with the network. We present CueT, a system that uses interactive machine learning to constantly learn from the triaging decisions of operators. It then uses that learning in novel visualizations to help them quickly and accurately triage alarms. Unlike prior interactive machine learning systems, CueT handles a highly dynamic environment where the groups of interest are not known a priori and evolve constantly. Our evaluations with real operators and data from a large network show that CueT significantly improves the speed and accuracy of alarm triage.

Purpose Although there are studies analyzing barriers related to new technological concepts, it turns out that there are only a few studies on barriers to machine learning (ML) applications, and none of them consider the implications for smart logistics. Therefore, the purpose of this study is to reveal and analyze the barriers to ML applications in smart logistics from both industry and academic perspectives. Design/methodology/approach To achieve this aim, first, various barriers to smart logistics activities based on the Industry 4.0 perspective are identified. Later, the relative importance of these barriers critical to the success of smart logistics activities is determined. Finally, the interval-valued fuzzy (IVF) DEMATEL method is used to analyze the cause-and-effect relationship between each barrier based on industry and academic perspective. Findings Eleven barriers related to ML applications in smart logistics were evaluated by seven experts who are working in different positions. Results show that, the most crucial cause-and-effect barriers are integration and connection problems with value chain/network systems (B6), requirements of adapting new infrastructures (B11) and lack of transparency, safety and security (B3). Originality/value There is no study about determining barriers with merging smart logistics activities with the Industry 4.0 perspective. It is expected that the results of this study will contribute to the use of ML in the logistics sector by revealing significant concepts to which businesses should pay attention to prevent these barriers and by suggesting practical solutions to these problems.