Machine learning under the spotlight

When bioprocess engineering meets machine learning: A survey from the perspective of automated bioprocess development

The process input data which materials processing operations can collect for each unit of production is extensive. Large datasets have long been difficult to work with as computing power to execute analysis in a timely fashion was unavailable. Further, the great velocity at which the data is generated makes near real-time decision-making unwieldy without a new set of tools with which to do the work. When troubleshooting by a small dataset, such as the last few hours of production, observations made on the measured parameters can be misleading. Machine learning is opening doors to high-dimensional data analysis in material processing. In this work, high-pressure die-casting (HPDC) is explored as an exemplar of high-volume materials processing. HPDC process summary data from a full year of production data covering over 950,000 machine cycles is analyzed via supervised machine learning methods to successfully model the prediction of good parts and process scrap as determined by the die-casting machine. Additionally, the prediction of ultimate tensile strength via a classification method of extracted tensile bars is performed and the important features identified are discussed. Supervised learning is found to be a useful tool for materials processing applications.

Summary Advances in machine learning and data science have given rise to a new set of tools and methods for developing efficient fit-for-purpose proxy models. Two major difficulties in adopting these methods are (i) the lack of physical constraints and domain insight in the design and construction of these models, and (ii) their limited interpretability, which can seriously impede their broader application to scientific domains, including reservoir engineering. In this paper, neural network architectures are designed based on simple reservoir engineering insights to characterize inter-well connectivity and well production performance under different operational conditions. Two different approaches are presented for construction of a neural network proxy model. The first approach takes a global view by representing all well connections in a fully connected neural network to allow any injector to be connected to a producer. The resulting network is a highly redundant description as many of the included connections are not plausible. To systematically eliminate unrealistic connections, l1-norm regularization is adopted to sparsify the network topology (connections) during the training stage. The resulting sparse structure characterizes the inter-well connectivity and defines a network topology that is supported by the training data. In the second approach, a local view is taken in building the proxy model. In this case, each producer is assumed to be supported by very few surrounding injection wells and likely to have weak connections with distant wells. However, inter-well connectivity in complex large-scale reservoirs is not just a function of distance and rather difficult to determine. Therefore, we use a series of randomly sized neighborhoods around each producer to include different number of injectors in the local networks. The variability in the neighborhood size reflects the prior uncertainty about the potential connectivity between wells at different distances. This approach results in many local neural networks (several local networks per each producer) that can be aggregated into a single large neural network model with pre-defined topological structure to represent possible connections. The training is then used to estimate the weights in the resulting architecture. The methods are applied to predict inter-well connectivity and oil production in a large-scale mature field that undergoes water flooding. Examination of the estimated connectivity parameters inside the neural network shows that the retrieved inter-well connectivity map is consistent with the existing geological features. The results suggest that even simple engineering insights can lead to noticeable improvement in the performance of neural networks.

Detecting abnormal sensors via machine learning: An IoT farming WSN-based architecture case study

As machine learning systems move from computer-science laboratories into the open world, their accountability becomes a high priority problem. Accountability requires deep understanding of system behavior and its failures. Current evaluation methods such as single-score error metrics and confusion matrices provide aggregate views of system performance that hide important shortcomings. Understanding details about failures is important for identifying pathways for refinement, communicating the reliability of systems in different settings, and for specifying appropriate human oversight and engagement. Characterization of failures and shortcomings is particularly complex for systems composed of multiple machine learned components. For such systems, existing evaluation methods have limited expressiveness in describing and explaining the relationship among input content, the internal states of system components, and final output quality. We present Pandora, a set of hybrid human-machine methods and tools for describing and explaining system failures. Pandora leverages both human and system-generated observations to summarize conditions of system malfunction with respect to the input content and system architecture. We share results of a case study with a machine learning pipeline for image captioning that show how detailed performance views can be beneficial for analysis and debugging.

The CRISM imaging spectrometer orbiting Mars has been producing a vast amount of data in the visible to infrared wavelengths in the form of hyperspectral data cubes. These data, compared with those obtained from previous remote sensing techniques, yield an unprecedented level of detailed spectral resolution in additional to an ever increasing level of spatial information. A major challenge brought about by the data is the burden of processing and interpreting these datasets and extract the relevant information from it. This research aims at approaching the challenge by exploring machine learning methods especially unsupervised learning to achieve cluster density estimation and classification, and ultimately devising an efficient means leading to identification of minerals. A set of software tools have been constructed by Python to access and experiment with CRISM hyperspectral cubes selected from two specific Mars locations. A machine learning pipeline is proposed and unsupervised learning methods were implemented onto pre-processed datasets. The resulting data clusters are compared with the published ASTER spectral library and browse data products from the Planetary Data System (PDS). The result demonstrated that this approach is capable of processing the huge amount of hyperspectral data and potentially providing guidance to scientists for more detailed studies.

The fiber optical network offers high speed, large bandwidth, and a high degree of reliability. However, the development of optical communication technology has hit a bottleneck due to several challenges such as energy loss, cost, and system capacity approaching the Shannon limit. As a powerful tool, machine learning technology provides a strong driving force for the development of various industries and greatly promotes the development of society. Machine learning also provides a new possible solution to achieve greater transmission capacities and longer transmission distances in optical communications. In this article, we introduce the application of machine learning in optical communication network systems. Three use cases are presented to evaluate the feasibility of our proposed architecture. In the transmission layer, the principal-component-based phase estimation algorithm is used for phase noise recovery in coherent optical systems, and the K-means algorithm is adopted to reduce the influence of nonlinear noise in probabilistic shaping systems. As for the network layer, the long short-term memory algorithm and the genetic algorithm are suitable for making traffic predictions and determining reasonable placement locations of remote radio heads in centralized radio access networks. Extensive simulations and experiments are conducted to evaluate the proposed algorithm in comparison to the state-of-the-art schemes. The results show the performance of three use cases. Machine learning algorithms applied to the transmission layer can greatly promote the performance of digital signal processing without increasing the complexity. Machine learning algorithms applied to the network layer can provide a more appropriate channel allocation plan in the era of high-speed communication. Ultimately, the intent of this article is to serve as a basis for stimulating more research in machine learning in optical communications.

Mechanistic or dynamical models based on governing equations are ubiquitous throughout science and engineering. Such models, also referred to as “simulators”, are typically characterized by a forward mapping from some set of inputs or parameters to one or more output quantities of interest. In many cases, the inputs required for the forward model are either unknown or represent approximations of unresolved processes. Bayesian inference provides a natural framework for constraining this model uncertainty using observed data. Such a framework is especially valuable in cryospheric application domains such as permafrost research, where direct observations of many quantities of interest, e.g. subsurface temperature and soil moisture, are only sparsely available. Mechanistic models based on known physics therefore play an indispensable role in filling these gaps. However, virtually all methods for Bayesian inference require repeated evaluation of the forward model which is often computationally challenging, especially for dynamical systems. As a result, computational requirements of statistical inference very quickly become intractable even for systems of only moderate complexity. The burgeoning field of “simulation-based inference” (SBI) aims to leverage modern computational methods from machine learning (ML) and data assimilation (DA) to overcome these challenges and facilitate large scale uncertainty quantification in complex scientific models. In this work, we show how SBI can be seen as a unifying theoretical framework that bridges the gap between existing DA methods (e.g. variants of the ensemble Kalman filter) and full-fledged Bayesian inference with the goal of facilitating hybrid statistical-physical modeling of complex systems. We present a novel set of software tools for making SBI more accessible to researchers along with benchmarks of several methods drawn from both the ML and DA literature. Two of these benchmarks are based on use cases from Arctic land surface modeling: degree day approximation of snowmelt and geothermal inversion of historical climate change from boreholes in Arctic permafrost. We highlight the contributions that SBI can make to solving such inverse problems and discuss further potential applications in the cryosphere and beyond.

Recently, extensive research has been conducted to explore the utilization of machine learning (ML) algorithms in various direct-detected and (self)-coherent short-reach communication applications. These applications encompass a wide range of tasks, including bandwidth request prediction, signal quality monitoring, fault detection, traffic prediction, and digital signal processing (DSP)-based equalization. As a versatile approach, ML demonstrates the ability to address stochastic phenomena in optical systems networks where deterministic methods may fall short. However, when it comes to DSP equalization algorithms such as feed-forward/decision-feedback equalizers (FFEs/DFEs) and Volterra-based nonlinear equalizers, their performance improvements are often marginal, and their complexity is prohibitively high, especially in cost-sensitive short-reach communications scenarios such as passive optical networks (PONs). Time-series ML models offer distinct advantages over frequency-domain models in specific contexts. They excel in capturing temporal dependencies, handling irregular or nonlinear patterns effectively, and accommodating variable time intervals. Within this survey, we outline the application of ML techniques in short-reach communications, specifically emphasizing their utilization in high-bandwidth demanding PONs. We introduce a novel taxonomy for time-series methods employed in ML signal processing, providing a structured classification framework. Our taxonomy categorizes current time-series methods into four distinct groups: traditional methods, Fourier convolution-based methods, transformer-based models, and time-series convolutional networks. Finally, we highlight prospective research directions within this rapidly evolving field and outline specific solutions to mitigate the complexity associated with hardware implementations. We aim to pave the way for more practical and efficient deployment of ML approaches in short-reach optical communication systems by addressing complexity concerns.

OmicInt package: Exploring omics data and regulatory networks using integrative analyses and machine learning

Sentiment Analysis deals with the detection and analysis of affective content in written text. It utilizes methodologies, theories, and techniques from a diverse set of scientific domains, ranging from psychology and sociology to natural language processing and machine learning. In this chapter, we discuss the contributions of the field in social media analysis with a particular focus in online collective actions; as these actions are typically motivated and driven by intense emotional states (e.g., anger), sentiment analysis can provide unique insights into the inner workings of such phenomena throughout their life cycle. We also present the state of the art in the field and describe some of its contributions into understanding online collective behavior. Lastly, we discuss significant real-world datasets that have been successfully utilized in research and are available for scientific purposes and also present a diverse set of available tools for conducting sentiment analysis.

Lung adenocarcinoma (LUAD) is the most common subtype of lung cancer, which is the leading cause of cancer death. Dysregulation of cell proliferation and death plays a crucial role in the development of LUAD. As of recently, the role of a new form of cell death, cuproptosis, and it has attracted more and more attention. As of yet, it is not clear whether cuproptosis is involved in the progression of LUAD. An integrated set of bioinformatics tools was utilized to analyze the expression and prognostic significance of cuproptosis-related genes. Meanwhile, a robust risk signature was developed using machine learning based on prognostic cuproptosis-related genes and explored the value of prognostic cuproptosis-related signature for clinical applications, functional enrichment and immune landscape. Lastly, the dysregulation of the cuproptosis-related genes in LUAD was validated by in vitro experiment. In this study, first, cuproptosis-related genes were found to be differentially expressed in LUAD patients of public databases, and nine of them had prognostic value. Next, a cuproptosis-related model with five features (DLTA, MTF1, GLS, PDHB and PDHA1) was constructed to separate the patients into high- and low-risk groups based on median risk score. Internal validation set and external validation set were used for model validation and evaluation. What's more, Enrichment analysis of differential genes and the WGCNA identified that cuproptosis-related signatures affected tumor prognosis by influencing tumor immunity. Small molecule compounds were predicted based on differential expressed genes to improve poor prognosis in the high-risk group and a nomogram was constructed to further advance clinical applications. In closing, our data showed that FDX1 affected the prognosis of lung cancer by altering the expression of cuproptosis-related signature. A new cuproptosis-related signature for survival prediction was constructed and validated by machine learning algorithm and in vitro experiments to reflect tumor immune infiltration in LUAD patients. The purpose of this article was to provide a potential diagnostic and therapeutic strategy for LUAD.

This paper documents the development of ml.lib: a set of open-source tools designed for employing a wide range of machine learning techniques within two popular real-time programming environments, namely Max and Pure Data. ml.lib is a cross-platform, lightweight wrapper around Nick Gillian's Gesture Recognition Toolkit, a C++ library that includes a wide range of data processing and machine learning techniques. ml.lib adapts these techniques for real-time use within popular data-flow IDEs, allowing instrument designers and performers to integrate robust learning, classification and mapping approaches within their existing workflows. ml.lib has been carefully de-signed to allow users to experiment with and incorporate ma-chine learning techniques within an interactive arts context with minimal prior knowledge. A simple, logical and consistent, scalable interface has been provided across over sixteen exter-nals in order to maximize learnability and discoverability. A focus on portability and maintainability has enabled ml.lib to support a range of computing architectures---including ARM---and operating systems such as Mac OS, GNU/Linux and Win-dows, making it the most comprehensive machine learning implementation available for Max and Pure Data.

In this paper, we present our current work towards developing a context aware visual system with capabilities to generate knowledge using an adaptive cognitive model. Our goal is to assist people in their daily routines using the acquired knowledge in combination with a set of machine learning tools to provide prediction and individual routine understanding. This is useful in applications such as assistance to individuals with Alzheimer by helping them to maintain a daily routine based on historical data. The proposed cognitive model is based on simple exponential smoothing technique and provides real time detection of objects and basic relations in the scene. To fulfill these objectives we propose the integration of machine learning tools and memory based knowledge representation.KeywordsAssistive technologyRoboticsIntelligent environmentTime seriesDeep neural networksCognitive architectures

NMR is a valuable experimental tool in the structural biologist’s toolkit to elucidate the structures, functions, and motions of biomolecules. The progress of machine learning, particularly in structural biology, reveals the critical importance of large, diverse, and reliable datasets in developing new methods and understanding in structural biology and science more broadly. Biomolecular NMR research groups produce large amounts of data, and there is renewed interest in organizing these data to train new, sophisticated machine learning architectures and to improve biomolecular NMR analysis pipelines. The foundational data type in NMR is the free-induction decay (FID). There are opportunities to build sophisticated machine learning methods to tackle long-standing problems in NMR data processing, resonance assignment, dynamics analysis, and structure determination using NMR FIDs. Our goal in this study is to provide a lightweight, broadly available tool for archiving FID data as it is generated at the spectrometer, and grow a new resource of FID data and associated metadata. This study presents a relational schema for storing and organizing the metadata items that describe an NMR sample and FID data, which we call Spectral Database (SpecDB). SpecDB is implemented in SQLite and includes a Python software library providing a command-line application to create, organize, query, backup, share, and maintain the database. This set of software tools and database schema allow users to store, organize, share, and learn from NMR time domain data. SpecDB is freely available under an open source license at https://github.rpi.edu/RPIBioinformatics/SpecDB.