Black box methods for least squares problems

In recent years, deep neural networks (DNN) have become a highly active area of research, and shown remarkable achievements on a variety of computer vision tasks. DNNs, however, are known to often make overconfident yet incorrect predictions on out-of-distribution samples, which can be a major obstacle to real-world deployments because the training dataset is always limited compared to diverse real-world samples. Thus, it is fundamental to provide guarantees of robustness to the distribution shift between training and test time when we construct DNN models in practice. Moreover, in many cases, the deep learning models are deployed as black boxes and the performance has been already optimized for a training dataset, thus changing the black box itself can lead to performance degradation. We here study the robustness to the geometric transformations in a specific condition where the black-box image classifier is given. We propose an additional learner, REinforcement Spatial Transform learner (REST), that transforms the warped input data into samples regarded as in-distribution by the black-box models. Our work aims to improve the robustness by adding a REST module in front of any black boxes and training only the REST module without retraining the original black box model in an end-to-end manner, i.e. we try to convert the real-world data into training distribution which the performance of the black-box model is best suited for. We use a confidence score that is obtained from the black-box model to determine whether the transformed input is drawn from in-distribution. We empirically show that our method has an advantage in generalization to geometric transformations and sample efficiency.

One of the important production processes in the textile industry is the spinning process. Starting from cotton fibers, yarns are (usually) created on a rotor-spinning machine. The spinnability of a fiber is dependant on its quality and on the machine settings of the spinning machine. It would be a great benefit to be able to predict the spinnability and resulting strength of the yarn starting from a certain quality and from machine settings. To this end, two totally diiffferent modeling approaches can be considered: the so-called 'white' modeling and the so-called 'black box' modeling. In white modeling the process is described by mathematical equations, which are based upon (theoretical) physical knowledge of the process. Extensive physical information about the process is in this case available through physical, chemical or mechanical equations giving the user a thorough insight into the operation of the process. However, due to the large input (and output) dimensions of the fiber-to-yarn process and their complex interactions, no exact mathematical model of a spinning machine is known to exist nor is it likely that such a model will ever be constructed. A black box model, in contrast to white modeling, simply connects input parameters to the output without giving or containing any substantial physical information about the process itsellf Black box models have been successfully constructed by Pynckels et al. to predict the spinnability (Pynckels, 1995) and the characteristics (Pynckels, 1997) of the yarn using neural networks with a Backpropagation learning rule. Apart from the lack of physical information, these models also have no fault indication or measure of uncertainty about the results.

Opening the Black Box: The Promise and Limitations of Explainable Machine Learning in Cardiology.

One promising approach to reduce carbon foot print of internal combustion engines (ICEs) is using alternative fuels like hydrogen, particularly by converting medium and heavy-duty diesel engines to dual-fuel hydrogen-diesel engines. To minimize elevated NOx emissions from hydrogen-fueled engine, fast and accurate emission models are essential for engine model-based control and for engine calibration and optimization using hardware-in-the-loop (HIL) setups. In this study, a fast-response NOx emissions sensor is used to measure the transient NOx emissions from a dual-fuel hydrogen-diesel engine. Subsequently, steady-state models (SSMs), quasi steady-state models (QSSMs), and transient sequential models (TSMs) in the form of black-box (BB) and gray-box (GB) models are developed for transient NOx emissions prediction. GB models utilize both information from a one dimensional (1D) physical engine model and experimental data for training, while BB models only use experimental data. SSMs are optimized artificial neural networks (ANNs) trained using steady-state data, QSSMs are optimized ANNs trained using transient data, and TSMs are time-series networks trained using transient data. Long short-term memory (LSTM) and gated recurrent unit (GRU) networks are used as the time-series deep learning networks. The results showed that the 1D physical model has the poorest performance with successive model performance improvement from SSM to QSSM and from QSSM to TSM. The developed BB TSM model in this study can predict transient NOx emissions with an R2 value greater than 0.96 at 89,000 predictions per second which makes this model suitable for real-time engine model-based control where computational efficiency is crucial. The developed GB TSM model can predict transient NOx emissions with an R2 value greater than 0.97 but it is computationally more expensive. The extra accuracy of the GB TSM models makes them the best choice for HIL setups where more computational power is available, and accuracy is more crucial.

We present an approach to explain the decisions of black box models for image classification. While using the black box to label images, our explanation method exploits the latent feature space learned through an adversarial autoencoder. The proposed method first generates exemplar images in the latent feature space and learns a decision tree classifier. Then, it selects and decodes exemplars respecting local decision rules. Finally, it visualizes them in a manner that shows to the user how the exemplars can be modified to either stay within their class, or to become counter-factuals by “morphing” into another class. Since we focus on black box decision systems for image classification, the explanation obtained from the exemplars also provides a saliency map highlighting the areas of the image that contribute to its classification, and areas of the image that push it into another class. We present the results of an experimental evaluation on three datasets and two black box models. Besides providing the most useful and interpretable explanations, we show that the proposed method outperforms existing explainers in terms of fidelity, relevance, coherence, and stability.

Explainable Boosting Machine for Choosing Radiation Dose-Volume Constraints on Cardio-Pulmonary Substructures Associated with Overall Survival in the NRG Oncology RTOG 0617 Clinical Trial

Virtual sensors, or soft sensors, have greatly contributed to the evolution of the sensing systems in industry. The soft sensors are process models having three fundamental categories, namely white-box (WB), black-box (BB) and gray-box (GB) models. WB models are based on process knowledge while the BB models are developed using data collected from the process. The GB models integrate the WB and BB models for addressing the concerns, i.e., accuracy and intuitiveness, of industrial operators. In this work, various design aspects of the GB models are discussed followed by their application in the process industry. In addition, the changes in the data-driven part of the GB models in the context of enormous amount of process data collected in Industry 4.0 are elaborated.

This paper presents a novel framework for Offline Reinforcement Learning (RL) with online fine tuning for Heating Ventilation and Air-conditioning (HVAC) systems. The framework presents a method to do pre-training in a black box model environment, where the black box models are built on data acquired under a traditional control policy. The paper focuses on the application of Underfloor Heating (UFH) with an air-to-water-based heat pump. However, the framework should also generalize to other HVAC control applications. Because Black box methods are used is there little to no commissioning time when applying this framework to other buildings/simulations beyond the one presented in this study. This paper explores and deploys Artificial Neural Network (ANN) based methods to design efficient controllers. Two ANN methods are tested and presented in this paper; a Multilayer Perceptron (MLP) method and a Long Short Term Memory (LSTM) based method. It is found that the LSTM-based method reduces the prediction error by 45% when compared with a MLP model. Additionally, different network architectures are tested. It is found that by creating a new model for each time step, performance can be improved additionally 19%. By using these models in the framework presented in this paper, it is shown that a Multi-Agent RL algorithm can be deployed without ever performing worse than an industrial controller. Furthermore, it is shown that if building data from a Building Management System (BMS) is available, an RL agent can be deployed which performs close to optimally from the first day of deployment. An optimal control policy reduces the cost of heating by 19.4 % when compared to a traditional control policy in the simulation presented in this paper.

Modeling is essential for modern science, and science-based policies are directly affected by the reliability of model outputs. Artificial intelligence has improved the accuracy and capability of model simulations, but often at the expense of a rational understanding of the systems involved. The lack of transparency in black box models, artificial intelligence based ones among them, can potentially affect the trust in science driven policy making. Here, we suggest that a broader discussion is needed to address the implications of black box approaches on the reliability of scientific advice used for policy making. We argue that participatory methods can bridge the gap between increasingly complex scientific methods and the people affected by their interpretations

In an attempt to reduce the cost of breakwaters, several elements have been developed. Examples are Accropodes, Tetrapods and concrete cubes. Previous tests were performed with armour layers consisting of a double layer of cubes. This study is based on a single layer of cubes. The great benefit of a single layer of cubes is that it reduces the total cost of concrete. Another benefit is that because of the shape, cubes are easy to prefabricate. Three important aspects considering the stability of a single armour layer of a breakwater consisting of concrete cubes are addressed: The influence of the wave steepness on the stqbility, the influence of the slope on the stability of the single armour layer, and the influence of the packing density on the stability of the single armour layer. This study is based on a literature study and the results from a test program including a small-scale physical model tests. All conclusions in this thesis have been based on model tests, in which the cubes were placed by hand and placed in a stretching bond (half-steensverband). In total eighteen tests were performed in the wave flume of the Fluid Mechanics Laboratory of the Faculty of Civil Engineering and Geosciences. Two different slopes were tested together with three different packing densities and three different wave steepness. It was found that the gentler slope did not contribute to the stability in this setting using a stretching bond. In fact the model failed earlier than the model with a steeper slope in most cases. The best results were found using a slope of cot? = 1.5. Secondly, the influence of the packing density showed varying failure mechanism. When applying a large packing density (np = 0.20) the damage occurred below SWL. Contrary to small packing densities (np = 0.35) where damage occurred higher than SWL. It was found that, from the tested packing densities, a packing density of 0.28 gives the best results for both slopes. This conclusion is conform the findings of previous tests [Van Gent et al, 1999]. During these tests an optimum packing density of np = 0.25 – 0.30 was found. Although the cubes were placed randomly in the tests of Van Gent. In this study the cubes were placed in a stretching bond. Finally it was found that a wave steepness of s0p = 0.04-0.05 causes minimum stability for the armour layer. The tests with a single armour layer of cubes placed in a stretching bond indicated that high sta-bility numbers (Hs/?Dn) can be reached before failure occurs (Nod > 0.2). the tests show that sta-bility numbers as high as 4.5 can be realized before Nod > 0.2 is reached. This study shows that the use of a single top layer of cubes is feasible. The top layer becomes very stable when placed in a stretching bond. In this configuration it is recommended to use a single top layer of cubes instead of a double top layer of cubes. The results during the tests seem to have a strong correlation with pitched stones. Therefore the black box model as well as the analytical method for pitched stones (6-xi-rule) is treated in an attempt to optimize the design rules for different configurations. Finally, in combination with an adjusted 6-xi-rule and the formulae from Van der Meer for loose rock, formulae were developed based on curve fitting.

Black box techniques [12] are enabling exact linear algebra computations of a scale well beyond anything previously possible. The development of new and interesting algorithms has proceeded apace for the past two decades. It is time for the dissemination of these algorithms in an easily used software library so that the mathematical community may readily take advantage of their power. LinBox is that library. In this paper, we describe the design of this generic library, sketch its current range of capabilities, and give several examples of its use. The examples include a solution of Trefethen’s “Hundred Digit Challenge” problem #7 [14] and the computation of all the homology groups of simplicial complexes using the Smith normal form [8]. Exact black box methods are currently successful on sparse matrices with hundreds of thousands of rows and columns and having several million nonzero entries. The main reason large problems can be solved by black box methods is that they require much less memory in general than traditional eliminationbased methods do. This fact is widely used in the numerical computation area. We refer for instance to the templates for linear system solution and eigenvalue problems [2,1]. This has also led the computer algebra community to a considerable interest in black box methods. Since Wiedemann’s seminal paper [16], many developments have been proposed especially to adapt Krylov or Lanczos methods to fast exact algorithms. We refer to [5] and references therein for a review of problems and solutions. LinBox supplies efficient black box solutions for a variety of problems including linear equations and matrix normal forms with the guiding design principle of re-usability. The most essential and driving design criterion for LinBox is that it is generic with respect to the domain of computation. This is because there are many and various representations of finite fields each of which is advantageous to use for some algorithm under some circumstance. The integral and rational number capabilities depend heavily on modular

An uncertainty model has been proposed which is based on the black-box concept to evaluate the uncertainty of measurement and calibration in high voltage pulse divider. The initial model was proposed in principle, unnecessary variables in the formula could be omissible in the calibration experiment. For the model absent of uncertainty originating from calibration document, and resolving power of oscilloscope, the correction components which belong to the black-box model would be introduced to the formula to perfect the model. The uncertainties were combined after perfection of model according to the law of propagation of uncertainty. The results are compared with that combined from relative uncertainty. It is shown that the uncertainty combined from relative uncertainty will be correct if this model is only composed of multiplication and division of different variables, or addition and subtraction of different variables are also permitted if the mathematical expectation values of these items equal to zero. This is proven by calculating the voltage potential between two different positions according to the measured voltage pulse at those positions and analyzing the attenuation represented by dB scales. In the calibration experiment of high voltage divider, the combined standard uncertainty of dividers scaling factor is obtained by evaluating Type A standard uncertainty of the ratio of output to input. The influence of dispersion from signal source output on evaluation of standard uncertainty can be weakened with this method.

We explore a special case of rational identity testing and algorithmic versions of two theorems on noncommutative polynomials, namely, Amitsur's theorem [S.A Amitsur, 1966] and the Bresar-Klep theorem [Bresar and Klep, 2008] when the input polynomial is given by an algebraic branching program (ABP). Let f be a degree-d n-variate noncommutative polynomial in the free ring Q over rationals. 1) We consider the following special case of rational identity testing: Given a noncommutative ABP as white-box, whose edge labels are linear forms or inverses of linear forms, we show a deterministic polynomial-time algorithm to decide if the rational function computed by it is equivalent to zero in the free skew field Q . Given black-box access to the ABP, we give a deterministic quasi-polynomial time algorithm for this problem. 2) Amitsur's theorem implies that if a noncommutative polynomial f is nonzero on k x k matrices then, in fact, f(M_1,M_2,...,M_n) is invertible for some matrix tuple (M_1,M_2,...,M_n) in (M_k(ℚ))^n. While a randomized polynomial time algorithm to find such (M_1,M_2,...,M_n) given black-box access to f is simple, we obtain a deterministic s^{O(log d)} time algorithm for the problem with black-box access to f, where s is the minimum ABP size for f and d is the degree of f. 3) The Bresar-Klep Theorem states that the span of the range of any noncommutative polynomial f on k x k matrices over Q is one of the following: zero, scalar multiples of I_k, trace-zero matrices in M_k(Q), or all of M_k(Q). We obtain a deterministic polynomial-time algorithm to decide which case occurs, given white-box access to an ABP for f. We also give a deterministic s^{O(log d)} time algorithm given black-box access to an ABP of size s for f. Our algorithms work when k >= d. Our techniques are based on some automata theory combined with known techniques for noncommutative ABP identity testing [Ran Raz and Amir Shpilka, 2005; Michael A. Forbes and Amir Shpilka, 2013].

Mathematical modeling of a system plays a vital role in controller design, tuning to meet the desired specifications and process parameter optimization. In this paper, different identification algorithms are implemented in the four‐tank system to demonstrate the superiority of the recommended subspace system identification method (Numeric Algorithms for Subspace State Space System Identification [N4SID]) for the different input design. A system identification method was utilized to identify the model of a dynamic system. It is modeled using the First‐Principle Method (FPM), Prediction Error Method (PEM) and Numeric Algorithms for N4SID method. The identification is performed through MATLAB. The plant input and output is acquired through LabVIEW. The developed models are validated with a new dataset based on the model performance specifications: Mean Absolute Error (MAE), Root Mean Squared Error (RMSE) and Best Fit Rate (BFR). The results show that the proposed N4SID method with the amplitude‐modulated pseudorandom binary signal (APRBS) signal is better than the other methods with different inputs. © 2020 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.

Artificial intelligence and explanation: How, why, and when to explain black boxes