Multiplication of medium-density matrices using TensorFlow on multicore CPUs

Enabled by a rich ecosystem of Machine Learning (ML) libraries, programming using learned models , i.e., Software-2.0 , has gained substantial adoption. However, we do not know what challenges developers encounter when they use ML libraries. With this knowledge gap, researchers miss opportunities to contribute to new research directions, tool builders do not invest resources where automation is most needed, library designers cannot make informed decisions when releasing ML library versions, and developers fail to use common practices when using ML libraries. We present the first large-scale quantitative and qualitative empirical study to shed light on how developers in Software-2.0 use ML libraries, and how this evolution affects their code. Particularly, using static analysis we perform a longitudinal study of 3,340 top-rated open-source projects with 46,110 contributors. To further understand the challenges of ML library evolution, we survey 109 developers who introduce and evolve ML libraries. Using this rich dataset we reveal several novel findings. Among others, we found an increasing trend of using ML libraries: The ratio of new Python projects that use ML libraries increased from 2% in 2013 to 50% in 2018. We identify several usage patterns including the following: (i) 36% of the projects use multiple ML libraries to implement various stages of the ML workflows, (ii) developers update ML libraries more often than the traditional libraries , (iii) strict upgrades are the most popular for ML libraries among other update kinds, (iv) ML library updates often result in cascading library updates, and (v) ML libraries are often downgraded (22.04% of cases). We also observed unique challenges when evolving and maintaining Software-2.0 such as (i) binary incompatibility of trained ML models and (ii) benchmarking ML models. Finally, we present actionable implications of our findings for researchers, tool builders, developers, educators, library vendors, and hardware vendors.

Enhancements to array processors in the form of broadcast buses have been proposed for improving the speeds of algorithms in linear algebra, image processing and computational geometry. In this paper, we consider certain practical issues in such arrays which include fewer processors connected to the broadcast buses and finite time, namely log N time for broadcasting data to N processors. We propose a modified broadcast bus VLSI architecture which consists of a √N × √N mesh connected structure and a hierarchy of broadcast buses in each row and in each column such that every bus has k PE's. That is, in any row or column, in the first level there are \(\frac{{\sqrt N }}{k}\)buses with groups of k PE's connected to each bus. One PE from each of this group is connected to second level of buses in a similar manner. This is recursively done until there are only k PE's left which are connected by a broadcast bus. With this architecture, maximum, minimum, or sum of N values can be found in O(log N), median row of a binary picture in O(log N), shortest distance between two points in O(log N). This architecture is well suited for parallel processing of applications in Linear Algebra, Image Processing, Computational Geometry and Numerical Computations. These restricted connections to buses reduces the I/O ports significantly, and therefore, is more suitable for VLSI implementation.

Thanks to their massive computational power and their SIMT computational model, Graphics Processing Units (GPUs) have been successfully used to accelerate a wide variety of regular applications (linear algebra, stencil computations, image processing and bioinformatics algorithms, among others). However, many established and emerging problems are based on irregular data structures, such as graphs. Examples can be drawn from different application domains: networking, social networking, machine learning, electrical circuit modeling, discrete event simulation, compilers, and computational sciences. It has been shown that irregular applications based on large graphs do exhibit runtime parallelism; moreover, the amount of available parallelism tends to increase with the size of the datasets. In this work, we explore an implementation space for deploying a variety of graph algorithms on GPUs. We show that the dynamic nature of the parallelism that can be extracted from graph algorithms makes it impossible to find an optimal solution. We propose a runtime system able to dynamically transition between different implementations with minimal overhead, and investigate heuristic decisions applicable across algorithms and datasets. Our evaluation is performed on two graph algorithms: breadth-first search and single-source shortest paths. We believe that our proposed mechanisms can be extended and applied to other graph algorithms that exhibit similar computational patterns.

We use the computing technology digital image processing (DIP) to enhance the teaching of linear algebra so as to make the course more visual and interesting. Certainly, this visual approach by using technology to link linear algebra to DIP is interesting and unexpected to both students as well as many faculty.

Image modeling in terms of physical/mathematical formation and the human visual system plays an important role in developing new image processing techniques. Parameters such as scale, orientation, texture, morphology, and color must be taken into consideration in order to solve complicated image processing problems efficiently. Although the classical linear image processing (CLIP) approach has played a central role in image processing, it is not necessarily the best and even the right choice. General linear image processing (GLIP) approaches based on different image modeling techniques have been studied to overcome some of the problems associated with CLIP. Indeed, using general linear algebra, it is possible to develop entirely new general linear operators and transforms. It is even possible to define entirely new general linear operations (addition, scalar multiplication, convolution, etc.) in order to describe how images are combined, amplified, transformed, analyzed, compared, and measured.

Today's linear algebra and machine learning applications (ML) continue to grow in size and complexity, placing rapidly increasing demands on the underlying hardware and software systems. To address these issues, hardware designers have proposed using custom accelerators explicitly designed for these demanding workloads. However, realizing the full potential of custom accelerators requires effective hardware/software (HW/SW) co-design strategies that enable seamless integration and optimization across both domains, which is currently a major challenge in accelerator development. This thesis presents an integrated solution to facilitate HW/SW accelerator design. We also address issues in accelerator deployment, enabling rapid prototyping, integrated benchmarking, and comprehensive performance analysis of custom accelerators. In this thesis, we demonstrate the value of a lightweight system modeling library integrated into the build/execution environment of a production ML runtime engine, leveraging TensorFlow Lite for deployment. We explore custom accelerators performance by analyzing the trade-offs between various design choices, including parameter settings and accelerator configurations, from small to large scale, to identify the most effective combinations for specific workloads. Secondly, we employ the Multi-Level Intermediate Representation (MLIR) compiler framework to automatically partition host code from accelerator code, pre-optimizing the latter for improved high-level synthesis (HLS) designs and high-quality accelerated kernels. Lastly, we present compiler extensions to automate the generation and optimization of communication between the host CPU and AXI-based accelerators. We present novel solutions that enable more efficient and effective design space exploration, optimization, and deployment of custom accelerators. The utility of these approaches is demonstrated through experiments with specific accelerator designs and key linear algebra and ML workloads. Most importantly, these solutions empower high-level language users, such as domain scientists, to participate in the design of new accelerator features.--Author's abstract

Modern software systems are increasingly including machine learning (ML) as an integral component. However, we do not yet understand the difficulties faced by software developers when learning about ML libraries and using them within their systems. To fill that gap this thesis reports on a detailed (manual) examination of 3,243 highly-rated Q&A posts related to ten ML libraries, namely Tensorflow, Keras, scikitlearn, Weka, Caffe, Theano, MLlib, Torch, Mahout, and H2O, on Stack Overflow, a popular online technical Q&A forum. Our findings reveal the urgent need for software engineering (SE) research in this area. The second part of the thesis particularly focuses on understanding the Deep Neural Network (DNN) bug characteristics. We study 2,716 high-quality posts from Stack Overflow and 500 bug fix commits from Github about five popular deep learning libraries Caffe, Keras, Tensorflow, Theano, and Torch to understand the types of bugs, their root causes and impacts, bug-prone stage of deep learning pipeline as well as whether there are some common antipatterns found in this buggy software. While exploring the bug characteristics, our findings imply that repairing software that uses DNNs is one such unmistakable SE need where automated tools could be beneficial; however, we do not fully understand challenges to repairing and patterns that are utilized when manually repairing DNNs. So, the third part of this thesis presents a comprehensive study of bug fix patterns to address these questions. We have studied 415 repairs from Stack Overflow and 555 repairs from Github for five popular deep learning libraries Caffe, Keras, Tensorflow, Theano, and Torch to understand challenges in repairs and bug repair patterns. Our key findings reveal that DNN bug fix patterns are distinctive compared to traditional bug fix patterns and the most common bug fix patterns are fixing data dimension and neural network connectivity. Finally, we propose an automatic technique to detect ML Application Programming Interface (API) misuses. We started with an empirical study to understand ML API misuses. Our study shows that ML API misuse is prevalent and distinct compared to non-ML API misuses. Inspired by these findings, we contributed Amimla (Api Misuse In Machine Learning Apis) an approach and a tool for ML API misuse detection. Amimla relies on several technical innovations. First, we proposed an abstract representation of ML pipelines to use in misuse detection. Second, we proposed an abstract representation of neural networks for deep learning related APIs. Third, we have developed a representation strategy for constraints on ML APIs. Finally, we have developed a misuse detection strategy for both single and multi-APIs. Our experimental evaluation shows that Amimla achieves a high average accuracy of ∼80% on two benchmarks of misuses from Stack Overflow and Github.

The changes in the housing market are not only related to human beings’ daily life, but also have an important impact on the national economy. The prediction of housing price is one of the most widely concerned topics, which is linked to the formulation of national real estate policies and the analysis of the economic situation. In this context, this paper takes housing price prediction as the topic, selects the Eames housing price dataset in Iowa, and uses supervised multiple linear regression and machine learning algorithm to train and test the real estate price prediction model. Among them, there are 79 explanatory variables, which are related to housing attributes and the explanatory variable is housing price. 1460 data is included in the training set and 1459 in the test set. In the part of machine learning algorithm, PaddlePaddle deep learning framework is used in this paper to train and test the models with the help of AI Studio platform. The experimental results show that the scatter plots of the real values are clustered and distributed on both sides of the predicted line, and their direct differences are within 30 points. According to the analysis, the real estate price prediction model based on linear regression and machine learning is reliable and stable. This paper aims to provide some suggestions for the housing price prediction. These results shed light on guiding the reference direction for investors, so as to guide the formulation of relevant policies.

With the introduction of more powerful and massively parallel embedded processors, embedded systems are becoming HPC-capable. Heterogeneous on-chip systems (SoC) that couple a general-purposehost processor to a many-core accelerator are becoming more and more widespread, and provide tremendous peak performance/watt, well suited to execute HPC-class programs. The increased computation potential is however traded off for ease programming. Application developers are indeed required to manually deal with outlining code parts suitable for acceleration, parallelize them efficiently over many available cores, and orchestrate data transfers to/from the accelerator. In addition, since most many-cores are organized as a collection ofclusters, featuring fast local communication but slow remote communication (i.e., to another cluster's local memory), the programmer should also take care of properly mapping the parallel computation so as to avoid poor data locality. OpenMP v4.0 introduces new constructs for computation offloading, as well as directives to deploy parallel computation in a cluster-aware manner. In this paper we assess the effectiveness of OpenMP v4.0 at exploiting the massive parallelism available in embedded heterogeneous SoCs, comparing to standard parallel loops over several computation-intensive applications from the linear algebra and image processing domains.

The Dynamic, Exascale Global Address Space programming environment (DEGAS) project will develop the next generation of programming models and runtime systems to meet the challenges of Exascale computing. The Berkeley part of the project concentrated on communication-optimal code generation to optimize speed and energy efficiency by reducing data movement. Our work developed communication lower bounds, and/or communication avoiding algorithms (that either meet the lower bound, or do much less communication than their conventional counterparts) for a variety of algorithms, including linear algebra, machine learning and genomics. The Berkeley part of the project concentrated on communication-optimal code generation to optimize speed and energy efficiency by reducing data movement. Our work developed communication lower bounds, and/or communication avoiding algorithms (that either meet the lower bound, or do much less communication than their conventional counterparts) for a variety of algorithms, including linear algebra, machine learning and genomics.

The rapid growth of large-scale machine learning (ML) models has led numerous commercial companies to utilize ML models for generating predictive results to help business decision-making. As two primary components in traditional predictive pipelines, data processing, and model predictions often operate in separate execution environments, leading to redundant engineering and computations. Additionally, the diverging mathematical foundations of data processing and machine learning hinder cross-optimizations by combining these two components, thereby overlooking potential opportunities to expedite predictive pipelines. In this paper, we propose an operator fusion method based on GPU-accelerated linear algebraic evaluation of relational queries. Our method leverages linear algebra computation properties to merge operators in machine learning predictions and data processing, significantly accelerating predictive pipelines by up to 317x. We perform a complexity analysis to deliver quantitative insights into the advantages of operator fusion, considering various data and model dimensions. Furthermore, we extensively evaluate linear algebra query processing and operator fusion utilizing the widely-used Star Schema and TPC-DI benchmarks. Through comprehensive evaluations, we demonstrate the effectiveness and potential of our approach in improving the efficiency of data processing and machine learning workloads on modern hardware.

In recent studies, drowsiness of drivers while driving is seen as one of the most important causes of deaths due to traffic accidents. Therefore, in this study, an algorithm that determines driver fatigue by examining the driver's eye conditions in real-time is proposed. Python versions of OpenCV (Open Source Computer Vision Library) and machine learning libraries were used to create this proposed algorithm. In the study, the face and eyes were detected with the Cascade classifier using Haar features. Detected eyes were classified as “open” “closed” or “half open” with the model trained by the SVM (Support Vector Machine) method according to the HOG (Histogram of Oriented Gradient) feature descriptor. Driver fatigue was decided according to the PERCLOS (Percentage of Eyelid Closure) method, which examines the percentage of eyelid closure over time. In the proposed system, fatigue was detected according to the time the driver's eyes were in the “half-open” or “closed” state, and a related user interface was designed and audio and visual warnings were given to the driver.

Machine learning (ML) has been leveraged to tackle a diverse range of tasks in almost all branches of nuclear engineering. Many of the successes in ML applications can be attributed to the recent performance breakthroughs in deep learning and the growing availability of computational power, data, and easy-to-use ML libraries. However, these empirical successes have often outpaced our formal understanding of the ML algorithms. An important but underrated area is the uncertainty quantification (UQ) of ML. ML-based models are subject to approximation uncertainty when they are used to make predictions due to sources including but not limited to, data noise, data coverage, extrapolation, imperfect model architecture, and the stochastic training process. The goal of this paper is to clearly explain and illustrate the importance of the UQ of ML. We elucidate the differences in the basic concepts of UQ of physics-based models and data-driven ML models. Various sources of uncertainties in physical modeling and data-driven modeling are discussed, demonstrated, and compared. We also present and demonstrate a few techniques to quantify the ML prediction uncertainties, including Monte Carlo dropout, deep ensemble, Bayesian neural networks, Gaussian processes, and conformal prediction. Finally, we discuss the need for building a verification, validation, and UQ framework to establish ML credibility.

BackgroundLow-cost sensors (LCS) are widely used for air quality monitoring, but their accuracy depends on proper calibration. This study compares linear regression (LR) and machine learning (ML) techniques, particularly random forest (RF), to determine optimal calibration strategies.ObjectivesThis study aims to compare the effectiveness of LR and RF models in calibrating the Plantower PMS 3003 sensor under different environmental conditions. It also explores ways to streamline calibration efforts whilemaintaining accuracy.MethodsSensor data were collected in a controlled laboratory setting, with measurements compared against a reference monitor. LR and RF models were developed to calibrate the sensor, and their performance was evaluated based on RMSE, R2, and bias. Additionally, the study examined whether using fewer sensors for training could still produce reliable calibration models.ResultsBoth LR and RF models demonstrated strong calibration performance. LR models were effective for low to moderate PM2.5 concentrations and required fewer computational resources, making them suitable for large-scale monitoring with limited resources. RF models captured nonlinear relationships, showing superior accuracy at high PMconcentrations and in conditions with high relative humidity. The findings suggest that LR models trained on smallerdatasets can achieve practical accuracy, reducing the need for extensive individual sensor calibration.ConclusionsThe selection of a calibration model should be guided by study-specific requirements, including environmental conditions and resource availability. LR models are recommended for large-scale studies with constrained resources, while RF models may offer advantages in high-exposure environments due to their ability to model complex interactions. This study is the first to explore reducing sensor calibration efforts while maintaining accuracy, highlighting the potential for optimized strategies in resource-limited settings. Future research should validate these findings in real-world deployments to further refine calibration models for LCS applications.Graphical

Linear algebra is sub field of mathematics and contains matrix's, Operations on dataset, Vectors. Linear algebra is core base for purely statistics and mathematics person to achieve goals with basic logics of linear algebra using machine learning. As a machine learning aspirant, the aspirant should be best at linear algebra to work on creating a new module or bundle. In other areas, such as statistics, engineering, research, linear algebra plays an important role and has strongest impact Although solving any problem mathematically with linear algebra is crucial for experienced engineers, linear algebra is crucial in machine learning, and in the case of any practical implementation of linear algebra, it is quite probable that the technical person may not have mathematical knowledge. Easier implementation is provided by linear algebra. Coronavirus disease (COVID-19) has raised urgent questions regarding minimizing and improving effective analytical strategies for analyzing information obtained from standardized protocols. In order to understand what quality of care interventions are the most successful and to test them as quickly as possible, it becomes necessary to evaluate available data immediately. It is necessary to integrate and clean up data from large multi-center hospitals and needs sophisticated data processing. To collect and store their data in a standardized manner, case report forms (CRF) for patients with suspected or confirmed COVID-19 are required. The goal of this study is to determine if it is possible to avoid or postpone local outbreaks of COVID-19 by travel restrictions from abroad. The problem statement is that we are unable to gather clear suspicious data from big data of covid-19 patients, and during the corona outbreak it becomes difficult to distinguish symptotic and asymptotic patients. The number of beds required or ventilators needed for COVID-19 cases at this point is difficult to estimate.