Deep Learning With PyTorch

Differential testing offers a promising strategy to alleviate the test oracle problem by comparing the test results between alternative implementations. However, existing differential testing techniques for deep learning (DL) libraries are limited by the key challenges of finding alternative implementations (called \(counterparts\) ) for a given API and subsequently generating diverse test inputs. To address the two challenges, this paper introduces DLL ens , an LLM-enhanced differential testing technique for DL libraries. The first challenge is addressed by an observation that DL libraries are commonly designed to support the computation of a similar set of DL algorithms. Therefore, the counterpart of a given API’s computation could be successfully synthesized through certain composition and adaptation of the APIs from another DL library. DLL ens incorporates a novel counterpart synthesis workflow, leveraging a large language model (LLM) to search for valid counterparts for differential testing. To address the second challenge, DLL ens incorporates a static analysis technique that extracts the path constraints from the implementations of a given API and its counterpart to guide diverse test input generation. The extraction is facilitated by LLM’s knowledge of the concerned DL library and its upstream libraries. DLL ens incorporates validation mechanisms to manage the LLM’s hallucinations in counterpart synthesis and path constraint extraction. We evaluate DLL ens on two popular DL libraries, TensorFlow and PyTorch. Our evaluation shows that DLL ens synthesizes counterparts for 1.84 times as many APIs as those found by state-of-the-art techniques on these libraries. Moreover, under the same time budget, DLL ens covers 7.23% more branches and detects 1.88 times as many bugs as state-of-the-art techniques on 200 randomly sampled APIs. DLL ens has successfully detected 71 bugs in recent TensorFlow and PyTorch libraries. Among them, 59 are confirmed by developers, including 46 confirmed as previously unknown bugs, and 10 of these previously unknown bugs have been fixed in the latest version of TensorFlow and PyTorch.

Recent breakthroughs in deep learning and artificial intelligence technologies have enabled numerous mobile applications. While traditional computation paradigms rely on mobile sensing and cloud computing, deep learning implemented on mobile devices provides several advantages. These advantages include low communication bandwidth, small cloud computing resource cost, quick response time, and improved data privacy. Research and development of deep learning on mobile and embedded devices has recently attracted much attention. This paper provides a timely review of this fast-paced field to give the researcher, engineer, practitioner, and graduate student a quick grasp on the recent advancements of deep learning on mobile devices. In this paper, we discuss hardware architectures for mobile deep learning, including Field Programmable Gate Arrays (FPGA), Application Specific Integrated Circuit (ASIC), and recent mobile Graphic Processing Units (GPUs). We present Size, Weight, Area and Power (SWAP) considerations and their relation to algorithm optimizations, such as quantization, pruning, compression, and approximations that simplify computation while retaining performance accuracy. We cover existing systems and give a state-of-the-industry review of TensorFlow, MXNet, Mobile AI Compute Engine (MACE), and Paddle-mobile deep learning platform. We discuss resources for mobile deep learning practitioners, including tools, libraries, models, and performance benchmarks. We present applications of various mobile sensing modalities to industries, ranging from robotics, healthcare and multimedia, biometrics to autonomous drive and defense. We address the key deep learning challenges to overcome, including low quality data, and small training/adaptation data sets. In addition, the review provides numerous citations and links to existing code bases implementing various technologies. These resources lower the user’s barrier to entry into the field of mobile deep learning.

The outbreak of a global pandemic called coronavirus has created unprecedented circumstances resulting into a large number of deaths and risk of community spreading throughout the world. Desperate times have called for desperate measures to detect the disease at an early stage via various medically proven methods like chest computed tomography (CT) scan, chest X-Ray, etc., in order to prevent the virus from spreading across the community. Developing deep learning models for analysing these kinds of radiological images is a well-known methodology in the domain of computer based medical image analysis. However, doing the same by mimicking the biological models and leveraging the newly developed neuromorphic computing chips might be more economical. These chips have been shown to be more powerful and are more efficient than conventional central and graphics processing units. Additionally, these chips facilitate the implementation of spiking neural networks (SNNs) in real-world scenarios. To this end, in this work, we have tried to simulate the SNNs using various deep learning libraries. We have applied them for the classification of chest CT scan images into COVID and non-COVID classes. Our approach has achieved very high F1 score of 0.99 for the potential-based model and outperforms many state-of-the-art models. The working code associated with our present work can be found here.

Cellular automata (CA) models are effective tools for simulating future urban expansion. With the widespread use of high‐resolution geospatial data for CA simulation, the computational intensity of CA models has increased. Additionally, due to the continuous development of CA modeling research, many scholars have made improvements to the models to enhance their simulation accuracy, resulting in an increasing computational complexity of the model. Consequently, the simulation task based on CA requires vast computing time and memory space. In recent years, deep learning (DL) has experienced rapid development. Many open‐source DL frameworks support graphic processing unit (GPU) parallel computing and provide efficient application programming interfaces (APIs) that can be easily called to handle tasks of interest. In this study, a high‐performance CA model was constructed based on the similarity between the neighborhood effect calculation process of the CA model and the convolutional process in a convolutional neural network (CNN). The convolution function in the DL library is used to calculate the neighborhood effect of the CA model to reduce the time and memory consumption of CA‐based simulation. The experimental results show that compared with the conventional CA model, the execution time of the GPU‐convolution‐CA model proposed in this study has been reduced by more than 98%.

AI-enabled applications are prolific today. Deep Learning (DL) libraries, such as PyTorch and Tensorflow, provide the building blocks for the AI components of these applications. As any piece of software, these libraries can be buggy. An impressive number of bug-finding techniques to address this problem have been proposed, but the lack of a curated set of reproducible bugs in DL libraries hinders credible evaluation of these techniques. We present BugsInDLLs, a database of curated reproducible bugs to fill that gap. Unique challenges exist in this context, such as installing drivers of specific CUDA versions to reproduce certain GPU-related bugs. Our dataset currently consists of 112 environments to reproduce bugs across three popular DL libraries, namely, JAX, Tensorflow, and PyTorch.

Deep Learning is a subfield of machine learning methods based on artificial neural networks. Thanks to the increased data availability and computational power, such as Graphic Process Units (GPU), training deep networks - a time-consuming process - became possible. Cloud computing is an excellent option to acquire the computational power to train these models since it provides elastic products with a pay-per-use model. Amazon Web Services (AWS), for instance, has GPU-based virtual machine instances in its catalog, which differentiates themselves by the GPU type, number of GPUs, and price per hour. The challenge consists in determining which instance is better for a specific deep learning problem. This paper presents the implications, in terms of runtime and cost, of running two different deep learning problems on AWS GPU-based instances, and it proposes a methodology, based on the previous study cases, that analyzes instances for deep learning algorithms by using the information provided by the Keras framework. Our experimental results indicate that, despite having a higher price per hour, the instances that contain the NVIDIA V100 GPUs (p3) are faster and usually less expensive to use than the instances that contain the NVIDIA K80 GPUs (p2) for the problems we analyzed. Also, the results indicate that the performance of both applications did not scale well with the number of GPUs and that increasing the batch size to improve scalability may affect the final model accuracy. Finally, the proposed methodology provides accurate cost and estimated runtime for the tested applications on different AWS instances with a small cost.

Deep learning (DL) has shown explosive growth in its application to bioinformatics and has demonstrated thrillingly promising power to mine the complex relationship hidden in large-scale biological and biomedical data. A number of comprehensive reviews have been published on such applications, ranging from high-level reviews with future perspectives to those mainly serving as tutorials. These reviews have provided an excellent introduction to and guideline for applications of DL in bioinformatics, covering multiple types of machine learning (ML) problems, different DL architectures, and ranges of biological/biomedical problems. However, most of these reviews have focused on previous research, whereas current trends in the principled DL field and perspectives on their future developments and potential new applications to biology and biomedicine are still scarce. We will focus on modern DL, the ongoing trends and future directions of the principled DL field, and postulate new and major applications in bioinformatics.

Deep learning has gained substantial popularity in recent years. Developers mainly rely on libraries and tools to add deep learning capabilities to their software. What kinds of bugs are frequently found in such software? What are the root causes of such bugs? What impacts do such bugs have? Which stages of deep learning pipeline are more bug prone? Are there any antipatterns? Understanding such characteristics of bugs in deep learning software has the potential to foster the development of better deep learning platforms, debugging mechanisms, development practices, and encourage the development of analysis and verification frameworks. Therefore, we study 2716 high-quality posts from Stack Overflow and 500 bug fix commits from Github about five popular deep learning libraries Caffe, Keras, Tensorflow, Theano, and Torch to understand the types of bugs, root causes of bugs, impacts of bugs, bug-prone stage of deep learning pipeline as well as whether there are some common antipatterns found in this buggy software. The key findings of our study include: data bug and logic bug are the most severe bug types in deep learning software appearing more than 48% of the times, major root causes of these bugs are Incorrect Model Parameter (IPS) and Structural Inefficiency (SI) showing up more than 43% of the times.We have also found that the bugs in the usage of deep learning libraries have some common antipatterns.

Deep Learning (DL) has become a prominent machine learning technique due to the availability of efficient computational resources in the form of Graphics Processing Units (GPUs), large-scale datasets and a variety of models. The newer generation of GPUs are being designed with special emphasis on optimizing performance for DL applications. Also, the availability of easy-to-use DL frameworks—like PyTorch and TensorFlow— has enhanced productivity of domain experts to work on their custom DL applications from diverse domains. However, existing Deep Neural Network (DNN) training approaches may not fully utilize the newly emerging powerful GPUs like the NVIDIA A100—this is the primary issue that we address in this paper. Our motivating analyses show that the GPU utilization on NVIDIA A100 can be as low as 43% using traditional DNN training approaches for small-to-medium DL models and input data size. This paper proposes AccDP—a data-parallel distributed DNN training approach—to accelerate GPU-based DL applications. AccDP exploits the Message Passing Interface (MPI) communication library coupled with the NVIDIA’s Multi-Process Service (MPS) to increase the amount of work assigned to parallel GPUs resulting in higher utilization of compute resources. We evaluate our proposed design on different small-to-medium DL models and input sizes on the state-of-the-art HPC clusters. By injecting more parallelism into DNN training using our approach, the evaluation shows up to 58% improvement in training performance on a single GPU and up to 62% on 16 GPUs compared to regular DNN training. Furthermore, we conduct an in-depth characterization to determine the impact of several DNN training factors and best practices—including the batch size and the number of data loading workers— to optimally utilize GPU devices. To the best of our knowledge, this is the first work that explores the use of MPS and MPI to maximize the utilization of GPUs in distributed DNN training.

Programming modern heterogeneous systems is becoming more and more challenging due to their complexity. To simplify software development for such architectures, more advanced compilers are being designed. Such compilers automatically optimize code and hide the complexity of the target heterogeneous architecture from the developer. An example of problems that these compilers need to solve is to decide whether to map (run) a piece of code on CPU or GPU (Graphics Processing Unit). State-of-the-art compilers use accurate optimization heuristics to solve such problems and decide about how to optimize a code automatically. Traditional machine learning and Deep Learning approaches have both been used to build such heuristics. While traditional machine learning is well suited for training on small datasets, it is not well suited for extracting a set of high-quality features, on the other hand, deep learning can automatically extract features, but it needs a large amount of data to get satisfactory results.In this paper, we propose a new machine-learning-based model that allows a compiler to automatically decide whether to map a piece of code to CPU or GPU. Our model uses traditional machine learning and deep learning by exploiting the advantages of each of them. We show that our proposed model, which requires a small amount of data and training time, matches and outperforms state-of-the-art pre-trained deep learning models that require large amounts of data.

Introduction to the special issue on deep learning approaches for machine translation

Computer vision and deep learning are clearly demonstrating a capability to create engaging cognitive applications and services. However, these applications have been mostly confined to powerful Graphic Processing Units (GPUs) or the cloud due to their demanding computational requirements. Cloud processing has obvious bandwidth, energy consumption and privacy issues. The Eyes of Things (EoT) is a powerful and versatile embedded computer vision platform which allows the user to develop artificial vision and deep learning applications that analyse images locally. In this article, we use the deep learning capabilities of an EoT device for a real-life facial informatics application: a doll capable of recognizing emotions, using deep learning techniques, and acting accordingly. The main impact and significance of the presented application is in showing that a toy can now do advanced processing locally, without the need of further computation in the cloud, thus reducing latency and removing most of the ethical issues involved. Finally, the performance of the convolutional neural network developed for that purpose is studied and a pilot was conducted on a panel of 12 children aged between four and ten years old to test the doll.

This study presents the development of a mobile identification system that detects biological butterfly characteristics through deep learning by capturing images. The challenge identified is that butterfly identification and recognition are difficult tasks because there are too many species, and it is hard to classify the types of butterfly species. Butterflies are also difficult to differentiate from each other, and limited studies are done using computer database referrals for butterflies’ characterization. This study aims to develop an automated computer program to easily identify the species of butterflies. Deep learning in image processing is programmed, which can control the qualification, segmentation, and classification of images and automatically detect butterfly characterization. The design system consists of three stages: capture, feature extraction, and butterfly recognition. Then, multiple recognition clues such as shape, color, texture, and size are extracted and analyzed to analyze and recognize the butterfly. This approach is faster and less complex than the previous approach. The result successfully presents a convolutional neural network (CNN) to classify images after training and characterization. The graphics processing unit (GPU) that trains the image dataset presents 86% image accuracy in the allocated time. This research is significant in such a way that new butterfly species will be automatically collected and stored on the online server. The information could be treasured as a valuable butterfly database.

With the rapid development of computer computing power, as an important method in the field of artificial intelligence, deep learning has amazing learning ability, especially in dealing with massive data, which makes deep learning in the fields of image recognition, image classification, natural language processing, data mining and unmanned driving, Has shown an extraordinary role. In previous studies, the style transfer algorithm has not developed well due to the poor computing power of Computer, the basic configuration of computer hardware can not meet the minimum requirements and the poor image effect after migration. However, with the development of computer hardware and the rapid change of GPU computing power, the style transfer network based on deep learning has become a hot issue in the study of style transfer in recent years. According to the research, although the traditional style transfer method can obtain the texture, color and other information of the style image, the model needs to be learned every time a new target image is generated, and the time cost during this period is very high. In this way, the trained model is not repeatable, and the generated image is often very random and can not get good results. Therefore, the emergence of style transfer methods based on deep learning solves the limitations of traditional style transfer methods. Style transfer methods based on deep learning are faster than traditional style transfer methods, and the generalization of the model is better. The style transfer algorithms of main neural networks are divided into two categories, Slow style transfer based on image iteration and fast style transfer based on model iteration. VGG network model can combine style image and content image, and greatly improve the style transfer efficiency of image.

Researchers analyze data, information, and knowledge through many sources, formats, and methods. The dominant data format includes text and images. In the healthcare industry, professionals generate a large quantity of unstructured data. The complexity of this data and the lack of computational power causes delays in analysis. However, with emerging deep learning algorithms and access to computational powers such as graphics processing unit (GPU) and tensor processing units (TPUs), processing text and images is becoming more accessible. Deep learning algorithms achieve remarkable results in natural language processing (NLP) and computer vision. In this study, we focus on NLP in the healthcare industry and collect data not only from electronic medical records (EMRs) but also medical literature and social media. We propose a framework for linking social media, medical literature, and EMRs clinical notes using deep learning algorithms. Connecting data sources requires defining a link between them, and our key is finding concepts in the medical text. The National Library of Medicine (NLM) introduces a Unified Medical Language System (UMLS) and we use this system as the foundation of our own system. We recognize social media’s dynamic nature and apply supervised and semi-supervised methodologies to generate concepts. Named entity recognition (NER) allows efficient extraction of information, or entities, from medical literature, and we extend the model to process the EMRs’ clinical notes via transfer learning. The results include an integrated, end-to-end, web-based system solution that unifies social media, literature, and clinical notes, and improves access to medical knowledge for the public and experts.