Evaluating API-Level Deep Learning Fuzzers: A Comprehensive Benchmarking Study

Differential testing offers a promising strategy to alleviate the test oracle problem by comparing the test results between alternative implementations. However, existing differential testing techniques for deep learning (DL) libraries are limited by the key challenges of finding alternative implementations (called \(counterparts\) ) for a given API and subsequently generating diverse test inputs. To address the two challenges, this paper introduces DLL ens , an LLM-enhanced differential testing technique for DL libraries. The first challenge is addressed by an observation that DL libraries are commonly designed to support the computation of a similar set of DL algorithms. Therefore, the counterpart of a given API’s computation could be successfully synthesized through certain composition and adaptation of the APIs from another DL library. DLL ens incorporates a novel counterpart synthesis workflow, leveraging a large language model (LLM) to search for valid counterparts for differential testing. To address the second challenge, DLL ens incorporates a static analysis technique that extracts the path constraints from the implementations of a given API and its counterpart to guide diverse test input generation. The extraction is facilitated by LLM’s knowledge of the concerned DL library and its upstream libraries. DLL ens incorporates validation mechanisms to manage the LLM’s hallucinations in counterpart synthesis and path constraint extraction. We evaluate DLL ens on two popular DL libraries, TensorFlow and PyTorch. Our evaluation shows that DLL ens synthesizes counterparts for 1.84 times as many APIs as those found by state-of-the-art techniques on these libraries. Moreover, under the same time budget, DLL ens covers 7.23% more branches and detects 1.88 times as many bugs as state-of-the-art techniques on 200 randomly sampled APIs. DLL ens has successfully detected 71 bugs in recent TensorFlow and PyTorch libraries. Among them, 59 are confirmed by developers, including 46 confirmed as previously unknown bugs, and 10 of these previously unknown bugs have been fixed in the latest version of TensorFlow and PyTorch.

AI-enabled applications are prolific today. Deep Learning (DL) libraries, such as PyTorch and Tensorflow, provide the building blocks for the AI components of these applications. As any piece of software, these libraries can be buggy. An impressive number of bug-finding techniques to address this problem have been proposed, but the lack of a curated set of reproducible bugs in DL libraries hinders credible evaluation of these techniques. We present BugsInDLLs, a database of curated reproducible bugs to fill that gap. Unique challenges exist in this context, such as installing drivers of specific CUDA versions to reproduce certain GPU-related bugs. Our dataset currently consists of 112 environments to reproduce bugs across three popular DL libraries, namely, JAX, Tensorflow, and PyTorch.

Deep learning has gained substantial popularity in recent years. Developers mainly rely on libraries and tools to add deep learning capabilities to their software. What kinds of bugs are frequently found in such software? What are the root causes of such bugs? What impacts do such bugs have? Which stages of deep learning pipeline are more bug prone? Are there any antipatterns? Understanding such characteristics of bugs in deep learning software has the potential to foster the development of better deep learning platforms, debugging mechanisms, development practices, and encourage the development of analysis and verification frameworks. Therefore, we study 2716 high-quality posts from Stack Overflow and 500 bug fix commits from Github about five popular deep learning libraries Caffe, Keras, Tensorflow, Theano, and Torch to understand the types of bugs, root causes of bugs, impacts of bugs, bug-prone stage of deep learning pipeline as well as whether there are some common antipatterns found in this buggy software. The key findings of our study include: data bug and logic bug are the most severe bug types in deep learning software appearing more than 48% of the times, major root causes of these bugs are Incorrect Model Parameter (IPS) and Structural Inefficiency (SI) showing up more than 43% of the times.We have also found that the bugs in the usage of deep learning libraries have some common antipatterns.

Software Engineering (SE) communities such as Stack Overflow have become unwelcoming, particularly through members' use of offensive language. Research has shown that offensive language drives users away from active engagement within these platforms. This work aims to explore this issue more broadly by investigating the nature of offensive language in comments posted by users in four prominent SE platforms - GitHub, Gitter, Slack and Stack Overflow (SO). It proposes an approach to detect and classify offensive language in SE communities by adopting natural language processing and deep learning techniques. Further, a Conflict Reduction System (CRS), which identifies offence and then suggests what changes could be made to minimize offence has been proposed. Beyond showing the prevalence of offensive language in over 1 million comments from four different communities which ranges from 0.07% to 0.43%, our results show promise in successful detection and classification of such language. The CRS system has the potential to drastically reduce manual moderation efforts to detect and reduce offence in SE communities.

In the last decade, deep learning has seen exponential growth due to rise in computational power as a result of graphics processing units (GPUs) and a large amount of data due to the democratization of the internet and smartphones. This chapter aims to throw light on both the theoretical aspects of deep learning and its practical aspects using PyTorch. The chapter primarily discusses new technologies using deep learning and PyTorch in detail. The chapter discusses the advantages of using PyTorch compared to other deep learning libraries. The chapter discusses some of the practical applications like image classification and machine translation. The chapter also discusses the various frameworks built with the help of PyTorch. PyTorch consists of various models that increases its flexibility and accessibility to a greater extent. As a result, many frameworks built on top of PyTorch are discussed in this chapter. The authors believe that this chapter will help readers in getting a better understanding of deep learning making neural networks using PyTorch.

Genotypes of varietal trials for identification of promising genotypes of sugarcane undergoes location-based and multiphase testing for both plant and ratoon crop. Identification depends upon analytical studies of data collected from the trials. Data on more than twenty characters such as germination%, tillers, shoots, NMC, fibre, brix, sucrose, CCS, cane yield, etc are collected frequently, starting from germination stage till harvesting at different stages of crop. It is a quite complex and time-consuming task and information about some important parameters may remain unnoticed by experts despite best efforts. Deep learning algorithms can be developed for such complex task to extract high level features from trial data and make intelligent decisions based on it for identifying promising genotypes. Deep learning works on the principle of artificial neural networks in which multiple layers of processing are used to extract progressively higher-level features from data. In this paper, we have demonstrated use of deep learning for identification of promising genotypes. We have developed sequential model using tensorflow libraries in Python programming platform. To construct, train and test deep learning model, datasets of 181 genotypes accepted in coordinated trials of crop improvement programme for the duration 2016-21 have been used. Model uses crop characters viz. cane yield, sucrose%, CCS%, and CCS yield along with score of monitoring and red rot screening. Data management practices allowed to pre-process data for learning and testing model from it. Deep learning in this study consists of input layer, two hidden layers and output layer. Output classes are ’Promising” and “Non-promising” in binary form corresponding to promising genotype or otherwise. Model performed well with accuracy of prediction worked out to be 91.67% with loss value as 0.2832, while F-measure for both promising and non-promising genotypes got a high equivalent score of 0.91 and 0.92 respectively.

Deep learning subsumes algorithms that automatically learn compositional representations. The ability of these models to generalize well has ushered in tremendous advances in many fields such as natural language processing (NLP). Recent research in the software engineering (SE) community has demonstrated the usefulness of applying NLP techniques to software corpora. Hence, we motivate deep learning for software language modeling, highlighting fundamental differences between state-of-the-practice software language models and connectionist models. Our deep learning models are applicable to source code files (since they only require lexically analyzed source code written in any programming language) and other types of artifacts. We show how a particular deep learning model can remember its state to effectively model sequential data, e.g., Streaming software tokens, and the state is shown to be much more expressive than discrete tokens in a prefix. Then we instantiate deep learning models and show that deep learning induces high-quality models compared to n-grams and cache-based n-grams on a corpus of Java projects. We experiment with two of the models' hyper parameters, which govern their capacity and the amount of context they use to inform predictions, before building several committees of software language models to aid generalization. Then we apply the deep learning models to code suggestion and demonstrate their effectiveness at a real SE task compared to state-of-the-practice models. Finally, we propose avenues for future work, where deep learning can be brought to bear to support model-based testing, improve software lexicons, and conceptualize software artifacts. Thus, our work serves as the first step toward deep learning software repositories.

Recent breakthroughs in deep learning and artificial intelligence technologies have enabled numerous mobile applications. While traditional computation paradigms rely on mobile sensing and cloud computing, deep learning implemented on mobile devices provides several advantages. These advantages include low communication bandwidth, small cloud computing resource cost, quick response time, and improved data privacy. Research and development of deep learning on mobile and embedded devices has recently attracted much attention. This paper provides a timely review of this fast-paced field to give the researcher, engineer, practitioner, and graduate student a quick grasp on the recent advancements of deep learning on mobile devices. In this paper, we discuss hardware architectures for mobile deep learning, including Field Programmable Gate Arrays (FPGA), Application Specific Integrated Circuit (ASIC), and recent mobile Graphic Processing Units (GPUs). We present Size, Weight, Area and Power (SWAP) considerations and their relation to algorithm optimizations, such as quantization, pruning, compression, and approximations that simplify computation while retaining performance accuracy. We cover existing systems and give a state-of-the-industry review of TensorFlow, MXNet, Mobile AI Compute Engine (MACE), and Paddle-mobile deep learning platform. We discuss resources for mobile deep learning practitioners, including tools, libraries, models, and performance benchmarks. We present applications of various mobile sensing modalities to industries, ranging from robotics, healthcare and multimedia, biometrics to autonomous drive and defense. We address the key deep learning challenges to overcome, including low quality data, and small training/adaptation data sets. In addition, the review provides numerous citations and links to existing code bases implementing various technologies. These resources lower the user’s barrier to entry into the field of mobile deep learning.

Supply chain management (SCM) is a fast growing and largely studied field of research. Forecasting of the required materials and parts is an important task in companies and can have a significant impact on the total cost. To have a reliable forecast, some advanced methods such as deep learning techniques are helpful. The main goal of this chapter is to forecast the unit sales of thousands of items sold at different chain stores located in Ecuador with holistic techniques. Three deep learning approaches including artificial neural network (ANN), convolutional neural network (CNN), and long short-term memory (LSTM) are adopted here for predictions from the Corporación Favorita grocery sales forecasting dataset collected from Kaggle website. Finally, the performances of the applied models are evaluated and compared. The results show that LSTM network tends to outperform the other two approaches in terms of performance. All experiments are conducted using Python's deep learning library and Keras and Tensorflow packages.

fastai is a deep learning library which provides practitioners with high-level components that can quickly and easily provide state-of-the-art results in standard deep learning domains, and provides researchers with low-level components that can be mixed and matched to build new approaches. It aims to do both things without substantial compromises in ease of use, flexibility, or performance. This is possible thanks to a carefully layered architecture, which expresses common underlying patterns of many deep learning and data processing techniques in terms of decoupled abstractions. These abstractions can be expressed concisely and clearly by leveraging the dynamism of the underlying Python language and the flexibility of the PyTorch library. fastai includes: a new type dispatch system for Python along with a semantic type hierarchy for tensors; a GPU-optimized computer vision library which can be extended in pure Python; an optimizer which refactors out the common functionality of modern optimizers into two basic pieces, allowing optimization algorithms to be implemented in 4–5 lines of code; a novel 2-way callback system that can access any part of the data, model, or optimizer and change it at any point during training; a new data block API; and much more. We used this library to successfully create a complete deep learning course, which we were able to write more quickly than using previous approaches, and the code was more clear. The library is already in wide use in research, industry, and teaching.

With the development of information and technology, especially with the boom in big data, healthcare support systems are becoming much better. Patient data can be collected, retrieved, and stored in real time. These data are valuable and meaningful for monitoring, diagnosing, and further applications in data analysis and decision-making. Essentially, the data can be divided into three types, namely, statistical, image-based, and sequential data. Each type has a different method of retrieval, processing, and deployment. Additionally, the application of machine learning (ML) and deep learning (DL) in healthcare support systems is growing more rapidly than ever. Numerous high-performance architectures are proposed to optimize decision-making. As reliability and stability are the most important factors in the healthcare support system, enhancing the predicted performance and maintaining the stability of the model are always the top priority. The main idea of our study comes from ensemble techniques. Numerous studies and data science competitions show that by combining several weak models into one, ensemble models can attain outstanding performance and reliability. We propose three deep ensemble learning (DEL) approaches, each with stable and reliable performance, that are workable on the above-mentioned data types. These are deep-stacked generalization ensemble learning, gradient deep learning boosting, and deep aggregation learning. The experiment results show that our proposed approaches achieve more vigorous and reliable performance than traditional ML and DL techniques on statistical, image-based, and sequential benchmark datasets. In particular, on the Heart Disease UCI dataset, representing the statistical type, the gradient deep learning boosting approach dominates the others with accuracy, recall, F1-score, Matthews correlation coefficient, and area under the curve values of 0.87, 0.81, 0.83, 0.73, and 0.91, respectively. On the X-ray dataset, representing the image-based type, the deep aggregation learning approach shows the highest performance with values of 0.91, 0.97, 0.93, 0.80, and 0.94, respectively. On the Depresjon dataset, representing the sequence type, the deep-stacked generalization ensemble learning approach outperforms the others with values of 0.91, 0.84, 0.86, 0.8, and 0.94, respectively. Overall, we conclude that applying DL models using our proposed approaches is a promising method for the healthcare support system to enhance prediction and diagnosis performance. Furthermore, our study reveals that these approaches are flexible and easy to apply to achieve optimal performance.

Recently biometric systems have shown improved capabilities because of the remarkable success of deep learning in solving various computer vision tasks. In ear recognition, the use of deep learning techniques is seldom due to training data scarcity. The existing work has shown poor performance as the majority of techniques are based on either handcraft features or pre-trained models. Besides this, transfer-learning has also shown poor performance because of the diversity among the tasks. To circumvent the existing issues, in this work, we have presented an end-to-end framework for ear recognition. It consist of the Ear Mask Extraction (EME) network to segment the ear, a normalization algorithm to align the ear, and a novel siamese-based CNN (CG-ERNet) for deep ear feature learning. CG-ERNet exploits domain-specific knowledge by using Curvature Gabor filters and uses triplet loss, triplet selection, and adaptive margin for better convergence of the loss. For comparative analysis, we trained state-of-the-art deep learning models like Face-Net, VGG19, ResNet50, Inception, Exception, and Mobile-Net for ear-recognition. The performance is assessed using five well-known evaluation metrics. In the extensive experimentation, our proposed model (CG-ERNet) outperformed the deep learning models and handcrafted feature based methods on four different, publicly available, benchmark datasets. To make the results more interpretable, we employ the t-SNE visualization of learned features. Additionally, our proposed method has shown robustness to various environmental challenges like Gaussian noise, Gaussian blur, up to ± 30 degrees of rotation, and 20% of occlusion.

Remote-sensing images comprise massive amount of spatial and semantic data that can be employed for several applications. Presently, deep learning (DL) models for RS image processing become a familiar research area. Due to the advancements of recent satellite imaging sensors , the issue of huge amount of data processing becomes a challenging problem. To accomplish this, deep transfer learning (DTL) models are developed to resolve the semantic gap among various datasets This study develops a new DTL-based fusion model for environmental remote-sensing image classification, called DTLF-ERSIC technique. The proposed technique focuses on the design of fusion model to combine multiple feature vectors and thereby attains maximum classification performance. The DTLF-ERSIC technique incorporates the entropy-based fusion of three feature extraction techniques, namely, Discrete Local Binary Pattern (DLBP), Residual Network (ResNet50), and EfficientNet models. Besides, a rain optimization algorithm (ROA) with fuzzy rule-based classifier (FRC) is applied to predict the class labels of the test RS images and shows the novelty of the work. A comprehensive experimental analysis of the DTLF-ERSIC technique takes place on benchmark dataset and examined the results in terms of different performance measures. The simulation results reported the supremacy of the DTLF-ERSIC technique over the recent state-of-art techniques.

Recent years have seen a surge of interest in deep learning models that outperform other machine learning algorithms on benchmarks across many disciplines. Most existing deep learning libraries facilitate the development of neural nets by providing a mathematical framework that helps users implement their models more efficiently. This still represents a substantial investment of time and effort, however, when the intention is to compare a range of competing models quickly for a specific task. We present DEFIne, a fluent interface DSL for the specification, optimisation and evaluation of deep learning models. The fluent interface is implemented through method chaining. DEFIne is embedded in Python and is build on top of its most popular deep learning libraries, Keras and Theano. It extends these with common operations for data pre-processing and representation as well as visualisation of datasets and results. We test our framework on three benchmark tasks from different domains: heart disease diagnosis, hand-written digit recognition and weather forecast generation. Results in terms of accuracy, runtime and lines of code show that our DSL achieves equivalent accuracy and runtime to state-of-the-art models, while requiring only about 10 lines of code per application.

The newly developed double disk magnetic abrasive finishing (DDMAF) process has been recently executed to improve the finishing efficacy of polished silicon wafers. It is obvious to have induction of errors in the experimental processes for predicting the output response in form of surface roughness of polished silicon wafers. This work has been presented to model the actual experimental values of surface roughness of polished mono-crystalline silicon wafer with the predicted data of surface roughness by utilizing machine and deep learning libraries in Python’s JupyterLab. The experimental data of 31 experiments for surface roughness of polished mono-crystalline silicon wafer with chemically accompanied unconventional machining process known as DDMAF has been used here. The machine learning algorithm chosen for the specific data distribution was a polynomial regression from scikit-learn API. The data was split into 85% training and 15% testing for the same. The data was further modeled in Keras sequential neural nets, with the experimental data spitted into 85% training and 15% validation.KeywordsDouble disk magnetic abrasive finishing (DDMAF)Machine learning (ML)Deep learning (DL)