An Overview of Organs-on-Chips Based on Deep Learning.

Deep (machine) learning in recent years has significantly increased the predictive modeling strengths for applications in the areas of computer vision, speech recognition, and natural language processing. Being heavily data-driven, conventional general-purpose deep learning methods mainly rely on training data to learn task-specific data representations. In this thesis, we focus on leveraging novel learning algorithms in deep learning to tackle visual recognition challenges at the three key levels -- namely pixel, object, and image levels. Throughout the thesis, we demonstrate how each and every of our proposed methods can address a specific visual recognition problem. Firstly, we apply deep learning to improve the saliency detection performance of deep networks. A coarse saliency map of the input image's salient object is initially predicted by a convolutional-deconvolutional network (or equivalently a fully convolutional network). Following that, the coarse saliency map is sequentially refined by a separate network called Recurrent Attentional Convolutional-Deconvolutional Network (RACDNN) that shares its network weights across its sequential steps. At every sequential (temporal) step, the same RACDNN is applied to a newly selected/attended image sub-region and it performs saliency map refinement for that particular sub-region. Despite having many learnable network parameters and no access to additional training data, RADCNN can be trained well and it outperforms the "single-step" (coarse saliency detection) baseline and state-of-the-art saliency detection methods by large margins, on several saliency detection benchmark datasets. Secondly, we introduce auto-encoder weight transfer networks (AE-WTN+), a deep model for scaling up object detection. Conventional WTN transfers object class knowledge from the classification weights of a pretrained large-scale image classification network, to the classification weights of an object detection network. However, they suffer from under-fitting and are not able to preserve the rich object class information of novel classes. In this work, we propose a new WTN method -- AE-WTN+ which makes use of normalization techniques to ease network optimization. Inputs and intermediate features are normalized to improve trainability for reduced under-fitting. More importantly, a novel auto-encoding (reconstruction) loss is proposed to encourage information preservation of all classes in the weight predictions of AE-WTN+. This enables the predicted weights of AE-WTN+ to maintain the class discriminability and neighbourhood relationships contained in the pre-trained large-scale classification weights. Experiments on large-scale detection datasets validate the effectiveness of AE-WTN+ on both seen and novel classes. Thirdly, we improve the efficiency of cross-layer network connections for very deep convolutional network in the context of image classification. This gives rise to a new convolutional network architecture called Deluge Networks (DelugeNets) which pass the output (feature activations) of any preceding composite network layer as direct inputs to all succeeding composite layers. For the incoming feature activations of preceding composite layers, the succeeding composite layers first apply rather lightweight cross-layer depthwise convolutional weights on them for feature aggregation. The heavy lifting is carried out by the last convolutional weight layer in each preceding composite layer that performs cross-channel convolutions. The proposed technique enables unimpeded information flow among many layers in very deep DelugeNets with great efficiency. Experiments on small-scale and large-scale datasets show that DelugeNets perform competitively to state-of-the-art deep convolutional networks at lower computational and parameter costs. Fourthly, we propose Stochastic Downsampling (SDPoint), a method to overcome a major limitation suffered by conventional deep convolutional networks -- their computational costs during inference are fixed and cannot be changed. SDPoint downsamples the feature maps at a randomly selected point (layer index) in the network hierarchy during training. The many downsampling configurations known as SDPoint instances each entail a unique downsampling point and downsampling ratio. The same convolutional network weights are shared across the different SDPoint instances that downsample feature maps differently. During inference, the computational cost of a network trained with SDPoint can be instantaneously adjusted to fit a given computational budget, by manually selecting an appropriate SDPoint instance. Additionally, sharing weights between different feature map scales provides significant regularization benefits, making the networks trained with SDPoint less sensitive to scales and more competent. In image classification experiments, SDPoint demonstrates significant cost-accuracy and computational advantages, over independently-trained (non-weight-shared) baselines with various inference costs.

Great progress has been made in deep learning over the past few years, which drives the deployment of deep learning–based applications into cyber-physical systems. But the lack of interpretability for deep learning models has led to potential security holes. Recent research has found that deep neural networks are vulnerable to well-designed input examples, called adversarial examples . Such examples are often too small to detect, but they completely fool deep learning models. In practice, adversarial attacks pose a serious threat to the success of deep learning. With the continuous development of deep learning applications, adversarial examples for different fields have also received attention. In this article, we summarize the methods of generating adversarial examples in computer vision, speech recognition, and natural language processing and study the applications of adversarial examples. We also explore emerging research and open problems.

Nowadays, deep learning has been widely used in various fields, and great breakthroughs have been made in computer vision, speech recognition and natural language processing. Based on the deep learning algorithm, this paper studies the general optimization technology of OpenCL based on FPGA in detail, and puts forward a targeted hardware design scheme, which has the characteristics of high performance, low power consumption and programmability. By analyzing the algorithm of forward calculation and reverse calculation in convolutional neural network, the hardware acceleration module is designed. Compared with CPU, GPU and other platforms, FPGA has obvious advantages in power consumption, which proves that the scheme proposed in this paper has good feasibility and accuracy.

For the past few years, gender classification has been an active area of study and researchers have been putting a lot of effort to contribute quality research in this area. There is a big potential field of study as it can be used in monitoring, surveillance and human-computer interaction. However, there is still a lack of the performance of existing methods on real live images. The rise of deep learning algorithm has been showing a spectacular increase in performance lately. Many difficult tasks involving computer vision, speech recognition, and natural language processing are easily solved with deep learning. Therefore, the approach to deep learning notably growing and this also happens to be on image classification. Gender classification is an important subject in the face recognition process. This paper shows the results of classifying gender using Convolutional Neural Network based Deep Learning architectures using Tensorflow’s Deep Learning framework. We have used models provided by Keras with weights pre-trained on ImageNet. We have made a comparison of the different type of models which includes VGG16, ResNet-50, and MobileNet. Our own database consists of Asian faces inclusive of Malaysians and some Caucasians. Our trained model on a database consisting of 1000 images shows that VGG-16 delivered the highest recognition accuracy.

Artificial Intelligence in the American Healthcare Industry: Looking Forward to 2030

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

Artificial Intelligence (AI) has become one of the most transformative foundational technologies of our time cutting across industry and function, while machine learning has become its engine of maturation. The rapid advancement of AI applications in different domains including computer vision, speech recognition, and natural language processing (NLP) has led to a need for highly accurate, efficient, scalable, and deployable systems in the real world. The paper discusses advances needed to advance AI applicability in these three major areas, especially using advanced machine-learning technologies to improve AI efficiency in terms of the performance, energy requirements, and computational scalability of AI systems In order to seek out the accuracy and speed of the computer vision systems, the study’s investigation studies the works related to the modernization of machine learning techniques namely deep learning, reinforcement learning, and transfer learning. Such advances are critical for real-time applications including ampere nous driving, facial recognizance, and medical image diagnosis. In parallel, studies on the architecture optimization for speech recognition models are being conducted to enhance feature extraction, reduce computation and latency while maintaining performance. Such improvements are crucial for voice assistants, transcription services and multilingual communication tools. For NLP, these translations shed light on the latest in transformer-based architectures, better pre-trained language models, and methods for reducing the size and inference time of models, all so that these AI services can be reasonable in chatbots, content summarization, and sentiment analysis, etc. This work presents a detailed insight into how machine learning techniques can offer massive energy savings that can be derived from the analysis of state-of-the-art architectures, finding bottlenecks, and comparing performance. In addition, it underscores recent advances in lightweight models, distributed training, model compression and hardware-aware optimization engage computational burden while preserving or improving predictive performance. This brings these approaches together to maintain that AI technologies can be consolidated more sustainably across edge devices, cloud platforms, and embedded systems.

Learning a similarity function between pairs of objects is at the core of learning to rank approaches. In information retrieval tasks we typically deal with query-document pairs, in question answering -- question-answer pairs. However, before learning can take place, such pairs needs to be mapped from the original space of symbolic words into some feature space encoding various aspects of their relatedness, e.g. lexical, syntactic and semantic. Feature engineering is often a laborious task and may require external knowledge sources that are not always available or difficult to obtain. Recently, deep learning approaches have gained a lot of attention from the research community and industry for their ability to automatically learn optimal feature representation for a given task, while claiming state-of-the-art performance in many tasks in computer vision, speech recognition and natural language processing. In this paper, we present a convolutional neural network architecture for reranking pairs of short texts, where we learn the optimal representation of text pairs and a similarity function to relate them in a supervised way from the available training data. Our network takes only words in the input, thus requiring minimal preprocessing. In particular, we consider the task of reranking short text pairs where elements of the pair are sentences. We test our deep learning system on two popular retrieval tasks from TREC: Question Answering and Microblog Retrieval. Our model demonstrates strong performance on the first task beating previous state-of-the-art systems by about 3\% absolute points in both MAP and MRR and shows comparable results on tweet reranking, while enjoying the benefits of no manual feature engineering and no additional syntactic parsers.

Detecting objects from images is a challenging problem in the domain of computer vision and plays a very crucial role for wide range of real-time applications. The ever-increasing growth of deep learning due to availability of large training data and powerful GPUs helped computer vision community to build commercial products and services which were not possible a decade ago. Deep learning architectures especially convolutional neural networks have achieved state-of-the-art performance on worldwide competitions for visual recognition like ILSVRC, PASCAL VOC. Deep learning techniques alleviate the need of human expertise from designing the handcrafted features and automatically learn the features. This resulted into use of deep architectures in many domains like computer vision (image classification, visual recognition) and natural language processing (language modeling, speech recognition). Object detection is one such promising area immensely needed to be used in automated applications like self-driving cars, robotics, drone image analysis. This paper analytically reviews state-of-the-art deep learning techniques based on convolutional neural networks for object detection.

The advent of the fourth industrial revolution (industry 4.0) has sparked a transformation in manufacturing through the integration of deep learning (DL) and big data technologies. These innovations have enhanced the ability of manufacturers to process and analyze vast amount of data, providing insights that improve decision-making, efficiency, and overall productivity. Deep learning, a subset of artificial intelligence (AI), offers advanced capabilities in data pattern recognition and predictive modeling, while big data facilitates the management of large and complex datasets from various sources. However, literature on the integration of deep learning, big data and industry 4.0 is still limited in the manufacturing context. This paper provides a detailed overview of big data, deep learning and industry 4.0 in manufacturing. It explores the applications of deep learning and big data in manufacturing, highlighting their role in optimizing production processes, predictive maintenance, quality control, and supply chain management. Furthermore, the paper addresses the key challenges and solutions associated with the integration of these technologies, such as data privacy, security, computational complexity, and the need for skilled labor. KEYWORDS: Deep learning, Industry 4.0, Supply chain management, Big data, Predictive maintenance, Artificial intelligence, ERP systems.

As an important technology and research direction to achieve AI, deep learning has been widely applied in the fields of computer vision, speech recognition, and natural language processing. How to effectively accelerate the computing power of deep learning has always been the focus of scientific research. Among various acceleration technologies, FPGA has the advantages of reconfiguration, high performance, small size, and low latency. As more and more FPGA-based neural network accelerators are developed, we notice there is a lack of a complete and detailed overview. In this paper, we give a comparative study of DNN accelerators on FPGA from the aspects of hardware structures, design ideas and optimization strategies. We further compare the performance of different acceleration technologies in different models and present the prospects of the FPGA accelerators for deep learning.

Chapter 19 - Recent advances in deep learning enabled approaches for identification of molecules of therapeutics relevance

Undoubtedly, deep learning (DL) has made striking progress in different artificial intelligence research fields in the bygone decade. DL is a subset of artificial intelligence, where models depict mathematical changes over a wide range of layers. DL is rapidly becoming the state-of-the-art execution in a variety of assignments and it has indicated huge potential in different fields, for example, speech recognition, computer vision and natural language processing. It has additionally been effectively applicable in drug discovery which is one of the areas that can acquire many advantages from the accomplishment of DL. Drug discovery and advancement are among the main translational science activities that add to human well-being and prosperity. This chapter highlights fundamentals of DL, challenges, opportunities, and applications in drug discovery.

Chapter 3 - Sensor-enabled biomedical decision support system using deep learning and fuzzy logic

Recently, significant improvements have been made in artificial intelligence. The artificial neural network was introduced in the 1950s. However, because of the low computing power and insufficient datasets available at that time, artificial neural networks suffered from overfitting and vanishing gradient problems for training deep networks. This concept has become more promising owing to the enhanced big data processing capability, improvement in computing power with parallel processing units, and new algorithms for deep neural networks, which are becoming increasingly successful and attracting interest in many domains, including computer vision, speech recognition, and natural language processing. Recent studies in this technology augur well for medical and healthcare applications, especially in endoscopic imaging. This paper provides perspectives on the history, development, applications, and challenges of deep-learning technology.