An embedded intelligent system for real-time image classification with a neuro-inspired color constancy algorithm

Convolutional Neural Networks (CNNs) require massive parallelism due to the high-precision floating-point arithmetic operations they perform. So, demand of processing power in them is significantly higher than what a standard CPU can offer. This has traditionally made CNNs more suited for running on a Graphics Processing Unit (GPU). However, GPUs consume much more power than CPUs, rendering the former impractical for implementing CNNs in Edge AI (Artificial Intelligence), where restraining power consumption is paramount. On the other hand, FPGAs (Field Programmable Gate Arrays) are more suited for AI computing at the edge as they consume much lesser power than GPUs and even CPUs. Additionally, GPUs and CPUs are not suited for real-time AI applications, which require both high throughput and low latency at the same time and FPGAs excel at all these requirements. The purpose of this paper is to provide a study of the performance of FPGA as the most suitable platform for AI-based computing at the edge. To achieve this, we chose AlexNet, a popular CNN image classifier, for which we present a case study on four different platforms: CPU, GPU, embedded RISC core, and FPGA fabric. Then, we quantitatively measure and compare the performance in terms of inference time (time needed to classify an image) on all these platforms. Inference time using FPGA is reduced by almost 64, 1.6, and 1.1 times compared to a dual-core ARM, an i5-6400 CPU, and an Nvidia GPU, respectively.

In positron emission tomography (PET), 3D iterative image reconstruction methods have a huge computational burden. In this paper, we developed a list-mode image reconstruction method using graphics processing units (GPUs). Efficiency of acceleration for GPU implementation largely depends on the method chosen, where a reduced number of conditional statements and a reduced memory size are required. On the other hand, accurate system models are required to improve the quality of reconstructed images. Various accurate system models for conventional CPU implementation have been proposed, but these models basically require many conditional statements and huge memory size. Therefore, we developed a new system model which matches GPU implementation better. In this model, the detector response functions, which vary depending on each line of response (LOR), are pre-computed in CPUs and modeled by sixth-order polynomial functions in order to reduce the memory size occupied in GPUs. Each element of a system matrix is obtained on-the-fly in GPUs by calculating the distance between an LOR and a voxel. Therefore the developed system model enables efficient GPU implementation of the accurate system modeling with a reduced number of conditional statements and a reduced memory size. We applied the developed method to a small OpenPET prototype, in which 4-layered depth-of-interaction (DOI) detectors were used. For image reconstruction, we used the dynamic row-action maximum likelihood algorithm (DRAMA). Compared with a conventional model for GPU implementation, in which DRFs are given as a Gaussian function of fixed width, we saw no remarkable difference for DOI data, but for non-DOI data the proposed model outperformed the conventional at the peripheral region of the field-of-view. The proposed model had almost the same calculation time as the conventional model did. For further acceleration, we tried parallel GPU implementation, and we obtained 3.8-fold acceleration by using 4 GPUs.

Ratios of excitations in each cone-photoreceptor class produced by light reflected from pairs of surfaces in a scene are almost invariant under natural illuminant changes. The stability of these spatially defined ratios may explain the remarkable ability of human observers to efficiently discriminate illuminant changes from changes in surface reflectances. Spatial cone-excitation ratios are not, however, exactly invariant. This study is concerned with observers' sensitivity to these invariance violations. Simulations of Mondrian paintings with either 49 or two natural surfaces under Planckian illuminants were presented as images on a computer-controlled display in a two-interval experimental design: in one interval, the surfaces underwent an illuminant change; in the other interval, the surfaces underwent the same change but the images were then corrected so that, for each cone class, ratios of excitations were preserved exactly. Although the intervals with corrected images corresponded individually to highly improbable natural events, observers systematically misidentified them as containing the illuminant changes, the probability of error increasing as the violation of invariance in the other interval increased. For the range of illuminants and surfaces tested, sensitivity to violations of invariance was found to depend on cone class: it was greatest for long-wavelength-sensitive cones and least for short-wavelength-sensitive cones. Spatial cone-excitation ratios, or some closely related quantities, seem to be the cues preferred by observers for making inferences about surface illuminant changes.

Convolutional neural networks (CNNs) are classical models for computer vision and machine learning applications such as video surveillance, pattern recognition, weather forecasting, traffic, and safety. CNNs involve computationally intensive operations and require huge off-chip memory bandwidth, which makes it a challenging task to deploy on real-time embedded systems. Compared to central processing units and graphic processing units, field programmable gate arrays (FPGA)-based CNNs are gaining popularity owing to their flexibility and efficiency. In this work, we present an efficient CNN accelerator based on blocked Winograd-GEMM architecture with high performance. We implement ResNet-18 CNN model on XC7VX690T FPGA using proposed architecture. This implementation operates at a clock frequency of 200 MHz and gives average throughput of 383 GOPS which is comparable to other state-of-art implementations. This manuscript is an extended version of [S. Kala, J. Mathew, B. R. Jose, and S. Nalesh, “UniWiG: Unified Winograd-GEMM Architecture for Accelerating CNN on FPGAs,” in 2019 32nd International Conference on VLSI Design and 2019 18th International Conference on Embedded Systems (VLSID), Delhi, NCR, India, 2019, pp. 209–214. DOI: 10.1109/VLSID.2019.00055.].

Efficient methods for implementation of multi-level nonrigid mass-preserving image registration on GPUs and multi-threaded CPUs

Deep learning (DL) has shown explosive growth in its application to bioinformatics and has demonstrated thrillingly promising power to mine the complex relationship hidden in large-scale biological and biomedical data. A number of comprehensive reviews have been published on such applications, ranging from high-level reviews with future perspectives to those mainly serving as tutorials. These reviews have provided an excellent introduction to and guideline for applications of DL in bioinformatics, covering multiple types of machine learning (ML) problems, different DL architectures, and ranges of biological/biomedical problems. However, most of these reviews have focused on previous research, whereas current trends in the principled DL field and perspectives on their future developments and potential new applications to biology and biomedicine are still scarce. We will focus on modern DL, the ongoing trends and future directions of the principled DL field, and postulate new and major applications in bioinformatics.

Field programmable gate array (FPGA) is widely considered as a promising platform for convolutional neural network (CNN) acceleration. However, the large numbers of parameters of CNNs cause heavy computing and memory burdens for FPGA-based CNN implementation. To solve this problem, this paper proposes an optimized compression strategy, and realizes an accelerator based on FPGA for CNNs. Firstly, a reversed-pruning strategy is proposed which reduces the number of parameters of AlexNet by a factor of 13× without accuracy loss on the ImageNet dataset. Peak-pruning is further introduced to achieve better compressibility. Moreover, quantization gives another 4× with negligible loss of accuracy. Secondly, an efficient storage technique, which aims for the reduction of the whole overhead cache of the convolutional layer and the fully connected layer, is presented respectively. Finally, the effectiveness of the proposed strategy is verified by an accelerator implemented on a Xilinx ZCU104 evaluation board. By improving existing pruning techniques and the storage format of sparse data, we significantly reduce the size of AlexNet by 28×, from 243 MB to 8.7 MB. In addition, the overall performance of our accelerator achieves 9.73 fps for the compressed AlexNet. Compared with the central processing unit (CPU) and graphics processing unit (GPU) platforms, our implementation achieves 182.3× and 1.1× improvements in latency and throughput, respectively, on the convolutional (CONV) layers of AlexNet, with an 822.0× and 15.8× improvement for energy efficiency, separately. This novel compression strategy provides a reference for other neural network applications, including CNNs, long short-term memory (LSTM), and recurrent neural networks (RNNs).

Introduction to hardware accelerator systems for artificial intelligence and machine learning

Color constancy is the ability to assign a constant color to an object independent of changes in illumination. Color constancy is achieved by taking context information into account. Previous approaches that have used matching paradigms to quantify color constancy found degrees of constancy between 20% and 80%. Here, we studied color constancy in a color-naming task under different conditions of surround illumination and patch size. Observers categorized more than 400 patches for each illumination condition. This allows one to overcome inherent limitations in color naming and to study the changes in color categories under illumination changes. When small central test patches with a full context illumination were categorized, observers followed the illumination shift almost completely, showing a high degree of constancy (99%). Reducing the available context information or increasing the patch size decreased the degree of constancy to about 50%. Moderate degrees of constancy (66%) occurred even when the test patches were never viewed simultaneously but only in temporal alternation with the illumination. Boundaries between color categories were largely stable within and across observers under neutral illumination. Under changing illumination, there were small but systematic variations in the color category boundaries. Color category boundaries tended to rotate away from the illumination color. This variation was largest under full context conditions where highest degrees of color constancy were obtained.

Convolutional Neural Networks (CNNs) are at the base of many applications, both in embedded and in server-class contexts. While Graphics Processing Units (GPUs) are predominantly used for training, solutions for inference often rely on Field Programmable Gate Arrays (FPGAs) since they are more flexible and cost-efficient in many scenarios. However, existing approaches fall short to accomplish several conflicting goals, like efficiently using resources on multiple platforms while retaining deep configurability and allowing a quick Design Space Exploration (DSE) towards the best solution. This paper proposes a solution composed of highly configurable kernels designed for resources time-sharing with an analytical model of their resource/performance characteristics. Building on such models, we propose an Integer Linear Programming (ILP)-based approach to effectively identify pareto optimal kernel configurations in terms of throughput and resource consumption. We evaluate our DSE on two state-of-the-art CNNs, showing how it identifies hundreds of pareto optimal solutions in less than a minute. Guided from the DSE configurations of the AlexNet network, we quickly identified a candidate design for a Xilinx Virtex-7 XC7VX485T FPGA and achieved a peak throughput of 4.05 ms per image, while we measured a maximum estimation error of 6.69% with respect to the proposed analytical models.

Advances in genomics and sequencing technologies demand faster and more scalable analysis methods that can process longer sequences with higher accuracy. However, classical pairwise alignment methods, based on dynamic programming (DP), impose impractical computational requirements to align long and noisy sequences like those produced by PacBio and Nanopore technologies. The recently proposed wavefront alignment (WFA) algorithm paves the way for more efficient alignment tools, improving time and memory complexity over previous methods. However, high-performance computing (HPC) platforms require efficient parallel algorithms and tools to exploit the computing resources available on modern accelerator-based architectures. This paper presents WFA-GPU, a GPU (graphics processing unit)-accelerated tool to compute exact gap-affine alignments based on the WFA algorithm. We present the algorithmic adaptations and performance optimizations that allow exploiting the massively parallel capabilities of modern GPU devices to accelerate the alignment computations. In particular, we propose a CPU-GPU co-design capable of performing inter-sequence and intra-sequence parallel sequence alignment, combining a succinct WFA-data representation with an efficient GPU implementation. As a result, we demonstrate that our implementation outperforms the original multi-threaded WFA implementation by up to 4.3× and up to 18.2× when using heuristic methods on long and noisy sequences. Compared to other state-of-the-art tools and libraries, the WFA-GPU is up to 29× faster than other GPU implementations and up to four orders of magnitude faster than other CPU implementations. Furthermore, WFA-GPU is the only GPU solution capable of correctly aligning long reads using a commodity GPU. WFA-GPU code and documentation are publicly available at https://github.com/quim0/WFA-GPU.

We have implemented the Martini force field within Lawrence Livermore National Laboratory's molecular dynamics program, ddcMD. The program is extended to a heterogeneous programming model so that it can exploit graphics processing unit (GPU) accelerators. In addition to the Martini force field being ported to the GPU, the entire integration step, including thermostat, barostat, and constraint solver, is ported as well, which speeds up the simulations to 278-fold using one GPU vs one central processing unit (CPU) core. A benchmark study is performed with several test cases, comparing ddcMD and GROMACS Martini simulations. The average performance of ddcMD for a protein-lipid simulation system of 136k particles achieves 1.04 µs/day on one NVIDIA V100 GPU and aggregates 6.19 µs/day on one Summit node with six GPUs. The GPU implementation in ddcMD offloads all computations to the GPU and only requires one CPU core per simulation to manage the inputs and outputs, freeing up remaining CPU resources on the compute node for alternative tasks often required in complex simulation campaigns. The ddcMD code has been made open source and is available on GitHub at https://github.com/LLNL/ddcMD.

Parallel implementation provides a solution for the problem of accelerating cellular automata (CA)-based secret sharing schemes and make them appropriate for bulk data sharing and real-time applications. By presenting new platforms, we need new implementation techniques to run algorithms as fast as possible on the platform. In this paper, we present a new implementation of a CA-based secret sharing scheme using the Graphic Processing Unit (GPU). We propose a new data arrangement that reduces the total number of accesses to the memories in GPU. Our algorithm further reduces the amount of data required by each thread and at the same time achieves a high cache hit rate. Also, it can achieve coalesced memory accesses to optimal use of the global memory bandwidth. The proposed method obtains speedup up to four times faster than the best previous GPU implemented CA-based multi-secret sharing schemes.

This paper presents a study of exact string matching algorithms and their performance behavior when executed on dynamic parallelism enabled Kepler Graphics Processing Unit (GPU) by Nvidia. The algorithms considered in this paper are Quick search (QS), Horspool (HP), and Brute force (BF) string matching. Their efficient implementation on Kepler gives a remarkable improvement over their respective multi-core CPU and generic GPU implementations. In addition, the proposed work further optimizes the algorithm by exploiting the fundamental architectural aspects of the GPU like the memory hierarchy, lightweight threads and avoiding strided access. The optimization associated with the memory is called Binning and the one using memory alignment is named Chunking. As a result of the significant boost obtained by extracting the most out of these features, the newer methods are named as SWIFT. SWIFT algorithms give performance benefit of about 1.7X on an average and up to 5X in some cases, which will be discussed in the paper. Also, the paper proposes a hybrid algorithm that employs both regular GPU implementation and SWIFT based on a predefined condition that gives benefit for all pattern sizes. The experimental results for different pattern sizes using benchmark datasets are presented in the paper.

GPU parallelization of unstructured/hybrid grid ALE multigrid unsteady solver for moving body problems