Edge Computing and Hardware Acceleration for HSI in Clinical Workflows

Today, many enterprises are evaluating and in some cases deploying heterogeneous computing platforms that include some form of hardware acceleration or co-processing. Such systems typically consist of commodity computing clusters augmented by hardware accelerators like graphics processing units (GPUs), field programmable gate arrays (FPGAs), gaming platform chips like the Cell Processor, and other hardware accelerators.In this presentation we will identify desirable architectural features of a computing platform that includes FPGA co-processing, along with practical system design issues that arise when introducing an FPGA into a modern x86 computing blade, including mechanical, power and cooling issues.The main body of the presentation will highlight the promise of these systems using a concrete example. I will describe the architecture, technical capabilities, and limitations of a commercially available FPGA coprocessor, the XD1000™ coprocessor module from XtremeData. The XD1000 integrates a Stratix™ II FPGA into a multi-Opteron™ system by replacing one of the Opteron CPUs with the FPGA co-processor. The module communicates with other Opteron CPUs on the motherboard via point-to-point HyperTransport (HT) links with 3.2 GB/sec of bandwidth each. The FPGA coprocessor interfaces directly with motherboard DDR SDRAM memory DIMMs without the need for an intervening north bridge.Important architectural considerations include whether to provide and how to interface to local on-module memory (e.g. on-module SRAM), the ramifications of coherent vs. noncoherent HT protocol support on the coprocessor module, the coprocessor programming interface for the host system, FPGA configuration, and monitoring and test support for user debug. A block diagram of a system employing the XD1000 coprocessor is shown in Figure 1.

Producción Científica

Real-time performance is the primary requirement for edge computing systems. However, with the surge in data volume and the growing demand for computing power, a computing framework consisting solely of CPUs is no longer competent. As a result, CPU+ heterogeneous architecture using accelerators to improve edge computing systems' computing capacity has received great attention. The type of accelerators determines the performance of the edge computing system largely. The accelerators include Graphics Processing Unit (GPU), Application Specific Integrated Circuit (ASIC) and Field Programmable Gate Array (FPGA). FPGAs with its reconfigurability and high energy efficiency are widely used in many edge computing scenarios. Nontheless, the performance depends also on the scheduling efficiency between software tasks on CPUs and hardware tasks on FPGAs. Unfortunately, the existing strategies have not fully exploited the differences between hardware and software tasks, thus resulting in low scheduling efficiency. This paper proposes a task scheduling framework on the Dynamic Partial Reconfiguration (DPR) platform. We take full account of the characteristics of task switching overhead and predictable execution time of hardware tasks in DPR, and reduce the number of task-switching times and active tasks in the system, thus improving the scheduling efficiency. We conduct a set of experiments on the Zynq platform to verify the proposed framework. Experimental results demonstrate that when the execution time of the accelerator exceeds the reconfiguration cost by an order of magnitude, the efficiencies of all the cases are more than 98%, and the efficiencies can reach 90%-98% in the same order of magnitude.

Artificial neural networks are a group of artificial intelligence algorithms widely used contemporarily in the industry and research, implemented to solve a great variety of issues. Its recent popularity is in part due to the advance of hardware technologies, which provide computational resources for the resource-intensive processing necessary for its implementation. Historically, neural network software is executed in Central Processing Units (CPUs), which are general purpose devices. However, metrics such as inference speed, energy efficiency and circuit area are improved with the use of specialized hardware accelerators. In the context of edge computing, the processing on location of data gathered by Internet of Things (IoT) devices, there are significant restrictions as to those metrics. One of the devices frequently used for this purpose are Field-Programmable Gate Arrays (FPGAs), which offer integrated circuits whose functionality may be programmed for the synthesis of hardware specialized in specific algorithms, offering high performance and execution efficiency, as well as other benefits such as being able to be reconfigured after manufacturing, shorter design cycles, faster time-to-market and lower cost than the alternatives. One of the issues for the use of FPGAs is that its programming is difficult and requires specialist knowledge to fully make use of these devices characteristics, and not every team of artificial intelligence developers has such knowledge. Consequently, there is interest in systems that automatize the synthesis of neural network hardware accelerators in FPGAs, to bring the benefits of this technique to a wider audience. Another approach to solve the high restriction environment of edge computing is neural network quantization, which means reducing the precision of representation of parameters so they consume less memory, and operations using these numbers are faster. This work studies the state of the art of this manner of software, diagnosing existing gaps. The main objective of this research is the creation and validation of a proof of concept of automatic generation of neural network hardware accelerators that are using parameter quantization techniques to enable its synthesis on small FPGAs aimed towards edge computing. For the validation of this system, an accelerator was generated and its behavior was measured in metrics of latency, end result throughput, energy efficiency, circuit area and compared to the execution of the same neural network in a high-end personal computer CPU. The results indicate that the generated hardware accelerator is synthesizeable, significantly faster and consumes considerably less energy than the CPU implementation.

Independent component analysis (ICA) is a technique that extracts independent source signals by searching for a linear or nonlinear transformation which minimizes the statistical dependence between components. ICA has been used in a variety of signal processing applications including dimensionality reduction in hyperspectral image (HSI) analysis. Due to the computation complexities and convergence rates, ICA is very time-consuming for high volume or dimension data set like hyperspectral images. Hardware implementation provides not only an optimal parallelism environment, but also a potential faster and real-time solution. This work synthesizes a parallel ICA (pICA) algorithm on field programmable gate array (FPGA). In the proposed implementation method, the pICA is partitioned into three temporally independent functional modules, and each of which is synthesized individually with several ICA-related reconfigurable components (RCs) that are developed for reuse and retargeting purpose. All modules are then integrated into a design and development environment for performing many subtasks such as FPGA synthesis, optimization, placement and routing. In a case study, we synthesize the pICA algorithm for hyperspectral image dimensionality reduction on the pilchard reconfigurahle computing platform embedded with Xilinx: VIRTEX V1000E. The FPGA executes at the maximum frequency of 20.161 MHz, and the pilchard board transfers data directly with CPU on the 64-bit memory bus at the maximum frequency of 133MHz. The performance comparisons between the proposed and another two ICA-related FPGA implementations show that the proposed FPGA implementation of pICA has potential in performing complicated algorithms on large volume data sets.

Deep neural networks (DNNs) are promising solutions for most of the artificial intelligence and machine learning applications in various fields like safety and transportation, medical field, weather forecasting and many more. State-of-the-art deep neural networks can have hundreds of millions of parameters, and it makes them less than ideal for mass adoption in devices with constrained memory and power requirements like edge computing devices and mobile devices. Techniques like quantization and inducing sparsity, aims to reduce the total number of computations needed for deep learning inference. General purpose computing hardware like CPUs (Central Processing Units) and GPUs (Graphic Processing Units) are not optimized for portable embedded applications as they are not energy efficient. Field Programmable Gate Arrays (FPGAs) are suitable candidates for edge computing, which delivers decent power consumption, with flexibility. We propose an efficient sparse matrix-vector multiplication (SpMV) architecture that aims to make deep learning inference faster and more efficient and also reduces memory-bandwidth bottleneck. The multiplication process is handled by multiple multiply and accumulate (MAC) channels and the architecture can use the maximum available memory bandwidth of the computing device. The proposed sparse matrix multiplier architecture has been implemented on Zynq Ultrascale+ FPGA with an operating frequency of 270 MHz and gave performance gain upto 5× when compared with existing implementation.

Introduction to hardware accelerator systems for artificial intelligence and machine learning

Spectroradiometric imaging is based on the fact that all materials display a specific spectral signature when light impinges on them. That is, they reflect, absorb, and emit electromagnetic waves in a particular pattern depending on the wavelength, leading to a spectrum for the material that can be used to classify it. An imaging spectroradiometer captures images where the spectral information of every pixel is collected for a set of contiguous discrete spectral bands. It is customary to classify these instruments as ultraspectral, hyperspectral, multispectral, and panchromatic imagers. Typically, ultraspectral data sets are made up of thousands of spectral bands, while hyperspectral sets have hundreds, multispectral tens, and panchromatic sets just a few. Thus, each ultraspectral or hyperspectral image contains large amounts of data which can be viewed as a cube with two spatial and one spectral dimension. Regardless of the application of the images, the analysis methods employed must deal with large quantities of data efficiently [1]. Originally, imaging spectroradiometers were used as airborne or spaceborne remote sensing instruments. During the last decade, spectral imaging has become a common advanced tool for remote sensing of the surface of Earth. It is being used to collect crucial data about many activities of economic or scientific relevance [2] as, for instance, mineral exploitation [3], agriculture and forestry use [4], environmental pollution [5], ecosystem studies [6], assessment of natural resources [7], water resource management [8], monitoring of coastal zones [9] and many others. For all these remote sensing applications the hyperspectral images obtained usually contain mixtures of spectra in every pixel due to the poor spatial resolution of the images when taken a large distance from the target. Thus a single pixel could typically correspond to one hundred or more square meters of land. As a result, most analysis methods developed had as their main objective to provide the segmentation of the images in terms of the ratio of endmembers present in every pixel to improve the spatial discrimination of these systems when analyzing different types of land covers [10].

The availability of graphics processing units (GPUs) provides a low-cost solution to real-time processing, which may benefit many remote sensing applications. In this paper, a spectral–spatial classification scheme for hyperspectral images is specifically adapted for computing on GPUs. It is based on wavelets, extended morphological profiles (EMPs), and support vector machine (SVM). Additionally, a preprocessing stage is used to remove noise in the original hyperspectral image. The local computation of the techniques used in the proposed scheme makes them particularly suitable for parallel processing by blocks of threads in the GPU. Moreover, a block-asynchronous updating process is applied to the EMP to speedup the morphological reconstruction. The results over different hyperspectral images show that the execution can be speeded up to $8.2\times$ compared to an efficient OpenMP parallel implementation, achieving real-time hyperspectral image classification while maintaining the high classification accuracy values of the original classification scheme.

Hyperspectral images are used in different applications in Earth and space science, and many of these applications exhibit real- or near real-time constraints. A problem when analyzing hyperspectral images is that their spatial resolution is generally not enough to separate different spectrally pure constituents (endmembers); as a result, several of them can be found in the same pixel. Spectral unmixing is an important technique for hyperspectral data exploitation, aimed at finding the spectral signatures of the endmembers and their associated abundance fractions. The development of techniques able to provide unmixing results in real-time is a long desired goal in the hyperspectral imaging community. In this paper, we describe a real-time hyperspectral unmixing chain based on three main steps: 1) estimation of the number of endmembers using the hyperspectral subspace identification with minimum error (HySime); 2) estimation of the spectral signatures of the endmembers using the vertex component analysis (VCA); and 3) unconstrained abundance estimation. We have developed new parallel implementations of the aforementioned algorithms and assembled them in a fully operative real-time unmixing chain using graphics processing units (GPUs), exploiting NVIDIA’s compute unified device architecture (CUDA) and its basic linear algebra subroutines (CuBLAS) library, as well as OpenMP and BLAS for multicore parallelization. As a result, our real-time chain exploits both CPU (multicore) and GPU paradigms in the optimization. Our experiments reveal that this hybrid GPU–CPU parallel implementation fully meets real-time constraints in hyperspectral imaging applications.

Hyperspectral images with hundreds of spectral bands have been proven to yield high performance in material classification. However, despite intensive advancement in hardware, the spatial resolution is still somewhat low, as compared to that of color and multispectral (MS) imagers. In this paper, we aim at presenting some ideas that may further enhance the performance of some remote sensing applications such as border monitoring and Mars exploration using hyperspectral images. One popular approach to enhancing the spatial resolution of hyperspectral images is pansharpening. We present a brief review of recent image resolution enhancement algorithms, including single super-resolution and multi-image fusion algorithms, for hyperspectral images. Advantages and limitations of the enhancement algorithms are highlighted. Some limitations in the pansharpening process include the availability of high resolution (HR) panchromatic (pan) and/or MS images, the registration of images from multiple sources, the availability of point spread function (PSF), and reliable and consistent image quality assessment. We suggest some proactive ideas to alleviate the above issues in practice. In the event where hyperspectral images are not available, we suggest the use of band synthesis techniques to generate HR hyperspectral images from low resolution (LR) MS images. Several recent interesting applications in border monitoring and Mars exploration using hyperspectral images are presented. Finally, some future directions in this research area are highlighted.

SUMMARYIn wireless communication, Viterbi decoding algorithm (VDA) is the one of most popular channel decoding algorithms, which is widely used in WLAN, WiMAX, or 3G communications. However, the throughput of Viterbi decoder is constrained by the convolutional characteristic. Recently, the three‐point VDA (TVDA) was proposed to solve this problem. In TVDA, the whole procedure can be divided into three phases, the forward, trace‐back, and decoding phases. In this paper, we analyze the parallelism of TVDA and propose parallel TVDA on the multi‐core CPU, graphics processing unit (GPU), and field programmable gate array (FPGA). We demonstrate approaches that fully exploit its performance potential on CPU, GPU, and FPGA computing platforms. For CPU platforms, we perform two optimization methods, single instruction multiple data and multithreading to gain over 145 × speedup over the naive CPU version on a quad‐core CPU platform. For GPU platforms, we propose the combination of cached memory optimization, coalesced global memory accesses, codeword packing scheme, and asynchronous data transition, achieving the throughput of 404.65 Mbps and 12 × speedup over initial GPU versions on an NVIDIA GeForce GTX580 card and 7 × speedup over Intel quad‐core CPU i5‐2300, under the same manufacturing year and both with fully optimized schemes. In addition, for FPGA platforms, we customize a radix‐4 pipelined architecture for the TVDA in a 45‐nm FPGA chip from Xilinx (XC6VLX760). Under 209.15‐MHz clock rate, it achieves a throughput of 418.30 Mbps. Finally, we also discuss the performance evaluation and efficiency comparison of different flexible architectures for real‐time Viterbi decoding in terms of the decoding throughput, power consumption, optimization schemes, programming costs, and price costs.Copyright © 2013 John Wiley & Sons, Ltd.

Hardware-in-the-Loop (HWIL) simulation is becoming increasingly important for cost-effective testing of imaging infrared systems. DSTO is developing real-time scene generation and image processing capabilities within its HWIL simulation programs, based on the application of COTS desktop PCs equipped with Graphics Processing Unit (GPU) cards, and including limited use of Field Programmable Gate Arrays (FPGAs). GPUs and FPGAs are high-performance parallel computing machines but are fundamentally different types of hardware. To determine which hardware type should be used to implement a real-time solution of a given application, a methodology is required to expose the concurrency within the problem and to structure the problem in a way that can be mapped to the hardware types. In this paper we use parallel programming patterns to compare the architectures of recent generation GPUs and FPGAs. We demonstrate the decomposition of a parallel application and its implementation on GPU and FPGA hardware and present preliminary results.

Convolutional Neural Networks (CNNs) require massive parallelism due to the high-precision floating-point arithmetic operations they perform. So, demand of processing power in them is significantly higher than what a standard CPU can offer. This has traditionally made CNNs more suited for running on a Graphics Processing Unit (GPU). However, GPUs consume much more power than CPUs, rendering the former impractical for implementing CNNs in Edge AI (Artificial Intelligence), where restraining power consumption is paramount. On the other hand, FPGAs (Field Programmable Gate Arrays) are more suited for AI computing at the edge as they consume much lesser power than GPUs and even CPUs. Additionally, GPUs and CPUs are not suited for real-time AI applications, which require both high throughput and low latency at the same time and FPGAs excel at all these requirements. The purpose of this paper is to provide a study of the performance of FPGA as the most suitable platform for AI-based computing at the edge. To achieve this, we chose AlexNet, a popular CNN image classifier, for which we present a case study on four different platforms: CPU, GPU, embedded RISC core, and FPGA fabric. Then, we quantitatively measure and compare the performance in terms of inference time (time needed to classify an image) on all these platforms. Inference time using FPGA is reduced by almost 64, 1.6, and 1.1 times compared to a dual-core ARM, an i5-6400 CPU, and an Nvidia GPU, respectively.

With numerous and contiguous spectral bands acquired from visible light (400?1,000 nm) to (near) infrared (1,000?1,700 nm and over), hyperspectral imaging (HSI) can potentially identify different objects by detecting minor changes in temperature, moisture, and chemical content. As a result, HSI has been widely applied in a number of application areas, including remote sensing [1]. HSI data contains two-dimensional (2-D) spatial and one-dimensional spectral information, and naturally forms a three-dimensional (3-D) hypercube with a high spectral resolution in nanometers that enables robust discrimination of ground features. However, new challenges arise in dealing with extremely large data sets. For a hypercube with relatively small spatial dimension of 600 ? 400 pixels at 16 bits-per-band-per-pixel, the data volume becomes 120 MB for 250 spectral bands. In some cases, this large data volume can be linearly increased when multiple hypercubes are acquired across time to monitor system dynamics in consecutive time instants. When the ratio between the feature dimension (spectral bands) and the number of data samples (in vector-based pixels) is vastly different, high-dimensional data suffers from the well-known curse of dimensionality. For feature extraction and dimensionality reduction, principal components analysis (PCA) is widely used in HSI [2], where the number of extracted components is significantly reduced compared to the original feature dimension, i.e., the number of spectral bands. For effective analysis of large-scale data in HSI, conventional PCA faces three main challenges: