Runtime assignment of reconfigurable hardware components for image processing pipelines

This paper proposes a hardware oriented dropout algorithm for an efficient field-programmable gate array (FPGA) implementation. Dropout is a regularization technique, which is commonly used in neural networks such as multilayer perceptrons (MLPs), convolutional neural networks (CNNs), among others. To generate a dropout mask to randomly drop neurons during training phase, random number generators (RNGs) are usually used in software implementations. However, RNGs consume considerable FPGA resources in hardware implementations. The proposed method is able to minimize the resources required for FPGA implementation of dropout by performing a simple rotation operation to a predefined dropout mask. We apply the proposed method to MLPs and CNNs and evaluate them on MNIST and CIFAR-10 classification. In addition, we employ the proposed method in GoogLeNet training using own dataset to develop a vision system for home service robots. The experimental results demonstrate that the proposed method achieves the same regularized effect as the ordinary dropout algorithm. Logic synthesis results show that the proposed method significantly reduces the consumption of FPGA resources in comparison to the ordinary RNG-based approaches.

Image processing is an unceasingly growing area with a range of applications including cryptography, medicine, video surveillance, remote sensing, and many more. Implementing sophisticated algorithms to process the large amount of data using software solutions makes the response slower, and that's where hardware implementation comes into the picture. Field Programmable Gate Arrays (FPGAs) are getting popular due to low latency, connectivity, parallel computing, and flexibility. The unique architecture of the FPGA has made it possible to use the technology for applying in many applications to have better and faster results. This paper is aiming at providing a comprehensive survey on the hardware implementations of image processing algorithms to facilitate the improvement in efficiency using FPGAs. The widely used Xilinx PYNQ is also presented in this paper as they play a major role in reducing the development time.

This paper presents a thorough discussion of the proposed field-programmable gate array (FPGA) implementation for fringe pattern demodulation using the one-dimensional continuous wavelet transform (1D-CWT) algorithm. This algorithm is also known as wavelet transform profilometry. Initially, the 1D-CWT is programmed using the C programming language and compiled into VHDL using the ImpulseC tool. This VHDL code is implemented on the Altera Cyclone IV GX EP4CGX150DF31C7 FPGA. A fringe pattern image with a size of 512×512 pixels is presented to the FPGA, which processes the image using the 1D-CWT algorithm. The FPGA requires approximately 100 ms to process the image and produce a wrapped phase map. For performance comparison purposes, the 1D-CWT algorithm is programmed using the C language. The C code is then compiled using the Intel compiler version 13.0. The compiled code is run on a Dell Precision state-of-the-art workstation. The time required to process the fringe pattern image is approximately 1 s. In order to further reduce the execution time, the 1D-CWT is reprogramed using Intel Integrated Primitive Performance (IPP) Library Version 7.1. The execution time was reduced to approximately 650 ms. This confirms that at least sixfold speedup was gained using FPGA implementation over a state-of-the-art workstation that executes heavily optimized implementation of the 1D-CWT algorithm.

In this paper we describe a single-node, double precision Field Programmable Gate Array (FPGA) implementation of the Conjugate Gradient algorithm in the context of Lattice Quantum Chromodynamics. As a benchmark of our proposal we invert numerically the Dirac–Wilson operator on a 4-dimensional grid on three Xilinx hardware solutions: Zynq Ultrascale+ evaluation board, the Alveo U250 accelerator and the largest device available on the market, the VU13P device. In our implementation we separate software/hardware parts in such a way that the entire multiplication by the Dirac operator is performed in hardware, and the rest of the algorithm runs on the host. We find out that the FPGA implementation can offer a performance comparable with that obtained using current CPU or Intel’s many core Xeon Phi accelerators. A possible multiple node FPGA-based system is discussed and we argue that power-efficient High Performance Computing (HPC) systems can be implemented using FPGA devices only.

Independent component analysis (ICA) is a technique that extracts independent source signals by searching for a linear or nonlinear transformation which minimizes the statistical dependence between components. ICA has been used in a variety of signal processing applications including dimensionality reduction in hyperspectral image (HSI) analysis. Due to the computation complexities and convergence rates, ICA is very time-consuming for high volume or dimension data set like hyperspectral images. Hardware implementation provides not only an optimal parallelism environment, but also a potential faster and real-time solution. This work synthesizes a parallel ICA (pICA) algorithm on field programmable gate array (FPGA). In the proposed implementation method, the pICA is partitioned into three temporally independent functional modules, and each of which is synthesized individually with several ICA-related reconfigurable components (RCs) that are developed for reuse and retargeting purpose. All modules are then integrated into a design and development environment for performing many subtasks such as FPGA synthesis, optimization, placement and routing. In a case study, we synthesize the pICA algorithm for hyperspectral image dimensionality reduction on the pilchard reconfigurahle computing platform embedded with Xilinx: VIRTEX V1000E. The FPGA executes at the maximum frequency of 20.161 MHz, and the pilchard board transfers data directly with CPU on the 64-bit memory bus at the maximum frequency of 133MHz. The performance comparisons between the proposed and another two ICA-related FPGA implementations show that the proposed FPGA implementation of pICA has potential in performing complicated algorithms on large volume data sets.

In this study, Field Programmable Gate Array (FPGA) implementation of Conic Section Function Neural Network (CSFNN) for a classification problem focused on iris plant is presented. This work demonstrates for the first time to our knowledge, the feed-forward computation of CSFNN implementation on FPGA. Using l6-bit floating point arithmetic and the look-up tables (LUTs) for the sigmoid function and the square root function, 83% and 72% of slices and LUTs on Spartan 3-E XC3S1600E are used for the realization of CSFNN with five neurons. The classification results obtained from the FPGA implementation and software simulation show that the accuracy error between two platforms is only 0.1%.

Brillouin optical time domain analyzer (BOTDA) fiber sensors have shown strong capability in static long haul distributed temperature/strain sensing. However, in applications such as structural health monitoring and leakage detection, real-time measurement is quite necessary. The measurement time of temperature/strain in a BOTDA system includes data acquisition time and post-processing time. In this work, we propose to use hardware accelerated support vector regression (SVR) for the post-processing of the collected BOTDA data. Ideal Lorentzian curves under different temperatures with different linewidths are used to train the SVR model to determine the linear SVR decision function. The performance of SVR is evaluated under different signal-to-noise ratios (SNRs) experimentally. After the model coefficients are determined, algorithm-specific hardware accelerators based on field programmable gate arrays (FPGAs) are used to realize SVR decision function. During the implementation, hardware optimization techniques based on loop dependence analysis and batch processing are proposed to reduce the execution latency. Our FPGA implementations can achieve up to 42x speedup compared with software implementation on an i7-5960x computer. The post-processing time for 96,100 BGSs along 38.44-km FUT is only 0.46 seconds with FPGA board ZCU104, making the post-processing time no longer a limiting factor for dynamic sensing. Moreover, the energy efficiency of our FPGA implementation can reach up to 226.1x higher than software implementation based on CPU.

In real-time image processing, the demand for efficient solutions has surged with the proliferation of applications spanning from autonomous vehicles to medical diagnostics. This study addresses the imperative need for accelerated machine learning algorithms to enhance the processing speed of image-related tasks. The research focuses on leveraging Field-Programmable Gate Arrays (FPGAs) to implement hardware acceleration, exploiting their parallel computing capabilities. The advent of machine learning in image processing has revolutionized various industries, yet real-time applications encounter computational bottlenecks. This research delves into hardware acceleration using FPGAs to overcome these constraints, offering a novel approach to expedite machine learning algorithms. Traditional software implementations of machine learning algorithms often fall short in meeting real-time processing requirements. This research aims to bridge this gap by exploring FPGA-based hardware acceleration, addressing the performance limitations hindering the seamless integration of machine learning into real-time image processing systems. While existing literature acknowledges the potential of FPGA-based acceleration, a comprehensive exploration of its application for real-time image processing is lacking. This research fills the void by presenting a detailed method and empirical results, contributing to the limited body of knowledge on FPGA-accelerated machine learning in the of image processing. The study employs a systematic approach, integrating machine learning algorithms onto FPGAs through hardware description languages. The implementation is optimized to exploit parallelism inherent in FPGAs, resulting in a tailored hardware solution for real-time image processing. Comparative analyses against software implementations provide insights into the performance gains achieved. The experimental results demonstrate a significant enhancement in processing speed, validating the efficacy of FPGA-based hardware acceleration for machine learning algorithms in real-time image processing applications.

This thesis aims to develop an embedded image processing board, which can be used to realize various image feature extraction and processing algorithms for robotic applications. The circuitry of this image board can be divided into two parts, one is for a digital signal processor (DSP) and the other for a field programmable gate array (FPGA). A low power, small size CMOS image sensor with YUV4:2:2 image format and 512x480 pixel size is adopted for image acquisition. The DSP module is mainly used for implementing image processing algorithms, while the FPGA module for lower level image processing and sensor control functions. In this work, the TMS320C6414TGLZ7 DSP from Texas Instruments Corporation and the EP2C35F672C6 FPGA from Altera Corporation were selected for the image board. In this work, speeded-up robust features (SURF) algorithm has been successfully realized on the developed image board. An USB interface with average throughput of 22MB/sec works to transfer acquired image frames and extracted SURF descriptors to a personal computer, where feature matching is executed. Experimental results show that the refresh period of frame grabbing is about 33.5ms. When the number of SURF descriptor is equal to 64, the computing time is about 417.5ms; when the number of SURF descriptor is equal to 199, the computing time is about 985ms. The typical matching rate varies from 70.2% to 20%, depending on the threshold of the patterns used in feature extraction.

Hyperspectral imaging (HSI) can extract information from scenes on the earth surface acquired by airborne or spaceborne sensors. On-board processing of HSI is characterized by large datasets on one side and limited processing time and communication links on the other. The CCSDS-123 algorithm is a compression standard assembled for space-related application that allows compacted data transmission via a transmission link. In this paper, a low-complexity high-throughput field-programmable gate array (FPGA) implementation of CCSDS-123 compression algorithm with band interleaved by pixel ordering is presented. Hardware accelerators implemented in FPGAs are increasingly used for custom tasks due to their efficiency, low power, and reconfigurability. The proposed implementation of CCSDS-123 compression standard has been tested on ZedBoard development board containing a Zynq-7020 FPGA. The results are verified against an existing software implementation. The synthesized design can perform on-the-fly processing of hyperspectral images with maximum operating frequency of 147 MHz. The achieved throughput of 147 Msamples/s (2.35 Gb/s) is higher when compared with the throughput reported in recent state-of-the-art FPGA implementations.

Abstract: This study looks into the difficulties of using the Distributed Arithmetic (DA) method for Finite Impulse Response (FIR) filters on Field-Programmable Gate Arrays (FPGAs). It focuses on how coefficients are represented, balancing precision using fixed-point arithmetic and quantization. The research explores ways to optimize memory use, aiming to store data more efficiently within FPGA resources and reduce memory needs. To speed up computations, it examines how to make accessing lookup tables faster and suggests improvements in design. The study also considers how to manage FPGA resources effectively, balancing latency, throughput, and resource use. It looks at specific improvements to the DA method for FIR filters to make better use of resources and enhance performance. Overall, this work provides insights and solutions for algorithm design, memory use, lookup table speed, and FPGA architecture to make DA-based FIR filter implementations on FPGAs more efficient.

This paper presents a hardware implementation of multilayer feedforward neural networks (NN) using reconfigurable field-programmable gate arrays (FPGAs). Despite improvements in FPGA densities, the numerous multipliers in an NN limit the size of the network that can be implemented using a single FPGA, thus making NN applications not viable commercially. The proposed implementation is aimed at reducing resource requirement, without much compromise on the speed, so that a larger NN can be realized on a single chip at a lower cost. The sequential processing of the layers in an NN has been exploited in this paper to implement large NNs using a method of layer multiplexing. Instead of realizing a complete network, only the single largest layer is implemented. The same layer behaves as different layers with the help of a control block. The control block ensures proper functioning by assigning the appropriate inputs, weights, biases, and excitation function of the layer that is currently being computed. Multilayer networks have been implemented using Xilinx FPGA "XCV400hq240". The concept used is shown to be very effective in reducing resource requirements at the cost of a moderate overhead on speed. This implementation is proposed to make NN applications viable in terms of cost and speed for online applications. An NN-based flux estimator is implemented in FPGA and the results obtained are presented.

Programmable logic is emerging as an attractive solution for many digital image processing applications. As image sizes and bit depths grow larger, software has become less useful in the image processing, Field Programmable Gate Array (FPGA) technology has become a viable target for the implementation of algorithms suited to image processing applications, the unique architecture of the FPGA has allowed the technology to be used in many such applications encompassing all aspects of image processing. Image smoothing is one of image processing applications, it often done to reduce the effect of pixel noise in images. This paper presents VHDL architectures (that allow description of the structure design of FPGA) to implement two of image smoothing filters: a) averaging filters b) median filters This research is also applying the filters on FPGA. The results proves high-speed performance of the algorithms that rely on hardware and software compared to software alone, as it appeared very big difference in the speed of execution, depending on the hardware devices that showed the speed of the implementation of the scale of nanosecond, while the software application of algorithms is measured in seconds. The software was implemented in this research using MATLAB 2010 language code as well as the VHDL language to deal with use of FPGA device, which was of a kind (Xilinx XC3S500E Spartan-3E).

In this paper an implementation of the Inverse Numerical Laplace Transform (INLT) on a Field Programmable Gate Array (FPGA) is presented. A brief description of INLT algorithms based on FFT are made and its MATLAB implementation is performed in order to have a comparison basis with FPGA results. A fixed point representation is used and it is optimized using fi MATLAB object. INLT is basically composed by three stages: Windowing, FFT and exponential function multiplications, the last stage is considered the critical one because it can result in severely large errors, specially, when several samples are required. Errors can be reduced by increasing the number of bits, however this increment makes the required FPGA area greater. Windowing and FFT can be implemented with high accuracy on FPGA with CORDIC and Cooley Tukey algorithms respectively, making the implementation flexible. The implementation was made with Verilog description hardware language using Xilinx technology and Virtex 6 FPGA.

One of the main challenges in Systems designs is the ability to integrate real time high fidelity models on suitable and feasible hardware platforms. Because of its inherited parallelism, FPGA (Field Programmable Gate Arrays) technology achieves sample rates which are typically faster than real time. This can be seen as the last line of defence against the increasing requirements given by high fidelity models. But as most of the FPGA applications are specialised and the FPGA toolsets do not support basic control systems blocks, designs are constructed and optimised manually. This leads to significant effort required in finding feasible hardware FPGA implementations. Therefore, the work in this paper describes how to automatically optimise the most time consuming process found in generic FPGA implementations: the optimisation of the pipelining process. This is constructed on a rigorous mathematical model and achieved using drag and drop floating point HDL (Hardware Description Language) control systems blocks, under System Generator, in Simulink.