Reconfigurable Hardware for Machine Learning Applications

Cryptography in digital world must offer integrity and confidentiality using cryptographic algorithms which mainly involve multiplication operation in finite fields. Various algorithms and architectures are proposed in the literature to obtain efficient finite field multiplications in both hardware and software. Here, a modified interleaved multiplication algorithm with reduced computational complexity is proposed based on a novel pre‐computation (PC) technique to perform multiplication over for trinomials. Consequently, an m ‐bit systolic multiplier for trinomials (SMT) is designed by employing the proposed algorithm. Hardware and delay complexity analysis is performed and comparison of the proposed SMT structure with similar multipliers available in the literature is presented. The SMT structure achieves ∼28 and 17% improvement in hardware and area‐delay product, respectively, for m = 233 when compared with the best multiplier available in the literature. The functionality of the proposed SMT structure is also verified by implementing on field‐programmable gate array (FPGA) and application‐specific integrated circuit (ASIC) technologies. It can be observed from FPGA and ASIC implementation results that the proposed SMT structure shows improvement in area, power consumption, area‐delay, and power‐delay products when compared with similar multipliers available in the literature.

Deep learning (DL) has shown explosive growth in its application to bioinformatics and has demonstrated thrillingly promising power to mine the complex relationship hidden in large-scale biological and biomedical data. A number of comprehensive reviews have been published on such applications, ranging from high-level reviews with future perspectives to those mainly serving as tutorials. These reviews have provided an excellent introduction to and guideline for applications of DL in bioinformatics, covering multiple types of machine learning (ML) problems, different DL architectures, and ranges of biological/biomedical problems. However, most of these reviews have focused on previous research, whereas current trends in the principled DL field and perspectives on their future developments and potential new applications to biology and biomedicine are still scarce. We will focus on modern DL, the ongoing trends and future directions of the principled DL field, and postulate new and major applications in bioinformatics.

In recent years, artificial intelligence has been widely used in the field of meteorology. However, it is perhaps less known that AI-based tools have been used for over 30 years in the modeling of pollutant dispersion in the atmosphere.In the early 1990s, Slovenia and other parts of Europe faced significant air pollution problems caused by SO₂ emissions from coal-fired power plants without desulfurization systems. Due to the highly complex terrain, simple physical Gaussian dispersion models were not suitable for reconstructing the dispersion of SO₂ in the atmosphere, while complex numerical Lagrangian particle dispersion models were still in development.Therefore, as early as 1992, we modeled air pollution using a Multilayer Perceptron Artificial Neural Network (MLPANN). We developed the world's first comprehensive method for selecting training samples, features, and the topology of the neural network, enabling the model to learn from measured meteorological variables and pollutant concentrations around the power plant to predict SO₂ concentrations at a selected location in advance.This type of artificial neural network remains the foundation for various derived structures used for machine learning from data even today.In the following decades, we expanded the use of MLPANN to ozone prediction. Researchers from other Slovenian research groups also worked on this topic. Meanwhile, the use of these and similar tools in pollutant dispersion modelling began to grow on a global scale.Together with Brazilian researchers, we successfully applied MLPANN for predicting diffuse solar radiation. In this field, we also managed to develop a spatially transferable model, which is not a common capability for artificial neural network-based models.For the classification of wind fields based on measurements from ground-based meteorological stations, we used a Kohonen neural network.Later, we added another tool called "Gaussian processes" (which, like MLPANN, is a universal approximator) and the tool "decision trees." We expanded the use to point-based forecasting of basic meteorological variables. In recent years, with high computational capabilities available even without supercomputers, we have been using these tools for surrogate models that can represent or predict pollutant concentration fields in the atmosphere, not only for individual points but for entire areas around pollution sources.Other Slovenian groups from Slovenian Meteorological Agency, the Department of Meteorology, and the Faculty of Computer Science have, in recent years, used machine learning for global medium-range forecasts of daily averages of meteorological variables and for post-processing weather forecasts from physical models.In the presentation, we will explain the basic principles of building models based on artificial neural networks in a way that is understandable to laypeople. For experts in the field, we will showcase numerous examples of their application. For the latter group, we hope these examples will inspire others to expand the use of these modern tools even further.

Closed-loop stimulation of many neurological disorders, such as epilepsy, is an emerging technology and regarded as a promising alternative for surgical and drug treatment. In this paper, a real-time seizure detection algorithm based on STFT and support vector machine (SVM) and its field-programmable gate array (FPGA) implementation are proposed. With a two-stage patient-specific channel selection and feature selection mechanism, those redundant and uncorrelated spectral features are removed from the entire feature set. The evaluation results on CHB-MIT epilepsy database show that the mean detection latency of the proposed algorithm is 6 s, the sensitivity is 98.4&#x0025;, and the false detection rate is 0.356/h. The performance of our proposed algorithm is comparable to other existing seizure detection algorithms. Moreover, we implement the proposed seizure detection algorithm on Xilinx Zynq-7000 XC7Z020 with high level synthesis. Each classification of the input electroencephalography signal can be finished within 313 <inline-formula> <tex-math notation="LaTeX">$\mu \text{s}$ </tex-math></inline-formula>, and the power consumption of the programmable logic is only 380 mW at 100 MHz. In hardware implementation, an optimization strategy for the nested-loop structure within nonlinear SVM is proposed to improve pipeline efficiency. Compared with existing method, the experimental result shows that our method can speed up the nonlinear SVM by <inline-formula> <tex-math notation="LaTeX">$1.70\times $ </tex-math></inline-formula>, <inline-formula> <tex-math notation="LaTeX">$1.53\times $ </tex-math></inline-formula>, <inline-formula> <tex-math notation="LaTeX">$1.37\times $ </tex-math></inline-formula>, and <inline-formula> <tex-math notation="LaTeX">$1.26\times $ </tex-math></inline-formula> with the unroll factor equal to 1&#x2013;4 at the same DSP utilization rate. The evaluation results affirm the possibility of integrating the proposed algorithm and FPGA implementation into a wearable seizure control device.

An artificial neural network(ANN) is a type of computing system that mimics the functioning of the neural networks in a biological brain. In this paper, an ANN has been simulated on an Field Programmable Gate Array (FPGA) for the application of handwritten digit recognition using the sigmoid and hyperbolic tangent(tanh) activation functions. Both software and hardware simulations have been carried out, using python and Verilog HDL respectively. The two activation functions were implemented on an Artix-7 FPGA. A comparison has been made between the sigmoid and tanh activation functions based on speed, accuracy, and hardware required, and it has been inferred that the tanh function is best for the application of handwritten digit recognition as it has 3% higher accuracy and uses 5 less LUTs than the sigmoid activation function.

ICA (Independent Component Analysis) technique can be used to detect difference in sequence of images. However, the algorithm is usually highly computation intensive. In this paper, FPGA (Field Programmable Gate Array) implementation to speed up the computation of a non-iterative algebraic ICA is described. System level approach to the design of FPGA prototype is explained. Although FPGA does not offer an optimized hardware implementation when compared to ASIC (Application Specific Integrated Circuit), it allows short development time and enables verification of algorithms in hardware at a low cost.

In this study, Field Programmable Gate Array (FPGA) implementation of Conic Section Function Neural Network (CSFNN) for a classification problem focused on iris plant is presented. This work demonstrates for the first time to our knowledge, the feed-forward computation of CSFNN implementation on FPGA. Using l6-bit floating point arithmetic and the look-up tables (LUTs) for the sigmoid function and the square root function, 83% and 72% of slices and LUTs on Spartan 3-E XC3S1600E are used for the realization of CSFNN with five neurons. The classification results obtained from the FPGA implementation and software simulation show that the accuracy error between two platforms is only 0.1%.

An artificial neural network (ANN) model was developed to monitor the density of rigid foam PVC on-line during the extrusion process using an ultrasound sensor mounted on the extruder die. Acoustic properties of the polymer melt, measured by multiple ultrasound echoes propagating through the polymer melt, were used to train a multilayer perceptron (MLP) artificial neural network to estimate the density of the extruded foam. The foam density was varied by varying the processing conditions, i.e. heating zones and screw speed, during the extrusion process. A high correlation was found to exist between the acoustic properties of the polymer melt and the foam density using a three-layer multilayer perceptron artificial neural network.

This paper proposes a hardware oriented dropout algorithm for an efficient field-programmable gate array (FPGA) implementation. Dropout is a regularization technique, which is commonly used in neural networks such as multilayer perceptrons (MLPs), convolutional neural networks (CNNs), among others. To generate a dropout mask to randomly drop neurons during training phase, random number generators (RNGs) are usually used in software implementations. However, RNGs consume considerable FPGA resources in hardware implementations. The proposed method is able to minimize the resources required for FPGA implementation of dropout by performing a simple rotation operation to a predefined dropout mask. We apply the proposed method to MLPs and CNNs and evaluate them on MNIST and CIFAR-10 classification. In addition, we employ the proposed method in GoogLeNet training using own dataset to develop a vision system for home service robots. The experimental results demonstrate that the proposed method achieves the same regularized effect as the ordinary dropout algorithm. Logic synthesis results show that the proposed method significantly reduces the consumption of FPGA resources in comparison to the ordinary RNG-based approaches.

The relationships between physico-chemical and rheological properties are considered complex nonlinear systems. Thus, the artificial neural network (ANN) and regression models were used for the rheological characterization of Brazilian honeys, based on low-cost measurements of water content and temperature. The steady shear viscosity (η) performed well when measured in the test phase in a 2-12-1 neuron multilayer perceptron (MLP) ANN (model 1) with a root mean square error (RMSE) and correlation coefficient (r) equal to 0.0430 and 0.9681, respectively. The parameter loss modulus (G″), storage modulus (G′) and complex viscosity (η*) were predicted in the temperature sweep test by small amplitude oscillatory shear (SAOS) measurements during heating and cooling, and the MLP ANNs with architectures of 2-9-3 (model 2) and 2-3-3 (model 3) showed RMSE values equal to 0.0261 and 0.0387 in the test phase, respectively. For all the determined parameters, non-linear exponential models showed similar results to models 1, 2 and 3. An ANN with 3-9-3 architecture (model 4) showed RMSE and r for G′ equal to 0.0158 and 0.7301, for G″ equal to 0.0176 and 0.9581, and for η* equal to 0.0407 and 0.9647, respectively, in the test phase for date of the frequency sweep test obtained by SAOS. These results were far superior to those obtained by second-order multiple linear models. The acquisition of all models is an important application for the processing of honey and honey-based products, since these properties are essential in engineering calculations and quality control of products.

Despite the substantial impact of rivers on the global marine litter problem, riverine litter has been accorded inadequate consideration. Therefore, our objective was to detect riverine litter by utilizing middle-scale multispectral satellite images and machine learning (ML), with the Tisza River (Hungary) as a study area. The Very High Resolution (VHR) images obtained from the Google Earth database were employed to recognize some riverine litter spots (a blend of anthropogenic and natural substances). These litter spots served as the basis for training and validating five supervised machine-learning algorithms based on Sentinel-2 images [Artificial Neural Network (ANN), Support Vector Classifier (SVC), Random Forest (RF), Naïve Bays (NB) and Decision Tree (DT)]. To evaluate the generalization capability of the developed models, they were tested on larger unseen data under varying hydrological conditions and with different litter sizes. Besides the best-performing model was used to investigate the spatio-temporal variations of riverine litter in the Middel Tisza. According to the results, almost all the developed models showed favorable metrics based on the validation dataset (e.g., F1-score; SVC: 0.94, ANN: 0.93, RF: 0.91, DT: 0.90, and NB: 0.83); however, during the testing process, they showed medium (e.g., F1-score; RF:0.69, SVC: 0.62; ANN: 0.62) to poor performance (e.g., F1-score; NB: 0.48; DT: 0.45). The capability of all models to detect litter was bounded to the pixel size of the Sentinel-2 images. Based on the spatio-temporal investigation, hydraulic structures (e.g., Kisköre Dam) are the greatest litter accumulation spots. Although the highest transport rate of litter occurs during floods, the largest litter spot area upstream of the Kisköre Dam was observed at low stages in summer. This study represents a preliminary step in the automatic detection of riverine litter; therefore, additional research incorporating a larger dataset with more representative small litter spots, as well as finer spatial resolution images is necessary.

Recent advances in intelligence technology have boosted the development of micro- Unmanned Air Vehicles (UAVs) including Sliver Fox, Shadow, and Scan Eagle for various surveillance and reconnaissance applications. These affordable and reusable devices have to fit a series of size, weight, and power constraints. Cameras used on such micro-UAVs are therefore mounted directly at a fixed angle without any motion-compensated gimbals. This mounting scheme has resulted in the so-called <i>jitter effect</i> in which jitter is defined as sub-pixel or small amplitude vibrations. The jitter blur caused by the jitter effect needs to be corrected before any other processing algorithms can be practically applied. Jitter restoration has been solved by various optimization techniques, including Wiener approximation, maximum <i>a-posteriori</i> probability (MAP), etc. However, these algorithms normally assume a spatial-invariant blur model that is not the case with jitter blur. Szu et al. developed a smart real-time algorithm based on auto-regression (AR) with its natural generalization of unsupervised artificial neural network (ANN) learning to achieve restoration accuracy at the sub-pixel level. This algorithm resembles the capability of the human visual system, in which an agreement between the pair of eyes indicates "signal", otherwise, the jitter noise. Using this non-statistical method, for each single pixel, a deterministic blind sources separation (BSS) process can then be carried out independently based on a deterministic minimum of the Helmholtz free energy with a generalization of Shannon's information theory applied to open dynamic systems. From a hardware implementation point of view, the process of jitter restoration of an image using Szu's algorithm can be optimized by pixel-based parallelization. In our previous work, a parallelly structured independent component analysis (ICA) algorithm has been implemented on both Field Programmable Gate Array (FPGA) and Application-Specific Integrated Circuit (ASIC) using standard-height cells. ICA is an algorithm that can solve BSS problems by carrying out the all-order statistical, decorrelation-based transforms, in which an assumption that neighborhood pixels share the same but unknown mixing matrix <b>A</b> is made. In this paper, we continue our investigation on the design challenges of firmware approaches to smart algorithms. We think two levels of parallelization can be explored, including pixel-based parallelization and the parallelization of the restoration algorithm performed at each pixel. This paper focuses on the latter and we use ICA as an example to explain the design and implementation methods. It is well known that the capacity constraints of single FPGA have limited the implementation of many complex algorithms including ICA. Using the reconfigurability of FPGA, we show, in this paper, how to manipulate the FPGA-based system to provide extra computing power for the parallelized ICA algorithm with limited FPGA resources. The synthesis aiming at the pilchard re-configurable FPGA platform is reported. The pilchard board is embedded with single Xilinx VIRTEX 1000E FPGA and transfers data directly to CPU on the 64-bit memory bus at the maximum frequency of 133MHz. Both the feasibility performance evaluations and experimental results validate the effectiveness and practicality of this synthesis, which can be extended to the spatial-variant jitter restoration for micro-UAV deployment.

Electrocardiogram analysis using a combination of statistical, geometric, and nonlinear heart rate variability features

This paper presents a thorough discussion of the proposed field-programmable gate array (FPGA) implementation for fringe pattern demodulation using the one-dimensional continuous wavelet transform (1D-CWT) algorithm. This algorithm is also known as wavelet transform profilometry. Initially, the 1D-CWT is programmed using the C programming language and compiled into VHDL using the ImpulseC tool. This VHDL code is implemented on the Altera Cyclone IV GX EP4CGX150DF31C7 FPGA. A fringe pattern image with a size of 512×512 pixels is presented to the FPGA, which processes the image using the 1D-CWT algorithm. The FPGA requires approximately 100 ms to process the image and produce a wrapped phase map. For performance comparison purposes, the 1D-CWT algorithm is programmed using the C language. The C code is then compiled using the Intel compiler version 13.0. The compiled code is run on a Dell Precision state-of-the-art workstation. The time required to process the fringe pattern image is approximately 1 s. In order to further reduce the execution time, the 1D-CWT is reprogramed using Intel Integrated Primitive Performance (IPP) Library Version 7.1. The execution time was reduced to approximately 650 ms. This confirms that at least sixfold speedup was gained using FPGA implementation over a state-of-the-art workstation that executes heavily optimized implementation of the 1D-CWT algorithm.

State estimation is used currently in wide area electrical systems for real time analysis. Studies have shown that in HV transmission networks, where system-wide synchronized phasor measurement units are installed, voltage angle measurements can be included in the input measurements data set, with the result of improving the estimation precision. The authors developed in previous papers a SE algorithm based on Multilayer Perceptron Artificial Neural Networks. This paper extends this research by using PMU voltage magnitude and angle measurements in the input data for the ANN estimator, and shows in a case study that the estimation precision is improved. The development of advanced electricity markets in Europe and North America changed the operation and dispatch rules of power systems. In the United States and Canada, the ISO/RTO model is used to manage grid operation for approximately a half of North American Continent (1), while in Europe continuous efforts are made for coupling the national and regional electricity markets into a common market, markets supplying 75% of the demand using now a common day-ahead price mechanism and cross-border capacity management (2). High-performance tools are required for real time monitoring and control of such large systems. Enhanced state estimation (SE) algorithms are robust tools used today by system operators. Phasor measurement units (PMU) are used for synchronizing measurements used in state estimation analyses for systems covering wide geographical territories (3), and the trend is to generalize their deployment also at distribution level (4,5). Other approaches use distributed estimation, by splitting one large system in several areas estimated locally (6). The two approaches can be also combined (7). Recent research also use artificial neural networks (ANN) for state estimation (8,9). The authors proposed in (10) a SE approach which uses Multilayer Perceptron (MLP) artificial neural networks. A case study performed on a section of the Romanian HV distribution system was carried out, and the estimation results have shown a good precision. This paper extends the study by including PMU measurements into the measurement input set to improve the previous estimation results. The expected results are the improvement of estimation, a behaviour similar to classic algorithms.