LionHeart: A Layer-Based Mapping Framework for Heterogeneous Systems With Analog In-Memory Computing Tiles

A well-known vegetable crop farmed all throughout the world is the potato. The possibility of a large number of diseases, however, could also impair the harvest. In order to quickly deal with the crop, it is vital for the planter to recognise the type of contamination. It had been determined that the leaves were a simple indicator of many diseases. Many Machine Learning (ML) techniques as well as Convolution Neural Network (CNN) models are superior in the literature for the detection of tomato crop diseases. CNN models rely on deep learning and neural networks, which are superior than traditional machine learning methods like decision trees and kNN. Although well before CNN creates beautiful artwork, the wide range of subjects they cover renders them operationally intensive. This painting suggests a less complex CNN model with eight convolution layer At the publicly accessible dataset PlantVillage, the recommended compact model performs better than the standard machine learning techniques and pre-trained fashions with an accuracy of 98.4%. The PlantVillage dataset includes 39 classes of different crops, including tomato diseases. These crops include apple, potato, maize, grapes, and more. In comparison to VGG16, which has an amazing accuracy of 93% 5% in pretrained patterns, k-NN has the absolute best accuracy of 94% in conventional ML approaches. After image augmentation, picture pre-processing was used to improve the suggested CNN's overall performance by having the ability to modify the brightness of the image using the knowledge of a prospective variable involving a randomized width of the image. The suggested model, which has a 98% accuracy rate, is also incredibly effective with datasets other than PlantVillage.

Cyber-security experts often require the help of an automated process that filters and classifies network attacks. To apply specific preventive measures for securing networks, the classification of the attack type is the key. Many Machine Learning (ML) models have been proposed as a base for Network Intrusion Detection (NID) systems. However, their performance varies based on multiple factors. For instance, an ML model fitted on a highly imbalanced dataset can be biased toward over-represented attack types. On the other hand, paying attention only to the ML model’s performance in the minority classes can negatively affect its performance in the majority classes. This paper proposes an NID system that addresses the issue of imbalanced datasets and uses Convolutional Neural Networks (CNN) to classify the different attack types. We compare the performance of our proposed system to other systems that use: Random Over-Sampling (ROS), Synthetic Minority Oversampling TEchnique (SMOTE), Adaptive Synthetic Sampling (ADASYN), and Generative Adversarial Networks (GAN). Using the NSL-KDD and the BoT-IoT datasets for benchmarking, we show that our proposed system performs well in the minority classes: recall scores of 70.50% and 72.08% on the User to Root (U2R) and Remote to Local (R2L) attack classes of the NSL-KDD dataset, respectively, while maintaining an overall False Alarm Rate (FAR) of 6.50% and a recall of 90.46% on the binary classification task. Our proposed system scores a weighted average F1-Score of 99.45% on the multi-class classification task using the BoT-IoT dataset.

Processing-in-memory (PIM) architectures have demonstrated great potential in accelerating numerous deep learning tasks. Particularly, resistive random-access memory (RRAM) devices provide a promising hardware substrate to build PIM accelerators due to their abilities to realize efficient in-situ vector-matrix multiplications (VMMs). However, existing PIM accelerators suffer from frequent and energy-intensive analog-to-digital (A/D) conversions, severely limiting their performance. This paper presents a new PIM architecture to efficiently accelerate deep learning tasks by minimizing the required A/D conversions with analog accumulation and neural approximated peripheral circuits. We first characterize the different dataflows employed by existing PIM accelerators, based on which a new dataflow is proposed to remarkably reduce the required A/D conversions for VMMs by extending shift and add (S+A) operations into the analog domain before the final quantizations. We then leverage a neural approximation method to design both analog accumulation circuits (S+A) and quantization circuits (ADCs) with RRAM crossbar arrays in a highly-efficient manner. Finally, we apply them to build an RRAM-based PIM accelerator (i.e., \textbf{Neural-PIM}) upon the proposed analog dataflow and evaluate its system-level performance. Evaluations on different benchmarks demonstrate that Neural-PIM can improve energy efficiency by 5.36x (1.73x) and speed up throughput by 3.43x (1.59x) without losing accuracy, compared to the state-of-the-art RRAM-based PIM accelerators, i.e., ISAAC (CASCADE).

Modern convolutional neural networks (CNNs) in computer vision are trained on a large number of images from numerous categories to form rich discriminative feature extractors. Inference using such models on resource-constrained Internet-of-Things (IoT) platforms poses a challenge and an opportunity. Having limited computation, storage, and energy budgets, most IoT platforms are not capable of hosting such compute intensive models. However, typical IoT applications demand detection of a relatively small number of categories, albeit the specific categories of interest may change at runtime as the context evolves dynamically. In this letter, we take advantage of the opportunity to address the challenge. Specifically, we develop a novel transformation to the architecture of a given CNN, so that the majority of the inference workload is allocated to class-specific disjoint branches, which can be dynamically executed or skipped, based on the context, to fulfill the application requirements. Experiments demonstrate that our approach preserves the classification accuracy for the classes of interest, while proportionally decreasing the model complexity and inference workload.

Convolutional neural networks (CNN) is a widely known deep learning architecture and achieves higher accuracies in speech processing and computer vision applications. CNN has arisen as best option in the fields such as video surveillance, mobile robot vision, smart factories, and medical diagnostics. CNN has shown its tremendous potential for image understanding in Cardio-Vascular (CV) systems. But, the tremendous potential of CNNs came up with additional computational burden. To get an outstanding performance of CNN-based algorithms, it requires large amount of enormous applications and memory resources. Matrix multiplication (MM) is the most fundamental computational operation taking place in CNN. Researchers in this field found that multiplication operation is the most resource-intensive and power-hungry operation in CNN. This work aims to design an FPGA-based CNN using systolic array which can improve both the accuracy and hardware efficiency of convolutional neural network. Here, we made design optimisation in multiplier unit which brings a significant impact on overall performance of convolutional neural networks.

Predicting panoramic indoor lighting from a single perspective image is a fundamental but highly ill-posed problem in computer vision and graphics. To achieve locale-aware and robust prediction, this problem can be decomposed into three sub-tasks: depth-based image warping, panorama inpainting and high-dynamic-range (HDR) reconstruction, among which the success of panorama inpainting plays a key role. Recent methods mostly rely on convolutional neural networks (CNNs) to fill the missing contents in the warped panorama. However, they usually achieve suboptimal performance since the missing contents occupy a very large portion in the panoramic space while CNNs are plagued by limited receptive fields. The spatially-varying distortion in the spherical signals further increases the difficulty for conventional CNNs. To address these issues, we propose a local-to-global strategy for large-scale panorama inpainting. In our method, a depth-guided local inpainting is first applied on the warped panorama to fill small but dense holes. Then, a transformer-based network, dubbed PanoTransformer, is designed to hallucinate reasonable global structures in the large holes. To avoid distortion, we further employ cubemap projection in our design of PanoTransformer. The high-quality panorama recovered at any locale helps us to capture spatially-varying indoor illumination with physically-plausible global structures and fine details.

Convolutional Neural Network (CNN) has achieved great success in various fields of machine vision, such as image classification and recognition, image segmentation, and video analysis. In specific applications, it is often necessary to customize the network structure by altering image size, convolution kernel size, pooling size, network architecture, and the number of network layers. These customizations pose significant challenges to the architecture of CNN accelerator especially in real-time systems, and as the number of these network variables increases, the internal convolution calculations become extremely demanding. The inference speed and energy consumption of CNN are becoming more and more important, which requires the design of the accelerator to adapt to different network architecture and run efficiently. Field-programmable gate arrays (FPGAs) is an ideal choice for a CNN accelerator due to its high programmability and low power consumption. To address the aforementioned challenges, we present NRUCA, a novel network reconfigurable universal CNN accelerator based on FPGA. On one hand, we have designed a flexible architecture that can dynamically configure CNN’s network structure parameters. These parameters can be sent to the FPGA via a configuration file, enabling the same design to run different CNN networks and allowing to modify on the fly. This forms the basis of the architecture’s adaptability presented in this paper. On the other hand, we employ a multi-channel parallel and efficient pipeline matrix multiplication architecture to implement the convolution and fully-connected layers, which constitute the majority of the computational load in CNNs. By utilizing an innovative inverted matrix multiplication algorithm and an interleaving cache data method, we reduce the internal cache required for multi-channel data. Furthermore, most of the intermediate calculation data does not need to be output to DDR, significantly improving the operation efficiency of the accelerator. We also fully leverage the FPGA chip architecture to compile multiple calculation kernels, which can be flexibly scheduled and combined to cater to various application scenarios. Based on the flexible and efficient overall design architecture, the layout of various resources of FPGA is balanced, which enables the designed FPGA project to compile four acceleration kernels with the running clock up to 300MHz in Xilinx Vitis and test them on Xilinx Alveo U250. We verified the adaptability and acceleration capabilities of our FPGA project using LeNet, AlexNet, VGG11, VGG13, VGG16, and VGG19 networks. Experiments demonstrated that our FPGA architecture achieves 33X and 35X energy savings compared to the Intel Xeon 5220R CPU, and 1.08X and 1.05X energy savings compared to the Nvidia Tesla P100 GPU, when accelerating AlexNet and VGG16, respectively.

In this paper, we analyze the suitability of convolutional neural network (CNN) inference workloads on a phase-change memory (PCM) platform. CNN inference has an average of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$14\times$</tex> more read requests than write requests (i.e., read dominant) and a significantly low last-level cache misses per kilo instructions (LLC MPKI) of 2 on average (i.e., computation intensive). In addition, to compare the latency and energy of PCM and DRAM systems, we evaluate CNN inference workloads on two memory systems through a memory simulator. As a result, compared to DRAM, PCM can save total energy by 54% on average, but instruction per cycle (IPC) of PCM is reduced by an average of 28%. In conclusion, CNN inference is a workload suitable for PCM in terms of energy efficiency, but it must be accompanied by a scheme to improve IPC for practical use.

Computing-in-memory (CIM) with emerging non-volatile resistive memory devices has demonstrated remarkable performance in data-intensive applications, such as neural networks and machine learning. A crosspoint memory array enables naturally parallel computation of matrix–vector multiplication (MVM) in the analog domain, offering significant advantages in terms of speed, energy efficiency, and computational density. However, the intrinsic device non-ideality residing in analog conductance state distorts the MVM precision and limits the application to high-precision scenarios, e.g., scientific computing. Yet, a theoretical framework for guiding reliable computing-in-memory designs has been lacking. In this work, we develop an analytical model describing the constraints on bit precision and row parallelism for reliable MVM operations. By leveraging the concept of capacity from information theory, the impact of non-ideality on computational precision is quantitively analyzed. This work offers a theoretical guidance for optimizing the quantized margins, providing valuable insights for future research and practical implementation of reliable CIM.

Exploring the effective signal features of electroencephalogram (EEG) signals is an important issue in the research of brain-computer interface (BCI), and the results can reveal the motor intentions that trigger electrical changes in the brain, which has broad research prospects for feature extraction from EEG data. In contrast to previous EEG decoding methods that are based solely on a convolutional neural network, the traditional convolutional classification algorithm is optimized by combining a transformer mechanism with a constructed end-to-end EEG signal decoding algorithm based on swarm intelligence theory and virtual adversarial training. The use of a self-attention mechanism is studied to expand the receptive field of EEG signals to global dependence and train the neural network by optimizing the global parameters in the model. The proposed model is evaluated on a real-world public dataset and achieves the highest average accuracy of 63.56% in cross-subject experiments, which is significantly higher than that found for recently published algorithms. Additionally, good performance is achieved in decoding motor intentions. The experimental results show that the proposed classification framework promotes the global connection and optimization of EEG signals, which can be further applied to other BCI tasks.

Memristive crossbar arrays can be used to realize matrix-vector multiplication (MVM) operations in constant time complexity by exploiting the Kirchhoff's circuit laws. This is enabled by the parallel read of the entire array in a single time step. However, parallel writing is prohibitive in such arrays due to limitations on the current that could be accumulated along the wires. Hence, loading the matrix elements into such an array still incurs significant time penalty. Another key challenge is the achievable computational precision. To overcome these challenges, we propose a unit-cell array design where each unit-cell comprises four memristive devices each attached to a selection transistor. Moreover, the array is organized in such a way that the selection transistors can be turned on in a diagonal fashion. We experimentally demonstrated this concept by fabricating a 2 x 2 unit-cell array based on projected phase-change memory (PCM) devices in 90 nm CMOS technology. It is shown that using the diagonal connections, the write operations can be parallelized while maintaining the current limit of the back-end-of-the-line metallization. Moreover, the increase in write time due to having more devices per unit-cell is minimized through a combination of single-shot and iterative programming schemes. Finally, we present experimental results on MVM operations that demonstrate improved computational precision exceeding that of a 4-bit fixed-point implementation.

Convolutional neural networks (CNNs) are widely adopted in various applications. State-of-the-art CNN models deliver excellent classification performance, but they require a large amount of computation and data exchange because they typically employ many processing layers. Among these processing layers, convolution layers, which carry out many multiplications and additions, account for a major portion of computation and memory access. Therefore, reducing the amount of computation and memory access is the key for high-performance CNNs. In this study, we propose a cost-effective neural network accelerator, named CENNA, whose hardware cost is reduced by employing a cost-centric matrix multiplication that employs both Strassen’s multiplication and a naïve multiplication. Furthermore, the convolution method using the proposed matrix multiplication can minimize data movement by reusing both the feature map and the convolution kernel without any additional control logic. In terms of throughput, power consumption, and silicon area, the efficiency of CENNA is up to 88 times higher than that of conventional designs for the CNN inference.

In today’s technological landscape, embedded and IoT devices face escalating demands for performance and power efficiency in inference tasks employing Convolution Neural Networks (CNNs). Various methods have emerged to improve performance, including quantization and the adoption of fixed-point formats instead of floating-point formats. RISC-V Instruction Set Architecture (ISA) being open-source and with a modular specification poses as a suitable platform for targeting such applications for embedded processing. The RISC-V’s standard P-extension represents a significant advancement in integer-based computation, offering new ways for achieving higher speedups through integer-packed SIMD instructions. This work presents the first RISC-V processor based on RV32IM base ISA with an optional support of P-extension Version 0.9.11-draft-20211209 called the P-CORE. The P-CORE implements P-extension ISA for RV32 specification. The implemented processor is evaluated for Xilinx Zynq Ultrascale+ FPGA using different test case functions; matrix multiplication, convolution, max-pooling, and fully connected. A CNN used for detecting handwritten digits called Lenet-5 is also used as a test case to evaluate the P-CORE. The evaluation also presents Design Space Exploration (DSE) for P-extension by exploiting the level of data-level-parallelism; SIMD8 (8-bit (4-way)) and SIMD16 (16-bit (2-way)). Our investigation demonstrates that with the P-extension activated, CNN algorithm computations can achieve speedups of up to 17x for max-pooling, 7x for matrix multiplication, and 4.8x for fully connected networks, accompanied by similar enhancements in power efficiency. The processor achieved a maximum performance of 335.1 MIOPS and 490.6 MIOPS/W efficiency for matrix multiplication using -O3 compiler optimization.

Convolutional neural networks (CNNs) have demonstrated promising results in various applications such as computer vision, speech recognition, and natural language processing. One of the key computations in many CNN applications is matrix multiplication, which accounts for a significant portion of computation. Therefore, hardware accelerators to effectively speed up the computation of matrix multiplication have been proposed, and several studies have attempted to design hardware accelerators to perform better matrix multiplications in terms of both speed and power consumption. Typically, accelerators with either a two-dimensional (2D) systolic array structure or a single instruction multiple data (SIMD) architecture are effective only when the input matrix has shapes that are close to or similar to a square. However, several CNN applications require multiplications of non-squared matrices with various shapes and dimensions, and such irregular shapes lead to poor utilization efficiency of the processing elements (PEs). This study proposes a configurable engine for neural network acceleration, called CONNA, whose computation engine can conduct matrix multiplications with highly utilized computing units, regardless of the access patterns, shapes, and dimensions of the input matrices by changing the shape of matrix multiplication conducted in the physical array. To verify the functionality of the CONNA accelerator, we implemented CONNA as an SoC platform that integrates a RISC-V MCU with CONNA on Xilinx VC707 FPGA. SqueezeNet on CONNA achieved an inference performance of 100 frames per second (FPS) with 2.36 mm2 and 83.55 mW in a 65 nm process, improving efficiency by up to 34.1 times better than existing accelerators in terms of FPS, silicon area, and power consumption.

Analog in-memory computing (AIMC) using memristive devices is considered a promising Non-von Neumann approach for deep learning (DL) inference tasks. However, inaccuracies in the programming of devices, that are attributed to conductance variations, pose a key challenge toward achieving sufficient compute precision for DL inference. Fortunately, conduction variations in memristive devices, such as phase-change memory (PCM) devices, exhibit a strong state dependence. This state dependence can be exploited in synaptic unit cells that comprise more than one memristive device, to encode positive or negative weights. In such multi-memristive unit cells, we propose a method that optimally maps the weights to the device conductance values, by maximizing the number of devices at the stable SET and RESET states. We demonstrate that this method reduces the matrix-vector multiplication (MVM) error and is more resilient to non-ideal device retention characteristics. With this approach, we increase the mean experimental inference accuracy of a network trained for MNIST classification by 0.71% on two PCM-based AIMC cores, and the hardware-realistic simulated top-1 accuracy of a network trained for ImageNet classification by 0.28%, while significantly reducing variability across multiple experiment instances.