Massively parallel binary neural network inference for detecting ships in FPGA systems on the edge

Binary neural networks (BNNs) can substantially accelerate a neural network's inference time by substituting its costly floating-point arithmetic with bit-wise operations. Nevertheless, state-of-the-art approaches reduce the efficiency of the data flow in the BNN layers by introducing intermediate conversions from 1 to 16/32 bits. We propose a novel training scheme, denoted as BNN-Clip, that can increase the parallelism and data flow of the BNN pipeline; specifically, we introduce a clipping block that reduces the data width from 32 bits to 8. Furthermore, we decrease the internal accumulator size of a binary layer, usually kept using 32 bits to prevent data overflow, with no accuracy loss. Moreover, we propose an optimization of the batch normalization layer that reduces latency and simplifies deployment. Finally, we present an optimized implementation of the binary direct convolution for ARM NEON instruction sets. Our experiments show a consistent inference latency speed-up (up to 1.3 and 2.4× compared to two state-of-the-art BNN frameworks) while reaching an accuracy comparable with state-of-the-art approaches on datasets like CIFAR-10, SVHN, and ImageNet.

Binary neural networks have recently begun to be used as a highly energy- and computation-efficient image processing technique for computer vision tasks. This paper proposes a novel extension of existing binary neural network technology based on the use of a Hadamard transform in the input layer of a binary neural network. Previous state-of-the-art binary neural networks require floating-point arithmetic at several parts of the neural network model computation in order to maintain a sufficient level of accuracy. The Hadamard transform is similar to a Discrete Cosine Transform (used in the popular JPEG image compression method) except that it does not include expensive multiplication operations. In this paper, it is shown that the Hadamard transform can be used to replace the most expensive floating-point arithmetic portion of a binary neural network. In order to test the efficacy of this proposed method, three types of experiments were conducted: application of the proposed method to several state-of-the-art neural network models, verification of its effectiveness in a large image dataset (ImageNet), and experiments to verify the effectiveness of the Hadamard transform by comparing the performance of binary neural networks with and without the Hadamard transform. The results show that the Hadamard transform can be used to implement a highly energy-efficient binary neural network with only a miniscule loss of accuracy.

With the wide adoption of AI applications, there is a pressing need of\nenabling real-time neural network (NN) inference on small embedded devices, but\ndeploying NNs and achieving high performance of NN inference on these small\ndevices is challenging due to their extremely weak capabilities. Although NN\npartitioning and offloading can contribute to such deployment, they are\nincapable of minimizing the local costs at embedded devices. Instead, we\nsuggest to address this challenge via agile NN offloading, which migrates the\nrequired computations in NN offloading from online inference to offline\nlearning. In this paper, we present AgileNN, a new NN offloading technique that\nachieves real-time NN inference on weak embedded devices by leveraging\neXplainable AI techniques, so as to explicitly enforce feature sparsity during\nthe training phase and minimize the online computation and communication costs.\nExperiment results show that AgileNN's inference latency is >6x lower than the\nexisting schemes, ensuring that sensory data on embedded devices can be timely\nconsumed. It also reduces the local device's resource consumption by >8x,\nwithout impairing the inference accuracy.\n

S[formula omitted]NN: Time step reduction of spiking surrogate gradients for training energy efficient single-step spiking neural networks

The binarized neural network (BNN) inference accelerators show great promise in cost- and power-restricted domains. However, the performances of these accelerators are still severely limited by the significant redundancies in BNNs inference. In this brief, we introduce channel-aware sparse accelerator (CAA) to alleviate the performance degradations induced by the redundancies in BNNs while maintaining original accuracies. First, motivated by the observation that the convolution processes of our rebuilt rectangle kernels contain many redundant operations which can be skipped by exploiting the BNN-specific property, we convert the entire original XNOR-popcount convolutions of each neuron into channel-aware-popcount (CAP) operations for all binarized convolutional and fully-connected layers in CAA by employing rectangle kernel simplification strategy and eliminate the unnecessary operations. Meanwhile, these CAP operations can be implemented to directly gain the final output without any extra steps. Furthermore, inspired by our new observations on two specific kinds of properties of the CAP operations, we adopt group pruning approach to save the remaining redundant CAP operations. Experimental results show that our design evaluated on an embedded FPGA achieves 4.2- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$6.6{\times }$ </tex-math></inline-formula> inference-speedup, 3.6- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$5.5{\times }$ </tex-math></inline-formula> energy-efficiency enhancement, and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1.35{\times }$ </tex-math></inline-formula> resource-efficiency improvement compared with state-of-the-art works.

A mixed-signal binarized neural network (BNN) processor for the MNIST image classification is demonstrated based on analogue circuit networks. BNN algorithm for training neural networks with binary weights and activations reduces power consumption and memory size. This algorithm is a design to perform core operations of multi-layer perceptron (MLP) to reduce complexity and power consumption using analogue circuits. The mixed-signal BNN processor employs a current mirror neuron to perform multiply-and-accumulate (MAC) operations and sign activation functions. The near-threshold current mirror neuron that computes the key operations of the BNN algorithm is used to achieve low power consumption. The design occupies 0.065 mm2 in 28-nm CMOS with 560B of on-chip SRAM. The 28-nm CMOS test-chip achieves energy efficiency of 24.1 TOPS/W and 94% accuracy on the MNIST image classification. This design with binary weights and activations exhibits only 5% degradation in accuracy compared with the models with floating-point precision.

With the rise of data-centric applications such as edge AI and neuromorphic computing, there is increasing demand for memory solutions that overcome the limitations of conventional nonvolatile devices. Selector-only memory (SOM), which stores data through threshold voltage (Vth) modulation in chalcogenide-based selector materials, offers a compact and scalable alternative. However, narrow read window margins and Vth drift remain major reliability concerns. In this work, we introduce a Sn-doped GeSbSeTe (Sn-GSST) material system that enhances SOM performance by reducing trap depth, increasing the population of shallow band-tail trap states, and widening the Vth margin. These improvements enable stable multibit switching and improved endurance. We evaluate the device’s system-level applicability through binary neural network (BNN) inference on the German Traffic Sign Recognition Benchmark (GTSRB) dataset, where Vth-induced bit error rate (BER) are modeled using statistical distributions. Sn-GSST devices show significantly lower BER of less than 0.01, leading to improved inference robustness. Finally, Shannon entropy-based error correction code (ECC) analysis confirms that the reduced BER of Sn-GSST leads to lower redundancy overhead and higher inference efficiency. This study demonstrates how material-level engineering can directly translate to system-level reliability and performance in neuromorphic memory applications.

With the recent growth of the Internet of Things (IoT) and the demand for faster computation, quantized neural networks (QNNs) or QNN-enabled IoT can offer better performance than conventional convolution neural networks (CNNs). With the aim of reducing memory access costs and increasing the computation efficiency, QNN-enabled devices are expected to transform numerous industrial applications with lower processing latency and power consumption. Another form of QNN is the binarized neural network (BNN), which has 2 bits of quantized levels. In this paper, CNN-, QNN-, and BNN-based pattern recognition techniques are implemented and analyzed on an FPGA. The FPGA hardware acts as an IoT device due to connectivity with the cloud, and QNN and BNN are considered to offer better performance in terms of low power and low resource use on hardware platforms. The CNN and QNN implementation and their comparative analysis are analyzed based on their accuracy, weight bit error, RoC curve, and execution speed. The paper also discusses various approaches that can be deployed for optimizing various CNN and QNN models with additionally available tools. The work is performed on the Xilinx Zynq 7020 series Pynq Z2 board, which serves as our FPGA-based low-power IoT device. The MNIST and CIFAR-10 databases are considered for simulation and experimentation. The work shows that the accuracy is 95.5% and 79.22% for the MNIST and CIFAR-10 databases, respectively, for full precision (32-bit), and the execution time is 5.8 ms and 18 ms for the MNIST and CIFAR-10 databases, respectively, for full precision (32-bit).

We trained a binarized neural network (BNN) which can be used for handwritten digit recognition on software, and implemented this BNN on the FPGA using software training parameters without the help of high-level synthesis (HLS). This BNN contains an input layer, two convolutional layers, two pooling layers, a fully connected layer and an output layer. The first convolution layer contains 10 convolution kernels, each of which has a size of $5^{ \ast} 5$ . Each convolution kernel uses 25 arithmetic units to calculate in parallel, the calculation of 10 convolution kernels is also parallel. The size of the convolution kernels in the second convolutional layer is $3^{ \ast} 3$ , the input and output channels are 10, we use 900 XNOR gates for calculation here. Compared with the traditional CNN, it does not need to consume DSP resources and requires less storage space for the parameters. And the power consumption obtained by Quartus ii software of this BNN is 0.136 W. The time to identify a picture is $18 \mu \mathrm{s}$ , and the accuracy rate is 85% without the batch normalize (BN) layer.

Infrared remote sensing images are becoming increasingly popular due to their superior penetration and resistance to light interference. However, challenges still remain when applying them in real-world applications: 1) raw infrared images suffer from severe stripes interference, and the preprocessing techniques used to obtain standard image products for subsequent detection tasks tend to be time-consuming, which fails to meet the application requirements; 2) current destriping techniques may inevitably weaken the local contrast between some objects and the local background since they need to consider the gray consistency of the overall image; 3) in low-resolution images, dim and small infrared targets are challenging to discriminate, resulting in high false alarms. To address these challenges, we proposed a progressive task-based universal network for raw infrared image ship detection while simultaneously removing stripes. First, we built an integrated network consisting of two components: the stripe denoising component (SDC) and the object detection component (ODC). We also designed a feedback loss adjustment mechanism to enhance the focus of the SDC on the target area. Second, a directed two-branch network was constructed for efficient stripe noise removal, including an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">x</i> -direction branch for feature enhancement and a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">y</i> -direction branch for grayscale smoothing. Finally, a parallel network with two labels was designed to extract the inherent features of the target and the background, as well as their relationship features, to achieve refined ship detection. We conducted experiments on a self-assembled dataset from the GaoFen-1 satellite to validate our approach. The experimental results demonstrated that the proposed method outperformed other state-of-the-art methods in infrared image ship detection.

Process-in-Memory realized by nonvolatile Task-Scheduling and Resource-Sharing XNOR-Net hardware Accelerator architectures

This paper proposes an effective method of improving ship detection performance of a compact high-frequency (HF) radar system which has been primarily optimized for observing surface radial current velocities and bearings. Previously developed ship detection systems have been vulnerable to error sources such as environmental noise and clutter when they are applied in a compact HF radar optimized for observing surface current. In particular, the influences of error are reduced by applying a principle component analysis of the generated range-Doppler maps. A compact radar signal model is first developed by the data acquired from an operating compact HF radar site. The proposed method is then validated by comparing it to the conventional ship detection method in terms of detection and false alarm rates. The experimental results confirm that the proposed method shows superior performance in both simulated and practical environments.

In recent years, the growing demand for synthetic aperture radar (SAR) ship detection on mobile and satellite platforms has made lightweight detection models an important research focus. However, existing SAR ship detection methods are often limited by high computational cost and inefficient utilization of intrinsic SAR characteristics, particularly the loss of edge and high-frequency information during downsampling and the insufficient interaction of multi-scale features across network layers. To address these issues, we propose a lightweight and efficient detection framework named ESENet, which is designed from the perspective of preserving discriminative information, improving cross-scale feature interaction, and reducing structural redundancy. Specifically, a lightweight feature extraction module is introduced to alleviate high-frequency and edge information loss caused by conventional pooling operations, thereby enhancing the representation of small and low-contrast ship targets commonly present in SAR images. Furthermore, a Slender bidirectional feature pyramid network is designed by streamlining cross-layer connections and reducing redundant fusion paths, enabling effective multi-scale feature aggregation with lower computational complexity while strengthening edge-aware feature propagation. In addition, an optimized detection head with parameter sharing across scales is employed to further reduce model parameters and computational overhead without sacrificing detection accuracy or inference speed. Extensive experiments conducted on multiple public SAR ship detection datasets demonstrate the effectiveness of the proposed ESENet, achieving average accuracy rates of 91.1% on HRSID, 98.9% on SSDD, 97.3% on SSDD+, and 72.1% on synthetic and real-world small-object ship detection dataset, while maintaining a compact model size of only 1.3 M parameters, which confirms its suitability for resource-constrained SAR ship detection applications.

The increasing paradigm shift towards i ntermittent computing has made it possible to intermittently execute d eep neural network (DNN) inference on edge devices powered by ambient energy. Recently, n eural architecture search (NAS) techniques have achieved great success in automatically finding DNNs with high accuracy and low inference latency on the deployed hardware. We make a key observation, where NAS attempts to improve inference latency by primarily maximizing data reuse, but the derived solutions when deployed on intermittently-powered systems may be inefficient, such that the inference may not satisfy an end-to-end latency requirement and, more seriously, they may be unsafe given an insufficient energy budget. This work proposes iNAS, which introduces intermittent execution behavior into NAS to find accurate network architectures with corresponding execution designs, which can safely and efficiently execute under intermittent power. An intermittent-aware execution design explorer is presented, which finds the right balance between data reuse and the costs related to intermittent inference, and incorporates a preservation design search space into NAS, while ensuring the power-cycle energy budget is not exceeded. To assess an intermittent execution design, an intermittent-aware abstract performance model is presented, which formulates the key costs related to progress preservation and recovery during intermittent inference. We implement iNAS on top of an existing NAS framework and evaluate their respective solutions found for various datasets, energy budgets and latency requirements, on a Texas Instruments device. Compared to those NAS solutions that can safely complete the inference, the iNAS solutions reduce the intermittent inference latency by 60% on average while achieving comparable accuracy, with an average 7% increase in search overhead.

We experimentally validated the sub-threshold reading strategy in OxRAM+OTS crossbar arrays for low precision inference in Binarized Neural Networks. In order to optimize the 1S1R sub-threshold current margin, an experimental and theoretical statistical study on HfO <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</inf> -based 1S1R stacks with various OTS technologies has been performed. Impact of device features (OxRAM R <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">HRS</inf> , OTS non-linearity and OTS threshold current) on 1S1R sub-threshold reading is elucidated. Accuracy and power consumption of a Binarized Neural Network designed in 28nm CMOS have been estimated with Monte Carlo simulations. A gain of 3 orders of magnitude in power consumption is demonstrated in comparison with conventional threshold reading strategy, while preserving the same network accuracy.