Mixed-precision Quantization with Dynamical Hessian Matrix for Object Detection Network

Entropy calculation provides meaningful insight into the dynamics and complexity of temporal signals, playing a crucial role in classification tasks. These measures are able to describe intrinsic characteristics of temporal series, such as regularity, complexity or predictability. Depending on the characteristics of the signal under study, the performance of entropy as a feature for classification may vary, and not any kind of entropy calculation technique may be suitable for that specific signal. Therefore, we aim to increase entropy’s classification accuracy performance, specially in the case of Slope Entropy (SlpEn), by enhancing the information content of the patterns present in the data before calculating the entropy, with downsampling techniques. More specifically, we will be using both uniform downsampling (UDS) and non-uniform downsampling techniques. In the case of non-uniform downsapling, the technique used is known as Trace Segmentation (TS), which is a non-uniform downsampling scheme that is able to enhance the most prominent patterns present in a temporal series while discarding the less relevant ones. SlpEn is a novel method recently proposed in the field of time series entropy estimation that in general outperforms other methods in classification tasks. We will combine it both with TS or UDS. In addition, since both techniques reduce the number of samples that the entropy will be calculated on, it can significantly decrease the computation time. In this work, we apply TS or UDS to the data before calculating SlpEn to assess how downsampling can impact the behaviour of SlpEn in terms of performance and computational cost, experimenting on different kinds of datasets. In addition, we carry out a comparison between SlpEn and one of the most commonly used entropy calculation methods: Permutation Entropy (PE). Results show that both uniform and non-uniform downsampling are able to enhance the performance of both SlpEn and PE when used as the only features in classification tasks, gaining up to 13% and 22% in terms of accuracy, respectively, when using TS and up to 10% and 21% when using UDS. In addition, when downsampling to 50% of the original data, we obtain a speedup around ×2 with individual entropy calculations, while, when incorporating the downsampling algorithms into time count, speedups with UDS are between ×1.2 and ×1.7, depending on the dataset. With TS, these speedups are above ×2, while maintaining accuracy levels similar to those obtained when using the 100% of the original data. Our findings suggest that most temporal series, specially medical ones, have been measured using a sampling frequency above the optimal threshold, thus obtaining unnecessary information for classification tasks, which is then discarded when performing downsampling. Downsampling techniques are potentially beneficial to any kind of entropy calculation technique, not only those used in the paper. It is able to enhance entropy’s performance in classification tasks while reducing its computation time, thus resulting in a win-win situation. We recommend to downsample to percentages between 20% and 45% of the original data to obtain the best results in terms of accuracy in classification tasks.

With the evolution of Internet standards and advancements in various Internet and mobile technologies, especially since web 4.0, more and more web and mobile applications emerge such as e-commerce, social networks, online gaming applications and Internet of Things based applications. Due to the deployment and concurrent access of these applications on the Internet and mobile devices, the amount of data and the kind of data generated increases exponentially and the new era of Big Data has come into existence. Presently available data structures and data analyzing algorithms are not capable to handle such Big Data. Hence, there is a need for scalable, flexible, parallel and intelligent data analyzing algorithms to handle and analyze the complex massive data. In this article, we have proposed a novel distributed supervised machine learning algorithm based on the MapReduce programming model and Distance Weighted k-Nearest Neighbor algorithm called MR-DWkNN to process and analyze the Big Data in the Hadoop cluster environment. The proposed distributed algorithm is based on supervised learning performs both regression tasks as well as classification tasks on large-volume of Big Data applications. Three performance metrics, such as Root Mean Squared Error (RMSE), Determination coefficient (R2) for regression task, and Accuracy for classification tasks are utilized for the performance measure of the proposed MR-DWkNN algorithm. The extensive experimental results shows that there is an average increase of 3% to 4.5% prediction and classification performances as compared to standard distributed k-NN algorithm and a considerable decrease of Root Mean Squared Error (RMSE) with good parallelism characteristics of scalability and speedup thus, proves its effectiveness in Big Data predictive and classification applications.

The vector vortex beam (VVB) has attracted significant attention due to its intrinsic diversity of information and has found great applications in both classical and quantum communications. However, a VVB is unavoidably affected by atmospheric turbulence (AT) when it propagates through the free-space optical communication environment, which results in detection errors at the receiver. In this paper, we propose a VVB classification scheme to detect VVBs with continuously changing polarization states under AT, where a diffractive deep neural network (DDNN) is designed and trained to classify the intensity distribution of the input distorted VVBs, and the horizontal direction of polarization of the input distorted beam is adopted as the feature for the classification through the DDNN. The numerical simulations and experimental results demonstrate that the proposed scheme has high accuracy in classification tasks. The energy distribution percentage remains above 95% from weak to medium AT, and the classification accuracy can remain above 95% for various strengths of turbulence. It has a faster convergence and better accuracy than that based on a convolutional neural network.

Convolutional neural network simplification via feature map pruning

<em>This paper presents a proposed supervised classification technique namely flexible partial histogram Bayes (fPHBayes) learning algorithm. In our previous work, partial histogram Bayes (PHBayes) learning algorithm showed some advantages in the aspects of speed and accuracy in classification tasks. However, its accuracy declines when dealing with small number of instances or when the class feature distributes in wide area. In this work, the proposed fPHBayes solves these limitations in order to increase the classification accuracy. fPHBayes was analyzed and compared with PHBayes and other standard learning algorithms like first nearest neighbor, nearest subclass mean, nearest class mean, naive Bayes and Gaussian mixture model classifier. The experiments were performed using both real data and synthetic data considering different number of instances and different variances of Gaussians. The results showed that fPHBayes is more accurate and flexible to deal with different number of instances and different variances of Gaussians as compared to PHBayes.</em>

Robust gas sensing is fundamental to safety, environmental monitoring, and industrial control. However, the design of intelligent gas analysis algorithms remains constrained by a key instrumentation challenge: the lack of generalization across heterogeneous sensor arrays, gas compositions, and operating conditions. This study addresses this gap by introducing two deep learning models—GraphCapsNet for mixture classification and GraphANet for concentration estimation—designed to operate directly on raw sensor data from structurally distinct platforms. GraphCapsNet integrates Graph Convolutional Networks (GCNs) with dynamic routing mechanisms to extract key features from temporal data, while GraphANet combines GCNs with self-attention mechanisms to identify concentration-related features. Unlike prior approaches that rely on dataset-specific retraining or feature engineering, both models are deployed under fixed configurations across two divergent datasets, encompassing different sensor types, chamber sizes, and gas species. GraphCapsNet achieved over 98.00% accuracy in classification tasks, and GraphANet attained an R<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> score exceeding 0.96 across various gas components. These results highlight the models’ exceptional accuracy, stability, and scalability, offering a potential foundation for real-world deployment in variable sensing environments.

Classification of Distribution Network Planning Documents Based on LSTM Neural Network

Voice disorders are among the most prevalent problems affecting a huge proportion of the population, with the majority of them resulting in communication issues and a deterioration in quality of life. Early detection leads to more effective treatment. Thus, better approaches for accurately diagnosing vocal abnormalities are essential. Machine learning techniques such as Convolutional Neural Networks (CNNs), Random Forests, and transfer learning approaches have all proven useful in this discipline. CNNs offer incredible audio signal processing capabilities, since they can convert raw voice data into spectrograms and automatically extract key properties. CNNs can detect minor patterns associated with numerous voice disorders, improving classification accuracy. Random Forest is a technique used in ensemble learning to enhance robustness and accuracy in classification tasks. The utility of this approach stems from its capacity to overcome existing data processing issues, making it a substantial contribution to the area. Thus, we propose the system with CNN and Random Forest as the classification models and obtained the 97.31% accuracy for transfer learning model.

Convolutional neural networks (CNNs) typically compress high‐resolution images to minimize computational requirements. However, this can lead to loss of information and reduced accuracy in classification tasks. This paper introduces WaveNet, a novel approach for processing high‐resolution images using wavelet‐domain inputs in CNNs. We address the challenge of maintaining classification accuracy with high‐resolution inputs while minimizing computational complexity. Our method employs wavelet packet transform (WPT) for image pre‐processing, extracting detailed multi‐scale and directional information from high‐resolution images. We propose a wavelet‐adaptive efficient channel attention (WAECA) module to dynamically select the most informative wavelet subbands. Popular CNN architectures like ResNet‐50 and MobileNetV2 are adapted by replacing their initial convolutional layers with wavelet‐transformed inputs, enabling direct learning in the wavelet domain. Experiments conducted on the Caltech‐256 and ALOT datasets demonstrate that WaveNet improves classification accuracy while reducing computational complexity. For instance, our wavelet‐enhanced ResNet‐50 achieves a Top‐1 accuracy of 72.47% on Caltech‐256, outperforming the baseline (70.65%) while reducing FLOPs from 16.52G to 3.98G. Similar improvements are observed across different architectures and datasets. We also evaluate various wavelet filters and ResNet backbones, finding that the bior1.1 filter and ResNet‐50 provide optimal performance. This work presents a practical solution for developing more accurate and efficient models for high‐resolution inputs without extensive computational resources or complex architectural modifications.

Bionic visual processing hardware serves as the core technology for mimicking the efficient information processing of biological vision systems and forms the foundation for perceiving and recognizing both static and dynamic scenes. However, most current bionic visual hardware remains at the level of individual devices and simple arrays. Full hardware implementation of bionic vision chips, as well as comprehensive demonstrations in practical application scenarios, is still scarce. This work proposes a complete optoelectronic insect‐inspired visual sensor. We realize a Si QDs/ReS 2 heterogeneous integrated neuromorphic chip fabricated using a 180‐nm CMOS process technology and demonstrate its capabilities on both static and dynamic visual tasks. Based on the wavelength‐dependent synaptic properties of ReS 2 , this neuromorphic photoreceptor chip exhibits high‐fidelity discrimination between purple and red features. Through light pulse sequence modulation, it achieves temporal encoding of dynamic visual information and can precisely identify leaf movement trajectories in eight directions. We further achieved 3D perception capabilities, attaining 99.4% accuracy in classification tasks. This work establishes a novel 1D/2D/3D heterogeneously integrated material platform and a CMOS‐compatible integration strategy for a low‐power, multifunctional brain‐inspired neuromorphic sensor chip, advancing the application of optoelectronic devices in artificial intelligence and machine vision.

Deep learning has achieved significant success in a variety of modern applications, but this remarkable achievement is often accompanied by inefficiency in terms of memory storage, computing resources, and inference speed. Such inefficiency prevents wide applications of modern deep models in practical problems, especially on resource-constrained computing platforms. Particularly, developing highly efficient neural networks while maintaining high prediction accuracy with much-reduced computing requirements is challenging and urgent. Meanwhile, theoretical analysis and an intuitive understanding of the underlying mechanism of performance bottleneck of efficient models would provide practical guidance and technical solutions to alleviate trade-offs between accuracy and efficiency. In addition, for more special cases, such as some specific hardware with non-conventional computing architectures, the problem can be more complicated. Such hardware has the potential of much higher energy efficiency and much faster inference speed, but the harmful effects induced by hardware non-idealities, such as limited computing precision and storage size, as well as noise and nonlinearity inherent in the analog interface circuits. Software solutions for these hardware effects can be more economical and can reduce the iteration period. In this thesis, we will explore the potential for simultaneously achieving high efficiency and high prediction accuracy in neural networks, with several different techniques and theoretical analyses, for both conventional and innovative computing platforms. In the first part, we explore the implementation of highly efficient generative adversarial networks (GANs) to guarantee their capability of generating high-quality images within a predefined computational budget. The key challenges lie in identifying the optimal architecture for the generative models, while preserving the quality of the generated images from the compressed models, despite their reduced computational cost and storage capability. To achieve this goal, we introduce inception blocks into the generative models to enlarge the searching space and propose a novel neural architecture search (NAS) algorithm with much faster searching speed. Also, we design a new knowledge distillation technique, where extra layers for distillation are avoided. With our proposed techniques, we achieve better quality for generated images than the original huge models. Our work is the first in the world to achieve better image generation quality than the original huge model, demonstrating that for generative models, decoupling the optimization of high performance and high efficiency can be technically achieved. In the second part, we investigate quantization techniques for deep neural networks and study the necessity of high-precision multiplications in quantized models. Specifically, we study if we could achieve the same accuracy for classification tasks without using either floating-point multiplication or the 32-bit fixed-point multiplication for the scaling operation. We find that by properly training, we are able to achieve the same performance with 8-bit fixed-point multiplication alone. Specifically, we conduct some statistical analysis of weights and activations, and based on the semi-empirical analysis, we propose an algorithm to determine the fixed-point formats automatically during training. Also, we study how to fuse batch normalization layers with the associated convolution layers and derive the effective weights after layer fusion. For models with residual connections, we find that for those sibling layers with the same preceding layer, assigning private fractional lengths to them is crucial to achieving good performance. We demonstrate that high-precision multiplication is not indispensable for quantized neural networks to achieve good accuracy. In the third part, we study the problem of training neural networks for innovative computing platforms, especially in-memory computing systems. To this end, we study the extra quantization step caused by limited precision in the computing platform and analyze how to make the operation differentiable to facilitate the optimization process. We also study the impact of PIM quantization on the output scale, and by analyzing the training dynamics, we propose rescaling techniques to achieve convergent training. For non-ideal effects of noise and nonlinearity introduced by the analog converting interface, we adopt batch normalization calibration to reduce their impact. Our proposed framework is generic and applicable for different PIM quantization schemes, including native, differential, and bit-serial, and is verified with a real-chip prototype.--Author's abstract

Sequential Quadratic Programming (SQP) for optimal control in direct numerical simulation of turbulent flow

Quantized neural networks are well known for reducing the latency, power consumption, and model size without significant harm to the performance. This makes them highly appropriate for systems with limited resources and low power capacity. Mixed-precision quantization offers better utilization of customized hardware that supports arithmetic operations at different bitwidths. Quantization methods either aim to minimize the compression loss given a desired reduction or optimize a dependent variable for a specified property of the model (such as FLOPs or model size); both make the performance inefficient when deployed on specific hardware, but more importantly, quantization methods assume that the loss manifold holds a global minimum for a quantized model that copes with the global minimum of the full precision counterpart. Challenging this assumption, we argue that the optimal minimum changes as the precision changes, and thus, it is better to look at quantization as a random process, placing the foundation for a different approach to quantize neural networks, which, during the training procedure, quantizes the model to a different precision, looks at the bit allocation as a Markov Decision Process, and then, finds an optimal bitwidth allocation for measuring specified behaviors on a specific device via direct signals from the particular hardware architecture. By doing so, we avoid the basic assumption that the loss behaves the same way for a quantized model. Automatic Mixed-Precision Quantization for Edge Devices (dubbed AMED) demonstrates its superiority over current state-of-the-art schemes in terms of the trade-off between neural network accuracy and hardware efficiency, backed by a comprehensive evaluation.

The inference of deep neural networks (DNNs) on resource-constrained embedded systems introduces non-trivial trade-offs among model accuracy, computational latency, and hardware limitations, particularly when real-time constraints must be satisfied. This paper presents ALADIN, an accuracy-latency-aware design-space inference analysis framework for mixed-precision quantized neural networks (QNNs) targeting scratchpad-based AI accelerators. ALADIN enables the evaluation and analysis of inference bottlenecks and design trade-offs across accuracy, latency, and resource consumption without requiring deployment on the target platform, thereby significantly reducing development time and cost. The framework introduces a progressive refinement process that transforms a canonical QONNX model into platform-aware representations by integrating both platform-independent implementation details and hardware-specific characteristics. ALADIN is validated using a cycle-accurate simulator of a RISC-V based platform specialized for AI workloads, demonstrating its effectiveness as a tool for quantitative inference analysis and hardware-software co-design. Experimental results highlight how architectural decisions and mixed-precision quantization strategies impact accuracy, latency, and resource usage, and show that these effects can be precisely evaluated and compared using ALADIN, while also revealing subtle optimization tensions.

Studies into the effects of coarse quantisation on the correlation function of astronomical signals have typically assumed that the undesirable part of the signal will be white noise with a Gaussian distribution. Due to the advent of the modern communications era, interference from man-made satellites has made it necessary to reinvestigate these effects using interference distributions which are non-Gaussian. This paper presents the results of an investigation into the effects of a) an interfering sine tone, and b) an interfering phase-shift keyed sine tone, on the normalised correlation coefficients of both continuum and spectral line sources, before and after the application of a coarse quantisation scheme. It was found that the effect of this interference is to offset the unquantised correlation coefficients by an amount, which is proportional to both the interfering signal strength and the value of its auto-correlation function at the delay introduced by array geometry. The effect of quantisation on signals containing such non-Gaussian interference was found to be approximately the same as for the Gaussian case.