ARLP: Automatic multi-agent transformer reinforcement learning pruner for one-shot neural network pruning

Relevance. When solving problems of stabilization of dynamic objects, classical model predictive control is widely used. It provides high quality control by solving the optimization problem at each step, but it has significant computing costs, which limits its application in real-time systems with high requirements for update frequency. Therefore, the question of investigating the applicability of a neural network regulator trained on a model predictive regulator (MPC) when solving the problem of stabilizing the position of a dynamic object with a limited computational and time resource is relevant. Goal.  The purpose of the presented work was to develop and study a neural network regulator trained on the basis of an MPC regulator to stabilize the position of a dynamic object on a mobile platform. Methods. When performing the work, methods of system analysis, simulation modeling, as well as experimental tests on the bench were used. Results and conclusions. As part of the study, a neural network regulator was developed and trained that approximates the behavior of MPC based on data obtained when controlling a real balancing platform. The training was conducted on the input and output data of the MPC without using the internal model of the system, which made it possible to reproduce the dynamics of the regulator at significantly lower computational costs. Experimental results showed that the neural network model provides a stabilization quality comparable to the original MPC, while the calculation time was reduced from 47 ms to 1.6 ms, which amounted to an acceleration value of 29 times. The proposed approach demonstrates the potential of neural network control methods in the problems of replacing complex optimization regulators for systems with limited computing resources.

Forest fires pose a serious threat to the environment with the potential of causing ecological harm, financial losses, and human casualties. While research suggests that climate change will increase the frequency and severity of these fires, recent developments in deep learning and convolutional neural networks (CNN) have greatly enhanced fire detection techniques and capability. These models can be leveraged by unmanned aerial vehicles (UAVs) to automatically monitor burning areas. However, drones can carry only limited computational and power resources; therefore, on-board computing capabilities are constrained by hardware limitations. This work focuses on the design of segmentation models to identify and localize active burning areas from aerial RGB images processed on limited computing resources. To achieve this goal, the research compares the performance of different variants of the DeepLabv3 neural network model for fire segmentation when trained and tested with the FLAME dataset using a k-fold cross validation approach. Experimental results are compared with U-Net, a benchmark model used with the FLAME dataset, by implementing this model in the same codebase as the DeepLabv3 model. This work demonstrates that a refined version of DeepLabv3, with a MobileNetv2 backbone using pretrained layers and a simplified atrous spatial pyramid pooling (ASPP) module, yields a similar performance to U-Net, with a precision of 87.8% and a recall of 83.2%, while only requiring 20% of the number of parameters involved with the U-Net topology. This significantly reduces memory and power consumption, enabling longer UAV flight duration and reducing the processing overhead associated with sensor input, making it more suitable for deployment on unmanned aerial vehicles. The model’s compact architecture, implemented using TensorFlow and Keras for model design and training, along with OpenCV for image preprocessing, makes it portable and easy to integrate with edge devices such as NVIDIA Jetson boards.

In the ocean, submesoscale physical phenomena O(100m) to O(1km) have been reported to play a key role in ocean oxygen ventilation, nutrient supply to the surface ocean, and carbon export, as well as the transfer of energy to larger scales [1]. However, due to limitations in computational resources, current ocean general circulation models are frequently run at resolutions on the order of O(10km) to O(100km) and cannot directly resolve submesoscale turbulence (i.e., subgrid-scale phenomena). Therefore, parameterization schemes are required to simulate these subgrid-scale phenomena.Recent advances in machine learning have triggered the active exploration of data-driven approaches to parameterization for subgrid-scale phenomena that utilize data from observations and simulations. In previous studies [2, 3], the neural network is trained directly using the same variables as the neural network's output, such as viscosity and diffusivity coefficients. However, this approach does not guarantee that the inferred model parameters accurately represent the state of subgrid-scale phenomena they aim to reproduce. We propose a novel parameterization method for estimating diffusivity and viscosity parameters to parameterize subgrid-scale phenomena and have implemented this method in MITgcm, an ocean simulator [4, 5]. This method trains a neural network using the state variables (i.e., velocity fields, potential temperature, and salinity) derived from the simulation results at a resolution that can directly resolve subgrid-scale phenomena. Therefore, unlike previous studies, the diffusivity and viscosity parameters inferred by the trained network can reproduce the global state of subgrid-scale phenomena.The ocean simulator MITgcm is implemented in Fortran, which does not have a built-in package to compute gradients within the neural network, in contrast to deep learning libraries (e.g., PyTorch) like Python. In our previous work [4, 5], we used a quasi-newton optimization method, which does not require computation of these gradients. However, the optimization performance of this method was limited. In this study, we use adjoint code within MITgcm to compute gradients for optimizing neural networks and examine the effect of different optimizers on training performance. AcknowledgmentsThis work used computational resources of supercomputer Fugaku provided by the RIKEN Center for Computational Science through the HPCI System Research Project (Project ID: hp240394).

<p>Deep neural networks have achieved tremendous success in many domains (e.g., computer vision~\cite{Alexnet12,vggnet15,fastrcnn15}, speech recognition~\cite{hinton2012deep,dahl2012context}, natural language processing~\cite{dahl2012context,collobert2011natural}, games~\cite{silver2017mastering,silver2016mastering}), however, there are still many challenges in deep learning comunity such as how to speed up training large deep neural networks, how to compress large nerual networks for mobile/embed device without performance loss, how to automatically design the optimal network structures for a certain task, and how to further design the optimal networks with improved performance and certain model size with reduced computation cost.</p> <p>To speed up training large neural networks, we propose to use multinomial sampling for dropout, i.e., sampling features or neurons according to a multinomial distribution with different probabilities for different features/neurons. To exhibit the optimal dropout probabilities, we analyze the shallow learning with multinomial dropout and establish the risk bound for stochastic optimization. By minimizing a sampling dependent factor in the risk bound, we obtain a distribution-dependent dropout with sampling probabilities dependent on the second order statistics of the data distribution. To tackle the issue of evolving distribution of neurons in deep learning, we propose an efficient adaptive dropout (named evolutional dropout) that computes the sampling probabilities on-the-fly from a mini-batch of examples.</p> <p>To compress large neural network structures, we propose a simple yet powerful method for compressing the size of deep Convolutional Neural Networks (CNNs) based on parameter binarization. The striking difference from most previous work on parameter binarization/quantization lies at different treatments of $1\times 1$ convolutions and $k\times k$ convolutions ($k>1$), where we only binarize $k\times k$ convolutions into binary patterns. By doing this, we show that previous deep CNNs such as GoogLeNet and Inception-type Nets can be compressed dramatically with marginal drop in performance. Second, in light of the different functionalities of $1\times 1$ (data projection/transformation) and $k\times k$ convolutions (pattern extraction), we propose a new block structure codenamed the pattern residual block that adds transformed feature maps generated by $1\times 1$ convolutions to the pattern feature maps generated by $k\times k$ convolutions, based on which we design a small network with $\sim 1$ million parameters. Combining with our parameter binarization, we achieve better performance on ImageNet than using similar sized networks including recently released Google MobileNets.</p> <p>To automatically design neural networks, we study how to design a genetic programming approach for optimizing the structure of a CNN for a given task under limited computational resources yet without imposing strong restrictions on the search space. To reduce the computational costs, we propose two general strategies that are observed to be helpful: (i) aggressively selecting strongest individuals for survival and reproduction, and killing weaker individuals at a very early age; (ii) increasing mutation frequency to encourage diversity and faster evolution. The combined strategy with additional optimization techniques allows us to explore a large search space but with affordable computational costs.</p> <p>To further design the optimal networks with improved performance and certain model size under reduced computation cost, we propose an ecologically inspired genetic approach for neural network structure search , that includes two types of succession: primary and secondary succession as well as accelerated extinction. Specifically, we first use primary succession to rapidly evolve a community of poor initialized neural network structures into a more diverse community, followed by a secondary succession stage for fine-grained searching based on the networks from the primary succession. Accelerated extinction is applied in both stages to reduce computational cost. In addition, we also introduce the gene duplication to further utilize the novel block of layers that appeared in the discovered network structure.</p>

Grapheme-to-phoneme (G2P) mapping plays a vital role in the development of text-to-speech systems, particularly for languages with complex morphology and limited computational resources such as Kannada. Existing G2P techniques based on handcrafted rules or supervised machine learning depend heavily on linguistic knowledge or large volumes of labeled data, making them difficult to scale for low-resource languages. To address these challenges, this work introduces a reinforcement learning–driven approach for Kannada grapheme-to-phoneme conversion. The task is modeled as a stepwise decision process in which an intelligent agent incrementally predicts phoneme sequences from written text by learning an optimal policy guided by a reward function that reflects pronunciation correctness and phonological coherence. By learning through interaction rather than direct supervision, the proposed framework adapts effectively to novel word forms and pronunciation variations. Experimental evaluation on a Kannada text dataset shows that the reinforcement learning model produces more accurate phoneme sequences and lower error rates when compared to conventional rule-based and statistical G2P methods. These findings demonstrate the potential of reinforcement learning as a flexible and data-efficient solution for building robust G2P systems in low-resource Indian languages, ultimately enhancing the clarity and naturalness of synthesized Kannada speech.

Pneumatic Artificial Muscle (PAM) is an attractive device to be used as an actuator for humanoid robots because of its high power-to-weight ratio and good flexibility. However, both the modeling and the controlling of PAM-driven robots are challenging due to the high nonlinearities of a PAM's air pressure dynamics and its mechanical structure. This paper focuses on applying Reinforcement Learning (RL) to the control of a PAM-driven robots without our knowledge of its model. We propose a new RL algorithm, Local Update Dynamic Policy Programming (LUDPP), as an extension of Dynamic Policy Programming (DPP). This algorithm exploits the nature of smooth policy update of DPP to considerably reduce the computational complexity in both time and space: at each iteration, this algorithm only updates the value function locally throughout the whole state-action space. We applied LUDPP to control one finger (2 DOFs with a 12-dimensional state-action space) of Shadow Dexterous Hand, a PAM-driven humanoid robot hand. Experimental results suggest that our method can achieve successful control of such a robot with a limited computational resource whereas other conventional value function based RL algorithms (DPP, LSPI) cannot.

S[formula omitted]NN: Time step reduction of spiking surrogate gradients for training energy efficient single-step spiking neural networks

In the evolving realm of autonomous vehicle navigation, the integration of fuzzy logic and neural networks presents a formidable challenge, particularly in the context of real-time, on-the-fly neural network training. This paper addresses the gap in dynamic and adaptable training methods necessary for navigating unpredictable environments with limited computational resources. The primary objective of our study is to empirically validate a hybrid training approach that combines fuzzy logic with back-propagation learning algorithms, aiming to optimize neural network performance under hardware constraints. Our methodology leverages a fuzzy logic trainer to provide initial training sets dynamically, which guide the neural network in adjusting its weights in real time, thus facilitating adaptive learning during navigation tasks. The findings reveal that this integrated approach not only enhances the learning efficiency of neural networks but also significantly improves navigation accuracy in real-time scenarios. These advancements contribute to the field by demonstrating the feasibility of deploying more adaptable and robust autonomous navigation systems, potentially expanding their application in more diverse and challenging environments.

Common challenges in the area of robotics include issues such as sensor modeling, dynamic operating environments, and limited on-broad computational resources. To improve decision making, robots need a dependable framework to facilitate communication between different modules and the optimal action for real-world applications. The Robotics Operating System (ROS) and Reinforcement Learning (RL) are two promising approaches that help accomplish precise control, seamless integration of sensors-actuators, and exhibit learned behavior. The ROS enables seamless communication between heterogeneous components, while RL focuses on learning optimal behaviors through trial-and-error scenarios. Combining the ROS and RL offers superior decision making, improved perception, enhanced automation, and reliability. This work focuses on investigating ROS-based RL applications across various domains, aiming to enhance understanding through comprehensive discussion, analysis, and summarization. We base our evaluation on the application area, type of RL algorithm used, and degree of ROS–RL integration. Additionally, we provide summary of seminal works that define the current state of the art, along with GitHub repositories and resources for research purposes. Based on the review of successfully implemented projects, we make recommendations highlighting the advantages and limitations of RL techniques for specific applications and environments. The ultimate goal of this work is to advance the robotics field by providing a comprehensive overview of the recent important works that incorporate both the ROS and RL, thereby improving the adaptability of these emerging techniques.

It is undoubtful that artificial intelligence (AI) is being the trend of computer science and this trend is still ongoing in the far future even though technologies are being developed suddenly fast because computer science does not reach the limitation of approaching biological world yet. Machine learning (ML), which is a branch of AI, is a spearhead but not a key of AI because it sets first bricks to build up an infinitely long bridge from computer to human intelligence, but it is also vulnerable to environmental changes or input errors. There are three typical types of ML such as supervised learning, unsupervised learning, and reinforcement learning (RL) where RL, which is adapt progressively to environmental changes, can alleviate vulnerability of machine learning but only RL is not enough because the resilience of RL is based on iterative adjustment technique, not based on naturally inherent aspects like data mining approaches and moreover, mathematical fundamentals of RL lean forwards swing of stochastic process. Fortunately, artificial neural network, or neural network (NN) in short, can support all three types of ML including supervised learning, unsupervised learning, and RL where the implicitly regressive mechanism with high order through many layers under NN can improve the resilience of ML. Moreover, applications of NN are plentiful and multiform because three ML types are supported by NN; besides, NN training by backpropagation algorithm is simple and effective, especially for sample of data stream. Therefore, this study research is an introduction to NN with easily understandable explanations about mathematical aspects under NN as a beginning of stepping into deep learning which is based on multilayer NN. Deep learning, which is producing amazing results in the world of AI, is undoubtfully being both spearhead and key of ML with expectation that ML improved itself by deep learning will become both spearhead and key of AI, but this expectation is only for ML researchers because there are many AI subdomains are being invented and developed in such a way that we cannot understand exhaustedly. It is more important to recall that NN, which essentially simulates human neuron system, is appropriate to the philosophy of ML that constructs an infinitely long bridge from computer to human intelligence.

Transcranial magnetic stimulation (TMS) is a growing therapy for a variety of psychiatric and neurological disorders that arise from or are modulated by cortical regions of the brain represented by singular 3D target points. These target points are often determined manually with assistance from a pre-operative T1-weighted MRI, although there is growing interest in automatic target point localisation using an atlas. However, both approaches can be time-consuming which has an effect on the clinical workflow, and the latter does not take into account patient variability such as the varying number of cortical gyri where these targets are located. This paper proposes a multi-resolution convolutional neural network for point localisation in MR images for a priori defined points in increasingly finely resolved versions of the input image. This approach is both fast and highly memory efficient, allowing it to run in high-throughput centres, and has the capability of distinguishing between patients with high levels of anatomical variability. Preliminary experiments have found the accuracy of this network to be [Formula: see text]mm, compared to [Formula: see text]mm for deformable registration and [Formula: see text]mm for a human expert. For most treatment points, the human expert and proposed CNN statistically significantly outperform registration, but neither statistically significantly outperforms the other, suggesting that the proposed network has human-level performance. The human-level performance of this network indicates that it can improve TMS planning by automatically localising target points in seconds, avoiding more time-consuming registration or manual point localisation processes. This is particularly beneficial for out-of-hospital centres with limited computational resources where TMS is increasingly being administered.

Satellite edge computing can provide communication and computing services to areas lacking ground network coverage, enhancing the computational capabilities of terrestrial users. However, due to the relatively limited computational resources available on satellites, cloud-edge collaborative computing in Low Earth Orbit (LEO) satellite communication scenarios emerges as a more optimal solution. In this paper, a cloud-edge collaborative offloading algorithm based on Deep Deterministic Policy Gradient (DDPG) algorithm is proposed to solve the offloading decision and resource allocation problems. The optimization problem is modeled as Markov decision process, and the comprehensive delay and energy consumption cost of computing tasks completed by ground users are optimized. Simulation results validate that the proposed algorithm has good convergence properties and lower comprehensive cost than processing all tasks locally and offloading all tasks to LEO satellites.

Bayesian networks are a popular diagnosis method, whose structures are usually defined by human experts and parameters are learned from data. For the increasing complexity of modern systems, building their structures based on physical behaviors is becoming a difficult task. However, the improvement of data collection techniques motivates learning their structures from data, where greedy search is a typical iterative method. In each iteration, it generates multiple structure candidates by modifying one edge, evaluates these structures by scores based on data and selects the best structure for the next iteration. This method is costly because there are too many structures to be evaluated. To solve this problem, we frame the traditional greedy search by Markov decision process and propose an efficient Bayesian network learning approach by integrating reinforcement learning into it. In our approach, a convolutional neural network is employed as the value function to approximate scores. Before evaluating structures using data, the neural network is used to predict scores. Structure candidates with a low predicted score are discarded. By avoiding unnecessary computation, the cooperation of reinforcement learning and greedy search effectively improves the learning efficiency. Two systems, a 10-tank system with 21 monitored variables and the classic Tennessee Eastman process with 52 variables, are employed to demonstrate our approach. The experiment results indicate that the computation cost of our method was reduced by 30%~50%, and the diagnosis accuracy was almost the same.

The rapid proliferation of Internet of Things (IoT) devices has led to a surge in energy consumption, necessitating innovative strategies to enhance energy efficiency. Machine Learning (ML) has emerged as a promising approach to optimize energy utilization in IoT networks by enabling intelligent data processing, adaptive resource allocation, and predictive maintenance. This paper explores various ML techniques, including supervised, unsupervised, and reinforcement learning, for optimizing energy consumption in IoT ecosystems. It highlights key challenges such as data heterogeneity, real-time processing constraints, and limited computational resources while discussing potential solutions like federated learning, edge computing, and energy-aware neural networks. The study also presents recent advancements and case studies that demonstrate the effectiveness of ML-driven energy-efficient IoT frameworks. By integrating ML-based approaches, IoT networks can achieve enhanced sustainability, prolonged device lifespan, and improved operational efficiency. Future research directions focus on lightweight ML models, decentralized learning paradigms, and AI-driven energy harvesting techniques to further advance energy-efficient IoT networks.

The growing number of low-power smart devices in the Internet of Things is coupled with the concept of "Edge Computing", that is moving some of the intelligence, especially machine learning, towards the edge of the network. Enabling machine learning algorithms to run on resource-constrained hardware, typically on low-power smart devices, is challenging in terms of hardware (optimized and energy-efficient integrated circuits), algorithmic and firmware implementations. This paper presents FANN-on-MCU, an open-source toolkit built upon the Fast Artificial Neural Network (FANN) library to run lightweight and energy-efficient neural networks on microcontrollers based on both the ARM Cortex-M series and the novel RISC-V-based Parallel Ultra-Low-Power (PULP) platform. The toolkit takes multi-layer perceptrons trained with FANN and generates code targeted at execution on low-power microcontrollers either with a floating-point unit (i.e., ARM Cortex-M4F and M7F) or without (i.e., ARM Cortex M0-M3 or PULP-based processors). This paper also provides an architectural performance evaluation of neural networks on the most popular ARM Cortex-M family and the parallel RISC-V processor called Mr. Wolf. The evaluation includes experimental results for three different applications using a self-sustainable wearable multi-sensor bracelet. Experimental results show a measured latency in the order of only a few microseconds and a power consumption of few milliwatts while keeping the memory requirements below the limitations of the targeted microcontrollers. In particular, the parallel implementation on the octa-core RISC-V platform reaches a speedup of 22x and a 69% reduction in energy consumption with respect to a single-core implementation on Cortex-M4 for continuous real-time classification.