HAS-RL: A Hierarchical Approximate Scheme Optimized With Reinforcement Learning for NoC-Based NN Accelerators

Routing algorithm design for power- and temperature-aware NoCs

Modern multicore processors increasingly rely on network-on-chip (NoC) architectures to support high-bandwidth and low-latency communication. Traditional mesh-based NoC topologies suffer from uneven load distribution, with central router nodes frequently becoming congestion hot spots. The existing approaches mainly focus on introducing bypass links or independent ring interconnects. However, bypass links can only alleviate congestion between fixed routers, and routers not included in the ring still rely on conventional mesh forwarding, limiting the overall performance improvement. To address these challenges, this article proposes a reinforcement learning-based routing for ring-augmented mesh NoC (RLRM) to improve communication efficiency in multicore processors. First, we design a ring-augmented <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$8\times 8$</tex-math> </inline-formula> mesh topology that integrates horizontal and vertical ring interconnects to achieve balanced load distribution and mitigate congestion hotspots. Second, we equip each router with a topology-aware reinforcement learning (RL) agent that dynamically selects paths based on congestion conditions, thereby proactively avoiding performance bottlenecks. Extensive experiments demonstrate that the proposed solution provides a scalable and efficient architecture for high-performance computing and multicore NoC systems. Experimental results under realistic benchmark traffic patterns show that RLRM reduces average latency by 22.78% compared to conventional mesh and static hybrid routing schemes.

NoC designs can be accepted to provision general communications between multiple IPs above multi-processor schemes on chip. In this effort we demonstrate the forming and imitation based exploration of some current architectures intended for Network on Chip (NoC). Specifically in this paper focus on the time synchronization based network, it hasthe Routing Integrated Time Synchronization protocol (RITS) that integrates post-facto time sync into a routing service also its achieved high result for QoS improvement in network. RITS, as well as reactive techniques in general, is superior to many proactive time sync protocols with respect to communication overhead. The best effort (BF) analysis of a representative reactive technique, the Routing Integrated Time Synchronization protocol (RITS) shows that in the general case, the presence of slack time cause RITS to Guaranteed throughput with the size of the network. We have using a shortest path based Traffic aware algorithm to time based data transmission quickly and efficiently on their network. It has more secure and easy to transfer data from one mobile node to another mobile node on the network.

In this paper we consider autonomous control of a snake-like robot using reinforcement learning. Conventional methods of reinforcement learning have significant problems in practical use. That is curse of dimensionally and lack of generality. To solve these problems, we focus on design of the mechanical body of the snake-like robot, and abstract necessary small state-action space from complex environments by utilizing the function of the body. To discuss the function of the body, experiments have been conducted and transition probability has been identified. As the result, we confirmed that by the function of the body, learning machine can observe different complex environments as similar simple environments.

Multiprocessor system-on-chips (MPSoCs) and chip multiprocessors (CMPs) are moving towards the integration of hundreds of heterogeneous cores on a single chip. Network-on-Chip (NoC) has been the superior interconnect fabric for MPSoCs and CMPs. Due to the rise of dark silicon with the end of Dennard Scaling, it becomes essential to design energy-efficient and high performance reconfigurable NoC between heterogeneous components based on the applications workloads (e.g., task communication demands). Because of the large and complex NoC design space in heterogeneous architectures (link bandwidth, big/little cores and routers, number of components, etc.), it becomes difficult to explore within a reasonable time for optimal trade-offs of energy-performance. Furthermore, reactive resource configuration of NoC is not effective in reducing power consumption. Therefore, we propose reinforcement learning (RL) technique to proactively configure the NoC link-bandwidths in heterogeneous architectures for energy-efficiency. NoC link-bandwidths are configured dynamically based on the learning of the system (e.g., link utilization, buffer utilization). RL agent automatically learns from past and current experiences and take the configuration decision intelligently on heterogeneous architectures to achieve its objective. Simulations under real-world benchmarks show that RL based technique reduces the energy-delay product by 30% and improves the throughput by 50% compared to non-machine learning based solutions.

Network on Chip (NoC) is a communication framework for interconnection of processing element (PE's). The excessive and parallel communication requirements of heterogeneous PE's in NoC have made the communication structure very complex. The size of the devices are scaled down to support the complexity but the size of interconnects remains the same. To support the complexity of NoC many fault tolerant routing techniques have been proposed to cope with the faulty nodes and interconnects which occur during the communication. All the routing algorithms lack the adaptiveness, robustness and they also have the drawbacks of congestion, delayed communication and latency issues. In this paper a synaptogenesis bases bio-inspired algorithm is proposed for NoC. This algorithm is fault tolerant as it adapts the mechanism of biological brain. The results show that the bio-inspired algorithm quickly recovers from the fault and it is robust and fault tolerant. The average packet network latency was increased by 11.64%, average bandwidth was reduced by 0.07% and throughput is drop to 43.30% during the network recovery from faults. While comparing with literature technique the average throughput and accepted traffic is increased by 76.92%.

Managing a Network-on-Chip (NoC) in an efficient way is a challenging task. To succeed, the operating system (OS) needs to be tuned to the capabilities and the needs of the NoC. Only by creating a tight interaction can we combine the necessary flexibility with the required efficiency. This paper illustrates such an interaction by detailing the management of communication resources in a system containing a packet-switched NoC and a closely integrated OS. Our NoC system is emulated by linking an FPGA to a PDA. We show that, with the right NoC support, the OS is able to optimize communication resource usage. Additionally, the OS is able to diminish or remove the interference between independent applications sharing a common NoC communication resource.

Knee rehabilitation programs often rely on fixed thresholds and static feedback, limiting their adaptability to dynamic patient responses. This project explores the independent application of machine learning (ML) and reinforcement learning (RL) techniques to enhance the adaptability and effectiveness of EMG-based rehabilitation, while laying the groundwork for future hybrid ML-RL solutions. Initially, curve fitting and ML models, including ridge regression, lasso regression, decision tree, random forest, and XGBoost, were used to predict key muscle performance metrics despite limited data. Subsequently, the study focused on an RL-based approach using deep Q-learning with experience replay and a target network, tailored to simulate dynamic muscle performance during rehabilitation. The state space, action space, and reward function were carefully tailored to reflect these variations, ensuring realistic interactions. The results highlight the potential of RL for real-time, patient-centric rehabilitation. While ML models provide baseline assessments, the RL model’s adaptability makes it ideal for dynamic scenarios. This thesis emphasizes the need for future implementation of hybrid ML-RL approaches to advance adaptive and personalized rehabilitation solutions, with the potential to revolutionize current practices and improve patient outcomes in the context of knee rehabilitation.

Emerging DSP applications have different latency, energy consumption and Quality of Service (QoS) requirements. An implementation of such applications requires a large number of intellectual property (IP) cores, communicating with each other, meeting the energy and latency constraints. Network-on-Chip (NoC) architectures is able to accommodate a large number of IP cores in the same chip implementing a set of complex applications. This leads to different usage of the available buffer space in the routers of the NoC system. In this work we propose power and the systematic design of novel NOC-based architectures, which realize DSP applications. Additionally, we present an integrated node resource management technique that combines priority assignment and buffer sizing so that the NoC system to best serve requirements of the considered Finally, to best of our knowledge, the implementation of DSP applications in 3D NOC architectures took place for first time. DSP applications. The proposed approach has been evaluated both on 2D and 3D mesh topologies by employing an NoC simulator and four real DSP/multimedia applications gaining an average of 34% on energy×delay product for each application. Finally, to best of our knowledge, the implementation of DSP applications in 3D NOC architectures took place for first time.

Modern network-on-chip (NoC) systems face reliability issues due to process and environmental variations. The power supply noise (PSN) in the power delivery network of a NoC plays a key role in determining reliability. PSN leads to voltage droop, which can cause timing errors in the NoC. This paper makes a novel contribution towards formally analyzing PSN in NoC systems. We present a probabilistic model checking approach to analyze key features of PSN at the behavioral level in a \(2\times 2\) mesh NoC with a uniform random traffic load. To tackle state explosion, we apply incremental abstraction techniques, including a novel probabilistic choice abstraction, based on observations of NoC behavior. The Modest Toolset is used for probabilistic modeling and verification. Results are obtained for several flit injection patterns to reveal their impacts on PSN. Our analysis finds an optimal flit pattern generation with zero probability of PSN events and suggests spreading flits rather than releasing them in consecutive cycles in order to minimize PSN.

It is undoubtful that artificial intelligence (AI) is being the trend of computer science and this trend is still ongoing in the far future even though technologies are being developed suddenly fast because computer science does not reach the limitation of approaching biological world yet. Machine learning (ML), which is a branch of AI, is a spearhead but not a key of AI because it sets first bricks to build up an infinitely long bridge from computer to human intelligence, but it is also vulnerable to environmental changes or input errors. There are three typical types of ML such as supervised learning, unsupervised learning, and reinforcement learning (RL) where RL, which is adapt progressively to environmental changes, can alleviate vulnerability of machine learning but only RL is not enough because the resilience of RL is based on iterative adjustment technique, not based on naturally inherent aspects like data mining approaches and moreover, mathematical fundamentals of RL lean forwards swing of stochastic process. Fortunately, artificial neural network, or neural network (NN) in short, can support all three types of ML including supervised learning, unsupervised learning, and RL where the implicitly regressive mechanism with high order through many layers under NN can improve the resilience of ML. Moreover, applications of NN are plentiful and multiform because three ML types are supported by NN; besides, NN training by backpropagation algorithm is simple and effective, especially for sample of data stream. Therefore, this study research is an introduction to NN with easily understandable explanations about mathematical aspects under NN as a beginning of stepping into deep learning which is based on multilayer NN. Deep learning, which is producing amazing results in the world of AI, is undoubtfully being both spearhead and key of ML with expectation that ML improved itself by deep learning will become both spearhead and key of AI, but this expectation is only for ML researchers because there are many AI subdomains are being invented and developed in such a way that we cannot understand exhaustedly. It is more important to recall that NN, which essentially simulates human neuron system, is appropriate to the philosophy of ML that constructs an infinitely long bridge from computer to human intelligence.

Fault tolerance and adaptive capabilities are challenges for modern networks-on-chip (NoC) due to the increase in physical defects in advanced manufacturing processes. Two novel adaptive routing algorithms, namely coarse and fine-grained (FG) look-ahead algorithms, are proposed in this paper to enhance 2-D mesh/torus NoC system fault-tolerant capabilities. These strategies use fault flag codes from neighboring nodes to obtain the status or conditions of real-time traffic in an NoC region, then calculate the path weights and choose the route to forward packets. This approach enables the router to minimize congestion for the adjacent connected channels and also to bypass a path with faulty channels by looking ahead at distant neighboring router paths. The novelty of the proposed routing algorithms is the weighted path selection strategies, which make near-optimal routing decisions to maintain the NoC system performance under high fault rates. Results show that the proposed routing algorithms can achieve performance improvement compared to other state of the art works under various traffic loads and high fault rates. The routing algorithm with FG look-ahead capability achieves a higher throughput compared with the coarse-grained approach under complex fault patterns. The hardware area/power overheads of both routing approaches are relatively low which does not prohibit scalability for large-scale NoC implementations.

ECTM: A network-on-chip communication model to combine task and message schedulability analysis

Reinforcement learning, as a branch of machine learning, has been gradually applied in the control field. However, in the practical application of the algorithm, the hyperparametric approach to network settings for deep reinforcement learning still follows the empirical attempts of traditional machine learning (supervised learning and unsupervised learning). This method ignores part of the information generated by agents exploring the environment contained in the updating of the reinforcement learning value function, which will affect the performance of the convergence and cumulative return of reinforcement learning. The reinforcement learning algorithm based on dynamic parameter adjustment is a new method for setting learning rate parameters of deep reinforcement learning. Based on the traditional method of setting parameters for reinforcement learning, this method analyzes the advantages of different learning rates at different stages of reinforcement learning and dynamically adjusts the learning rates in combination with the temporal-difference (TD) error values to achieve the advantages of different learning rates in different stages to improve the rationality of the algorithm in practical application. At the same time, by combining the Robbins–Monro approximation algorithm and deep reinforcement learning algorithm, it is proved that the algorithm of dynamic regulation learning rate can theoretically meet the convergence requirements of the intelligent control algorithm. In the experiment, the effect of this method is analyzed through the continuous control scenario in the standard experimental environment of ”Car-on-The-Hill” of reinforcement learning, and it is verified that the new method can achieve better results than the traditional reinforcement learning in practical application. According to the model characteristics of the deep reinforcement learning, a more suitable setting method for the learning rate of the deep reinforcement learning network proposed. At the same time, the feasibility of the method has been proved both in theory and in the application. Therefore, the method of setting the learning rate parameter is worthy of further development and research.

A reinforcement learning based artificial bee colony algorithm with application in robot path planning