Deep Reinforcement Learning-based Power Distribution Network Structure Design Optimization Method for High Bandwidth Memory Interposer

Switching operation of 10kV line's ring network is one of the effective technical measures to reduce power failure and improve supply reliability of urban power distribution network. The switching operation of ring network in urban power distribution network is not only affected by operation mode of power system and structure of distribution network, but also restrained by factors such as load and voltage difference prior to closing loop operation, and the overlarge circulating power load during closing loop operation can endanger the safety of power distribution network and equipments. Based on the complicated structure of power distribution network with multiple segments and connections in Nanyang city, the mathematic model of switching operation of ring network in power distribution network is established, which conducts load flow calculation and analysis of closing loop operation with the integrated analysis system of power system for 10kV circuits of different 220kV systems and under different operation modes. This is to protect the setting value and dynamic stability limitation checking of the equipment. Through practicing, method of reducing power flow of closing loop can be proposed and the management system of urban distribution ring network operation standard can be established.

The impedance characteristics of a power distribution system are analyzed in the previous chapter based on a one-dimensional circuit model. While useful for understanding the principles of the overall operation of a power distribution system, a one-dimensional model is not useful in describing the distribution of power and ground across a circuit die. The size of an integrated circuit is usually considerably greater than the wavelength of the signals in the power distribution network. Furthermore, the power consumption of on-chip circuitry (and, consequently, the current drawn from the power distribution network) varies across the die area. The voltage across the on-chip power and ground distribution networks is therefore non-uniform. It is therefore necessary to consider the two-dimensional structure of the on-chip power distribution network to ensure that target performance characteristics of a power distribution system are satisfied. The on-chip power distribution network should also be considered in the context of a die-package system as the properties of the die-package interface significantly affect the constraints imposed on the electrical characteristics of the on-chip power distribution network.

This paper focuses on the modeling and analysis of different power distribution network (PDN) structures in 3D TSV ICs, including 3D full wave model and equivalent circuit model. In addition, a simplified equivalent circuit model of stacked PDN is proposed to analysis aspects influencing stacked PDN impedance, such as number of ground TSVs, number of TSV pairs and number of stacked layers. Analyses about how these aspects influencing total capacitance, inductance of stacked PDN and causing resonance in PDN self-impedance (Z <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">11</sub> ) are made in detail. What's more, power noise in different stacked PDN structures is calculated and analyzed in time domain to evaluate their power integrity features. Analysis in this paper can offer instructions for the design and analysis of on-chip PDN.

A semiconductor industry has been encountered a memory bandwidth bottleneck toward a high density and high bandwidth system. In order to overcome those limitations, a 3D stacked high bandwidth memory (HBM) based on a through silicon via (TSV) and fine pitch interposer technology is lately introduced. By adopting this structure, thousands numbers of input/output (I/O) channels with a fine pitch can be integrated on HBM interposer which enables a terabyte/s bandwidth system. On the HBM interposer, significant numbers of I/O are integrated and they tend to operate at the same time which leads to severe simultaneous switching noise (SSN). When SSN occurs, the performance of system can be heavily degraded. Total SSN is strongly related to the self-noise and transfer-noise. In this point of view, a proper PDN design to manage transfer noise which is closely related to transfer-impedance must be taken into account. The analysis of power distribution network (PDN) impedance of HBM interposer must be performed since it generally affects power supply to the chips as well as signal integrity (SI). In this paper, HBM interposer with five layers is designed to analyze PDN. For PDN impedance analysis, Z-parameters depending on the various physical dimensions are simulated and compared. PDN impedance of HBM interposer is simulated and analyzed in the interest of frequency range dominated by interposer PDN. In order to suppress SSN, we suggest a metal-insulator-metal (MIM) de-cap scheme which can be commonly available for HBM interposer to reduce PDN impedance. Based on the designed physical dimension and material properties of HBM interposer, we successfully shows the suppression of SSN.

Distributed generators are usually connected to medium and low voltage distribution networks. The structure of distribution networks has changed from a single radial network to a two-terminal and multi-terminal network with multiple power sources. The characteristics of power flow and voltage distribution in power networks have become more complex. Aiming at the problem of voltage control of distributed generation in distribution network, this paper firstly theoretically analyses the coupling relationship between voltage and active/reactive power in distribution network, and then compares the reactive and active power control methods under different R/X conditions. Then, by dividing the operation of distribution network into three states: normal, early warning and emergency, coordination control strategy of multi-distributed photovoltaic power under different conditions are proposed, which is validated by the IEEE standard example. The coordinated control strategy proposed in this paper has important application value for ensuring reliable and stable operation of distribution network and distributed power supply.

In coupled power distribution and transportation (CPT) system, a joint scheduling framework for mobile energy storage systems (MESSs) and Volt/VAR control (VVC) ensures reliable power distribution grid operations while supporting electric vehicle loads at electric vehicle charging stations (EVCSs). However, conventional model-based optimization methods for MESS scheduling and VVC may yield suboptimal solutions and greater computation times because of MESS operation and VVC in uncertain environment of CPT systems. To resolve this issue, this study proposes a model-free deep reinforcement learning (DRL) framework. In this framework, smart inverters of MESSs and solar photovoltaic (PV) systems cooperate to minimize the real power loss and mitigate the violations of both MESSs’ state of charge (SOC) and voltage in the power distribution network, while MESSs travel via the transportation network to satisfy EV loads at EVCSs. A MESS routing algorithm based on Dijkstra’s algorithm is developed to determine the optimal destinations of the MESSs. In addition, two safety modules are developed to ensure that neither SOC nor voltage violations occur by adjusting real and/or reactive power of MESSs and PV systems during the training process. The developed MESS routing algorithm and safety modules are integrated into the proposed DRL framework, wherein the DRL agent performs the desired MESS scheduling and VVC through safe exploration during the training procedure. The proposed approach is tested in coupled IEEE 33-bus power distribution and 15-node transportation systems and coupled IEEE 57-bus power distribution and 42-node transportation systems. Numerical examples demonstrate the advantages of the proposed approach in terms of training convergence, real power loss, and SOC/voltage violation.

With the access of distributed generators (DG), the power flow direction and structure of distribution networks have changed, and therefore many traditional fault location methods are no longer applicable. Single-phase grounding fault is a common fault in power distribution network, which may bring secondary fault and even blackout. Starting from the relationship between the wavelet eigenvalues of traveling waves and the fault section of distribution lines with DG, the optimal fault features were selected by dimension reduction of linear discriminant analysis (LDA), and then the Bayesian classification model based on kernel distribution was used to realize the new method of Single phase grounding fault location. The IEEE 33-bus model with DGs was constructed to test the faults in different sections of active distribution network, the locating accuracy of the optimal three-dimensional feature sample is 97.9%. The result shows the method can achieve accurate fault location.

EVs are recognized as distributed loads and power sources with random motion characteristics in the power distribution networks (PDN); therefore, they serve as connecting points for the distribution network and transportation network (PDN-TN), and have significant impact on the stability and reliability of the routine operation of the coupled networks. In order to guarantee the safe operation of PDN-TN, it is necessary to evaluate the reliability of this coupled network. In this paper, the trip chaining theory is applied to describe the travel plan of EVs, and semi-dynamic flow allocation (SDTA) method is adopted to propose a simulation calculation method of energy flow between two networks based on user equalization (UE) criterion. The new reliability evaluation index for energy flow between the coupled networks is further proposed to better evaluate the reliability. Taking into account the daily trip characteristics of EVs, vehicles types, and charging modes, a coupled network reliability evaluation method based on sequential Monte Carlo simulation is put forward. Finally, the practicability of the proposed method is verified by an IEEE-RBTS BUS-6 test system and a 12-node transportation network.

Ensemble decision trees for high impedance fault detection in power distribution network

The rapid advancement of artificial intelligence (AI) has accelerated the semiconductor industry, driving chip performance requirements to unprecedented levels. However, conventional system-on-chip (SoC) designs are increasingly constrained by the physical and economic limits of monolithic scaling, where continued transistor miniaturization yields diminishing returns and rising fabrication costs. To address these challenges, the industry has shifted toward chiplet-based heterogeneous integration and advanced packaging technologies. By interconnecting multiple specialized chiplets through high-density, high-bandwidth interfaces, this modular approach improves scalability, reusability, and cost efficiency while enabling faster time-to-market. However, chiplet-based systems introduce significant design complexity, particularly in signal integrity (SI) and power integrity (PI), where highly constrained interconnects and power delivery networks must meet stringent electrical requirements. Traditional design methodologies rely heavily on expert-driven manual tuning and extensive full-wave electromagnetic simulations using electronic design automation (EDA) tools, resulting in long design cycles that can span weeks or months. To overcome these limitations, this thesis investigates the application of AI, particularly deep reinforcement learning (RL), to automate and optimize chiplet-based system design with a focus on SI and PI challenges in advanced packaging technologies. The research initially explores a supervised learning approach using convolutional neural networks (CNNs) to classify signal impairments from eye diagrams. While effective, this approach reveals a critical dependence on large labeled datasets, which are scarce in SI/PI domains, motivating a shift toward RL for autonomous design space exploration without labeled data. The first major contribution proposes an RL framework for automated power distribution network (PDN) design in a four-chiplet system. Two RL algorithms, Dueling Double Deep Q-Network (DDDQN) and Proximal Policy Optimization (PPO), are evaluated under different reward strategies. Results show that dense analytical rewards based on power domain squareness and bump proximity enable the RL agent to achieve lower self-impedance than human-designed PDNs, ensuring power integrity across 28 power domains. The complete PDN layout, which typically requires three months of manual effort, is generated in under one hour using a single GPU. The second contribution focuses on signal integrity optimization for chiplet interconnects in CoWoS redistribution layers (RDLs). An RL agent is trained using a reward function that balances eye width, voltage transfer function (VTF) crosstalk, and interconnect density. The agent explores design variations across data rates from 4 to 32 GT/s using a simulation environment integrated with commercial EDA tools. Experimental results demonstrate up to 24.67% improvement in eye width and over 22 dB reduction in VTF crosstalk while satisfying UCIe electrical specifications. The training process also generates labeled datasets for future supervised learning. The third contribution introduces an RL-based routing framework for UCIe bridge interconnects in advanced System-in-Package (SiP). Using Proximal Policy Optimization, the framework learns to generate routing layouts that comply with constraints such as ground-signal-ground structures, layer assignment, and congestion avoidance. It supports all UCIe module configurations and produces design rule check-compliant solutions within 100 training episodes, reducing design time from four weeks to less than one hour while improving signal distribution and reducing via usage. In summary, this thesis presents a unified AI-driven methodology for addressing SI and PI challenges in chiplet-based systems. The proposed RL frameworks significantly outperform traditional design approaches in both efficiency and quality, demonstrating the potential of deep RL to enable scalable and intelligent design automation for advanced packaging and chiplet-based architectures.

This paper proposes a deep reinforcement learning (DRL) framework that learns a reusable policy to find the optimal placement of decoupling capacitors (decaps) on power distribution network (PDN) with an arbitrary probing port. The proposed DRL framework trains a policy parameterized by pointer network, which is a sequence-to-sequence neural network, based on REINFORCE algorithm. The policy finds the positional combination of a pre-defined number of decaps that best suppresses self-impedance of a given probing port on PDN with randomly assigned keep-out regions. Verification was done by allocating 20 decaps on ten randomly generated test sets with an arbitrary probing port and randomly selected keep-out regions. Performance of the policy generated by the proposed DRL framework was evaluated based on the magnitude of probing port self-impedance suppression followed by decap placement over 434 frequencies between 100MHz and 20GHz. The policy generated by the proposed framework achieves greater impedance suppression with fewer samples in comparison to random search heuristic method.

Growing penetration of Plug-in Electric Vehicles (PEVs) in the transportation fleet and their subsequent charging demands introduce substantial intermittency to the electric load profile which imposes techno-economic challenges on power distribution networks. To address the uncertainty in demand, a novel deep learning-based approach equipped with a hybrid classification task is developed which can take into account the travel characteristics of the PEV owners. The classification structure helps us scrutinize the PEVs demand by allocating a specific forecasting network to each cluster of travel behavior patterns. In our hybrid classification task, first, an unsupervised classifier discerns hidden travel-behavior patterns between the historical PEVs data by clustering them; then, a supervised classifier directs each new PEV data to its appropriate cluster-specific forecasting network. The deep learning-based forecasting and classification networks are constructed based on the Long Short-Term Memory networks to investigate long- and short term features in PEV behaviors. The data-driven structure of our proposed method enables us to observe and preserve the correlation between PEV travel data parameters (departure time, arrival time and traveled distance) and avoid the generation of unrealistic travel samples found in scenario-based approaches. To verify the effectiveness of the proposed method in practical environments, we have studied the impact of the precise forecasting of the PEVs demand in an aggregator’s financial profit in the energy market of the California Independent System Operator market. The numerical results confirm the outstanding performance of our proposed deep learning-based method in forecasting PEVs demand against benchmark approaches in this field such as Monte Carlo, Quasi-Monte Carlo, and Copula with only a 6.77% error in comparison with real data.

A new method is proposed to identify the location of transient events, including incipient faults, in power distribution systems, by using synchronized measurements from an emerging class of sensors, called waveform measurement units (WMUs). WMUs capture the voltage and current waveforms in time domain. The proposed method consists of three steps. The first step is to characterize the oscillatory modes of the transient components of all the captured synchronized voltage and current waveforms from all WMUs, by conducting a multi-signal modal analysis. The second step is to construct a circuit model for the underlying distribution feeder at the identified dominant mode(s) of the transient event. The final step is to identify the location of the transient event with the means of a method that involves certain forward and backward analyses of the constructed circuit model. The proposed method requires installing as few as only two WMUs. It can also utilize several synchronized waveform measurements when several WMUs are available. The performance of the proposed method is assessed on the IEEE 33-bus test system; for different cases of transient events, such as sub-cycle incipient faults, multi-cycle incipient faults, permanent faults, as well as benign yet informative events such as capacitor bank switching. Both the accuracy and the robustness of the proposed method are verified. The analysis and results in this paper provide new insights on possible applications of synchronized WMU measurements; while they also address a highly challenging problem in power distribution networks.

Wen-Sheng Zhao from Hangzhou University, China, talks to Electronics Letters about the paper ‘Novel electromagnetic bandgap structure for wideband suppression of simultaneous switching noise’, page 1243 Wen-Sheng Zhao My research fields include interconnect modelling, multiphysics simulations and microwave sensor design. My major research topic is signal and power analysis of silicon via (TSV)-based 3D integrated systems. In recent years, the fourth industry revolution has shown an urgent need to develop next generation memory modules. As a key technology of industry 4.0, TSV-based high-bandwidth memory (HBM) has attracted a lot of interest from both the research and industrial worlds. To guarantee the system performance, it is necessary to perform signal/power/thermal integrity analysis for an HBM module. We are currently working on the issues related to the physical design of HBM modules (such as TSV modeling, equaliser design, and power noise suppression). Modern circuits are developing towards miniaturisation, high frequency and low power consumption, which puts strict constraints on the signal and power integrity design. For example, the operating voltages are continuously lowered, thereby limiting noise margin in power distribution network (PDN). Under such circumstances, power/ground noise is becoming a serious problem. To resolve this issue, various techniques including decoupling capacitor and electromagnetic bandgap (EBG) have been proposed and are under study. In comparison with decoupling capacitor, EBG structure can provide wideband noise suppression, and has been widely explored in the chip, package, and board. To improve the system performance, it is always desirable to increase the stopband bandwidth and rejection level of EBG structure. In this Letter, a power plane with coplanar EBG etched in the S-bridge EBG was proposed. With the proposed EBG structure, an ultrawide bandgap from 0.28 to 25 GHz with a high suppression level of -45 dB was achieved. Moreover, the signal integrity performance of the proposed EBG power plane was examined. The signal integrity can be ensured by utilising differential traces as the transmission structure. The proposed design can be used to reduce power/ground noise efficiently for mixed signal systems. We expect the proposed design to provide performance improvements in various mixed-signal systems in the immediate future. Currently, we are also working on developing EBG structures for silicon interposer and on-chip PDN. In the longer term, we expect that the EBG structures can be further explored and used in the physical design optimisation of HBM systems. As the clock frequency of the circuit system is continuously increased, the frequency of the simultaneous switching noise (SSN) also increases. However, the low-frequency components of the noise are not reduced accordingly. Therefore, the major issue we faced was developing a wide stopband EBG structure with a low cutoff frequency. Moreover, as grooves etched on the power/ground plane have an adverse impact on the signal transmission, we had to pay special attention to ensure signal integrity in the EBG design procedure. When we began studying TSV technologies, there was very little real-world deployment. Although several semiconductor giants such as Samsung presented some TSV based memory modules, these systems remained in the laboratory stage. Nowadays, TSV technology has matured, and the TSV-based HBM modules have been used in many graphics processing units since 2015. We are currently working on the physical design and optimisation (including signal and power integrity analysis) for TSV-based integrated systems. We have seen the introduction of many kinds of techniques (e.g., EBG, equaliser, differential signaling, and active decoupling) which will be helpful for future designs. There is no doubt that further improvements are always desirable to promote wide applications of TSV based integrated systems.

The selection of decoupling capacitors (decap) is a critical but tedious process in power distribution network (PDN) design. In this paper, an improved decap-selection algorithm based on deep reinforcement learning (DRL), which seeks the minimum number of decaps through a self-exploration training to satisfy a given target impedance, is presented. Compared with the previous relevant work: the calculation speed of PDN impedance is significantly increased by adopting an impedance matrix reduction method; also, the enhanced algorithm performs a better convergence by utilizing the techniques of double Q-learning and prioritized experience replay; furthermore, a well-designed reward is proposed to facilitate long-term convergence when more decaps are required. The proposed algorithm demonstrates the feasibility of achieving decent performance using DRL with pre-trained knowledge for more complicated engineering tasks in the future.