Performance implications of deadlock avoidance techniques in torus networks

A Hierarchical Torus Network (HTN) is a 2D-torus network of multiple basic modules, in which the basic modules are 3D-torus networks that are hierarchically interconnected for higher level networks. The dynamic communication performance of the HTN using the dimension-order routing under common traffic patterns have been evaluated, and have been shown to be good. However, dynamic communication performance of HTN under adverse traffic patterns has not been evaluated yet. In this paper, we evaluate the dynamic communication performance of HTN using a deadlock-free dimension order routing with 3 virtual channels under adverse traffic patterns, and compare it with H3D-mesh, mesh, and torus networks. It is shown that even under adverse traffic patterns, the HTN yields high throughput and low average transfer time, which provide better dynamic communication performance than H3D-mesh, mesh, and torus networks.

Current Network-on-Chip (NoC) architectures sometimes employ mesh or torus topology with the dimension-order routing. In this paper, we propose a deadlock-free routing algorithm, referred to as Balanced Dimension-Order Routing (BDOR), which provides the balanced minimal paths to each destination based on the simple routing regulations. Since the BDOR has the similar path regularity to that of the dimension-order routing, its implementation can be lightweight, and most of its modules can be borrowed from the router for the dimension-order routing. Evaluation results show that the BDOR router increases by 3.4% hardware amount compared with the router for the dimension-order routing. Also show that the throughput of the BDOR outperforms on average up to 14% that of the dimension-order routing on two-dimensional mesh and torus.

Cluster networks are seen as the future access networks for multimedia streaming, e-commerce, network storage, etc. For these applications, performance and high availability are particularly crucial. Regular topologies are preferred when performance is the primary concern. However, due to spatial constraints or fault-related issues, the network structure may become irregular, which makes more difficult to find deadlock-free minimal paths. Over the recent years, several solutions have been proposed. One of them is the LASH routing, which enables minimal routing by assigning paths to different virtual layers. In this paper, we propose an extension of LASH in order to reduce the number of required virtual layers by allowing transitions between virtual layers. Evaluation results show that the new routing scheme (LASH-TOR) is able to obtain full minimal routing with a reduced number of virtual channels. For torus and mesh networks, with only two virtual channels, LASH throughput is increased by an average factor of improvement of 3.30 for large networks. For regular networks with some unconnected (faulty) links, equal performance improvements are achieved. Even for highly irregular networks of size up to 128 switches the new routing scheme only needs three virtual channels for guaranteeing minimal routing. Besides, LASH-TOR performs well compared to dimension order routing for mesh and torus networks.

The static network performance and dynamic communication performance of the Hierarchical Torus Network (HTN) using dimension-order routing algorithm have already been evaluated and shown to be superior to the performance of other interconnection networks. However, the assessment of the dynamic communication performance improvement of HTN by the efficient use of both the physical link and virtual channels has not yet been evaluated. This paper addresses three adaptive routing algorithms - link-selection, channel-selection, and a combination of link-selection and channel-selection - for the efficient use of physical links and virtual channels of an HTN to enhance dynamic communication performance. It also proves that the proposed adaptive routing algorithms are deadlock-free with 3 virtual channels. The dynamic communication performances of an HTN is evaluated by using dimension-order routing and proposed adaptive routing algorithms under various traffic patterns. It is found that the dynamic communication performance of an HTN using these adaptive routing is better than when the dimension-order routing is used, in terms of network throughput.

The availability of efficient and reliable simulation tools is one of the mission-critical technologies in the fast-moving field of computational neuroscience. Research indicates that higher brain functions emerge from large and complex cortical networks and their interactions. The large number of elements (neurons) combined with the high connectivity (synapses) of the biological network and the specific type of interactions impose severe constraints on the explorable system size that previously have been hard to overcome. Here we present a collection of new techniques combined to a coherent simulation tool removing the fundamental obstacle in the computational study of biological neural networks: the enormous number of synaptic contacts per neuron. Distributing an individual simulation over multiple computers enables the investigation of networks orders of magnitude larger than previously possible. The software scales excellently on a wide range of tested hardware, so it can be used in an interactive and iterative fashion for the development of ideas, and results can be produced quickly even for very large networks. In contrast to earlier approaches, a wide class of neuron models and synaptic dynamics can be represented.

This paper presents efficient algorithms that implement one-to-many, or multicast, communication in wormhole-routed torus networks. By exploiting the properties of the switching technology and the use of virtual channels, a minimum-time multicast algorithm is presented for n-dimensional torus networks that use deterministic, dimension-ordered routing of unicast messages. The algorithm can deliver a multicast message to m - 1 destinations in [log_2 m] message-passing steps, while avoiding contention among the constituent unicast messages. Performance results of a simulation study on torus networks are also given.

We develop on-line routing and wavelength assignment (RWA) algorithms for WDM bidirectional ring and torus networks with N nodes. The algorithms dynamically support all k-allowable traffic matrices, where k denotes an arbitrary integer vector [k/sub 1/, k/sub 2/,... k/sub N/], and node i, 1 /spl les/ i /spl les/ N, can transmit at most k/sub i/ wavelengths and receive at most k/sub i/ wavelengths. Both algorithms support the changing traffic in a rearrangeably nonblocking fashion. Our first algorithm, for a bidirectional ring, uses [(/spl Sigma//sub i=1//sup N/ k/sub i/)/3] wavelengths in each fiber and requires at most three lightpath rearrangements per new session request regardless of the number of nodes N and the amount of traffic k. When all the k/sub i/'s are equal to k, the algorithm uses [kN/3] wavelengths, which is known to be the minimum for any off-line rearrangeably nonblocking algorithm. Our second algorithm, for a torus topology, is an extension of a known off-line algorithm for the special case with all the k/sub i/'s equal to k. For an R /spl times/ C torus network with R /spl ges/ C nodes, our on-line algorithm uses [kR/2] wavelengths in each fiber, which is the same as in the off-line algorithm, and is at most two times a lower bound obtained by assuming full wavelength conversion at all nodes. In addition, the on-line algorithm requires at most C - 1 lightpath rearrangements per new session request regardless of the amount of traffic k. Finally, each RWA update requires solving a bipartite matching problem whose time complexity is only O (R), which is much smaller than the time complexity O(kCR/sup 2/) of the bipartite matching problem for an off-line algorithm.

The one-to-all broadcast is the most primary operation in a multicomputer network. In this paper, we consider this problem in a wormhole-routed 2D torus network which provides the dimension-ordered routing and all-port communication capability. With the distance-insensitive characteristic of wormhole routing, we model the broadcast as a sequence of communication phases. We propose a novel scheme to solve this problem. The scheme requires at most two communication phases more than the optimal number of phases when the torus is square, or at most four or five communication phases more than optimum when the torus is nonsquare. The best known results, as compared to ours, can only be applied to square tori of very restricted network sizes and either rely on an undesirable non-dimension-ordered routing or require more numbers of phases.

Many different factors, such as topology, routing technique, selection function, flow control policy, complexity of router design, contribute to the performance of networks on chip (NoCs). Among these factors, the choice of topology and routing function has a significant effect on the average packet latency and saturation behavior. Torus is popular in many application domains, while the problem of virtual channel misbalance caused by its deadlock avoidance scheme brings lots of performance pathologies. In this paper, we present a novel deadlock avoidance scheme based on draining scheme, and propose a deadlock-free routing scheme for torus networks. We quantify the effects of the proposed routing scheme on the overall network performance by presenting simulation results for 1-D torus and 2-D torus NoCs. Experimental results show that the novel routing algorithm for torus networks could make more efficient use of virtual channel resources, and hence cutting down packet latency and boosting throughput. Note although this paper has considered 1-D and 2-D torus networks in the experimental evaluation, the new scheme is flexible enough to support highradix torus networks.

Modern supercomputers are massively parallel systems: they embody thousands of computing nodes and sometimes several millions. The torus topology has proven very popular for the interconnect of these high‐performance systems. Notably, this network topology is employed by the supercomputer ranked number one in the world as of November 2020, the supercomputer Fugaku. Given the high number of compute nodes in such systems, efficient parallel processing is critical to maximise the computing performance. It is well known that cycles harm the parallel processing capacity of systems: for instance, deadlocks and starvations are two notorious issues of parallel computing that are directly linked to the presence of cycles. Hence, network decycling is an important issue, and it has been extensively discussed in the literature. We describe in this paper a decycling algorithm for the 3‐dimensional k‐ary torus topology and compare it with established results, both theoretically and experimentally. (This paper is a revised version of Antoine Bossard (2020)).

Compressionless Routing (GR) is a new adaptive routing framework which provides a unified framework for efficient deadlock-free adaptive routing and fault-tolerance. CR exploits the tight-coupling between wormhole routers for flow control to detect potential deadlock situations and recover from them. Fault-tolerant Compressionless Routing (FCR) extends Compressionless Routing to support end-to-end fault-tolerant delivery. Detailed routing algorithms, implementation complexity and performance simulation results for CR and FCR are presented.CR has the following advantages: deadlock-free adaptive routing in torus networks with no virtual channels, simple router designs, order-preserving message transmission, applicability to a wide variety of network topologies, and elimination of the need for buffer allocation messages. FCR has the following advantages: tolerates transient faults while maintaining data integrity (nonstop fault-tolerance), tolerates permanent faults, can be applied to a wide variety of network topologies, and eliminates the need for software buffering and retry for reliability. These advantages of CR and FCR not only simplify hardware support for adaptive routing and fault-tolerance, they also can simplify communication software layers.

The increased communication bandwidth demands of high performance computing (HPC) systems calling at the same time for reduced latency and increased power efficiency have designated optical interconnects as the key technology in order to achieve the target of exascale performance. In this realm, technology advances have to be accompanied by the development of corresponding design and simulation tools that support end-to-end system modeling in order to evaluate the performance benefits offered by optical components at system scale. In this paper, we present recent advances on electro-optical printed circuit boards (EOPCB) technology development pursued within the European FP7 PhoxTroT research program and directed toward system-scale performance benefits in real HPC workload applications. We report on high-density and multilayered EOPCBs together with all necessary building blocks for enabling true optical blade technology, including multimode polymer-based single- and dual-layer EOPCBs, a board-compatible optically interfaced router chip, and passive board-level connectors. We also demonstrate a complete optical blade design and evaluation software simulation framework called OptoHPC that tailors optical blade technology development toward optimized performance at HPC system scale, allowing for its validation with synthetic workload benchmark traffic profiles and for reliable comparison with existing HPC platforms. The OptoHPC simulator is finally utilized for evaluating and comparing a 384-node HPC system relying on optically enabled blades with the state-of-the-art Cray XK7 HPC network when performing with a range of synthetic workload traffic profiles, revealing the significant throughput and delay improvements that can be released through application-oriented optical blade technology.

Modern supercomputers are massively parallel systems: they include hundreds of thousands of processors. Processor connection is realised by the interconnection network of the system. In recent years, the torus topology has proven very popular as interconnection network: the Fujitsu K, IBM Blue Gene/L, IBM Blue Gene/P and Cray Titan supercomputers are examples of devices relying on this topology. In this paper, we address the fundamental problem of pairwise disjoint-path routing in a torus network. This problem is about selecting mutually node disjoint paths between given node pairs. Solving this problem has very important implications such as increased system dependability and more efficient data transfers. The correctness and complexities of the proposed routing algorithm to solve this problem are formally established. In an n-dimensional k-ary torus (n $c$ (≤ n) node pairs with mutually node disjoint paths of lengths at most 2k(c - 1) + n [k / 2], and the worst-case time complexity of the algorithm is O (nk2c3).

Torus topology is one of the preferred topologies for the interconnection network in high-performance clusters and supercomputers. Cost and scalability are some of the properties that make torus suitable for systems with a large number of nodes. The 3D torus is the version more extended due to its excellent nearest neighbor. However, some of the last supercomputers have been built using a torus network with five or six dimensions. To obtain an $nD$ torus, $2n$ ports per node are needed, which can be offered by a single or several cards per node. In the second case, there are multiple ways of assigning the dimension and direction of the card ports. In previous work we defined and characterized the 3D Twin (3DT) torus which uses two four-port cards per node. In this paper we extend that previous work to define the $n$ -dimensional Twin ( $n$ DT) torus topology. In this case, we formally obtain the optimal port configuration when ( $n$ + 1)-port cards are used instead of $2n$ -port cards. Moreover, we explain how deadlock problem can appear and propose a simple solution. Finally, we include evaluation results which show performance increases when an $nDT$ torus is used instead of an $nD$ torus with fewer dimensions and with the same computational resources.

We propose a deterministic fault-tolerant and deadlock-free routing protocol in 2-dimensional (2-D) meshes based on dimension-order routing and the recently proposed odd-even turn model. The proposed protocol, called extended X-Y routing, does not use any virtual channels by prohibiting certain locations of faults and destinations. Faults are contained in a set of disjointed rectangular regions called faulty blocks. The number of faults to be tolerated is unbounded as long as nodes outside faulty blocks are connected in the mesh network. The extended X-Y routing can also be used under a special convex fault region called orthogonal faulty block, which can be derived from a given faulty block by activating some nonfaulty nodes in the block. Extensions to partial adaptive routing, traffic- and adaptivity-balanced using virtual networks, and routing without constraints using virtual channels and virtual networks are also discussed.