High Performance Hierarchical Torus Network Under Adverse Traffic Patterns

The static network performance and dynamic communication performance of the Hierarchical Torus Network (HTN) using dimension-order routing algorithm have already been evaluated and shown to be superior to the performance of other interconnection networks. However, the assessment of the dynamic communication performance improvement of HTN by the efficient use of both the physical link and virtual channels has not yet been evaluated. This paper addresses three adaptive routing algorithms - link-selection, channel-selection, and a combination of link-selection and channel-selection - for the efficient use of physical links and virtual channels of an HTN to enhance dynamic communication performance. It also proves that the proposed adaptive routing algorithms are deadlock-free with 3 virtual channels. The dynamic communication performances of an HTN is evaluated by using dimension-order routing and proposed adaptive routing algorithms under various traffic patterns. It is found that the dynamic communication performance of an HTN using these adaptive routing is better than when the dimension-order routing is used, in terms of network throughput.

USM, PULAU PINANG, 14 Mac 2018 – Lima pensyarah Universiti Sains Malaysia (USM) telah
\nmemenangi pingat dalam pertandingan sempena Ekspo Inovasi Islam (i-INOVA) 2018 yang
\nberlangsung pada 10-11 Mac lalu di Dewan Tuanku Canselor Universiti Sains Islam Malaysia (USIM),
\nBandar Baru Nilai, Negeri Sembilan.
\nIa merupakan kemenangan 100% kepada USM kerana kelima-lima produk/teknologi yang
\ndipertandingkan telah memenangi anugerah dan pingat.

Cluster networks are seen as the future access networks for multimedia streaming, e-commerce, network storage, etc. For these applications, performance and high availability are particularly crucial. Regular topologies are preferred when performance is the primary concern. However, due to spatial constraints or fault-related issues, the network structure may become irregular, which makes more difficult to find deadlock-free minimal paths. Over the recent years, several solutions have been proposed. One of them is the LASH routing, which enables minimal routing by assigning paths to different virtual layers. In this paper, we propose an extension of LASH in order to reduce the number of required virtual layers by allowing transitions between virtual layers. Evaluation results show that the new routing scheme (LASH-TOR) is able to obtain full minimal routing with a reduced number of virtual channels. For torus and mesh networks, with only two virtual channels, LASH throughput is increased by an average factor of improvement of 3.30 for large networks. For regular networks with some unconnected (faulty) links, equal performance improvements are achieved. Even for highly irregular networks of size up to 128 switches the new routing scheme only needs three virtual channels for guaranteeing minimal routing. Besides, LASH-TOR performs well compared to dimension order routing for mesh and torus networks.

An efficient all-to-all communication algorithm for torus and mesh networks, A2AT, was proposed. A2AT schedules message sending sequence so that all links are fully used by exploiting function of concurrent message transfer in the node. By using A2AT, the hop count of messages equals the maximum number of messages sharing a link in their routes for all message transfers. A2AT can therefore maintain synchronization without the need for phasing operation such as an MPI barrier. When the VOQ which is an ideal configuration for A2AT was used, communication times for mesh/torus network obtained by A2AT were roughly 1.20 and 1.09 times higher, on average, than those of the ideal times. When the networks had the minimum number of virtual channels and a small buffer, assuming a practical network, A2AT was able to reduce communication times by 12.5% and 36.0% compared with those of the conventional algorithm. When two controllers are used, A2AT reduced 28.2% and 55.7% communication time with those by A2AND on 15x15x15 (=3,375 nodes) mesh and torus networks respectively (18.6% and 44.8% in average). A2AT also reduced 15.1% and 41.9% of communication time with those by A2AND on the same mesh and torus networks respectively (14.4% and 37.5% in average) when six controllers are used.

In this paper, we have evaluated the dynamic communication performance of the mesh and torus networks using a open source TOPAZ simulator using virtual cut through switching under uniform traffic pattern. It is found that the torus network yields low latency and high throughput than that of mesh network. Therefore, The dynamic communication performance of the torus network is better than that of the mesh network. The well known phenomenon is justified using a new open source simulator, TOPAZ simulator.

Minimal DIstance MEsh with Wrap-around links (Midimew) connected Mesh Network (MMN) is a hierarchical interconnection network consists of several Basic Modules (BM), where the BM is a 2D-mesh network and the higher level network is a midimew network. In this paper, we present the architecture of MMN and evaluate the number of long wires, length of a long wire, and the total length for the long wire of MMN, TESH, and torus networks. It is shown that the proposed MMN possesses simple structure and moderate wire length. The long wire length of MMN is slightly higher than TESH network and far lower than that of 2D torus network. Overall performance suggests that, MMN is a good choice for future generation massively parallel computers.

We propose a deterministic fault-tolerant and deadlock-free routing protocol in 2-dimensional (2-D) meshes based on dimension-order routing and the recently proposed odd-even turn model. The proposed protocol, called extended X-Y routing, does not use any virtual channels by prohibiting certain locations of faults and destinations. Faults are contained in a set of disjointed rectangular regions called faulty blocks. The number of faults to be tolerated is unbounded as long as nodes outside faulty blocks are connected in the mesh network. The extended X-Y routing can also be used under a special convex fault region called orthogonal faulty block, which can be derived from a given faulty block by activating some nonfaulty nodes in the block. Extensions to partial adaptive routing, traffic- and adaptivity-balanced using virtual networks, and routing without constraints using virtual channels and virtual networks are also discussed.

Compressionless Routing (GR) is a new adaptive routing framework which provides a unified framework for efficient deadlock-free adaptive routing and fault-tolerance. CR exploits the tight-coupling between wormhole routers for flow control to detect potential deadlock situations and recover from them. Fault-tolerant Compressionless Routing (FCR) extends Compressionless Routing to support end-to-end fault-tolerant delivery. Detailed routing algorithms, implementation complexity and performance simulation results for CR and FCR are presented.CR has the following advantages: deadlock-free adaptive routing in torus networks with no virtual channels, simple router designs, order-preserving message transmission, applicability to a wide variety of network topologies, and elimination of the need for buffer allocation messages. FCR has the following advantages: tolerates transient faults while maintaining data integrity (nonstop fault-tolerance), tolerates permanent faults, can be applied to a wide variety of network topologies, and eliminates the need for software buffering and retry for reliability. These advantages of CR and FCR not only simplify hardware support for adaptive routing and fault-tolerance, they also can simplify communication software layers.

In this paper, we have proposed a new hierarchical interconnection network called Tori-connected Flattened Butterfly Network (TFBN). It is a hierarchical interconnection network which is able to interconnect millions of nodes in the network to achieve high performance and cost-effective design for massively parallel computing system. TFBN consists of multiple basic modules of flattened butterfly network whereby the basic modules are connected using 2D-torus network. This paper focuses on the architecture of the TFBN and the evaluation of its static network performance such as degree, diameter, cost, average distance, arc connectivity, bisection width and wiring complexity. The comparison of performance shows better performance in terms of distance parameters as well as other parameters.

Networks with torus interconnection topology are widely used due to the symmetry in traffic distribution. In order to ensure deadlock-freedom and provide adaptive routing in torus, at least two Virtual Channels (VCs) per physical channel are required to break the cyclic channel dependencies. However, VCs increase the arbitration latency and consume large power/area overheads which is undesirable, particularly for on-chip networks with limited power/area budgets. In this paper, we propose a novel technique for routing in wormhole-switched 2D torus networks. The proposed method relies on the Abacus Turn Model (AbTM) and Worm-Bubble Flow Control (WBFC) to support adaptive and deadlock-free routing without using VCs. Furthermore, the network blocking is reduced by providing on-demand routing adaptiveness through reconfiguration. The experimental results demonstrate the efficiency of the proposed scheme in terms of performance and hardware overhead.

A balanced ring (b-ring) proposed by Gu et al. [Journal of Systems Architecture, 53, (2007), 902–912.] is more effective than a fault ring (f-ring) in providing fault tolerance for mesh and torus networks. By properly utilizing the b-ring, an efficient fault tolerant routing algorithm can be achieved without using any virtual channel. Traffic congestion can also be significantly reduced. However, the algorithm presented by Gu et al. for the formation of b-ring has the demerits that it involves all the nodes of the network each time a b-ring needs to be formed, and thereby increasing the overhead to the whole system. In this paper, we first present an improved algorithm for the formation of b-ring by considering only the nodes on the fault ring in contrast to all. We next present a distributed algorithm for the same, which is also based on the local knowledge, thereby avoiding overhead and thus, enhancing the overall network performances. We show that our sequential algorithm requires O(|f-ring|) time and the distributed algorithm runs in constant time.

Deadlock free routing techniques for torus topologies have been a subject of deep study in the field of HPC interconnects and many proposals exist in the literature. Practical deadlock avoidance techniques can be classified into two main categories, requiring either a segregation of traffic in non-cyclic virtual networks or some form of injection control. Simulating large high-dimension tori networks using application traffic is challenging. Most proposals use either large low-dimension tori, or synthetic traffic. Currently, tori of five and six dimensions are being used in actual supercomputers, such as the Fujitsu K Computer, which was ranked first in the Top 500 in two consecutive lists (June 2011 and November 2011). To our knowledge, there are no published papers comparing the performance implications of deadlock avoidance techniques for large high-dimension tori using traffic typical of parallel applications. We chose two well established deadlock-avoidance techniques in tori with dimension-order routing, dateline resource allocation and bubble injection restriction. The simulation tools had to be adapted to scale to simulate these large networks. In this paper we analyze network performance for tori of up to 6 dimensions comprising up to 4096 nodes when dealing with both synthetic and HPC-specific workloads.

ROMM is a class of Randomized, Oblivious, Multi--phase, Minimal routing algorithms. ROMM routing offers a potential for improved performance compared to both fully randomized algorithms and deterministic oblivious algorithms, under both light and heavy loads. ROMM routing also offers close to best case performance for many common routing problems. In previous work, these claims were supported by extensive simulations on binary cube networks [30, 31]. Here we present analytical and empirical results for ROMM routing on wormhole routed mesh and torus networks. Our simulations show that ROMM algorithms can perform several representative routing tasks 1.5 to 3 times faster than fully randomized algorithms, for medium--sized networks. Furthermore, ROMM algorithms are always competitive with deterministic, oblivious routing, and in some cases, up to 2 times faster.

A fault-tolerant routing algorithm for torus networks that uses only three virtual channels is presented. The proposed algorithm is based on the block fault model, which is suitable for modelling faults at the board level in networks with grid structures. Messages are routed via shortest paths when there are no faults. However, if a message is blocked by a faulty block, the message will use a detour path to route around the faulty block. Previously at least six virtual channels were needed to achieve the same fault-tolerant ability. Simulation results using various workloads and fault patterns are presented.

Two new algorithms for worm-hole routing in the hypercube are presented. The first hypercube algorithm is adaptive, but non-minimal in the sense that some derouting is permitted. Then another deadlock-free adaptive worm-hole based routing algorithm for the hypercube interconnection is presented which is minimal. Finally some well-known worm-hole algorithms for the hypercube were evaluated together with the new ones on a hypercube of 2/sup 10/ nodes. One oblivious algorithm, the Dimension-Order, or E-Cube routing algorithm (W. Dally, C. Seitz, 1987) was tried. In addition, three partially adaptive algorithms were considered: the Hanging algorithm (Y. Birk, P. Gibbons, D. Soroker, J. Sanz, 1989 and S. Konstantinidou, 1990), the Zenith algorithm (S. Konstantinidou, 1990), and the Hanging-Order algorithm (G.-M. Chia, S. Chalasani, C.S. Raghavendra, 1991). Finally, a fully adaptive minimal algorithm presented independently by L. Gravano, G. Pifarre, S.A. Felperin and J. Sanz (1991) and J. Duato was tried. This algorithm allows each message to choose adaptively among all the shortest paths from its source to its destination. Only four virtual channels per physical link are needed to achieve this. This technique is referred to as Fully. The results obtained show that the two new algorithms are good candidates as a choice for worm-hole routing in the hypercube network.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>