Parmesan: Efficient Partitioning and Mapping Flow for DNN Training on General Device Topology

Complex networks are a technique for the modeling and analysis of large data sets in many scientific and engineering disciplines. Due to their excessive size conventional algorithms and single core processors struggle with the efficient processing of such networks. Employing multi-core graphic processing units (GPUs) could provide sufficient processing power for the analysis of such networks. However, commonly designed algorithms cannot exploit these massively parallel processing power for the analysis of such networks. In this paper, we present the Multi Layer Network Decomposition (MLND) approach which provides a general approach for parallel network analysis using multi-core processors via efficient partitioning and mapping of networks onto GPU architectures. Evaluation using a 336 core GPU graphic card demonstrated a 16x speed-up in complex network analysis relative to a CPU based approach.

In prokaryotic genomes the number of transcriptional regulators is known to be proportional to the square of the total number of protein-coding genes. A toolbox model of evolution was recently proposed to explain this empirical scaling for metabolic enzymes and their regulators. According to its rules, the metabolic network of an organism evolves by horizontal transfer of pathways from other species. These pathways are part of a larger “universal” network formed by the union of all species-specific networks. It remained to be understood, however, how the topological properties of this universal network influence the scaling law of functional content of genomes in the toolbox model. Here we answer this question by first analyzing the scaling properties of the toolbox model on arbitrary tree-like universal networks. We prove that critical branching topology, in which the average number of upstream neighbors of a node is equal to one, is both necessary and sufficient for quadratic scaling. We further generalize the rules of the model to incorporate reactions with multiple substrates/products as well as branched and cyclic metabolic pathways. To achieve its metabolic tasks, the new model employs evolutionary optimized pathways with minimal number of reactions. Numerical simulations of this realistic model on the universal network of all reactions in the KEGG database produced approximately quadratic scaling between the number of regulated pathways and the size of the metabolic network. To quantify the geometrical structure of individual pathways, we investigated the relationship between their number of reactions, byproducts, intermediate, and feedback metabolites. Our results validate and explain the ubiquitous appearance of the quadratic scaling for a broad spectrum of topologies of underlying universal metabolic networks. They also demonstrate why, in spite of “small-world” topology, real-life metabolic networks are characterized by a broad distribution of pathway lengths and sizes of metabolic regulons in regulatory networks.

In this paper, we propose a distributed algorithm for scheduling divisible loads originating from single site on arbitrary networks. We first propose a mathematical model and formulate the scheduling problem as an optimization problem with an objective to minimize the processing time of the loads. A number of theoretical results on the solution of the optimization problem are derived. On the basis of these results, we propose our algorithm using the concept of virtual routing. The proposed algorithm has three attractive features - distributed working style, simple structure, in terms of implementation ease, and offers generalized approach for handling divisible load scheduling for any network topology. This is the first time in the divisible load scheduling literature that a distributed strategy is attempted.

We propose a solution to the problem of calculating deterministic end-to-end delay bounds in arbitrary topology networks, which employ the generalized processor sharing (GPS) discipline at each node. We give a general framework under network calculus by extending the service curve model of GPS and constructing the end-to-end service curve for each session, from which exact values of delay and backlog bounds can be computed. Network internal traffic characterization and traffic shaping are applied to support local performance bound computation. Sessions are allowed to be arbitrary weighted and virtual feedback is handled by the iterative approach of Barta, Nemeth, Szabo and Bird (see ICC2001). Numerical examples are shown to illustrate our algorithmic approach for per-session end-to-end delay bound computation.

Preface: Volume 81

Preface: Volume 52, Issue 2

Distribution systems (DS) are a focal connection point for existing and new renewable energy sources. DS have increasing sizes; they connect increasing numbers of renewable energy sources and have a mandate to operate as financial entities seeking to maximize their profits in an electricity market. To achieve these goals, DS require highly efficient planning tools. In this paper, we propose a successive fuzzy mixed integer linear programming (MILP) method for optimally placing and sizing capacitors in a DS that explicitly models induction run wind electric generators (WEGs). The optimization problem is linearized, transformed into a fuzzy MILP model and is solved successively while updating the solution vector. The fuzzy MILP method employs fuzzy models of voltage constraints that are often violated in a planned system that lacks reactive power support. The use of fuzzy constraints makes the proposed solution method very robust by avoiding infeasibility and reducing the need for user expertise. Further, the fuzzy MILP method represents the dual objectives of capacitor cost and annual energy loss minimization as fuzzy sets with individual satisfaction functions. The method maximizes the minimum of these satisfaction functions during optimization. Tests on 69-bus and 2002-bus DS show that the method performs well and is robust.

We describe distributed tracking of a nonlinear dynamical system via networked sensors. The sensors communicate with each other by means of a multihop protocol over a communication network. We derive in-network processing algorithms to deal with arbitrary network topology and then extend these results to account for communication delays and packet losses. We show that these algorithms are optimal in the linear setting and achieve centralized performance. The proposed techniques differ from existing techniques in two important aspects: a) there is no designated leader/fusion node and each sensor attempts to optimally track the system trajectory based on its local observations and time-dependent information available from other sensors in the network; b) the message computation at each sensor is structurally identical. Consequently, the sensor network can be queried at any time and at any node to obtain optimal estimates for the state of the dynamical system. We then present two multihop protocols - one based on gossip and another token-based - for distributed implementation of the in-network processing techniques. We show several advantages of token-based schemes over gossip protocols: a) message complexity is significantly smaller for achieving the same performance; b) they are well-suited for situations where target and network data aggregation time-scales are comparable; and c) they are well-suited for random geometric graphs with nodes having small communication-connectivity radius - a scenario typical of ad-hoc wireless networks. This is because they can fuse data only from the set of nodes that can be visited in any time period.

We describe distributed tracking of a linear dynamical system via networked sensors. The networked sensors communicate with each other by means of a multi-hop protocol over a communication network. MMSE-optimal solution is Kalman filtering when measurements are available centrally, but new methods are required to account for communication constraints. We derive optimal algorithms to deal with arbitrary network topology and then extend these results to account for communication delays and packet losses. The proposed techniques differ from related techniques proposed in two important aspects: a) there is no designated leader/fusion node and each sensor attempts to optimally track the system trajectory based on its local observations and time-dependent information available from other sensors in the network; b) the message computation at each sensor is structurally identical, where the computed message from each sensor is the innovation in the state conditioned on all the information available upto that time at each sensor. Consequently, the sensor network can be queried at any time and at any node to obtain optimal estimates for the state of the dynamical system

Total and partial amplitude death in networks of diffusively coupled oscillators

The network coding capacity of a single multicast traffic is characterized by the min-cut/max-flow (mcMF) theorem, which can be achieved by random linear network coding (RLNC). Nonetheless, the graph-theoretic characterization for multiple unicast/multicast traffic remains an open problem. This paper proposes and studies a new class of intersession-network-coding schemes: RLNC with selective cancelling (SC), which inherits the complexity advantage of RLNC once the set of selective cancelling edges is decided. A graph-theoretic characterization is provided for the achievable rates of RLNC with SC for the general multiple multicast setting. The findings contain most existing achievability results as special cases, including the mcMF theorem of the single multicast traffic and the existing characterization of pairwise intersession network coding. One prominent feature of the proposed approach is its focus on the achievability analysis with arbitrary network topology, arbitrary inter-session packet-mixing capability, and arbitrary traffic demands, which distinguishes the results from the special case analysis, capacity outer bound constructions, and the pattern-based (butterfly-based) superposition arguments.

Solutions for time synchronization based on coupled oscillators operate in a self-organizing and adaptive manner and can be applied to various types of dynamic networks. In this demo, we present the actual implementation of our guaranteed self-organizing synchronization algorithm on packet based networks like wireless sensor networks as well as on iPhone devices (by using acoustic signals). The algorithm is purposed for pulse couple oscillators and exploits the interactions between them to guarantee synchronization on arbitrary network topologies. It supports the design of scalable, minimal control toter and energy efficient communication protocols for fully distributed synchronization.

On the delay bound for coordination of multiple generic linear agents under arbitrary topology with time delay

We study the long term (steady state) performance of a simple, randomized, local load balancing technique. We assume a system of n processors connected by an arbitrary network topology. Jobs are placed in the processors by a deterministic or randomized adversary. The adversary knows the current and past load distribution in the network and can use this information to place the new tasks in the processors. The adversary can put a number of new jobs in each processor, in each step, as long as the (expected) total number of new jobs arriving at a given step is bounded by /spl lambda/n. A node can execute one job per step, and also participate in one load balancing operation in which it can move tasks to a direct neighbor in the network. In the protocol we analyze here, a node equalizes its load with a random neighbor in the graph. We first study the stability of a system running our load balancing protocol. Clearly, if /spl lambda/ > 1 the system cannot be stable. We show that for any /spl lambda/ < 1, and any connected network topology, the system is stable. When the system is stable, the next performance parameter of interest is the waiting time of jobs. We develop high probability bounds and bounds on the expectation of the waiting time of jobs in terms of the network topology. In particular, if the network is an expander graph the expected wait of a task is O(log n), and the waiting time of a task that enters the network at an arbitrary time is O(log n) with high probability. We contrast these results with the work stealing load balancing protocol, where we show that, in sparse networks, the load in the system and the waiting time can be exponential in the network size.

We describe new methods for multi-hop mobile wireless network design, so as to meet performance specifications. We introduce an implicitly defined approximate loss model that couples the physical, MAC and routing layers' effects. The model provides quantitative statistical relations between the loss parameters that are used to characterize multiuser interference and physical path conditions on the one hand and the traffic rates and types between origin-destination pairs on the other. The model considers effects of hidden nodes, node scheduling algorithms, MAC and PHY layer failures and unsuccessful packet transmission attempts at the MAC layer in arbitrary network topologies where multiple paths share nodes. We consider both contention-based as well as scheduling-based MAC protocols. The new methods are based on implicit loss network models developed using fixed point methods. We describe the application of Automatic Differentiation (AD) to these implicit performance models, and develop a methodology for sensitivity analysis and parameter optimization for wireless protocols. We also introduce new methods that utilize on-line feedback from the PHY layer to simplify the joint MAC and routing protocol design. We provide several examples, with realistic scenarios, that demonstrate the benefits of applying these new methods in an integrated manner. We close with current and future work that extends these methods and algorithms towards a Component-Based design, performance analysis and validation of distributed protocols for mobile wireless networks. We argue that this set of ideas provides a very promising approach to 'clean slate' design for such networks and the resulting hybrid broadband Internet.