Allocating modules to processors in a distributed system

This dissertation focuses on the problem of assigning the modules of a program to the processors in a distributed system with the goal of minimizing the overall cost of running the program. The cost depends on the execution times of the modules on the processors and on the cost of communication between modules. This module allocation problem arises in a variety of situations where one is interested in making optimum use of available computer resources. The general module allocation problem is intractable; however it becomes polynomially-solvable when the communication graph is restricted. In this dissertation, we restrict our attention to k-trees. As the first problem, we study parametric module allocation on partial k-trees. We allow the costs, both execution and communication, to vary linearly as functions of a real parameter t. We show that if the number of processors is fixed, the sequence of optimum assignments that are obtained, as t varies from zero to infinity, can be constructed in polynomial time. As an auxiliary result, we develop a linear-time algorithm to find a separator in a k-tree. We discuss the implications of our results for parametric versions of the weighted vertex cover, independent set, and 0-1 quadratic programming problems on partial k-trees. Next, we consider two variants of the assignment problem. The first problem is to find a minimum-cost assignment when one of the processors has a limited memory. The second is to find an assignment that minimizes the maximum processor load. We present exact dynamic programming algorithms for both problems, which lead to approximation schemes for the case where the communication graph is a partial k-tree. Faster algorithms are presented for trees with uniform costs. In contrast to these results, we show that, for arbitrary graphs, no fully polynomial time approximation schemes exist unless P = NP. Both dynamic programming algorithms have been implemented. The implementation details and our experimental results are presented.

The problem of allocating modules to processors in a distributed system to minimize total costs when the underlying communication graph is a partial k-tree and all costs are linear functions of a real parameter t is considered. It is shown that if the number of processors is fixed, the sequence of optimum assignments that are obtained as t varies from zero to infinity can be constructed in polynomial time. As an auxiliary result, a linear time separator algorithm for k-trees is developed. The implications of the results for parametric versions of the weighted vertex cover, independent set, and 0-1 quadratic programming problems on partial k-trees are discussed. >

This paper considers the problem of task assignment in heterogeneous distributed computing systems with the goal of minimizing the total execution and communication costs. An iterated local search algorithm is proposed for finding the suboptimal task assignment in a reasonable amount of computation time. We study the performance of the proposed algorithm over a wide range of parameters such as the problem scales, the ratio of average communication time to average computation time, and task interaction density of applications. The effectiveness of the algorithm is manifested by comparing it with other competing algorithms in the relevant literature.

The task allocation problem with constant communication

Given n heterogeneous traffic sources which generate multiple types of traffic among themselves, we consider the problem of finding a set of disjoint clusters to cover n traffic sources. The objective is to minimize the total communication cost for the entire system in the context that the intracluster communication is less expensive than the intercluster communication. Different from the general graph partitioning problem, our work takes into account the physical topology constraints of the linear arrangement of physical cells in highway cellular systems and the hexagonal mesh arrangement of physical cells in cellular systems. In our partitioning schemes, the optimal partitioning problem is transformed into an equivalent problem with a relative cost function, which generates the communication cost differences between the intracluster communications and the intercluster communications. For highway cellular systems, we have designed an efficient optimal partitioning algorithm of O(mn/sup 2/) by dynamic programming, where m is the number of clusters of n base stations. The algorithm also finds all the valid partitions in the same polynomial time, given the size constraint on a cluster and the total allowable communication cost for the entire system. For hexagonal cellular systems, we have developed four heuristics for the optimal partitioning based on the techniques of moving or interchanging boundary nodes between adjacent clusters. The heuristics have been evaluated and compared through experimental testing and analysis.

Distributed computing systems are of current interest due to the advancement of microprocessor technology and computer network. The prime function of effective utilization of distributed system is accurately mapping of task and tasks their scheduling on different processors for reducing their total finish time. Total runtime is taken time for all module with the runtime of tasks and their communication cost among tasks. In Distributed processing system, partitioning of a task into modules and proper allocation of module among processors are more important factor for efficient utilization of resources. In this paper, we discuss allocation of a set of 'm' module of a task to a set of 'n' processors (where m > n) to get an optimal utilization of available processors. We partition the task in different size of module and design an algorithm for allocation of these modules in distributed processing environment. While designing the model Per Bit Processing Rate, Inter Task Communication Cost, Task Size has been taken into consideration. We consider the constraints of minimizing execution cost and their Inter task communication and maximize the overall throughput of the system to be taking as constraints in such a way that allocated load on all the processor is balances. Keyword: Distributed computing, Task Allocation, Module distribution, Execution Cost, Communication Cost, Task Size.

For a given graph G and p pairs (s i, ti), 1≤i≤p, of vertices in G, the edge-disjoint paths problem is to find p pairwise edge-disjoint paths P i, 1≤i≤p, connecting s i and t i. Many combinatorial problems can be efficiently solved for partial k-trees (graphs of treewidth bounded by a fixed integer k), but it has not been known whether the edge-disjoint paths problem can be solved in polynomial time for partial k-trees unless p=O(1). This paper gives two algorithms for the edge-disjoint paths problem on partial k-trees. The first one solves the problem for any partial k-tree G and runs in polynomial time if p=O(log n) and in linear time if p=O(1), where n is the number of vertices in G. The second one solves the problem under some restriction on the location of terminal pairs even if p ≥ log n.Key wordsedge-disjoint pathspartial k-treebounded tree-widthpolynomial-time algorithmedge-coloring

In Federated Learning (FL), two-way model exchanges are required between the server and the workers every training round. Due to the large size of machine learning models, communications between them lead to high training delay and economic cost. At present, communication-efficient FL methods, for examples, top-k sparsification and quantization, taking advantages of the sparseness of model gradients and the fact that gradient-based model updating can tolerance small deviations, effectively reduce the communication cost of single training round. However, these gradient-based communication-efficient schemes cannot be applied to downlink communication. In addition, they cannot be used in conjunction with those communicationfrequency-suppressed methods, e.g., FedAvg, which hinders them from further improving training efficiency. In this paper, we propose FedCS, a compressive sensing based FL method, which can effectively compress and accurately reconstruct non-sparse model (both local and global) parameters (iveights), and can reduce the overall communication cost up to 10 $\times$ as compared to FedAvg without decreasing test accuracy. We introduce 1) a dictionary learning scheme with a quasi-validation set, which helps to project non-sparse parameters onto a sparse domain; 2) ajoint reconstruction scheme, by using which the server recovers global model parameters by executing the reconstruction algorithm only once a round, regardless of the number of compressed local models; 3) a compression ratio adjustment strategy, which balances the trade-off between total communication cost and model accuracy. We perform FedCS on three image classification tasks, and compare it with FedAvg, FedPAQ and T-FedAvg (two improvements of FedAvg). Experimental results demonstrate that FedCS outperforms comparison methods in all tasks, and always maintains a comparable test accuracy to FedAvg, even using a small quasi-validation set and on Non-IId data.

In a given project network, execution of each activity in normal duration requires utilization of certain resources. If faster execution of an activity is desired then additional resources at extra cost would be required. Given a project network, the cost structure for each activity and a planning horizon, the project compression problem is concerned with the determination of optimal schedule (duration) of performing each activity while satisfying given restrictions and minimizing the total cost of project execution. This paper considers the project compression problem with time dependent cost structure for each activity. The planning horizon is divided into several regular time intervals over which the cost structure of an activity may vary. But the cost structure of the activities remains the same (constant) within a time interval. Key events of the project attract penalty for finishing earlier or later than the corresponding target times. The objective is to find an optimal project schedule minimizing the total project cost. We present a mathematical model for this problem, develop some heuristics and an exact branch and bound algorithm. Using simulated problems we provide an insight into the computational performances of heuristics and the branch and bound algorithm.

Mobile edge computing (MEC) technology guarantees the privacy and security of large-scale data in the Narrowband-IoT (NB-IoT) by deploying MEC servers near base stations to provide sufficient computing, storage, and data processing capacity to meet the delay and energy consumption requirements of NB-IoT terminal equipment. For the NB-IoT MEC system, this paper proposes a resource allocation algorithm based on deep reinforcement learning to optimize the total cost of task offloading and execution. Since the formulated problem is a mixed-integer non-linear programming (MINLP), we cast our problem as a multi-agent distributed deep reinforcement learning (DRL) problem and address it using dueling Q-learning network algorithm. Simulation results show that compared with the deep Q-learning network and the all-local cost and all-offload cost algorithms, the proposed algorithm can effectively guarantee the success rates of task offloading and execution. In addition, when the execution task volume is 200KBit, the total system cost of the proposed algorithm can be reduced by at least 1.3%,and when the execution task volume is 600KBit, the total cost of system execution tasks can be reduced by 16.7% at most.

Graph computing plays an important role in mining data at large scale. Partition is the primary step when we process large graph in a distributed system. A good partition has less communication and memory cost as well as more balanced load to take advantage of the whole system. Traditional edge cut methods introduce large communication cost for realistic power law graphs. Current vertex cut methods perform poorly with little consideration on load balance especially for online streaming vertex cut partition. In this paper, we formulate the total cost (partition cost, communication cost and computing cost) of graph computing especially that in iterating algorithms and analyze the cost of current partitioning methods. In addition, we explore a novel vertex cut method to ensure lower total cost. It has more balanced load with fewer communications. Experiments show that our method outperforms in state of the art graph computing frameworks at an average of 10 percent.

In this paper, we address an optimization problem that arises in context of cache placement in sensor networks. In particular, we consider the cache placement problem where the goal is to determine a set of nodes in the network to cache/store the given data item, such that the overall communication cost incurred in accessing the item is minimized, under the constraint that the total communication cost in updating the selected caches is less than a given constant. In our network model, there is a single server (containing the original copy of the data item) and multiple client nodes (that wish to access the data item). For various settings of the problem, we design optimal, near-optimal, heuristic-based, and distributed algorithms, and evaluate their performance through simulations on randomly generated sensor networks.KeywordsSensor NetworkSensor NodeGreedy AlgorithmData ItemSteiner TreeThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

In this work, we study the computational perspective of network coding, focusing on two issues. First, we address the computational complexity of finding a network code for acyclic multicast networks. Second, we address the issue of reducing the amount of computation performed by the network nodes. In particular, we consider the problem of finding a network code with the minimum possible number of encoding nodes, i.e., nodes that generate new packets by combining the packets received over incoming links. We present a deterministic algorithm that finds a feasible network code for a multicast network over an underlying graph G(V, E) in time O(|E|kh+|V|k <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> h <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> +h <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">4</sup> k <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> (k+h)), where k is the number of destinations and h is the number of packets. This improves the best known running time of O(|E|kh+|V|k <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> h <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> (k+h)) of Jaggi et al. (2005) in the typical case of large communication graphs. In addition, our algorithm guarantees that the number of encoding nodes in the obtained network code is bounded by O(h <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> k <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ). Next, we address the problem of finding a network code with the minimum number of encoding nodes in both integer and fractional coding networks. We prove that in the majority of settings this problem is NP-hard. However, we show that if h=O(1) and k=O(1) and the underlying communication graph is acyclic, then there exists an algorithm that solves this problem in polynomial time.

Generalized coloring for tree-like graphs

In this paper, we are interested in path optimization for robotic communication operations in obstacle environments. Consider a robot that needs to perform a given communication task (e.g., data uploading, broadcasting, or relaying) while navigating from a start position to a designated final position, avoiding obstacles, and minimizing its total motion and communication costs. Our goal is to develop a general path planning algorithm applicable to various robotic communication scenarios in which a robot operates in realistic channel fading environments and in the presence of obstacles. We show how we can adapt the traditional Rapidly-Exploring Random Tree Search Star (RRT*) path planning algorithm to jointly consider both communication and motion objectives in realistic channel environments that contain obstacles. We further show that our proposed approach can provide theoretical optimality guarantees while being computationally efficient. We extensively evaluate our proposed approach in realistic wireless channel environments for various transmission settings and communication tasks. The results demonstrate the efficacy of our proposed approach.