Scheduling in Heterogeneous Distributed Computing Systems Based on Internal Structure of Parallel Tasks Graphs with Meta-Heuristics

First we show that all genetic algorithms can be approximated by an algorithm which keeps the population in linkage equilibrium. Here the genetic population is given as a product of univariate marginal distributions. We describe a simple algorithm which keeps the population in linkage equilibrium. It is called the univariate marginal distribution algorithm (UMDA). Our main result is that UMDA transforms the discrete optimization problem into a continuous one defined by the average fitness W(p1, . . . , p n ) as a function of the univariate marginal distributions p i. For proportionate selection UMDA performs gradient ascent in the landscape defined by W(p). We derive a difference equation for p i which has already been proposed by Wright in population genetics. We show that UMDA solves difficult multimodal optimization problems. For functions with highly correlated variables UMDA has to be extended. The factorized distribution algorithm (FDA) uses a factorization into marginal and conditional distributions. For decomposable functions the optimal factorization can be explicitly computed. In general it has to be computed from the data. This is done by LFDA. It uses a Bayesian network to represent the distribution. Computing the network structure from the data is called learning in Bayesian network theory. The problem of finding a minimal structure which explains the data is discussed in detail. It is shown that the Bayesian information criterion is a good score for this problem.

Like Darwinian-type genetic algorithms, there also exists genetic drift in Univariate Marginal Distribution Algorithm (UMDA). Since the universal analysis of genetic drift in UMDA is very difficult, in this paper, we just approach a certain kind of problem (WOneMax Problem). For WOneMax Problem, The individual space in UMDA can be denoted as a full binary tree, and the selecting process in UMDA can be considered as a process of cutting branch. We employ this binary tree to calculate the probability change of each variable between two adjacent generations. Comparing this change with our experimental data, we find that when the population size is limited, there exists genetic drift in UMDA. In order to avoid genetic drift, we model the probability of each variable as a signal with noise, and then use smoothing filter to eliminate genetic drift. Numerical results show this method is effective.

Feature subset selection for unsupervised learning, is a very important topic in artificial intelligence because it is the base for saving computational resources. In this implementation we use a typical testor's methodology in order to incorporate an importance index for each variable. This paper presents the general framework and the way two hybridized meta-heuristics work in this NP-complete problem. The evolutionary mechanisms are based on the Univariate Marginal Distribution Algorithm (UMDA) and the Genetic Algorithm (GA). GA and UMDA --- Estimation of Distribution Algorithm (EDA) use a very useful rapid operator implemented for finding typical testors on a very large dataset and also, both algorithms, have a local search mechanism for improving time and fitness. Experiments show that EDA is faster than GA because it has a better exploitation performance; nevertheless, GA' solutions are more consistent.

Shared memory programming and distributed memory programming, are the most prominent ways of parallelize applications requiring high processing times and large amounts of storage in High Performance Computing (HPC) systems; parallel applications can be represented by Parallel Task Graphs (PTG) using Directed Acyclic Graphs (DAGs). The scheduling of PTGs in HPCS is considered a NP-Complete combinatorial problem that requires large amounts of storage and long processing times. Heuristic methods and sequential programming languages have been proposed to address this problem. In the open access paper: Scheduling in Heterogeneous Distributed Computing Systems Based on Internal Structure of Parallel Tasks Graphs with Meta-Heuristics, the Array Method is presented, this method optimizes the use of Processing Elements (PE) in a HPCS and improves response times in scheduling and mapping resource with the use of the Univariate Marginal Distribution Algorithm (UMDA); Array Method uses the internal characteristics of PTGs to make task scheduling; this method was programmed in the C language in sequential form, analyzed and tested with the use of algorithms for the generation of synthetic workloads and DAGs of real applications. Considering the great benefits of parallel software, this research work presents the Array Method using parallel programming with OpenMP. The results of the experiments show an acceleration in the response times of parallel programming compared to sequential programming when evaluating three metrics: waiting time, makespan and quality of assignments.

In DNA and RNA, five types of nitrogenous bases are present; these are adenine (A), uracil (U), guanine (G), thymine (T), and cytosine (C). Sometimes, these arrangements are altered which is known as single nucleotide polymorphism. These polymorphisms appear due to two causes (i) mutation and (ii) disease. By classifying these two types of SNPs, we can conclude a disease-causing SNP. In this paper, we describe genetic analysis of simple and complex disease evaluation by various methods. Here, we describe Apriori algorithm, genetic algorithm, machine learning approach like support vector machine (SVM). An approach of Apriori-Gen algorithm is also discussed which evaluates statistical interaction between several SNPs to find association among them. Univariate marginal distribution algorithm (UMDA) and support vector machine (SVM) are also used to find disease identification. USVM is used not only for its redundancy feature but also it solves parameters selecting problem of SVM.

A hybrid heuristic–genetic algorithm for task scheduling in heterogeneous processor networks

Executing complicated computations in parallel increases the speed of computing and brings user delight to the system. Decomposing the program into several small programs and running multiple parallel processors are modeled by Directed Acyclic Graph. Scheduling nodes to execute this task graph is an important problem that will speed up computations. Since task scheduling in this graph belongs to NP-hard problems, various algorithms were developed for node scheduling to contribute to quality service delivery. The present study brought a heuristic algorithm named looking ahead sequencing algorithm (LASA) to cope with static scheduling in heterogeneous distributed computing systems with the intention of minimizing the schedule length of the user application. In the algorithm proposed here, looking ahead is considered as a criterion for prioritizing tasks. Also, a property called Emphasized Processor has been added to the algorithm to emphasize the task execution on a particular processor. The effectiveness of the algorithm was shown on few workflow type applications and the results of the algorithm implementation were compared with two more heuristic and meta-heuristic algorithms.

Evolutionary computation and Wright's equation

With the rapid development of high-throughput genotyping technologies, more and more attentions are paid to the disease association study identifying DNA variations that are highly associated with a specific disease. One main challenge for this study is to find the optimal subsets of Single Nucleotide Polymorphisms (SNPs) which are most tightly associated with diseases. Feature selection has become a necessity in many bioinformatics applications. In this paper, we propose a wrapper algorithm named USVM which combines Univariate Marginal Distribution Algorithm (UMDA) and Support Vector Machine (SVM) for disease association study. USVM not only eliminates the redundancy of feature, but also solves the problem of SVM's parameters selection. We use USVM to analyze the Crohn's disease (CD) dataset including 387 samples and each one has 103 SNPs. The experimental results show that our algorithm outperforms the current algorithms including DNF, CSP, ORF and so on.

Coarse-grained reconfigurable architectures (CGRAs) are increasingly employed as domain-specific accelerators due to their efficiency and flexibility. However, the existing CGRA architectures suffer from low hardware resource utilization and performance due to the limitations of the scheduling scheme. In this article, an adaptive scheduler (denoted as DPTM) for concurrent and multi-task interleaved pipelining-oriented CGRA is introduced, which exploits timeslot matching and release methods to avoid the pipeline conflicts and improve the scheduling performance. The characteristics of task scheduling based on directed acyclic graph (DAG) are analyzed, and several performance-influencing factors are extracted to build a scheduling performance model for reducing the time cost of scheduling and guiding the design of scheduling schemes. Moreover, the scoreboard method of dynamic instruction schedulers is optimized to control the entry time of multiple tasks into the pipeline, and then a timeslot matching method is proposed to provide non-conflict pipelining for the multiple tasks. Further, a timeslot release method is presented to release the timeslots for unscheduled sub-tasks dynamically, which can adapt the parallel processing of multiple tasks and decrease the scheduling time. Then, an adaptive scheduling scheme combines the dynamic priority-based task assignment method, timeslot matching method, and timeslot release method to schedule massive tasks for CGRA. Finally, the overall architecture of DPTM is introduced and designed to validate the efficacy of the proposed scheduling scheme. Experimental results show that the proposed timeslot matching/release approach reduces 84% total scheduling time and decreases 40% average scheduling time at most compared to the non-timeslot-matching scheduling schemes, the proposed task assignment approach decreases 8% total scheduling time and lowers 3% average scheduling time compared to the existing approaches, and the proposed scheduler decreases 51% critical path delay, lowers 35% area overhead, and reduces 12% power consumption at most compared with the existing schedulers.

This paper addresses the problem of static load balancing in heterogeneous distributed computing systems taking into account both memory and communication capacity constraints. It first models the load balancing problem as an optimization problem. It then presents a modified genetic algorithm, called Adaptive Genetic Algorithm (AGA), to solve the problem. The performance of the proposed algorithm is evaluated by simulation studies on randomly generated instances and the results are compared with that obtained by applying both the Genetic Algorithm (GA) and the Simulated Annealing (SA). Also, the qualities of the results are compared with the optimal solutions that obtained by applying the Brach-and-Bound (BB) algorithm.

The compact Genetic Algorithm (cGA) evolves a probability distribution favoring optimal solutions in the underlying search space by repeatedly sampling from the distribution and updating it according to promising samples. We study the intricate dynamics of the cGA on the test function OneMax, and how its performance depends on the hypothetical population size K, which determines how quickly decisions about promising bit values are fixated in the probabilistic model. It is known that the cGA and the Univariate Marginal Distribution Algorithm (UMDA), a related algorithm whose population size is called lambda, run in expected time O(n log n) when the population size is just large enough (K = varTheta (sqrt{n}log n) and lambda = varTheta (sqrt{n}log n), respectively) to avoid wrong decisions being fixated. The UMDA also shows the same performance in a very different regime (lambda =varTheta (log n), equivalent to K = varTheta (log n) in the cGA) with much smaller population size, but for very different reasons: many wrong decisions are fixated initially, but then reverted efficiently. If the population size is even smaller (o(log n)), the time is exponential. We show that population sizes in between the two optimal regimes are worse as they yield larger runtimes: we prove a lower bound of varOmega (K^{1/3}n + n log n) for the cGA on OneMax for K = O(sqrt{n}/log ^2 n). For K = varOmega (log ^3 n) the runtime increases with growing K before dropping again to O(Ksqrt{n} + n log n) for K = varOmega (sqrt{n} log n). This suggests that the expected runtime for the cGA is a bimodal function in K with two very different optimal regions and worse performance in between.

Detecting diseases associated SNPs is the central goal of genetics and molecular biology. However, highthroughput techniques often provide long lists of disease SNPs candidates, and the identification of disease SNPs among the candidates set remains timeconsuming and expensive. In addition, contrasting to the number of SNPs involved, the available datasets (samples) generally have fairly small sample size and the datasets used in these studies are often imbalanced (the number of cases and controls are not equal). Therefore, it is necessary to develop a robust tool to identify the disease associated SNPs. In this paper, we proposed a modified univariate marginal distribution algorithm (MUMDA) taking into account the imbalanced ratios of case/control for selecting the associated SNPs. In addition, the sparse vector encoding method was used in this paper. We illustrated the effectiveness of our algorithm using Crohn’s disease (CD) dataset which includes 144 cases and 243 controls (each one has 103 SNPs). Our observations suggested that the proposed method could capture important features of the genetic architecture of CD and our algorithm outperformed other current methods.

Several approaches have been developed into evolutionary algorithms to deal with dynamic optimization problems, of which memory and random immigrants are two major schemes. This paper investigates the application of a direct memory scheme for univariate marginal distribution algorithms (UMDAs), a class of evolutionary algorithms, for dynamic optimization problems. The interaction between memory and random immigrants for UMDAs in dynamic environments is also investigated. Experimental study shows that the memory scheme is efficient for UMDAs in dynamic environments and that the interactive effect between memory and random immigrants for UMDAs in dynamic environments depends on the dynamic environments.

It has been widely perceived that a univariate Gaussian model for evolutionary search can be used to solve separable problems only. This paper explores whether and how the univariate Gaussian model may also be used to solve inseparable problems. The analysis is followed up with experimental tests. The results show that the univariate Gaussian model stipulates no inclination towards separable problems. Further, it is revealed that the model is not only an efficient but also an effective method for solving multimodal inseparable problems. To verify its relative convergence speed, a restart strategy is applied to a univariate Gaussian model (the univariate marginal distribution algorithm) on inseparable problems. The results confirm that the univariate Gaussian model outperforms the five peer algorithms studied in this paper.