Logical Instantaneity and Causal Order: Two “First Class” Communication Modes for Parallel Computing

Distributed memory parallel machines are increasingly being used by the scientific community to provide computational power for solving problems. Building a distributed memory parallel machine offers a tremendous advantage in terms of cost and scalability over building a shared memory parallel machine. Unfortunately, distributed memory machines are much more difficult to program, mainly due to the lack of a single global address space. Communication between processors in a distributed memory machine is accomplished by message passing. The time cost for acquiring data from another processor's memory is significantly greater than the time cost of acquiring data from local memory. To assist the programmer in using a distributed memory machine, compilers have been developed which alleviate the need for the programmer to manage the lack of a global address space. These compilers will generate semantically correct code containing communication routines to produce a logically shared memory machine on a physically distributed memory machine. However, the runtime performance of the code emitted from some of these compilers can be further improved. This thesis is primarily concerned with advanced compilation techniques which will provide better runtime performance of programs written in imperative languages for regular scientific programs. The optimization of communication generation for distributed memory machines is used to produce this result. We look at two "state of the art" compilation systems and find that improvement in the communication code generated by these compilers is still possible. We have simplified existing descriptions of the various formalisms found in these compilers, while still providing enough formalism to study the problems in depth. We have extended some existing optimization techniques by enhancing the elimination of masks, by extending the notion of relaxing the owner computes rule, by extending the notion of replicating computation, and by extending the notion of replicating data. This thesis has discussed a novel technique to split loops so that they do not contain any guarded statements. Finally, we have suggested avenues for future research, as it is clear from this thesis that more work is required before compilation systems for distributed memory parallel machines will be widely used for general purpose computing.

Mobile agent group concept is a versatile software paradigm capable of providing group task execution flexibility and collaborative adaptability in various application fields of distributed networked systems. However, as the size of these fields is rapidly increasing, improving the performance of the agent group communication in Internet-scale infrastructures should be reconsidered to be suitable for their scale. For this purpose, some appropriate inter-agent group communication protocols are required in distributed agent-based systems. Causal order delivery to a broadcast group is a very important issue for these systems. However, the existing message delivery protocols for mobile agent groups may not satisfy this important requirement. This paper presents a new causally ordered group communication protocol for mobile agent groups to consider weaker message ordering consistency than atomic broadcast semantics while ensuring reliability even with the assumption of unreliable networks for exchanging messages to agent groups and agent mobility. The simulation results show the proposed protocol performs better than the existing atomic broadcast protocol in terms of the total message delivery latency.

Loops are the main source of parallelism in scientific programs. Hence, several techniques were developed to detect parallelism in these loops and transform them into parallel forms. In this dissertation, compile time transformations and efficient parallel execution of loops with various type of dependencies are investigated. First, Doacross loops with uniform dependencies are considered for execution on distributed memory parallel machines (multicomputers). Most known Doacross loop execution techniques can be applied efficiently only to shared memory parallel machines. In this thesis, code reordering technique, improvements to partitioning strategies, and finding a balance between communication and parallelism are presented to reduce the execution time of Doacross loops on multicomputers. As with most parallelizing transformation techniques, only single loopnests are considered in the first part of this dissertation. However, parallelizing each loopnest in a program separately, even if an optimal execution can be obtained for each loopnest, may not result in an efficient execution of all the code in the program because of communication overhead across the loops in a multicomputer environment. Hence, across loop data dependence representation and analysis are introduced in this work to improve the parallel execution of the whole code. Our contribution consists of finding and representing data dependencies whose sources and destinations are subspaces of the iteration space mainly common across the loops. This type of dependence information is used in this thesis to improve global iteration space partitioning, automatic generation of communication statements across loops, and index alignment. The final part of this dissertation presents new parallelizing techniques for loops with irregular and complex dependencies. Various data dependence analysis algorithms can be found in the literature even for loops with complex array indexing. However, the improvement in data dependence testing has not been followed by similar amelioration in restructuring transformations for loops with complex dependencies. Such loops are mostly executed in serial mode. Our parallelizing techniques for these loops consists of identifying regions of the iteration space where all iterations can be executed in parallel. The advantages of all the transformations presented in this dissertation are: (1) they significantly reduce the execution time of loops with various types of dependencies as shown in this work using the MasPar machine (2) they can be implemented at compile, time which makes the task of parallel programming easier.

Over the last year, a considerable portion of FEFLO, a general-purpose legacy CFD code operating on unstructured grids, was ported to run on GPUs. Like so many other legacy codes, FEFLO is an adaptive, edge-based nite element code for the solution of compressible and incompressible ow, which was primarily written in Fortran 77 and has previously been ported to vector, shared memory parallel and distributed memory parallel machines. Due to the large size of FEFLO and the likelihood of human error in porting, as well as the desire for continued development within a single codebase, a specialized Python script, based on FParser (Peterson, 2009), was written to perform automated translation from the OpenMP-parallelized loops to GPU kernels implemented in CUDA, along with GPU memory management, while integrating with the existing framework for distributed memory parallelism via MPI. The present paper describes extensions of the script and algorithmic techniques that enable the ecient running on GPUs of the modules that are not straightforward to port. In particular, we consider LU-SGS algorithms, linelet preconditioning and particle-mesh algorithms.

PurposeThe purpose of the paper is to investigate the effects of service quality and service satisfaction on intention in a business‐to‐business setting.Design/methodology/approachThis research addresses three unanswered questions regarding satisfaction and service quality: the distinction between customer satisfaction and perceived service quality; their causal ordering; and their relative impact on intentions. The data were collected using a large survey of buyers in a business setting.FindingsThe data were analyzed using structural equation modeling. The results show that service quality has a larger impact on intentions than does customer satisfaction. The results also show that the effects of individual transactions on intentions are mediated by corresponding cumulative constructs.Research limitations/implicationsThe primary implications for theory include demonstrating the distinction between satisfaction and service quality; specifying, based on theory and logic, the causal ordering between transaction constructs and cumulative constructs, and between service quality and satisfaction; and assessing their relative impact on behavioral intentions.Originality/valueThe results show that one negative transaction outcome may not be sufficient to cause the customer to switch if the cumulative levels are sufficiently positive. Thus, a negative outcome may be discounted by the user if it is seen as a unique occurrence. However, a series of successive negative transaction outcomes may cause the cumulative constructs to become less positive, resulting in lower intentions to repurchase from the same supplier.

A note on the correctness of the causal ordering algorithm

Vector autoregressions (VARs) are economically interpretable only when identified by being transformed into a structural form the (SVAR) in which the contemporaneous variables stand in a well-defined causal order. These identifying transformations are not unique. It is widely believed that practitioners must choose between them using a priori theory or other criteria not rooted in the data under analysis. We show how to apply graph-theoretical approach of searching for causal structure based on relations of conditional independence to select the possible causal orders – or at least to reduce the admissible them to a narrow equivalence class. This study investigates the dynamic structure of four major stock markets using an error correction model and directed acyclic graphs (DAG). The DAG representation provides a structure of causality among these markets in a contemporaneous time. Building this contemporaneous structure and the estimated error correction model, innovation accounting techniques are applied.

The implementation and performance of a parallel spatial direct numerical simulation (PSDNS) approach on the Intel iPSC/860 hypercube and IBM SP1 and SP2 parallel computers is documented. Spatially evolving disturbances associated with laminar-to-turbulent transition in boundary-layer flows are computed with the PSDNS code. The feasibility of using the PSDNS to perform transition studies on these computers is examined. The results indicate that PSNDS approach can effectively be parallelized on a distributed memory parallel machine by remapping the distributed data structure during the course of the calculation. The performance of the PSDNS indicates that 32 nodes on the IBM SP1 or SP2 are required to match a comparable computation on the CRAY C-90 single processor; the hypercube was found to be two times slower than the CRAY Y/MP for this application.

An algorithm for FIFO message delivery among migrating tasks

LOCA, the Library of Continuation Algorithms, is a software library for performing stability analysis of large-scale applications. LOCA enables the tracking of solution branches as a function of a system parameter, the direct tracking of bifurcation points, and, when linked with the ARPACK library, a linear stability analysis capability. It is designed to be easy to implement around codes that already use Newton's method to converge to steady-state solutions. The algorithms are chosen to work for large problems, such as those that arise from discretizations of partial differential equations, and to run on distributed memory parallel machines. This manual presents LOCA's continuation and bifurcation analysis algorithms, and instructions on how to implement LOCA with an application code. The LOCA code is being made publicly available at www.cs.sandia.gov/loca.

Array remapping are useful to many applications on distributed memory parallel machines. They are available in High Performance Fortran, a Fortran-based data-parallel language. This paper describes techniques to handle dynamic mappings through simple array copies: array remapping are translated into copies between statically mapped distinct versions of the array. It discusses the language restrictions required to do so. The remapping graph which captures all remapping and liveness information is presented, as well as additional data-flow optimizations that can be performed on this graph, so as to avoid useless remapping at run time. Such useless remapping appear for arrays that are not used after a remapping. Live array copies are also kept to avoid other flow-dependent useless remappings. Finally the code generation and runtime required by our scheme are discussed. These techniques are implemented in our prototype HPF compiler.

Abstract: Processing and analyzing large volumes of data plays an increasingly important role in many domains of scientific research. We are developing a compiler that processes data intensive applications written in a dialect of Java and compiles them for efficient execution on distributed memory parallel machines. In this paper, we focus on the problem of generating correct and efficient communication for data intensive applications. We present static analysis techniques for 1) extracting a global reduction function from a data parallel loop, and 2) determining if a subscript function is monotonic. We also present a runtime technique for reducing the volume of communication during the global reduction phase. We have experimented with two data intensive applications to evaluate the efficacy of our techniques. Our results show that 1) our techniques for extracting global reduction functions and establishing monotonicity of subscript functions can successfully handle these applications, 2) significant reduction in communication volume and execution times is achieved through our runtime analysis technique, 3) runtime communication analysis is critical for achieving speedups on parallel configurations.

Abstract: Processing and analyzing large volumes of data plays an increasingly important role in many domains of scientific research. We are developing a compiler that processes data intensive applications written in a dialect of Java and compiles them for efficient execution on distributed memory parallel machines. In this paper, we focus on the problem of generating correct and efficient communication for data intensive applications. We present static analysis techniques for 1) extracting a global reduction function from a data parallel loop, and 2) determining if a subscript function is monotonic. We also present a runtime technique for reducing the volume of communication during the global reduction phase. We have experimented with two data intensive applications to evaluate the efficacy of our techniques. Our results show that 1) our techniques for extracting global reduction functions and establishing monotonicity of subscript functions can successfully handle these applications, 2) significant reduction in communication volume and execution times is achieved through our runtime analysis technique, 3) runtime communication analysis is critical for achieving speedups on parallel configurations.

Pollutant dispersion models in the atmosphere can describe by Eulerian or Lagrangian approaches. Lagrangian models belong to the class of Monte Carlo methods. This type of method is very flexible, solving more complex problems, however this computational cost is greater than Eulerian models, as it is well established in the atmospheric pollutant and nuclear engineering communities. A parallel version of the Lagrangian particle model - LAMBDA - is developed using the MPI message passing communication library. Performance tests were executed in a distributed memory parallel machine, a multicomputer based on IA-32 architecture. Portions of the pollutant in the air are considered particles emitted from a pollutant source, evolving under stochastic forcing. This yields independent evolution equations for each particle of the model that can be executed by a different processor in a parallel implementation. Speed-up results show that the parallel implementation is suitable for the used architecture.

Towards the Parallelisation of Pressure Correction Method on Unstructured Grids