A brief summary on formalizing parallel tensor distributions redistributions and algorithm derivations.

Cross-scale efficient tensor contractions for coupled cluster computations through multiple programming model backends

Tensor contractions are ubiquitous in computational chemistry and physics, where tensors generally represent states or operators and contractions express the algebra of these quantities. In this context, the states and operators often preserve physical conservation laws, which are manifested as group symmetries in the tensors. These group symmetries imply that each tensor has block sparsity and can be stored in a reduced form. For nontrivial contractions, the memory footprint and cost are lowered, respectively, by a linear and a quadratic factor in the number of symmetry sectors. State-of-the-art tensor contraction software libraries exploit this opportunity by iterating over blocks or using general block-sparse tensor representations. Both approaches entail overhead in performance and code complexity. With intuition aided by tensor diagrams, we present a technique, irreducible representation alignment, which enables efficient handling of Abelian group symmetries via only dense tensors, by using contraction-specific reduced forms. This technique yields a general algorithm for arbitrary group symmetric contractions, which we implement in Python and apply to a variety of representative contractions from quantum chemistry and tensor network methods. As a consequence of relying on only dense tensor contractions, we can easily make use of efficient batched matrix multiplication via Intel’s MKL and distributed tensor contraction via the Cyclops library, achieving good efficiency and parallel scalability on up to 4096 Knights Landing cores of a supercomputer.

Tensors (also known as mutidimensional arrays or N-way arrays) are used in a variety of applications ranging from chemometrics to psychometrics. We describe four MATLAB classes for tensor manipulations that can be used for fast algorithm prototyping. The tensor class extends the functionality of MATLAB's multidimensional arrays by supporting additional operations such as tensor multiplication. The tensor as matrix class supports the 'matricization' of a tensor, i.e., the conversion of a tensor to a matrix (and vice versa), a commonly used operation in many algorithms. Two additional classes represent tensors stored in decomposed formats: cp tensor and tucker tensor. We descibe all of these classes and then demonstrate their use by showing how to implement several tensor algorithms that have appeared in the literature.

Intra-query parallelism is a well-established mechanism for achieving high performance in (object-) relational database systems. However, the methods have yet not been applied to the upcoming field of multidimensional array databases. Specific properties of multidimensional array data require new parallel algorithms. This paper presents a number of new techniques for parallelizing queries in multidimensional array database management systems. It discusses their implementation in the RasDaMan DBMS, the first DBMS for generic multidimensional array data. The efficiency of the techniques presented is demonstrated using typical queries on large multidimensional data volumes.

Mining change patterns can give unique understanding on the evolution of dynamically changing systems like social relation graphs, weblinks, hardware descriptions and models. A more recent focus is source code change pattern mining that may qualitatively justify expected or uncover unexpected patterns. These patterns then offer a basis, e.g., for program language evolution or auto-completion support. We present a change pattern mining method that greatly expands the limits of input data and pattern complexity, over existing methods. We propose scalability solutions on conceptual and algorithmic level, thereby evolving the state-of-the-art sampling-based frequent subgraph mining method FS <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> , resulting in 75% reduction in memory consumption and a speedup of 6500 for a large scale dataset. Patterns can have 100,000s of occurrences for which manual review is impossible and may lead to misinterpretation. We propose the novel content track approach for interactively exploring pattern contents in context, based on marginal distributions. We evaluate our approach by mining 1,000 open source projects contributing a total of 558 million changes and 2 billion contextual connections among them, thereby, demonstrating its scalability. A manual interpretation of 19 patterns shows sensible mined patterns allowing to deduct implications for language design and demonstrating the soundness of the approach.

Two novel paralleled Probabilistic Latent Semantic Analysis (PLSA) algorithms based on the MapReduce model are proposed, which are P2LSA and P2LSA+, respectively. When dealing with a large-scale data set, P2LSA and P2LSA+ can improve the computing speed with the Hadoop platform. The Expectation-Maximization (EM) algorithm is often used in the traditional PLSA method to estimate two hidden parameter vectors, while the parallel PLSA is to implement the EM algorithm in parallel. The EM algorithm includes two steps: E-step and M-step. In P2LSA, the Map function is adopted to perform the E-step and the Reduce function is adopted to perform the M-step. However, all the intermediate results computed in the E-step need to be sent to the M-step. Transferring a large amount of data between the E-step and the M-step increases the burden on the network and the overall running time. Different from P2LSA, the Map function in P2LSA+ performs the E-step and M-step simultaneously. Therefore, the data transferred between the E-step and M-step is reduced and the performance is improved. Experiments are conducted to evaluate the performances of P2LSA and P2LSA+. The data set includes 20000 users and 10927 goods. The speedup curves show that the overall running time decrease as the number of computing nodes increases.Also, the overall running time demonstrates that P2LSA+ is about 3 times faster than P2LSA.KeywordsParalleled PLSAPLSAMapReduce

A new hardware-agnostic contraction algorithm for tensors of arbitrary symmetry and sparsity is presented. The algorithm is implemented as a stand-alone open-source code libxm. This code is also integrated with general tensor library libtensor and with the Q-Chem quantum-chemistry package. An overview of the algorithm, its implementation, and benchmarks are presented. Similarly to other tensor software, the algorithm exploits efficient matrix multiplication libraries and assumes that tensors are stored in a block-tensor form. The distinguishing features of the algorithm are: (i) efficient repackaging of the individual blocks into large matrices and back, which affords efficient graphics processing unit (GPU)-enabled calculations without modifications of higher-level codes; (ii) fully asynchronous data transfer between disk storage and fast memory. The algorithm enables canonical all-electron coupled-cluster and equation-of-motion coupled-cluster calculations with single and double substitutions (CCSD and EOM-CCSD) with over 1000 basis functions on a single quad-GPU machine. We show that the algorithm exhibits predicted theoretical scaling for canonical CCSD calculations, O(N6 ), irrespective of the data size on disk. © 2017 Wiley Periodicals, Inc.

The advent of high-throughput short read technology is revolutionizing life sciences by providing an inexpensive way to sequence genomes at high coverage. Exploiting this technology requires the development of a de novo short read assembler, which is an important open problem that is garnering significant research effort. Current methods are largely limited to microbial organisms, whose genomes are two to three orders of magnitude smaller than complex mammalian and plant genomes. In this paper, we present the design and development of a parallel de novo short read assembler that can scale to large genomes with high coverage. Our approach is based on the string graph formulation. Input reads are mapped to short paths, and the genome is reconstructed as a superpath anchored by distance constraints inferred from read pairs. Our method can handle a mixture of multiple read sizes and multiple paired read distances. We present parallel algorithms for string graph construction, string graph compaction, graph based error detection and removal, and computing aggregate summarization of paired read links across graph edges. Using this, we navigate the final graph structure to reproduce large contiguous sequences from the underlying genome. We present a validation of our framework on experimental and simulated data from multiple known genomes and present scaling results on IBM Blue Gene/L.

: System biology problems such as whole-genome network construction from large-scale gene expression data are sophisticated and time-consuming. Therefore, using sequential algorithms are not feasible to obtain a solution in an acceptable amount of time. Today, by using massively parallel computing, it is possible to infer large-scale gene regulatory networks. Recently, establishing gene regulatory networks from large-scale datasets have drawn the noticeable attention of researchers in the field of parallel computing and system biology. In this paper, we attempt to provide a more detailed overview of the recent parallel algorithms for constructing gene regulatory networks. Firstly, fundamentals of gene regulatory networks inference and large-scale datasets challenges are given. Secondly, a detailed description of the four parallel frameworks and libraries including CUDA, OpenMP, MPI, and Hadoop is discussed. Thirdly, parallel algorithms are reviewed. Finally, some conclusions and guidelines for parallel reverse engineering are described.

We consider the problem of low-rank approximation of massive dense nonnegative tensor data, for example, to discover latent patterns in video and imaging applications. As the size of data sets grows, single workstations are hitting bottlenecks in both computation time and available memory. We propose a distributed-memory parallel computing solution to handle massive data sets, loading the input data across the memories of multiple nodes, and performing efficient and scalable parallel algorithms to compute the low-rank approximation. We present a software package called Parallel Low-rank Approximation with Nonnegativity Constraints, which implements our solution and allows for extension in terms of data (dense or sparse, matrices or tensors of any order), algorithm (e.g., from multiplicative updating techniques to alternating direction method of multipliers), and architecture (we exploit GPUs to accelerate the computation in this work). We describe our parallel distributions and algorithms, which are careful to avoid unnecessary communication and computation, show how to extend the software to include new algorithms and/or constraints, and report efficiency and scalability results for both synthetic and real-world data sets.

This paper presents a methodology for performance prediction of parallel algorithms and illustrates its use on a large scale computational chemistry application. The performance prediction uses a component time characterization technique which splits up the sequential code into computational components and measures the time for each of them. The parallel algorithm is built from these components by adding communication routines. A “Processor Activity Graph” (PAG) providing a graphical representation of the parallel algorithm runtime behaviour is used for predicting the execution time. For a case study a Self Consistent Field (SCF) computation has been selected which forms the basis of many computational chemistry packages [4, 5]. The performance model of SCF computation has been built and the prediction have been compared with the results of measurements. The measurements have been provided on a mesh connected distributed memory parallel computer (128 T800 Parsytec SuperCluster). The prediction error is less than 10%. Performance optimisation of the application has been achieved by reducing the communication overhead and changing the data representation.

Matrix factorization (MF), as one of the most accurate and scalable approaches in dimension reduction techniques, has become popular in the collaborative filtering (CF) recommender systems, social network and graph communities. Currently, Kullback-Lerbler Non-negative Matrix Factorization (KL-NMF) is one of the most famous approaches for MF, due to its representative non-negativity feature of the CF model. However, it is non-trivial to obtain high performance KL-NMF on Graphic Processing Units (GPU) for large-scale problems, due to the redundant large-scale intermediate data, frequent matrices manipulation, and access of sparse and irregular entries characteristic of KL-NMF. In this work, we propose single-thread-based KL-NMF, which depends on the involved feature tuples multiplication and summation, and then, we present L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> norm regularized single-thread-based KL-NMF. On that basis, a novel CUDA parallelization KL-NMF approach (CuKL-NMF) is presented for GPU computing. Furthermore, to process large-scale CF data sets and make advantages of GPU computation power, we propose multi-GPU CuKL-NMF (MCuKL-NMF). Compared with state-of-the-art parallel algorithms, e.g., CUMF, CCD++, MCuKL-NMF obtains the highest performance.

Tensor algebra operations such as contractions in computational chemistry consume a significant fraction of the computing time on large-scale computing platforms. The widespread use of tensor contractions between large multi-dimensional tensors in describing electronic structure theory has motivated the development of multiple tensor algebra frameworks targeting heterogeneous computing platforms. In this paper, we present Tensor Algebra for Many-body Methods (TAMM), a framework for productive and performance-portable development of scalable computational chemistry methods. TAMM decouples the specification of the computation from the execution of these operations on available high-performance computing systems. With this design choice, the scientific application developers (domain scientists) can focus on the algorithmic requirements using the tensor algebra interface provided by TAMM, whereas high-performance computing developers can direct their attention to various optimizations on the underlying constructs, such as efficient data distribution, optimized scheduling algorithms, and efficient use of intra-node resources (e.g., graphics processing units). The modular structure of TAMM allows it to support different hardware architectures and incorporate new algorithmic advances. We describe the TAMM framework and our approach to the sustainable development of scalable ground- and excited-state electronic structure methods. We present case studies highlighting the ease of use, including the performance and productivity gains compared to other frameworks.

In this paper, we give efficient parallel and distributed algorithms for the topological sort problem on acyclic graphs with n vertices. Our parallel algorithm solves the problem on a CREW PRAM in O(log/sup 2/ n) time with O(M(n)/log n) processors, where M(n) denotes the number of processors needed to multiply two n/spl times/n integer matrices over the integer ring. The best known upper bound of M(n) is O(n/sup 2.376/). The parallel algorithm can also solve the problem on processor arrays with reconfigurable bus systems in O(1) time and O(n/sup 3/) processors. Our distributed algorithm solves the topological sort problem of an arbitrary asynchronous network with communication complexity O(n/sup 2/).

We describe the design philosophy, structure, and supporting tool kits of the NWChem computational chemistry package. The primary purpose of this effort was to develop efficient parallel algorithms for a broad range of methods commonly used in computational chemistry. To facilitate this, we developed a shared nonuniform access memory model which simplifies parallel programming while at the same time providing for portability across both distributed- and shared-memory machines. In addition to this specific focus on parallelization, a substantial effort has been made to simplify the general problem of large-scale software development, which is common to many research groups. We find that this simplification can be achieved through judicious use of ideas from the computer science field of software engineering in the design and implementation of the program with minimal extra effort on the part of the chemist. © 1995 John Wiley & Sons, Inc.