Recursive blocked data formats and BLAS’s for dense linear algebra algorithms

SUMMARYThe objective of this paper is to analyze the dynamic scheduling of dense linear algebra algorithms on shared‐memory, multicore architectures. Current numerical libraries (e.g., linear algebra package) show clear limitations on such emerging systems mainly because of their coarse granularity tasks. Thus, many numerical algorithms need to be redesigned to better fit the architectural design of the multicore platform. The parallel linear algebra for scalable multicore architectures library developed at the University of Tennessee tackles this challenge by using tile algorithms to achieve a finer task granularity. These tile algorithms can then be represented by directed acyclic graphs, where nodes are the tasks and edges are the dependencies between the tasks. The paramount key to achieve high performance is to implement a runtime environment to efficiently schedule the execution of the directed acyclic graph across the multicore platform. This paper studies the impact on the overall performance of some parameters, both at the level of the scheduler (e.g., window size and locality) and the algorithms (e.g., left‐looking and right‐looking variants). We conclude that some commonly accepted rules for dense linear algebra algorithms may need to be revisited. Copyright © 2011 John Wiley & Sons, Ltd.

This paper addresses the efficient exploitation of task-level parallelism, present in many dense linear algebra operations, from the point of view of both computational performance and energy consumption. The strategies considered here, referred to as the Slack Reduction Algorithm (SRA) and the Race-to-Idle Algorithm (RIA), adjust the operation frequency of the cores during the execution of a collection of tasks (in which many dense linear algebra algorithms can be decomposed) with very different approaches to save energy. The procedures are evaluated using an energy-aware simulator, which is in charge of scheduling/mapping the execution of these tasks to the cores, leveraging dynamic frequency voltage scaling featured by current technology. Experiments with this tool and the practical integration of the RIA strategy into a runtime show the energy gains for two versions of the QR factorization.

We show how to exploit high-level information, available as part of the derivation of provably correct algorithms, so that SMP parallelism can be systematically identified. Recent research has shown that loop-based dense linear algebra algorithms can be systematically derived from the mathematical specification of the operation. Fundamental to the methodology is the determination of loop-invariants (in the sense of Dijkstra and Hoare) from which correct loops can be systematically derived. We show how the high-level specification of the operation together with these loop-invariants can be exploited to detect the independence of loop iterations. This in turn then allows a Workqueuing Model to be used to implement and parallelize the algorithms using a feature proposed for OpenMP 3.0, task queues. Although performance is not the main feature of this paper, performance is reported on a 4 CPU Itanium2 server for a concrete example, the symmetric rank-k update operation.

This note describes some experiments on simple, dense linear algebra algorithms. These experiments show that the CRAY X-MP is capable of small-grain multitasking arising from standard implementations of $LU$ and Cholesky decomposition. The implementation described here provides the “fastest” execution rate for $LU$ decomposition, 718 MFLOPS for a matrix of order 1000.

A Biograph object is a data frame of individual life history data. All information on the life history of a person is stored in a single record. That data format is known as wide format. The wide format differs from the long format, in which information on an episode or a transition is contained in a separate record. The structure of the data frame is described in Sect. 3.2 of this chapter. In a Biograph object, life history data are organised chronologically starting with the first reported state of the life course and the first transition. That data structure is consistent with the life course as a sequence of events and a sequence of states. Converting raw data from surveys, registers or follow-up studies into a Biograph object can be cumbersome. Most surveys are not organised from a life history perspective but from a life domain perspective. In Sect. 3.3, I describe how to convert the GLHS data into a Biograph object. The GLHS data structure is only one of the many possible data structures. It is not possible to develop a single conversion method for all known data structures. More on how to create a Biograph object and several examples may be found in Annex A. Data analysis may require some operations on the data before the analysis can start. In multistate modelling, that may involve the removal from the data of transitions to the same state (intrastate transitions). In the GLHS data, job changes are intrastate transitions. Other operations may change the observation window. Biograph includes functions to change the observation window. One function selects transitions between two time points (calendar time) or between two ages. Another function delineates an observation window by two events or by an event and the survey date. These operations change the structure of the data. Data restructuring is the subject of Sect. 3.4. In Sect. 3.5 of this chapter, I review formats of life history data and list functions that may be used to convert one data format into another. A description of life histories includes information on states occupied during the period of observation, on transitions between states and on the dates of transition. Different ways exist to report dates. Most people use calendar dates, but dates may also be measured as time elapsed since a reference date or a reference event. The Century Month Code (CMC), used in the GLHS, measures dates as the number of months elapsed since 1 January 1900. Other surveys may express dates differently. Section 3.5 covers different date formats and R functions, including Biograph functions, for converting one date format into another.

We designed a data structure to promote the reuse of experimental data collected from multiple research institutions in a research project on solid electrolytes for next-generation batteries. Leveraging data from multiple research institutions is extremely challenging because the data formats and items differ, preventing their integration and joint use. Therefore, a data structure is required that meets the following criteria: (1) it is readily usable by researchers, (2) it introduces commonality into the data, and (3) it is easily understood by third parties. By analysing experimental data from several institutions, we decomposed each process into ‘elementary process units’ and proposed a ‘process‑chain’ data structure created by concatenating these units. Furthermore, by adopting RDE – the cloud‑based data acquisition system developed by NIMS – and linking the tabular format to the RDE format, we streamlined data collection and management. The proposed data structure is not limited to solid electrolytes but can also be applied to the management of experimental data for other materials, making it useful as a cross-disciplinary method for accumulating material data.

Language for the definition and exchange of biological data sets

We present a framework for compiling recurrence equations into native code. In our framework, users specify a system of recurrences, the types of data structures that store inputs and outputs, and scheduling commands for optimization. Our compiler then lowers these specifications into native code that respects the dependencies in the recurrence equations. Our compiler can generate code over both sparse and dense data structures, and determines if the recurrence system is solvable with the provided scheduling primitives. We evaluate the performance and correctness of the generated code on several recurrences, from domains as diverse as dense and sparse matrix solvers, dynamic programming, graph problems, and sparse tensor algebra. We demonstrate that the generated code has competitive performance to hand-optimized implementations in libraries. However, these handwritten libraries target specific recurrences, specific data structures, and specific optimizations. Our system, on the other hand, automatically generates implementations from recurrences, data formats, and schedules, giving our system more generality than library approaches.

\n Contains fulltext :\n 295725.pdf (Publisher’s version ) (Open Access)\n

Applications continue to increase in size and complexity which makes debugging and program understanding more challenging. Programs written in managed languages, such as Java, C#, and Ruby, further exacerbate this challenge because they tend to encode much of their state in the heap. This paper introduces dynamic shape analysis, which seeks to characterize data structures in the heap by dynamically summarizing the object pointer relationships and detecting dynamic degree metrics based on class. The analysis identifies recursive data structures, automatically discovers dynamic degree metrics, and reports errors when degree metrics are violated. Uses of dynamic shape analysis include helping programmers find data structure errors during development, generating assertions for verification with static or dynamic analysis, and detecting subtle errors in deployment. We implement dynamic shape analysis in a Java Virtual Machine (JVM). Using SpecJVM and DaCapo benchmarks, we show that most objects in the heap are part of recursive data structures that maintain strong dynamic degree metrics. We show that once dynamic shape analysis establishes degree metrics from correct executions, it can find automatically inserted errors on subsequent executions in microbenchmarks. These suggests it can be used in deployment for improving software reliability.

As clearly evidenced by Grady Booch in his book Software Components with Ada (Booch, 1987), data structures are an important part of reusable software components. The need for some of these software components arises, because of limitations in programming language design. In particular, the lack of recursive data structures in many languages requires contortions on the part of programmers that hampers the creation, the reuse, the portability, and the reliability of software components. We describe a preprocessor that translates definitions of recursive data structures to Ada.

This paper presents a framework to reason about the effects of assignments in recursive data structures. We define an operational semantics for a core language based on Meyer’s ideas for a semantics for the object-oriented language Eiffel. A series of field accesses, e.g. f1 ∙ f2 ∙ ··· ∙ fn , can be seen as a path on the heap. We provide rules that describe how these multidot expressions are affected by an assignment. Using multidot expressions to construct an abstraction of a list, we show the correctness of a list reversal algorithm. This approach does not require induction and the reasoning about the assignments is encapsulated in the mentioned rules. We also discuss how to use this approach when working with other data structures and how it compares to the inductive approach. The framework, rules and examples have been formalised and proven correct using the PVS proof assistant.

Automatic parallelization is a promising strategy to improve application performance in the multicore era. However, common programming practices such as the reuse of data structures introduce artificial constraints that obstruct automatic parallelization. Privatization relieves these constraints by replicating data structures, thus enabling scalable parallelization. Prior privatization schemes are limited to arrays and scalar variables because they are sensitive to the layout of dynamic data structures. This work presents Privateer, the first fully automatic privatization system to handle dynamic and recursive data structures, even in languages with unrestricted pointers. To reduce sensitivity to memory layout, Privateer speculatively separates memory objects. Privateer's lightweight runtime system validates speculative separation and speculative privatization to ensure correct parallel execution. Privateer enables automatic parallelization of general-purpose C/C++ applications, yielding a geomean whole-program speedup of 11.4x over best sequential execution on 24 cores, while non-speculative parallelization yields only 0.93x.

There are two reasons to have an efficient algorithm for identifying all right-maximal Lyndon substrings of a string: firstly, Bannai et al. introduced in 2015 a linear algorithm to compute all runs of a string that relies on knowing all right-maximal Lyndon substrings of the input string, and secondly, Franek et al. showed in 2017 a linear equivalence of sorting suffixes and sorting right-maximal Lyndon substrings of a string, inspired by a novel suffix sorting algorithm of Baier. In 2016, Franek et al. presented a brief overview of algorithms for computing the Lyndon array that encodes the knowledge of right-maximal Lyndon substrings of the input string. Among those presented were two well-known algorithms for computing the Lyndon array: a quadratic in-place algorithm based on the iterated Duval algorithm for Lyndon factorization and a linear algorithmic scheme based on linear suffix sorting, computing the inverse suffix array, and applying to it the next smaller value algorithm. Duval’s algorithm works for strings over any ordered alphabet, while for linear suffix sorting, a constant or an integer alphabet is required. The authors at that time were not aware of Baier’s algorithm. In 2017, our research group proposed a novel algorithm for the Lyndon array. Though the proposed algorithm is linear in the average case and has O(nlog(n)) worst-case complexity, it is interesting as it emulates the fast Fourier algorithm’s recursive approach and introduces τ-reduction, which might be of independent interest. In 2018, we presented a linear algorithm to compute the Lyndon array of a string inspired by Phase I of Baier’s algorithm for suffix sorting. This paper presents the theoretical analysis of these two algorithms and provides empirical comparisons of both of their C++ implementations with respect to the iterated Duval algorithm.

This document proposes a notational convention to express CBOR data structures (RFC 7049, Concise Binary Object Representation). Its main goal is to provide an easy and unambiguous way to express structures for protocol messages and data formats that use CBOR or JSON.