Numerical Algorithms Group (United Kingdom)

ExaFEL is an HPC-capable X-ray Free Electron Laser (XFEL) data analysis software suite for both Serial Femtosecond Crystallography (SFX) and Single Particle Imaging (SPI) developed in collaboration with the Linac Coherent Lightsource (LCLS), Lawrence Berkeley National Laboratory (LBNL) and Los Alamos National Laboratory. ExaFEL supports real-time data analysis via a cross-facility workflow spanning LCLS and HPC centers such as NERSC and OLCF. Our work therefore constitutes initial path-finding for the US Department of Energy’s (DOE) Integrated Research Infrastructure (IRI) program. We present the ExaFEL team’s 7 years of experience in developing real-time XFEL data analysis software for the DOE’s exascale supercomputers. We present our experiences and lessons learned with the Perlmutter and Frontier supercomputers. Furthermore we outline essential data center services (and the implications for institutional policy) required for real-time data analysis. Finally we summarize our software and performance engineering approaches and our experiences with NERSC’s Perlmutter and OLCF’s Frontier systems. This work is intended to be a practical blueprint for similar efforts in integrating exascale compute resources into other cross-facility workflows.

NeuroLF is a dedicated brain PET system with an octagonal prism shape housed in a scanner head that can be positioned around a patient’s head. Because it does not have MR or CT capabilities, attenuation correction based on an estimation of the attenuation map is a crucial feature. In this article, we demonstrate this method on [18F]FDG PET brain scans performed with a low-resolution proof of concept prototype of NeuroLF called BPET. We perform an affine registration of a template PET scan to the uncorrected emission image, and then apply the resulting transform to the corresponding template attenuation map. Using a whole-body PET/CT system as reference, we quantitively show that this method yields comparable image quality (0.893 average correlation to reference scan) to using the reference µ-map as obtained from the CT scan of the imaged patient (0.908 average correlation). We conclude from this initial study that attenuation correction using template registration instead of a patient CT delivers similar results and is an option for patients undergoing brain PET.

Mesophotic coral reefs, generally defined as deep reefs between 30 and 150 m, are found worldwide and are largely structured by changes in the underwater light field. Additionally, it is increasingly understood that reef-to-reef variability in topography, combined with quantitative and qualitative changes in the underwater light field with increasing depth, significantly influence the observed changes in coral distribution and abundance. Here, we take a modeling approach to examine the effects of the inherent optical properties of the water column on the irradiance that corals are exposed to along a shallow to mesophotic depth gradient. In particular, the roles of reef topography including horizontal, sloping and vertical substrates are quantified, as well as the differences between mounding, plating and branching colony morphologies. Downwelling irradiance and reef topography interact such that for a water mass of similar optical properties, the irradiance reaching the benthos varies significantly with topography (i.e. substrate angle). Coral morphology, however, is also a factor; model results show that isolated hemispherical colonies consistently ‘see’ greater incident irradiances across depths, and throughout the day, compared to plating and branching morphologies. These modeled geometric-based differences in the incident irradiances on different coral morphologies are not, however, consistent with actual depth-dependent distributions of these coral morphotypes, where plating morphologies dominate as you go deeper. Other factors, such as the cost of calcification, arguably contribute to these differences, but irradiance-driven patterns are a strong proximate cause for the observed differences in mesophotic communities on sloping versus vertical reef substrates.

The solution of large sparse linear systems is one of the most time consuming kernels in many 4 numerical simulations. The domain decomposition community has developed many efficient and robust methods in 5 the last decades. While many of these solvers fall into the abstract Schwarz (aS) framework, their robustness has 6 originally been demonstrated on a case-by-case basis. In this paper, we propose a bound for the condition number 7 of all deflated aS methods provided that the coarse grid consists of the assembly of local components that contain 8 the kernel of some local operators. We show that classical results from the literature on particular instances of 9 aS methods can be retrieved from this bound. We then show that such a coarse grid correction can be explicitly 10 obtained algebraically via generalized eigenproblems, leading to a condition number independent of the number of 11 domains. This result can be readily applied to retrieve or improve the bounds previously obtained via generalized 12 eigenproblems in the particular cases of Neumann-Neumann (NN), Additive Schwarz (AS) and optimized Robin but 13 also generalizes them when applied with approximate local solvers. Interestingly, the proposed methodology turns 14 out to be a comparison of the considered particular aS method with generalized versions of both NN and AS for 15 tackling the lower and upper part of the spectrum, respectively. We furthermore show that the application of the 16 considered grid corrections in an additive fashion is robust in the AS case although it is not robust for aS methods in 17 general. In particular, the proposed framework allows for ensuring the robustness of the AS method applied on the 18 Schur complement (AS/S), either with deflation or additively, and with the freedom of relying on an approximate 19 local Schur complement. Numerical experiments illustrate these statements.

Workshops are used to explore a specific topic, to transfer knowledge, to solve identified problems, or to create something new. In funded research projects and other research endeavours, workshops are the mechanism used to gather the wider project, community, or interested people together around a particular topic. However, natural questions arise: how do we measure the impact of these workshops? Do we know whether they are meeting the goals and objectives we set for them? What indicators should we use? In response to these questions, this paper will outline rules that will improve the measurement of the impact of workshops.

Loop collapsing is a well-known loop transformation which combines some loops that are perfectly nested into one single loop. It allows to take advantage of the whole amount of parallelism exhibited by the collapsed loops, and provides a perfect load balancing of iterations among the parallel threads. However, in the current implementations of this loop optimization, as the ones of the OpenMP language, automatic loop collapsing is limited to loops with constant loop bounds that define rectangular iteration spaces, although load imbalance is a particularly crucial issue with non-rectangular loops. The OpenMP language addresses load balance mostly through dynamic runtime scheduling of the parallel threads. Nevertheless, this runtime schedule introduces some unavoidable executiontime overhead, while preventing to exploit the entire parallelism of all the parallel loops. In this paper, we propose a technique to automatically collapse any perfectly nested loops defining non-rectangular iteration spaces, whose bounds are linear functions of the loop iterators. Such spaces may be triangular, tetrahedral, trapezoidal, rhomboidal or parallelepiped. Our solution is based on original mathematical results addressing the inversion of a multi-variate polynomial that defines a ranking of the integer points contained in a convex polyhedron. We show on a set of non-rectangular loop nests that our technique allows to generate parallel OpenMP codes that outperform the original parallel loop nests, parallelized either by using options “static” or “dynamic” of the OpenMPschedule clause.

The aim of this study was to evaluate cost-effectiveness of Acceptance and Commitment Therapy (ACT) and workplace dialogue intervention (WDI), both as stand-alone interventions and in combination, compared with treatment as usual (TAU), for employees on sickness absence with mental disorders. Employees (n = 352, 78.4% females) on sickness absence were randomized to one of four groups. Cost-utility analyses were conducted from a health care perspective and a limited societal perspective. All groups reported significant improvements in health-related quality-of-life (HRQoL) and there were no significant differences in HRQoL or costs between groups. The probability of cost-effectiveness for ACT+WDI was 50% compared with ACT, indicating that both treatment alternatives could be considered equally favorable for decision-makers. TAU and WDI were rejected due to less economic efficiency. Adding WDI to ACT cannot be recommended on the basis of our study results.

SummaryAs the computational power of high‐performance computing systems continues to increase by using a huge number of cores or specialized processing units, high‐performance computing applications are increasingly prone to faults. In this paper, we present a new class of numerical fault tolerance algorithms to cope with node crashes in parallel distributed environments. This new resilient scheme is designed at application level and does not require extra resources, that is, computational unit or computing time, when no fault occurs. In the framework of iterative methods for the solution of sparse linear systems, we present numerical algorithms to extract relevant information from available data after a fault, assuming a separate mechanism ensures the fault detection. After data extraction, a well‐chosen part of missing data is regenerated through interpolation strategies to constitute meaningful inputs to restart the iterative scheme. We have developed these methods, referred to as interpolation–restart techniques, for Krylov subspace linear solvers. After a fault, lost entries of the current iterate computed by the solver are interpolated to define a new initial guess to restart the Krylov method. A well‐suited initial guess is computed by using the entries of the faulty iterate available on surviving nodes. We present two interpolation policies that preserve key numerical properties of well‐known linear solvers, namely, the monotonic decrease of the A‐norm of the error of the conjugate gradient or the residual norm decrease of generalized minimal residual algorithm for solving. The qualitative numerical behavior of the resulting scheme has been validated with sequential simulations, when the number of faults and the amount of data losses are varied. Finally, the computational costs associated with the recovery mechanism have been evaluated through parallel experiments. Copyright © 2016 John Wiley & Sons, Ltd.